Not every small company has to store 10 terabytes of data under its roof, but then not every business is working to uncover the secrets of human life. Agencourt BioSciences of Beverly, Mass., provides human-genome sequencing to researchers, a service that accumulates dizzying amounts of data.
"We manage about 80,000 customer samples every day, seven days a week, and because we have to create complete audit trails for customers, that amounts to close to 800,000 database records every day," says Brendan McKernan, vice president of operations. And over the next two years, he expects that 10T of data under storage to double.
Agencourt's challenge isn't just data volume—it must also call up critical data quickly. "Even though everything we do is scientific-based, our end product is information," McKernan says.
An application such as Agencourt's demands a storage system optimized for both high speed and reliability. But a company doesn't have to store 10T of data to benefit from real-time storage systems, which offer not only faster access times but also increased redundancy. Delays in querying a client file during a support call, analyzing a financial history for quick credit approval or processing transactions on a Web site, for example, can drive customers away. Although real-time storage will initially cost more, not upgrading could have a larger eventual cost.
A lethargic storage system can end up costing tens of thousands of dollars, says Greg Schulz, author of Resilient Storage Networks: Designing Flexible Scalable Data Infrastructures and a senior analyst with the storage analysis company Evaluator Group of Greenwood Village, Colo.
Is It for Me?
How can small companies use real-time storage effectively without spending too much? Four steps are key:
Step 1: Spend the extra money where it will pay the highest dividends. Use high-performance storage systems only for applications that directly generate revenue, such as customer data exchanges or running production lines. Companies can trade off the additional cost of a speedy system by tightening budgets for second-tier applications.
Step 2: Choose enterprise-class hard drives and communications technology. This means using high-speed Small Computer Systems Interface (SCSI) hard drives that have mean time between failure ratings of around 1.5 million hours.
Agencourt runs clusters of speedy SCSI drives that are connected by Fibre Channel pipelines, a 2-gigabit-per-second networking topology long used for demanding storage applications. "Because we're moving vast amounts of data, we have to get it off the storage servers very quickly," McKernan says.
WH Trading LLC, a Chicago financial trading company, relies on 15,000-rpm Ultra320 SCSI drives for each of the dozen servers that host the company's core trading applications.
"We need real-time computing power to perform computations immediately and report the results to traders," says Mike Madigan, chief technology officer for the 40-person company. Rather than sending data over a Fibre Channel network, Madigan pairs a SCSI drive to each server to avoid data transfer lag time.
Step 3: Build in redundancy and high availability. Speed isn't the only factor for high performance. Real-time storage systems must be reliable, and drives with even the highest failure ratings can and do fail. Businesses should build in redundancy by installing dual controllers—the direct connection between hard drives and computers. If one controller stops working, management software can immediately route traffic over the spare connection.
Storage analyst Schulz also recommends installing extra power supplies and cooling fans to keep these components from bringing expensive storage gear to a standstill.
Step 4: Consider the broader IT environment. For example, build in further redundancy by having multiple servers that pull data from the storage systems. "We have redundant machines so we can flip over if a problem occurs," Madigan says.
Real-time storage comes at a price premium so businesses will have to evaluate cost against improved service to decide whether it's a worthwhile return on investment. Schulz estimates that companies will typically pay 50 percent to 100 percent more for the extra technology than for near-time or slower-access storage. But businesses can usually justify the extra cost if they are running the storage equipment for core business systems—and "it can be a cheap insurance policy," Schulz says.
|First quarter 2005||Change since first quarter 2004|
|Total Hard Drive Revenue||$5.5 billion||6.0%|
|Network Storage Revenue||$2.3 billion||16.7%|
Source: International Data Corp.
Sometimes, it makes more sense for a small business to buy rather than build real-time storage. That approach made sense for Akimbo Systems.
A growing 35-person company, Akimbo sells video-on-demand products that let customers download digital movies, TV shows and Major League Baseball games to TV set-top boxes or, in the near future, to their PCs.
By purchasing third-party storage capacity, the San Mateo, Calif., company gets high availability without the headache of managing the company's burgeoning storage growth itself.
"We're getting close to processing a terabyte of data a month," says Tom Hammer, chief technology officer and vice president of engineering.
Processing for Akimbo primarily means digitizing and compressing thousands of hours of video content in preparation for delivery to customers. "Eventually we could have hundreds of terabytes as we add hundreds of thousands of hours of content to our library," Hammer says.
While price was a factor in the decision to outsource storage, an even more important consideration was reliability. "The benefit I get is built-in redundancy for my content farm because the content is replicated on servers throughout the country," Hammer says. "That built-in replication alleviates my need to mirror my servers for purposes of disaster recovery."
Before you make the jump to storage on demand: