Big Data Is More Than Hype, It'll Change the Way We Do Business

The flow of digital data through the global economy has grown from a trickle to a torrent over the last decade. Consulting firm McKinsey & Company estimates that the typical large enterprise now holds somewhere in the neighborhood of 200 terabytes of stored data.

Today, organizations capture trillions of bytes of transactional data about customers, constituents, students, suppliers and business operations. This is done via vast networks of computers, mobile phones, sensors embedded in machines and an array of other sources.

Although there’s no exact definition for big data, McKinsey and other consultancies generally apply the tag to datasets that extend beyond the range and scope of a single database or application. Typically, an organization will require an overarching strategy as well as tools and technologies to put the various databases, repositories and data sources to work in a deeper way.

The ultimate goal is to create greater transparency across systems and provide insights that wouldn’t be possible using conventional approaches. Typically, big data combines structured and unstructured data — and incorporates the use of metadata to make information actionable.

In some cases, big data uses artificial intelligence and specialized algorithms to analyze overall data patterns — and the relationship with data points against other data points.

The Big Data Challenge

What makes today’s data environment so challenging is that it’s impossible to build a database big enough to accommodate all the data sets that can potentially fuel innovation or a competitive advantage. What’s more, it’s nearly impossible to identify the specific data to tap into at any given moment and know how to apply the data to real-world problems. Much of the power of big data is applying combinations of data sets in new and unusual ways.

To be sure, the technology allows organizations to develop complex computer models and sophisticated simulations that wouldn’t have been possible only a few years ago. For instance, big data might help a retailer understand what conditions affect sales of certain products or when consumer sentiment is changing in regard to a company’s brand and reputation. Or, it might help a manufacturer predict when a piece of equipment will fail or a financial services firm identify when fraud takes place.

According to McKinsey, big data creates value in several ways:

• It creates transparency by allowing an enterprise to zero in on core issues and understand how to operate the business more efficiently
• It enables experimentation and the exploration of what-if scenarios
• It allows organizations to segment populations and customize actions
• It replaces or supports human decision-making with automated algorithms

The end result, McKinsey reports, is a business culture that ratchets up innovation while reducing risk.