Big data: looking inward, looking outward

  • Many companies are using big data to look inward and improve operations.

  • These efforts can increase effectiveness and efficiency.

  • A successful project can springboard inward-to-outward efforts.

Big data in the enterprise has a trajectory for success. Nemertes’ 2014-15 Enterprise Technology Benchmark shows that nearly two-thirds of organizations now claim to have a big data effort, and most start with an inward-facing objective. Although the media hype is about how big data helps you better understand your customers, your public image, or the effectiveness of your advertising, our research shows most organizations are instead looking at operational data. Their goal is to improve their ability to execute on their core missions, or to cut overhead; that is, how to become more effective or more efficient.

There are good reasons for this operational focus:

  • They have the data. Although they may be bringing together for analysis streams of data previously kept separate (e.g., card key swipe records and patient care outcomes in a hospital), they are usually not generating or acquiring entirely new streams. To be sure, new data may become available. A trucking company may put GPS locators in all its trucks, for example, or a manufacturer may get more data about the materials in its supply chain via expanded feeds from suppliers—but the data is a byproduct of instrumented operational systems.
  • They understand it. That is, they know how to interpret each record and can even understand relationships among records easily. What they are looking for is patterns and relationships involving the newly joined data streams; how does this information from a pipeline system match up with this information from a storage and shipping facility, and that information from consumer meters, to show where natural gas is leaking from the network?
  • They have volumes that let them work with tools they already understand. They may be bringing more records into their Oracle or SQL Server relational database, for example, or expanding use of a special purpose big data tool such as Splunk, but they have what they need and can work with it already.

Efforts to improve operationally that require minimal new investment in tools, systems, or staff are easier to advance than efforts aiming at external targets, using new external data, and requiring new tools and a new staff skill set. However, they can be a springboard to get the organization more data-powered and hungry for more data on which to base decisions in a broader set of contexts.

Our benchmark research also shows that the most successful big data efforts are those with outward-facing goals, such as entering or exiting markets. Those wondering how to use big data to make a difference in their organization should consider plotting an inward-to-outward path for their effort.

Do you have questions about your existing or upcoming big data effort? Leave them in comments below and I’ll do my best to answer them.

John Burke Principal Research Analyst Nemertes Research About John