Big data-KM’s bowl, water or goldfish?
Big data poses one of those puzzles with which my Psych 101 professor taunted the class. Consider this question: Is big data the bowl, the water in the bowl, or the goldfish in the water in the bowl?
Big data is everywhere. People and companies are striving to figure out what big data means to their organizations, how to make sense from hourly multiterabyte flows of information, and what to do with big data once some of it is in hand.
For knowledge management professionals, is big data like “water” for knowledge management applications? Most KM systems were not designed to cope with massive flows of open source and third-party digital information.
Tata Consultancy Services published “Managing Knowledge from Big Data Analytics in Product Development,” a white paper focusing on new product development (tcs.com/SiteCollectionDocuments/White%20Papers/Knowledge-Big-Data-Analytics-Product-Development-1213-1.pdf). It states: “In order to benefit from this data, organizations must have a well-defined strategy to collect, store, synthesize and disseminate it in the form of knowledge required for various business functions. For example, product ideas, customer behavior patterns, voice of customer (VoC) data, quality function deployment (QFD) data, and product trends from social networks and listening platforms can help design product strategy and portfolio. Warranty, quality and testing data, data from computer-aided design (CAD) systems and manufacturing process data can help in product design and validation, as information is fed back into the new product development (NPD) process.”
Tata’s approach is that big data is the environment in which new product development and other enterprise functions take place. Are organizations equipped to “collect, store, synthesize and disseminate it [on point information] in the form of knowledge ... ?”
The bowl and the fish
What about the “bowl”? Organizations often find cloud technology forced upon them. Cloud solutions generate images of balance sheets that present sharply reduced costs for computer hardware, storage devices and network infrastructure. Amazon’s marketing of its AWS product and service line has stressed the value of its low-cost cloud computing systems. Google and Microsoft have followed Amazon’s lead in order to capture a share of the market. Dell, Hewlett-Packard and IBM also offer cloud solutions. Those companies find themselves in a dogfight.
When price is the differentiator, traditional value-based pricing clashes with the low-cost strategy. Dell and EMC provide storage-centric solutions. Organizations tapping big data face stark choices: Finance or buy dedicated storage systems or use commodity storage available from the cloud.
The bowl is where the big data reside. Unlike the goldfish bowl in my office, the bowl required to handle flows of big data has to be able to handle larger and larger flows of data. Getting larger bowls is expensive. Adding storage can be costly too.
What about the “fish”? I like to think of myself and those working in organizations as comfortable in a particular environment. Big data is a larger and more important part of what I call the “datasphere.” Just 10 years ago, only a small number of organizations (primarily governmental entities engaged in intelligence operations) and data-centric companies like FedEx and UPS were trying to adapt to what Vivisimo’s Raul Valdes-Perez described as the “volume, velocity and variety”of big data. After IBM acquired Vivisimo in 2012, IBM added “veracity.” See the infographic (Download graphic which is also on pge 16, February KMWorld, Vol 24, Issue 1), prepared by IBM.
Figuring out what is going on with information that manifests the four Vs is a difficult job. The fish like me may not be able to adapt to the new information environment. The implications of the impact of big data on knowledge workers may be an issue requiring considerable management attention.
The question about the bowl, the water or the fish is one that would spark considerable discussion among knowledge management professionals. The challenge boils down to four issues.
Four questions
What big data does an organization need? Is it tens of millions of Twitter messages, flows of third-party clickstream information, analyses of Facebook postings for positive and negative signals about a company’s products and services, or data from customers using devices that phone home real-time information about location, searches for information and purchases?
What business processes can make use of big data? If big data influences every aspect of a business, how can organizations afford the supporting infrastructure to allow a purchasing department to track data from automated failure analysis systems to identify substandard suppliers in real time? Amazon and airlines can vary prices dynamically to maximize yield. How can an automobile insurance company match collision claims with policy pricing?
What systems and subject matter experts are required to select, procure, configure and maintain a next-generation information system? Robust solutions are available from BAE Systems, Leidos (formerly SAIC) and Knowlesys, among others. Most of those companies’ products are not widely known among U.S. commercial enterprises. Specialist knowledge is required to know what companies to contact for assistance in building a big data infrastructure that hooks into an organization’s business processes.
How can companies and government agencies adapt to the rapid change in the big data source streams? Institutions respond less quickly than a small startup. Big data’s four Vs stress institutions at business process weak spots: knowledge workers.