Is Knowledge Management a Substitute or Complement to Big Data?
For a number of years, the distinction between data, information and knowledge could be stated with some level of precision. Data are raw facts and figures that can become information when massaged and placed in the right context. The value adding activities from data to information are largely the domain of information systems. Knowledge, however, adds experience and expertise to the information, and often resides in tacit form in people’s heads. So, the Knowledge Management System (KMS) popularity that started in the 1990s, was intended to capture knowledge (largely tacit) and put it in a system that could benefit others in the organization. KM processes include externalization (taking tacit knowledge and representing it in a KMS) and internalization (making this knowledge accessible to people who might need it). For example, a consulting firm might have a team that concluded a multi- year project in Malaysia – and their experiences, successes, failures, precautions, and guidance in a KMS, could be invaluable for other teams initiating projects in that region. KM practices included creating the right incentives for knowledge to flow between people and the system, as well as embedding knowledge into products and services offered.
The advent of “big data” in recent years, along with advanced analytics and machine learning has created a disruption in the practice of KM. Big data, through its sheer volume, velocity and variety, along with computationally intensive analysis, offers opportunities for generating new insights. Mining of big data and digital streams can yield fresh perspectives for decision makers, optimize and automate processes, and discover new ways to understand and fulfill customers’ needs. However, in observing the ready embrace of big data and analytics by business, there are also signs that some companies might fall into a trap where they see data replacing knowledge, blurring the distinctions in the trichotomy described above. The argument implicit in this, is that knowledge was largely a correspondence between our observation of the world and its interpretation in our brain, in the form of models that we used to make sense of observations. Now, with the plethora of data, we don’t need to understand the world to know it. All knowledge can be extracted through the data. So, in this view, knowledge is not internalized through human assessment, but externalized through data.
The advent of “big data” in recent years, along with advanced analytics and machine learning has created a disruption in the practice of KM
While many may not endorse the data is knowledge view, companies that intensively invest in data and analytical capabilities, could be accepting a data culture that comes at the cost of human interpretation and judgment. Companies where everything revolves around data science and “show me the data” is the prevailing edict, they might be overshooting the mark. There are caveats with equating big data management with knowledge management. We present three, but there could be more.
First, big data and analytics is largely about prediction; knowledge is about explanation and causality. So, the presumption is that that if data can be used to predict accurately, then there is little reason to understand why the prediction works. While this could be true for machine learning algorithms that read an MRI, the vast majority of businesses deal with customers and employees, and human behaviors. Often big data sets are digitally captured (e.g., on apps, websites) for transactional purposes, and not for predicting certain outcomes. Therefore, such data sets are replete with correlates that could yield high predictive accuracy but be fallacious. Shark attacks are good predictors of ice-cream sales on the beach – which could lead to ridiculous decision for an ice-cream seller. An obvious correlate in this confounding is the temperature. However, to decipher this, a data set is needed that includes that correlate, and expertise is needed to identify the correlation-causality fallacy.
Second, big data by definition is historical…and subject to biases. Predictions are only as good as the data. So, predicting the profile of effective US Senators based on historical data on prior senators, will inevitably have “male” in the profile as the dominant group in the population, reinforcing prior prejudices. Similarly, evaluations of products and services could be due to herd effects, where users provide good ratings because other ratings are positive. Understanding limitations of prior data requires human knowledge, expertise and experience in the domain of interest.
Thirdly, the data is often subject to the “streetlight effect” where access has primacy over the questions asked. Instead of developing strategic questions, the inclination would be to mine the data to see what questions can be addressed. Or, a manager might simply use accessible data over having a systematic data capture strategy. For example, a supermarket chain manager engages in data mining to identify per-product profitability using historical sales and cost data. Then, based on the results, a decision is made to purge over a dozen product lines. The decision might be absolutely wrong, if a broader data set that examined shopping behaviors of customers who purchased these products on a single shopping trip. Such data might reveal that some products being purged could be highly profitable as high-end customers were drawn to the store due to that product and brought a number of high-margin products on the same trip. Here again, understanding and domain experience would have been invaluable in asking better questions and spreading a wider data net.
The key point here is the knowledge management involves identifying and codifying knowledge that is replete with experience and expertise. Big data offers tremendous potential, but companies that see it as a panacea to replace the insights and intuition, often accumulated through education and experience would be a mistake. So human judgment and domain expertise is needed on both sides of big data and its analysis – to frame the right questions, access the right data sources, and to interpret the veracity and viability of the results for decision making. So, companies that can create synergies between their KMS and their big data and analytical systems are more likely to do better. This can occur in many ways, and in both directions. Knowledge codified in a KMS can be subject to enhancement through big data analysis. For example, externalizing knowledge from top sales people in a KMS on what are their points of emphasis for selling successfully to customers, can be enhanced through text analysis of customer feedback or reviews on what customers like. Similarly, strong prescriptive implications emerging from big data analysis can be represented as knowledge in the KMS. A trucking company running optimization algorithms across massive data in its fleet might derive useful prescriptions for fuel conservation that can be codified in a KMS.
Big data is not knowledge, but if captured and analyzed well, it can offer important tactical predictions. However, to truly leverage the power of big data, companies should not simply jump on this technology driven bandwagon, but carefully develop the multi-level structures and mechanisms so that big data systems can work synergistically with human knowledge –answering the rhetorical question in the title.