How Knowledge Management Will Change With the Advent of Machine Learning and Cognitive Search

Jeff Fried, Director of Product Management, InterSystems and Chair of the Advisory Board for BA Insight
Jeff Fried, Director of Product Management, InterSystems and Chair of the Advisory Board for BA Insight

Jeff Fried, Director of Product Management, InterSystems and Chair of the Advisory Board for BA Insight

Knowledge management is a dynamic field, and a new wave of technology is changing the possibilities and the practices in content curation, search and discovery, and knowledge analytics. In this article, we discuss how the technology is advancing in those areas and how this translates to the promise of improved organizational efficiency and competitiveness.

  Effective knowledge management can dramatically improve an organization's efficiency and competitiveness 

With all the hype around artificial intelligence (AI) and cognitive computing, it’s hard to distill the real advances and understand how to actually apply those new technologies to improve knowledge management. We've previously written about separating the hype from the reality and how to leverage these technologies as they advance. Machine learning and cognitive search are two of the most ready-for-prime-time technologies, and both apply directly to knowledge management.

Knowing what you know: better content curation

The industry has evolved techniques for curating enterprise content, labeling it with the key concepts involved, linking it to related content, and indicating the degree of authority and applicability.However, manual curation is extremely costly, and traditional automation also requires more effort than many organizations are able or willing to provide. Most organizations are therefore limited in how much and how well they curate, which in turn limits how effective organizations are at getting the right content in front of users when and where they need it.

Machine learning has advanced rapidly in its ability to automate content curation by using algorithms to scan content items, find similarities between sources, and cluster them in logical groups. For the most part, recent advances aren't actually new algorithms - they are the result of fast, cheap computing, the availability of huge sets of data, open-source software and models, and easier administration and development environments. Machine curation is not perfect, but it can now be applied economically to a much broader range of content - both inside and outside of organizations.

The flip side of knowing what you know is knowing what you don't know. Machine learning and cognitive systems also have the potential to help organizations better understand their content and pinpoint areas where they have insufficiencies or gaps. Where traditional techniques tended to use fixed taxonomies and ontologies, the new generation of systems can support dynamic content structures that adapt to and reflect ongoing changes in vocabularies and knowledge domains. This will make it easier to break down organizational and content silos, re-use content in different contexts, and tap into trends and knowledge from outside the organization.

Finding what you know: improved search and discovery

In the majority of organizations, enterprise search has failed to live up to the high expectations set by public web search engines.Despite remarkable improvements in search technology, the volume of content and the demand for comprehensive answers without information overload aren't keeping up; employees still complain that they can’t find what they need. 

The possibilities for cognitive computing to enhance enterprise search and discovery processes are enormous. While many intriguing opportunities exist for cognitive applications, 40 percent of KM professionals believe that search carries the most potential in the context of KM. Interaction with search is becoming much more natural, as systems use natural language processing to interpret the meaning of requests and extend their reach to new data sets and scenarios. People are becoming accustomed to digital assistants in their personal lives - from Apple's Siri, Amazon's Alexa, Microsoft's Cortana, and Google's Duplex as well as a rising tide of specialized voice and chatbot services. This makes them ready to interact this way with their knowledge management system at work - and more likely to expect it. 

The ability to pose questions in natural language and receive explicit answers in narrative form will not only save employees time searching for information;it will also provide new insights - because the time and cost of analysis and exploration is going down quickly. This applies not only to finding information, but also to expertise location - finding the people with the right knowledge.

Finding patterns through knowledge analytics

New insights will also come from the ability for machines to assimilate more data than humans can, and to identify patterns in knowledge that might be invisible to humans. There are many areas in knowledge management that can benefit from this.Lessons-learned programs can be highly beneficial, but are too often skipped because of the effort involved, and may miss important connections that could be identified by machine learning. Knowledge-sharing programs can benefit from machine recommendations of what knowledge to share with which groups at what time. Automated scanning of project artifacts can identify risks, potential compliance issues, and surface trends. The combination of reduced effort through automation and new insight through machine-assisted analysis promises many benefits.

Machine-assisted data visualization (sometimes referred to as Visual Analytics) uses AI techniques coupled with data visualization to make knowledge exploration faster and easier. In the visual analytics paradigm, a human enters into a discourse with a software system about some data, querying it and receiving results back in visual form, including images that are constructed from text as well as numerical data, and machine-generated captions. This type of knowledge analytics is interactive, allowing users to answer specific questions, or just get a feel for what knowledge and data are out there and how things are connected.

New technologies in a mature framework

Effective knowledge management can dramatically improve an organization's efficiency and competitiveness, by ensuring that staff has the information they need when they need it. The overall framework of knowledge management encompasses more than technology - it includes many cultural and human elements as well. While the products and practices of knowledge management are in the midst of rapid change as they tap into a new wave of technology, the overall framework for knowledge management will remain much the same.

One important thing to keep in mind: Machine learning and cognitive search aren't magic. They rely on high-quality data, and lots of it. As you adopt these new technologies, remember that learning starts with the basics, and use a continual-improvement approach to make this technology work for you.

Read Also

Technology in Legal Industry, a Far Fledged Dream

Technology in Legal Industry, a Far Fledged Dream

Scott Rechtschaffen, Chief Knowledge Officer, Littler Mendelson
Knowledge Management - A Pragmatic View

Knowledge Management - A Pragmatic View

Christopher Harrison, CTO, Nova Southeastern University
Create Constant Growth for QA Professionals (or Perish)

Create Constant Growth for QA Professionals (or Perish)

Tim Harrison, Principal & Chief Knowledge Officer, SQAsquared
Current Market Trends Shaping The Knowledge Management Space

Current Market Trends Shaping The Knowledge Management Space

Dora M. Tynes, Chief Knowledge Officer, Sheppard Mullin Richter & Hampton LLP