Today, organisations across industries have access to increasingly large volumes of data. Often, this data is in the form of large, complex documents that contain industry-specific information. The sheer volume and complexity of the data available introduces both challenges and opportunities – When organisations have so much data, it can be difficult for stakeholders, employees, or even customers to find the information they’re looking for. When organisations find techniques to leverage the data they have, however, that data can be a valuable tool. Across industries, AI-enabled search is already enabling organisations to unlock numerous possibilities, from improving information access to enhancing research and business processes.
To understand what AI-enabled search offers, we can compare it to traditional search engines like Google. Traditional search engines crawl through the Internet, building up an index of hundreds of billions of web pages. This allows users to search through a variety of content available on the web. AI-enabled search also allows users to search through indexed data – However, it unlocks search possibilities far beyond those of traditional search engines.
Unlike traditional search, AI-enabled search technology is built on large language models (LLMs) and uses natural language processing to consume and index a variety of data in various formats. This allows organisations to search through their internal organisational data, rather than just external, publicly available data. This functionality allows organisations to answer business-specific questions and develop a more comprehensive understanding of their operations.
Additionally, AI-enabled search provides another benefit over traditional search engines – By using natural language processing, it can understand and interpret data by recognising synonyms, relationships, and context. Rather than simply returning a list of sources, it can provide a summary of the data returned. This ability makes AI-enabled search invaluable for businesses aiming to extract insights from large, complex documents.
With AI-enabled search, organisations can also search through and summarise large amounts of data in a way that previously was not feasible without a large degree of manual labour. As a result, this tool unlocks unprecedented access to data and insights, while cutting down on the time-consuming and error-prone nature of manual search.
With all these capabilities, AI-enabled search technology is emerging as a powerful tool for organisations across the private and public sectors. In fact, you’ve likely already encountered this technology through tools like OpenAI’s Chat-GPT platform, powered by GPT-3 and GPT-4 LLMs and Google’s Bard AI, powered by LaMDA. Those tools are only a small subset of how organisations are using AI-enabled search, however – With the ability to search through internal and external data sources and understand data in ways traditional search engines cannot, AI-enabled search is unlocking new, exciting capabilities across the public and private sectors.
Applications of AI-Enabled Search
An increasing number of organisations across the private and public sectors are incorporating AI-enabled search into their internal and external operations. Let’s dive into a few real-world examples of how organisations are already leveraging this technology.
Facilitating document search and retrieval
Public institutions often have access to more records than they can manually archive. AI-enabled search can accelerate the process of archiving these resources, as well as make them more accessible to the public. The National Archives and Records Administration in the UK, for example, has explored the use of AI to facilitate the process of identifying documents. Because the institution maintains UK government records, they’re responsible for managing a massive quantity of documents. AI-enabled search provides a promising approach to making these documents easier to archive, as well as more accessible to the public. While AI-enabled search certainly cannot replace the skills of human archivists, it can aid in the process of finding records, especially in unstructured collections.
The Stonnington Libraries in Australia have also pursued the use of AI-enabled search to make their data more accessible. Before incorporating AI into their systems, library personnel were manually archiving all their data. However, with the help of AI, they were able to extract meta information from their files automatically, creating a searchable catalogue. With this technology, the library system has cut down on the time library personnel need to spend manually archiving data, while making resources much more accessible.
Improving access to organisational knowledge
An organisation’s internal data is often difficult for employees to index and search. With AI-enabled search, organisations can develop systems that allow employees to search through their internal data more efficiently. This is especially useful for organisations that have large amounts of data that employees must search through and reference.
For example, engineers at the Australian energy company Woodside Energy rely on a mixture of company knowledge, historical context, and procedural information to effectively accomplish their work. However, in the past, a great deal of that knowledge was maintained by senior experts, making it difficult for junior employees to locate, analyse, and learn from the relevant knowledge. This meant that, on average, engineers spent up to 80% of their time researching solutions and hazards, and only 20% on engineering work.
To address this limitation, Woodside Energy leveraged AI technology to create an easily searchable database for its employees, creating a solution that allowed easier access to knowledge. After incorporating this system into their work, employees reduced their time spent researching to only 20%, with the remaining time freed up for their engineering work. By making their internal data easy to search through, the energy company made it possible for employees to find the information they needed faster and more easily. Additionally, they saved over 10 million USD worth of time.
Morgan Stanley, a leader in financial services, is another company that pursued a similar solution. The company has a massive internal content library, which contains documents covering investment strategies, market research and commentary, and analyst insights, that financial advisors often have to scan through to manually find the information they need. Because this is such a time-consuming process, Morgan Stanley developed an internal-facing chatbot that allowed financial advisors to search through their internal content library. By interfacing with this chatbot rather than searching for data directly, advisors can now find the relevant information they need quicker and more efficiently.
Optimising business processes
We’ve just covered how AI-enabled search can help to provide employees with faster and easier access to information. However, that’s not the only way that organisations are using AI-enabled search to improve their efficiency. Organisations can also use AI-enabled search to automate other parts of their business processes, freeing up time for employees to work on more complex tasks.
Aviva, for example, is a British insurance company that is using AI-enabled search to improve its process of providing quotes for insurance coverage. With the large amount of historical and current data that Aviva possesses, they can automatically provide accurate insurance rates to potential clients, while also ensuring that clients have the appropriate level of coverage. With AI-enabled search, Aviva can leverage their wealth of data to allow their data brokers, as well as their clients, to make well-informed decisions about the insurance coverage they need.
Enhancing scientific research
Organisations can also incorporate AI-enabled search into their research process, enabling researchers to conduct searches more efficiently and accurately, as well as accelerating other aspects of the research process. The consulting company Deloitte, for example, has proposed the promise of using AI to accelerate the research process by automating data collection and management. This approach can have numerous benefits for public institutions conducting medical research, as well as private pharmaceutical development companies.
AI-enabled search can make it easier for researchers to manage and link various clinical trial data, such as body sensor data, historical health Information, and reported health data. With AI-enabled search, these various types of data can be easily searchable through a single platform. By managing data across the clinical trial process, these tools can also make it easier for researchers to auto-populate reports and analyses. Additionally, researchers can more easily search through past and current clinical trials to generate insights. By making these aspects of the research process more efficient, AI-enabled search can allow researchers to focus on higher-level tasks and accelerate the research process.
Outside of clinical trials, AI-enabled search also provides an opportunity for researchers to search through medical data and health publications more efficiently. For example, Amazon Web Services created CORD-19 Search, a tool that allows users to quickly and easily search through COVID-19 research papers and documents by simply entering questions. Public organisations like the UK’s National Institute for Health Research can leverage AI-enabled search to develop similar tools. By utilising natural language processing, AI-enabled search can analyse and retrieve information from research publications, making it easier for researchers to find the relevant information they’re looking for. This can enable researchers to conduct even more thorough literature reviews, uncovering sources that they may not have encountered otherwise.
Providing enhanced customer service
In the private sector, customer-facing chatbots and external search tools are a popular application of AI-enabled search. Instead of requiring customers to read an FAQ or search through pages and pages of documentation, these virtual agents can help customers quickly and easily find the answers they need.
Customer-facing chatbots search through organisational documentation to find an answer to customer requests, providing a variety of benefits to both the organisation and the customer. By allowing customers to receive real-time support and to quickly and easily find the answers they need, chatbots can contribute to higher customer satisfaction. Additionally, by allocating simple questions to chatbots, organisations can free up time for the support team to respond quickly to more complex requests, while providing a cost-effective solution for simple requests.
While chatbots are especially common in the retail space, that’s not their only application. Prudential, a financial company focused on investment and insurance, developed an AI-powered virtual assistant that can answer routine customer questions, allowing consumers to get the answers they need in real-time, quickly and easily. Meanwhile, companies like Microsoft are using chatbots to provide personalised technical support. Microsoft’s Virtual Agent troubleshoots technical problems by asking the user questions, before returning a potential solution to the issue, including links to relevant troubleshooting articles in their knowledge base. This agent can answer the majority of questions across a variety of Microsoft products and services.
The Future of AI-Enabled Search
In this article, we covered just a small set of the ways that organisations in the public and private sectors are using AI-enabled search to enhance their operations, improving information access to enhancing research and business processes. But what does the future of AI-enabled search look like?
With AI increasingly breaking into the mainstream, the development of large language models that form the bedrock of AI-enabled search is rapidly accelerating! With the release of popular tools like Chat-GPT, models like GPT-3 and GPT-4 have become more accessible. As competitors continue to develop various models and tools, organisations will have even more opportunities to incorporate AI-enabled search into their internal and external processes.
The development of large language models, and by extent AI-enabled search, is far from done, and there are still many opportunities for future development. A recent Forbes article predicted that future research would involve the development of large language models that can improve and fact-check themselves, as well as the exploration of new model architectures.
One particularly intriguing development is research into the use of sparse expert models architecture. Today’s well-known models are all built on the same core infrastructure, dense transformer-based models. When models built on this architecture are run, all of its parameters are used – GPT-3, for example, uses 175 billion model parameters. Sparse expert models, on the other hand, only call upon the parameters that are helpful to handle their input. Because these models only rely on a subset of parameters, they may provide a way to build bigger, less computationally-dense models that return higher-quality search results faster.
The ongoing research into large language models, as well as the growing adoption of AI-enabled search tools, suggest the future of AI-enabled search is bright. AI-enabled search is already driving innovation across the private and public sectors, with more AI advancements to come.