LLM Response Selector

Introduction

Since the launch of OpenAI’s ChatGPT, the use of Large Language Model (LLM) chatbots have surged. OpenAI’s ChatGPT website alone receives approximately 1.63 billion visits to per month – this does not account for the substantial traffic to other LLM chatbots developed by organizations like Meta and Google, which also likely attract billions of visitors.   

Large Language Models (LLMs) are a fundamental component of Generative AI, a branch of artificial intelligence (AI) focused on creating new content ranging from text to images to music and computer code. These models utilize deep learning architectures to produce human-like text that is coherent, contextually relevant, and often indistinguishable from text written by people. 

LLMs have revolutionized the way AI chatbots interact with humans – allowing them to engage in conversations across a wide variety of topics, and providing information and assistance to users. This enables them to be useful for a variety of use cases such as customer service and support.  

Not All is Perfect with LLM Chatbots

In typical conversational AI systems, LLM chatbots generates responses based on inputs. These responses are often contextually relevant, and mimic human conversations. However, sometimes LLM chatbots can generate incorrect or misleading information that appears convincing – this is referred to as an “hallucination”.  

In 2022, a Canadian court held Air Canada liable when its AI chatbot gave a customer bad advice; resulting in the Canadian airline compensating the customer for damages and legal costs. In 2023, a New York lawyer used an LLM chatbot to help file a legal brief in federal court. Consequently, the LLM chatbot cited a series of fake cases with “bogus judicial decisions with bogus quotes and bogus internal citations.” In 2024, the Government Digital Service (GDS) of GOV.UK observed some cases of hallucination and inaccuracies in responses after experimenting with generative AI. 

LLM chatbots may also be trained on large datasets that have inherent biases or prejudices, which can result in discriminatory or biased responses. Additionally, LLMs chatbots have raised privacy concerns where it inadvertently generates or reveals personally identifiable information (PII) and intellectual property (IP) issues due to copyright or originality of content. This has led to major organizations – like Samsung, Apple and Microsoft – restricting the use of LLM chatbots in the workplace.  

LLM Technologies in the Public Sector

Despite some challenges with LLM chatbots, they remain powerful tools for enhancing user experience by providing contextually aware interactions that mimic human conversations. Recognizing the many benefits observed in the private sector, public sector organizations are increasingly looking to safely adopt LLM chatbots to modernize their services and interactions with citizens. Governments are utilizing these AI tools to facilitate access to information, automate routine inquiries – like renewing driver’s licenses – and provide services more efficiently.

Many levels of governments across the United States are creating or already have created legislation and an AI Task Force to guide the safe, responsible, and ethical use of AI. In fact, NeuroSoph is a member of Commonwealth of Massachusetts AI Strategic Task Force Government Working Group and helps advise the Massachusetts government on how to implement AI into various applications like chatbots, intelligent document processing (IDP), and assisted intakes safely and responsibly. 

The safe adoption in the public sector not only aims to enhance the efficiency, protect rights, and accessibility of government services but also seeks to ensure that these services are available to a broader audience, including those with disabilities, limited access to traditional modes of communication, or who require services outside of normal business hours.  

Implementing LLMs Safetly for the Public Sector

For public sector organizations, the implementation of LLM chatbots must adhere to very high standards. Factual accuracy is critical; government LLM chatbots must not hallucinate or provide any inaccurate information. Governments should demand near perfection on the chatbot’s response accuracy rate – anything less is unacceptable. Additionally, the use of these AI chatbots must comply with legislation to ensure fairness and equality, preventing any discrimination or bias in responses.   

LLM Response Selector

At NeuroSoph, we believe in the power of cutting-edge generative AI technologies; however, they should be used and implemented in a safe, responsible, and controlled manner. Our hybrid approach, called the LLM Response Selector, embodies this approach. The LLM Response Selector consists of two main components: an LLM to help understand the question and context; and human oversight to create a list of contextually appropriate pre-approved responses from which the LLM can choose. This approach keeps humans in the loop and prevents the LLM chatbot from generating responses which can lead to hallucinations or inaccurate responses.   The approach of using an LLM Response Selector was recently shared to StateScoop by Mark Raymond, Connecticut CIO, at a technology conference session about generative AI governance:  
“We shouldn’t be using [generative AI] to create new content and responses, because we have to curate that, we have to make sure that that’s good,” Raymond said. “One of the really powerful things large language models can do is you can implement that in a chatbot to understand the question and get them to already-curated content, instead of providing hallucinated content. That’s a great use case that helps improve government in ways that you would really have a difficult time to do on our own.”
Mark Raymond
Connecticut CIO

Using an LLM Response Selector is an effective method in ensuring the safe and responsible implementation of AI. It helps to ensure responses from government agencies and their tools are factual and relevant.  

NeuroSoph's LLM Response Selector

NeuroSoph’s LLM Response Selector chatbot harnesses the power of LLM technology safely and responsibly, while keeping humans in the loop. This LLM chatbot is specifically designed for government agencies and minimizes risk of inaccurate responses.   

NeuroSoph is a trusted and reliable government AI solutions provider. Using our proprietary, secure, and cutting-edge Specto AI platform, we help streamline government communication and delivers digital government services to citizens. NeuroSoph is an ITS75 Category 3 provider. For more information about our products and services, please contact us today – let’s extend intelligence in your organization.    

Contact NeuroSoph