Ground Control: A Practical Guide For Hallucinations Prevention
Large language models are known to "hallucinate" - provide seemingly intelligent answers that turn out to be erroneous or irrelevant. People do that too. The solutions to both biases are the same.
When talking about Large Language Models (LLMs), “Grounding” refers to generating text in context, or in layman's terms - making sure that the model generates “real” and relevant answers. In other words, grounding means establishing a link between the text produced by these sophisticated models and real-life entities, events, and situations. Grounding is a real challenge in LLMs and generative AI. It is particularly important for virtual assistants, where grounding is the actual context of the conversation, and an ungrounded model will simply generate irrelevant, nonsensical, or even misleading responses.
Since we believe that there are many similarities between LLMs and our cognitive system and the brain, a better understanding of grounding methodologies is very important both practically and theoretically.
Photo by Mark Glancy
LLMs exhibit remarkable capabilities in numerous natural language processing tasks. However, they often have difficulty generating coherent, contextually precise responses anchored in commonsense reasoning. Consequently, a model may produce a very convincing output, that seems perfectly fluent at first sight, but upon examination, this text may turn out irrelevant, incorrect, or even entirely unhinged. This phenomenon is often referred to as “hallucinations”. Interestingly, people do that too. We judge the truthfulness of a statement, or even a thought, based on how fluent, familiar, or coherent it feels. As people, we have various mechanisms to stop us from expressing these untrue ideas, or framing them as fiction.
Models like GPT are statistical by nature, and (roughly speaking) are pre-trained by ingesting huge amounts of text, to generate the most plausible completion to a prompt. However, what is most probable to the model, due to some learned pattern in the training material, or some bias in the training process, is not necessarily based in reality, nor is it helpful, appropriate, or even makes sense.
Addressing the problem of grounding in large language models is thus crucial for advancing the field of artificial intelligence and enabling the development of more context-aware, reliable, and useful language-based applications.
What Can We Do?
There are several ways to address the issue of grounding and hallucinations. We can divide the techniques based on the time they are applied:
Training - The first time we expose the model to the world
Fine-tuning - After the learning phase, we can adjust the model to the world
During inference - When asking the model a question, we can form it differently to help it form a grounded answer.
Post-inference - After the model has generated a response, we can ground it or ask it to do that.
We will now go over the different stages and see how each of them can be improved to provide responses closer to “the truth”, and how the same processes occur in human learning.
Training
Training is the first time the model sees data and learns what to do with it.
Like a baby’s first interaction with the world in some models, or a traveler learning a new language from external cues in others, the model encounters many examples and learns from context and sometimes feedback.
In the process of training large language models, several procedures can improve the grounding of the resulting model:
Garbage In – Garbage Out
Selecting high-quality, diverse data, and filtering out biased, or inferior data sources may reduce hallucinations. If the erroneous information is not in the training material, there’s low probability for it to pop on inference.
For people, this is compared to learning language, or cultural references from low vs. high sociolinguistics speakers. Learning a new language from a non-native speaker will not yield as good results as learning it from a fluent speaker.
Regularization Methods
Well known methods like dropout, or weight decay help avoid overfitting to some specific parts of the training data. This allows the model to correctly learn the true patterns in the data, and thus improves generalization and grounding capabilities.
This process automatically happens in our minds, when building a new information structure. At first, we are strongly biased by each piece of information. Then, when we know more, we move to adjusting smaller pieces of our information structures, rather than shifting it all upon each encounter.
Multimodal Learning
Training the model on multimodal information, such as visual or auditory and textual information, allows the model to better understand the underlying contextual connections between the modalities, and thus - reality. This is also the strongest predictor of language acquisition with humans, and the reason psychologists and educational instructors are so fond of rich and versatile environments for toddlers. There are even claims that this type of approach is the next step for LLMs. Some examples of this type of models are GPT4 by OpenAI, PaLM-E by Google, and Kosmos-1 by Microsoft.
Fine Tuning
As training LLMs from scratch is prohibitive in time and cost, a lot of effort is put into fine-tuning pre-trained models on task-specific datasets. For example, you might take a set of pre- and post-summarization text pairs and fine-tune the LLM to improve its performance in the summarization task.
The human equivalent of it is learning a second language with an instructor - you know how to use it already but can improve.
Similarly, the most prominent LLM fine-tuning method nowadays is reinforcement learning from human feedback (RLHF). In this method, the model is given prompts (i.e., questions), responses, and human feedback on those responses (Is it true? Is it useful? etc.). This type of tuning improves the alignment of the LLM responses with human expectations, as it would have been from human learners. This type of fine-tuning is used ubiquitously, following the success by OpenAI models, which were admittedly fine-tuned with RLHF.
While this approach improves grounding, it also increases (by design) the bias toward human-preferred responses and potentially may inflate errors humans tend to make. Specifically, RLHF is highly dependent on the expertise and skills of the humans who are tagging the training data and on the sheer amount of tagged data. In addition, this has proven insufficient for high-level grounding, requiring additional post-filters, as implemented in chatGPT, Bard, and others.
Then, again, the same goes for human learners - from first language acquisition to academic degrees, medical procedures, and even artistic practices - we learn from our teachers and often unwillingly copy their thought patterns and behaviors.
Photo by Alena Darmel
During Inference
This is a family of methods that focus on incorporating external knowledge into the generation process. Specifically, the context for generation is fetched from encyclopedias, databases or other knowledge sources. This is done many times using semantic search and vector databases. Once the LLM is provided with factual information in context, it tends to produce better-grounded outputs.
For example, the question “what is grounding?” is answered by chatGPT with “In various contexts, "grounding" refers to the process of connecting with the present moment and physical reality. It involves focusing on sensory experiences, such as touch, sight, sound, smell, and taste, to center oneself and reduce stress or anxiety.”, while the question “what is grounding in the context of LLMs”, produces “the ability to connect language outputs to the physical and perceptual world”.
The human comparable is answering more detailed questions with some context rather than out of the blue.
Post-Inference
Post inference methods accept the fact that hallucinations cannot be avoided entirely, and instead attempt at verifying the generated outputs. These methods usually include classifiers that evaluate the generated text, and validate specific aspects such as inappropriate language, medical advice, racism, etc.
This process also happens in our minds and social environment. When starting to form answers, behaviors, or ideas, we often jump to very unrelated representations or acts. This mechanism is usually automatically corrected, and these errors are rarely expressed, or even reach conscious awareness. These processes are strongly evident in pathological cases, such as actual hellucinations and delusions or strong associative lossness; Yet, we can find them in anyone, in easier examples such as slips of the tongue (when you write one thing but actually mean your mother), erroneous feeling that we know something we don’t, or spontaneous responses and behaviors.
Photo by Manuel Keusch
Our Approach
In Xoltar we developed an innovative approach to improve grounding and reduce hallucinations. Our approach, grounded in the psychological literature, holds that by controlling the flow of the conversation, we can greatly reduce the tendency of the model to generate irrelevant or inappropriate responses. This does not mean that the conversation is limited from the user's perspective but that the generation follows some guidelines that constrain it in a way that keeps it grounded and appropriate.
We develop a combination of soft and hard mechanisms in conversations that enable us to steer the dialogue to maintain the user's freedom of interaction while still providing guidance. These guidelines ensure the system remains accurate regarding the factual nature of its responses and adheres to the goal of the conversation and the intermediate objectives.
The modular (hard) mechanism breaks down a conversation into parts, constraining the responses to the objectives of each segment while incorporating the broader context of the entire conversation. This approach makes the conversation contextual, as it remains aware of the ongoing discussion while also adhering to its goals. Consequently, the conversation stays grounded and focused. This modular method introduces a bias towards the relevant aspects of the conversation, increasing the likelihood of generating an accurate and appropriate response without being diverted by unrelated elements from other parts of the discussion.
The guided (soft) mechanisms prime the conversational models to follow some specific principles. Adhering to guidelines differs significantly from having scripted conversations or employing pre-defined responses. Guidelines enable precise, useful, and appropriate responses, facilitating a constructive conversation.
Humans behave in a similar manner when following strict protocols, as they do in some evidence-based speech therapies (such as cognitive-behavioral therapy, motivational-interviewing or 12-steps accountability partnership). These practices have strict sentences people repeat, while being guided between them based on soft instructions and agenda-based guidelines. In that way, the person keeps following their goals while holding a dynamic conversation.
Conclusion
Model grounding and hallucinations in large language models pose a significant challenge to developing truly intelligent and helpful conversational agents and systems. We propose means to significantly improve grounding, by intelligently constraining the context, while keeping the dialog itself unrestricted, like humans do. By controlling the conversation flow, our approach reduces the tendency of the model to generate irrelevant or inappropriate responses. Additionally, by contextualizing the model based on specific goals and sub-goals, this approach biases the model to remain on point. It effectively constrains it to the relevant aspects of the conversation, leading to more accurate, meaningful, and ethically sound responses. While RLHF alone is often insufficient, XOLTAR's approach has the power to mediate responses driven by the statistics of the corpus and the responses from human raters, making it a valuable contribution to the field of artificial intelligence.
