Large Language Models (LLMs) like GPT-4 exhibit impressive capabilities in text generation tasks such as summarization and question answering. However, they often produce “hallucinations,” generating content that is factually incorrect or contextually irrelevant. The problem is particularly acute when the LLMs are provided with correct facts but still produce inaccurate outputs, termed “contextual hallucinations.” These errors undermine the reliability of LLMs in applications where accuracy is critical, such as document-based question answering and summarization.
Prior work on detecting and mitigating hallucinations has focused on using the internal representations of LLMs, such as hidden states or attention outputs. Existing methods to combat hallucinations generally focus on scenarios without any input context, relying on the LLMs’ internal knowledge. Techniques to detect and mitigate hallucinations often utilize the LLM’s hidden states, attention block outputs, or entailment models trained on large annotated datasets. While effective in some cases, these methods do not specifically address contextual hallucinations, where the provided context is key.
To address this gap, the researchers from the Massachusetts Institute of Technology and the University of Washington propose a novel approach that leverages the attention maps of LLMs. The Lookback Lens, a unique and innovative solution, is based on the insight that contextual hallucinations are related to the extent to which the LLM attends to the provided context versus its own generated tokens. This novel approach introduces the ‘Lookback Lens,’ a simple yet effective hallucination detection model. The Lookback Lens uses the ratio of attention weights on the context versus the newly generated tokens, termed the ‘lookback ratio,’ as its primary feature.
The Lookback Lens operates by computing the lookback ratio at each time step during the generation process. This ratio is calculated for each head in each layer for a transformer model with multiple layers and heads. The lookback ratio is the attention weight focused on the context tokens divided by the total attention weight on both context and new tokens. These ratios are concatenated into a feature vector, which is then used to train a linear classifier to detect hallucinations.
The effectiveness of the Lookback Lens is validated through experiments on summarization and question-answering tasks. The results show that the Lookback Lens performs comparably to, or even better than, more complex detectors that use the entire hidden states of an LLM. Importantly, the Lookback Lens can be transferred across different models and tasks without retraining, demonstrating its robustness and generalizability. For instance, a detector trained on a 7B model can be applied to a 13B model, reducing hallucinations by 3.2% in the XSum summarization task.
The researchers propose a classifier-guided decoding strategy to further mitigate hallucinations during text generation. This approach incorporates the Lookback Lens into the decoding process, evaluating multiple token chunks at each step and selecting the one predicted to be least likely to cause hallucinations. This strategy reduces hallucinations by 9.6% in the XSum summarization task, highlighting the potential of the Lookback Lens in practical applications.
The problem of contextual hallucinations in LLMs is significant, affecting the reliability of these models in critical applications. The Lookback Lens provides a simple yet effective solution, leveraging attention maps to detect and mitigate hallucinations. Its ability to transfer across models and tasks without retraining further underscores its utility. This approach represents a promising step toward more accurate and reliable LLM-generated content.
Check out the Paper and GitHub. All credit for this research goes to the researchers of this project. Also, don’t forget to follow us on Twitter.
Join our Telegram Channel and LinkedIn Group.
If you like our work, you will love our newsletter..
Don’t Forget to join our 46k+ ML SubReddit
The post Enhancing LLM Reliability: The Lookback Lens Approach to Hallucination Detection appeared first on MarkTechPost.
#AIPaperSummary #AIShorts #Applications #ArtificialIntelligence #EditorsPick #LanguageModel #LargeLanguageModel #Staff #TechNews #Technology [Source: AI Techpark]