Reducing LLM Hallucination via Chain of Verification

Reducing Hallucination in Large Language Models: Chain of Verification Technique

 Introduction

Hallucination—where AI models generate incorrect information despite appearing confident—remains one of the significant challenges in deploying large language models (LLMs) in production environments. This article explores several techniques to reduce hallucination, with a special focus on the promising "Chain of Verification" method.

Four Key Techniques to Reduce Hallucination

 1. Through Moderation

When using Retrieval Augmented Generation (RAG), we ground language models to an organization's knowledge base. However, if the retrieval process returns no matching documents, we end up with a blank context. Including this blank context in the prompt often causes the model to hallucinate answers despite having no relevant information.

The solution is to implement a relevance moderator that checks whether any context was returned from the retrieval process. If no matching documents are found, the query isn't sent to the language model at all, effectively preventing hallucination.

2. Chain of Verification

Based on recent research, this technique follows a four-step process:

1. Draft an initial response

2. Create verification questions to fact-check the draft

3. Answer these verification questions independently (without bias from the initial draft)

4. Generate a final verified response based on these answers

3. Chain Polling

Similar to quorum concepts in cluster management (like Zookeeper or Kafka clusters), chain polling employs multiple LLMs as participants in a decision-making process. After generating a response, it's sent to multiple models which vote on whether hallucination occurred. The final decision is based on a majority vote, typically requiring a 2/3 consensus.

 4. Human in the Loop (HITL)

Currently the most reliable approach, HITL involves human reviewers checking AI-generated responses for hallucinations before they're delivered to end users. While effective, this method trades efficiency for accuracy.

Chain of Verification: A Deeper Dive

Implementation Approaches

Chain of Verification can be implemented through several approaches:

1. **Joint Approach**: Both planning verification questions and answering them are handled by a single LLM prompt. The disadvantage is that if the original response was hallucinated, there's a high chance the verification answers will also be hallucinated.

2. **Two-Step Process**: Planning and execution steps are separated with different LLM prompts. Verification questions are generated based on the original response, but answered by a different prompt using only the context, not the original response. This significantly reduces repeated hallucinations.

3. **Factored Approach**: Each verification question is answered by a separate LLM prompt. Though more expensive, this enhances the accuracy of verification.

4. **Factor and Revise**: An extension of the factored approach where another LLM cross-checks verification question responses against the original response to detect inconsistencies.

Practical Implementation: The Factored Approach

The article demonstrates the factored approach with a concrete example:

1. A deliberately incorrect context is provided: "Rahul Gandhi is the prime minister of India" (while in reality, it's Narendra Modi)

2. The model is asked: "Who is the prime minister of India?"

Implementation Steps:

1. **Draft Initial Response**: The model generates an initial answer, but ignores the provided context and correctly states that Narendra Modi is the prime minister.

2. **Generate Verification Questions**: A separate prompt creates verification questions to test the accuracy of the baseline response.

3. **Answer Verification Questions Independently**: Each verification question is answered separately using only the provided context.

4. **Generate Verified Response**: The original response is revised based on the verification question answers.

In this example, the final verified response correctly grounds itself to the provided context, stating "Rahul Gandhi is the prime minister of India" even though this is factually incorrect in the real world. This demonstrates how Chain of Verification helps ensure the model's response is properly grounded to the provided context, reducing the chance of the model hallucinating information from its training data.

Conclusion

The Chain of Verification technique offers a promising approach to reducing hallucinations in large language models. By implementing systematic verification steps, we can significantly improve the reliability of AI-generated responses, especially when using the factored approach.

Future explorations will dive deeper into other hallucination reduction techniques, building upon the foundation established with Chain of Verification.