Hallucination Reduction in.LLMs - Chain of Verification and Beyond

Reducing Hallucination in LLMs: Chain of Verification and Beyond

 Introduction

Hallucination in language models—where they generate information that isn't accurate or supported by the provided context—is a significant challenge. This post explores several techniques to reduce hallucination, with a special focus on the Chain of Verification approach.

Four Techniques to Combat Hallucination

1. Through Moderation

In Retrieval Augmented Generation (RAG) systems, language models are grounded to an organization's knowledge base. However, when the retrieval process returns no matching documents, adding this "blank context" to the prompt often results in hallucinations.

Solution: Implement a relevance moderator that checks whether any context was returned from the retrieval process. If no matching documents are found, the query isn't sent to the language model at all, effectively preventing hallucination by stopping the process before it begins.

 2. Chain of Verification

Based on academic research, this four-step approach systematically reduces hallucination:

1. **Draft an initial response**

2. **Generate verification questions** to fact-check the draft

3. **Answer these verification questions independently** (without bias from the initial draft)

4. **Create a final verified response** based on the verification answers

 3. Chain Polling

Similar to consensus mechanisms used in cluster management (like Zookeeper or Kafka clusters), chain polling implements a quorum approach:

1. Generate a response

2. Send the response to multiple LLMs (participants in the quorum)

3. Each LLM votes on whether hallucination occurred

4. The final decision is based on majority vote

 4. Human in the Loop (HITL)

The most reliable but resource-intensive approach: human reviewers check LLM responses for hallucinations before they're delivered to end users.

Deep Dive: Chain of Verification

The Chain of Verification can be implemented in several ways:

Joint Approach

Planning (generating verification questions) and execution (answering them) are handled by a single LLM prompt. This approach is simpler but may repeat hallucinations since the same model that potentially hallucinated is used for verification.

 Two-Step Approach

Planning and execution steps are separated with different LLM prompts. Verification questions are generated based on the original response, but answered using only the context, not the original response—reducing the chance of repeating hallucinations.

 Factored Approach

Each verification question is answered by a separate LLM prompt. While more expensive, this enhances accuracy by preventing cross-contamination between verification questions.

 Factored and Revise Approach

An extension of the factored approach where an additional LLM cross-checks verification responses against the original response to detect inconsistencies.

Implementation Example: The Factored Approach

The author demonstrated the factored approach using a deliberately incorrect context: "Rahul Gandhi is the prime minister of India" (while in reality, it's Narendra Modi).

Here's how the implementation worked:

1. **Initial Response**: When asked "Who is the prime minister of India?", the model (text-davinci) relied on its training rather than the provided context, correctly stating "Narendra Modi is the prime minister of India."

2. **Verification Questions**: The system generated verification questions about who is India's prime minister.

3. **Independent Answers**: Each verification question was answered separately using the provided context only, resulting in "Rahul Gandhi is the prime minister of India" (correct according to the provided context, though factually incorrect).

4. **Final Verified Response**: The system revised the original response to be consistent with the verification answers, producing "The prime minister of India is Rahul Gandhi."

This example shows how Chain of Verification successfully grounds the model to the provided context, even when that context contains incorrect information. This is valuable in RAG systems where we want models to faithfully represent the information in the retrieved documents rather than their own training data.

 Conclusion

Reducing hallucination in language models requires systematic approaches. While Chain of Verification provides a structured method for improving accuracy, other techniques like moderation, chain polling, and human review offer complementary benefits. The choice of technique depends on the specific use case, balancing accuracy requirements against computational and human resource costs.

For critical applications, a combination of these approaches may provide the most robust defense against hallucination.