AI Assistant Evolution - From Fast Thinking To Rational Decision Makers

AI Assistant Evolution: From Fast Thinking to Rational Decision Making

In today's world of AI development, we're witnessing a fascinating shift from simple predictive models to more complex systems that can simulate deeper reasoning. Let me transform this YouTube transcript into a structured blog post that explores this evolution.

The Inner Dialogue: System 1 vs. System 2 Thinking

Have you ever found yourself on the verge of making a controversial purchase? Just as you're about to click that "buy" button, an unexpected thought crosses your mind:

"Wait a minute... they look a little bit like Swiss cheese, don't they? No, no, no—they're absolutely beautiful, and Kanye West loves them! He wears them all the time! But if I like things that Kanye likes, is that really a good thing? Okay, I need to relax. Everything is fine, and buying these makes me a visionary, a trendsetter. Do these holes exist for ventilation purposes? Oh... okay, time for a break. I need to urgently destress from all this thinking with some Pringles... wait, is this really unhealthy?"

This inner dialogue represents what Daniel Kahneman, author of "Thinking Fast and Slow," calls **System 2 thinking**—a slow, conscious type of thinking that requires deliberate effort and time. The opposite is **System 1 thinking**—fast, subconscious, and automatic processing, like effortlessly recognizing a familiar face in a crowd.

The Current Limitations of AI

According to Andrej Karpathy, a renowned engineer at OpenAI, all current large language models (LLMs) are only capable of System 1 thinking. They function like "auto-predict on steroids." None of the current LLMs can take 40 minutes to process a request, think about a problem from various angles, and then offer a rational solution to a complex problem.

This rational or System 2 thinking is what we ultimately want from AI. Fortunately, some innovative approaches are helping to bridge this gap.

Simulating Rational Thinking in AI

Two primary methods have emerged to simulate more rational thinking in AI systems:

1. **Tree of Thought Prompting**: This involves forcing the LLM to consider an issue from multiple perspectives or from the viewpoints of various experts. These "experts" then make a final decision together by respecting everyone's contribution.

2. **Agent Systems**: Platforms like Crew AI allow anyone—even non-programmers—to build custom agents or experts that can collaborate, thereby solving complex tasks. You can tap into any model with an API or run local models through alternatives like Ollama.

Building Your Team of AI Agents

Let's explore how to assemble a team of smart AI agents to solve complex problems:

 Setting Up the Environment

First, you'll need to:

1. Open VS Code and a new terminal

2. Create and activate a virtual environment

3. Install Crew AI with pip

4. Import necessary modules and set up your OpenAI API key

Creating Specialized Agents

For this example, we'll create three agents to help analyze a startup concept:

1. **Market Researcher**: 

   - Goal: Understand if there's substantial need for the product

   - Provide guidance on reaching the widest possible target audience

2. **Technologist**:

   - Goal: Analyze technical feasibility

   - Suggest production methods

3. **Business Development Expert**:

   - Goal: Consider everyone's input

   - Create a comprehensive business plan

Each agent needs:

- A specific role

- A clearly defined goal

- A detailed backstory

Defining Tasks

Tasks should be specific with clear expected results. For our startup idea (creating elegant plugs for Crocs), we'll assign:

1. **Market analysis** to the Market Researcher

2. **Technical feasibility study** to the Technologist

3. **Business plan creation** to the Business Development Expert

Finally, we instantiate the crew (team of agents), include all agents and tasks, and define a sequential process where outputs from one agent become inputs for the next.

Making Your AI Agents Smarter

The real power comes from giving your agents access to real-world data. There are two approaches:

1. Using Built-in Tools

Add pre-built tools from libraries like LangChain:

- Eleven Labs text-to-speech

- Google search data access

- Wikipedia knowledge access

 2. Creating Custom Tools

For better results, create custom tools that scrape targeted information:

- Reddit scrapers for specific subreddits

- Email analyzers

- Custom data APIs

 Real-World Application: Creating an AI Newsletter

In a practical example, three specialized agents (Researcher, Technical Writer, and Writing Critic) can collaborate to create a detailed AI newsletter:

1. The Researcher uses Google search or Reddit scraping to gather information

2. The Technical Writer transforms this into a structured newsletter

3. The Writing Critic refines and polishes the content

This can reduce hours of work to just minutes, though results vary depending on the quality of information gathered and the LLM used.

 Cost Considerations and Private Alternatives

Running these agent systems with commercial APIs like GPT-4 can cost around $0.30 per run, which adds up quickly. To avoid these costs and keep conversations private, local models are an option.

Local Model Performance

After testing 13 open-source models:

- **Worst performers**: Llama 2 (7B parameters) and Phi-2

Bloggers Note: Can be FIXED with Unsloth (http://unsloth.ai) and Mergekit (https://github.com/arcee-ai/MergeKit)

- **Better performers**: OpenChat (7B), Mistral (with many emojis)

- **Surprise performer**: Regular Llama (13B) was the only model that actually processed Reddit data properly

Most local models struggled to understand complex tasks, though models with more parameters (13B+) performed better than smaller ones.

 Conclusion

The evolution of AI assistants from simple "auto-predict" tools to collaborative agents that can simulate rational thinking represents a significant step forward. While we're not yet at true System 2 thinking, these approaches offer increasingly powerful ways to leverage AI for complex problem-solving.

Whether you use commercial APIs or local models, the ability to create specialized agent teams opens up new possibilities for automation and decision support. As these technologies continue to develop, we can expect even more sophisticated reasoning capabilities from our AI assistants.