Prompt Engineering For Open Source LLMs

**Prompt Engineering for Open-Source LLMs: Why Transparency and Iteration Matter**

Prompt engineering is often misunderstood—especially when transitioning between closed and open-source large language models (LLMs). In a recent workshop hosted by deeplearning.ai and Lamini, Dr. Sharon Zhou, co-founder and CEO of Lamini and former Stanford AI faculty, shared her candid, practical insights on how to maximize performance from open-source LLMs. Her core message? **Prompt engineering is not software engineering, and prompts are just strings.** Here’s what you need to know.

---

**1. LLMs Need to “Wear Pants” (Prompt Settings Matter)**

**The Analogy: Why Prompts Are Like Pants**

Dr. Zhou kicked off with a memorable analogy: **LLMs need to “wear pants.”** Just as you wouldn’t leave the house without pants, LLMs need the right prompt settings to function as expected. These settings—often called *meta-tags* or *chat templates*—are strings that tell the model how to behave. Without them, responses can be off-topic, incoherent, or even nonsensical.

**Example:**

- **Without “pants” (prompt settings):**

  - *Mistral:* Responds in the third person, ignoring instructions.

  - *Llama 2:* Continues a sentence instead of answering a question.

- **With “pants”:**

  - Both models respond appropriately, as if following social norms.

**Key Takeaway:**

Every LLM (and even different versions of the same LLM) has unique prompt settings. For open-source models, these settings are often *transparent and customizable*—unlike closed models like ChatGPT, where they’re hidden behind APIs.

---

**2. Prompt Engineering ≠ Software Engineering**

 **Iterate Like a Google Search, Not Like Code**

Dr. Zhou emphasized that prompt engineering is closer to refining a Google search query than writing software.

 **There’s no “perfect design” upfront.** Instead:

- Start with a simple, even lazy prompt.

- Iterate based on the model’s output.

- Time-box your efforts—diminishing returns set in after ~100 iterations.

**Why?**

LLMs are probabilistic and sensitive to small changes (e.g., a single space can alter responses). Over-engineering prompts with complex frameworks often backfires. **Keep it simple: prompts are just strings.**

---

**3. Open vs. Closed LLMs: Transparency is Power**

 **Closed LLMs (e.g., ChatGPT)**

- Prompt settings are managed behind the scenes.

- Updates can break prompts without warning (no backward compatibility).

**Open-Source LLMs (e.g., Mistral, Llama)**

- **Prompt settings are exposed.** You can see and modify them.

- **Flexibility:** You can adapt prompts for specific use cases (e.g., JSON outputs, multi-turn conversations).

- **Risk:** Frameworks often obscure these settings, leading to poor performance.

**Dr. Zhou’s Advice:**

- **Avoid frameworks that hide prompts.** Transparency lets you debug and optimize.

- **Test prompts empirically.** What works for GPT-4 may fail for Mistral.

---

 **4. RAG (Retrieval-Augmented Generation) is Just Prompt Engineering**

RAG isn’t a separate discipline—it’s about **concatenating relevant strings (documents) to your prompt.** Dr. Zhou demonstrated a minimalist RAG implementation in ~80 lines of code using FAISS (Facebook’s similarity search library). Her approach:

1. **Chunk and embed** your data.

2. **Retrieve** the most relevant chunks for a query.

3. **Prepend** them to the prompt.

**Pro Tips:**

- **Debug visually:** Print the retrieved chunks to ensure they’re relevant.

- **Optimize embeddings:** Throughput matters—aim for high queries per second (QPS).

- **Add metadata:** Headers, titles, or document sources help the model contextualize chunks.

**Example Use Case:**

A fast-food drive-thru system fine-tuned to extract menu items from conversation snippets.

---

 **5. Customizing “Pants”: Fine-Tuning and Beyond**

**Fine-Tuning**

- Lets you define new “pants” (prompt settings) for your LLM.

- Useful for specialized tasks (e.g., guaranteed JSON outputs).

 **No Fine-Tuning? Trick the Model**

Dr. Zhou’s team built a custom inference engine to **force structured outputs** (e.g., JSON) without fine-tuning—akin to giving the LLM “new clothes.”

**When to Fine-Tune vs. Iterate:**

- **Lazy approach:** Use default settings for quick experiments.

- **Curious approach:** Fine-tune for production-grade performance.

---

---

 **Q&A Highlights**

**Q: Should system prompts for non-English apps be in English or the target language?**

**A:** *Test both.* Run 20–30 examples in each language and A/B test results. Model performance varies by training data.

**Q: How to prepare for future LLM releases?**

**A:** Demand transparency from creators (e.g., prompt templates, training data). Build test sets for your use case.

**Q: How to handle ambiguity in prompts?**

**A:** Clarify the prompt’s goal. If the model struggles, add context or constraints (e.g., “Respond as a health expert”).

---

 **Final Thought: Prompts Are Just Strings**

Dr. Zhou’s workshop debunked the myth that prompt engineering requires a PhD. **It’s about transparency, iteration, and treating prompts as what they are: strings.** Whether you’re debugging a chatbot or scaling RAG over millions of documents, the rules are the same:

1. **Put pants on your LLM** (use the right prompt settings).

2. **Keep prompts visible and editable.**

3. **Iterate relentlessly.**

---

**Want to dive deeper?**

- [Lamini’s open-source prompt engineering repo](https://github.com/lamini-ai) (mentioned in the talk).

- [DeepLearning.AI’s fine-tuning course](https://www.deeplearning.ai/courses/fine-tuning-large-language-models/) (taught by Dr. Zhou).

---

---