Conversations with LLMs: Patterns and Paradigms for Better Prompting

While Large Language Models (LLMs) like ChatGPT and Claude have become powerful tools for everyday work, their effectiveness largely depends on how we communicate with them. From Chain of Thought to Graph of Thoughts approaches, these structured methods can significantly improve the quality and reliability of AI-generated outputs. This article by Manuel Riesen and Peter von Niederhäusern provides an overview of key concepts, prompting techniques and paradigms to obtain better results.

A language model is a useful tool for everyday work, especially when working with source code for programming or text generation and analysis. With the  release of ChatGPT in 2022, OpenAI achieved mainstream adoption of LLMs [1]. As of Fall 2024, other models like Google Gemini [2], Claude [3] or LLaMA [4] are available as well. There is also an emerging trend to overcome the limitations of classic search engines like Google. Perplexity [5] combines the above into one tool.

This article moves beyond the common ‘prompting cheat sheets’ found online to explore scientifically grounded prompting patterns and prompting paradigms.

Figure 1: AI Prompt: “Woman in front of a computer, thinking; photorealistic”

Key Concepts

With or without following prompting patterns or certain paradigms, there are some key concepts to consider when formulating prompts.

Key Concepts

• Scope: It is important to set the right scope. Explaining your intent and what should not be done is helpful.
• Context: Context is key. Provide as much domain knowledge as possible. Language models benefit from in-context learning, allowing them to improve their performance through examples provided during prompting [6]. For instance, when requesting slide content for a presentation based on a document, providing examples from previous presentations can guide the language model toward the desired depth and structure of the output.
• Format: The desired length and structure of responses should align with your specific needs. While some language models default to creating lists, this isn’t always optimal. Clear instructions about your preferred format and response length help ensure the generated output is immediately useful for your purpose.

Prompting Patterns

Prompting patterns are comparable to software design patterns. Patterns aim to improve the quality of a prompt (or software) by applying proven techniques for recurring challenges. Which patterns to apply is dependent on the nature of the problem and the objective to achieve. Like software design patterns, prompting patterns are often applied intuitively, even without formal knowledge of them. Below, we explore several established patterns that can enhance your prompting strategy. [7]

When requiring output in non-natural language formats, such as computer code or graph- or other mathematical notation, the Meta Language Creation Pattern proves valuable. This pattern establishes a clear context for both input logic and expected output format, ensuring more accurate results. [7]

Another possibly helpful pattern is the Output Automater Pattern. This involves providing a general instruction about the expected properties of the output. For example, one could demand the language model to output only in a diff-format (lines prefixed with +/- for adding/removing) when editing a text. More practically, for programming tasks, the language model could be prompted to generate a script that adjusts the code files automatically, so that no manual editing must take place. A drawback here is the higher complexity for the language model, which may lead to errors. [7]

Sometimes, it is hard to define what information is needed by the language model to solve a certain task. Here, the Flipped Interaction Pattern can be beneficial. Instead of providing all information, the language model is prompted to ask questions about the problem so that it can solve the task. [7]

Figure 3: An interaction pattern, which inverses the Q & A session with an LLM.

In situations where one wants the language model to answer like a persona, the Persona Pattern comes into play. Telling the language model it should answer like someone in a specific field or with a certain role can help to get the right answers. [7]

Figure 4: Instructing the LLM to be a persona (avatar, proxy) for contextualizing the problem domain.

Prompting Paradigms

Prompting paradigms describe the (potentially complex) structure of a conversation with a language model.

The Chain of Thought (CoT) paradigm improves problem-solving by asking language models to show their work, similar to thinking aloud. This is reasonable because predicting the next token (or word) is easier with more context and a clear direction. Moreover, the user can see the “reasoning” steps of the language model, which improves transparency. CoT can be implemented either as a single prompt or as a series of sequential outputs called ‘thoughts’.[8]

Recently, OpenAI released their ChatGPT o1 model variants, which leverage this technique (in combination with Reinforcement Learning and a proprietary algorithm) in the background to refine the answers [9].

A more complex paradigm is the Tree of Thoughts (ToT), where the thoughts of a language model are structured as a tree [10]. For example, the language model can be prompted to refine an initial thought. Doing this multiple times for the same start thought leads to a branching of the tree. By letting the language model score its results, one can choose the best thought at the end.

While this paradigm leads to better results for certain tasks, it is not feasible to carry out the operations manually. Instead, an automated way, such as creating a Python script, is required.

A rather new paradigm is represented by the Graph of Thoughts (GoT). framework. Here the thoughts are structured as an arbitrary graph [11]. This enables the use of aggregating operations, such as combining two individual thoughts back together to a single one. Again, automation is required to leverage this paradigm in an efficient way. While the original authors of the GoT paper published their Python framework, we also provide an augmented implementation that aims for easier use and a more declarative approach rather than imperative [12].

Conclusion

These patterns and paradigms provide a systematic foundation for more effective prompting of language models. As LLMs evolve, structured approaches like these will be key to making the most of their potential in everyday tasks.

References

[1] OpenAI. Introducing ChatGPT. Nov. 2022. url: https://openai.com/index/chatgpt/ (visited on 11/23/2024).

[2] Google. Gemini – Supercharge your creativity and productivity. 2024. url:https://gemini.google.com/ (visited on 12/05/2024).

[3] Anthropic PBC. Meet Claude. 2024. url: https://www.anthropic.com/claude (visited on 11/23/2024).

[4] Meta. Introducing LLaMA 3.2. 2024. url: https : //www.llama.com/(visited on 11/23/2024).

[5] Perplexity. What is Perplexity? 2024. url: https://www.perplexity.ai/hub/faq/what-is-perplexity (visited on 12/05/2024).

[6] Qingxiu Dong et al. A Survey on In-context Learning. 2024. arXiv: 2301.00234 [cs.CL]. url: https://arxiv.org/abs/2301.00234.

[7] Jules White et al. A Prompt Pattern Catalog to Enhance Prompt Engineering with ChatGPT. arXiv:2302.11382 [cs]. Feb. 2023. doi: 10.48550/arXiv.2302.11382. url: http://arxiv.org/abs/2302.11382 (visited on 11/23/2024).

[8] Jason Wei et al. “Chain-of-thought prompting elicits reasoning in large language models”. In: Proceedings of the 36th International Conference on Neural Information Processing Systems. NIPS ’22. New Orleans, LA, USA: Curran Associates Inc., 2024. isbn: 9781713871088.

[9] OpenAI. Introducing OpenAI o1. 2024. url: https://openai.com/o1/ (visited on 11/23/2024).

[10] Shunyu Yao et al. “Tree of thoughts: Deliberate problem solving with large language models”. In: Advances in Neural Information Processing Systems 36 (2024).

[11] Maciej Besta et al. “Graph of Thoughts: Solving Elaborate Problems with Large Language Models”. In: Proceedings of the AAAI Conference on Artificial Intelligence 38.16 (Mar. 2024). Publisher: AAAI Press, pp. 17682–17690. doi: 10.1609/aaai.v38i16.29720. url: https://ojs.aaai.org/index.php/AAAI/article/view/29720.

[12] Manuel Riesen. Pure Graph of Thoughts. 2024. url: https://github.com/mriesen/pure-graph-of-thoughts (visited on 11/23/2024).