An Introduction to Prompt Engineering with LangChain

LangChain is an open-source framework designed to easily build applications using language models like GPT, LLaMA, Mistral, etc.

One of the most powerful features of LangChain is its support for advanced prompt engineering. Prompt engineering refers to the design and optimization of prompts to get the most accurate and relevant responses from a language model.

In this post, we will cover the basics of prompts, how Langchain utilizes prompts, prompt templates, memory, agents, and chains, and for each of these core concepts, we will see several code examples in Python so by the end of this tutorial you are comfortable in building your own applications using LangChain framework in Python.

Curious about how LangChain and GPT can be used to create a simple AI agent? Check out our Prompt Engineering with GPT & LangChain AI Code-Along:

What is Prompt Engineering?

Image by Author

We’ve got a full guide covering prompt engineering, but as a quick refresher, it’s the practice of creating text inputs that effectively communicate with generative AI models. These prompts, written in natural language, specify the task we want the AI to perform.

They can be diverse in form, ranging from simple questions like asking for an explanation of a mathematical theorem to more creative requests such as writing a poem about a specific theme.

In prompt engineering, the key is not just what you ask but how you ask it. This might include choosing the right wording, setting a particular tone or style, providing necessary context, or even defining a role for the AI, like asking it to respond as if it were a native speaker of a certain language.

In some cases, prompts can also include examples to guide the AI's learning process. This technique, known as few-shot learning, can be particularly effective for complex tasks.

When dealing with models that generate images or audio, prompts usually describe the desired output in detail, from the subject and style to the composition and mood of the final product. The art of prompt engineering lies in adjusting these elements to guide the AI toward producing the desired result.

The Basics of Prompts

Image by Author

Prompts are the inputs provided to large language models (LLMs) to elicit a desired response. Well-designed prompts are crucial to get useful outputs from LLMs. The goal of prompt engineering is to carefully construct prompts that result in accurate, relevant, and helpful LLM responses.

A good prompt typically contains these four components:

1. Instructions - Tell the model what to do, how to use the provided information, what to do with the query, and how to construct the output.
2. Example Input - Provide sample inputs to demonstrate to the model what is expected.
3. Example Output - Provide corresponding example outputs.
4. Query - The actual input you want the model to process.

Except for query, everything else is optional but may impact the quality of response significantly, especially the instructions that provide guidance to the LLM on the task and expected response.

Examples demonstrate the desired output format for a given example input.

The query is the question or request made to the LLM.

Constructing effective prompts involves creatively combining these elements based on the problem being solved.

LangChain Prompts

LangChain offers various classes and functions to assist in constructing and working with prompts, making it easier to manage complex tasks involving language models.

Langchain provides first-class support for prompt engineering through the `PromptTemplate` object.

Prompt templates serve as structured guides to formulating queries for language models.

These templates typically comprise directives, a selection of example inputs (few-shot examples), and tailored questions and contexts suitable for specific tasks.

The aim of LangChain is to design templates that are compatible with various language models, enhancing the ease of transferring templates between different models.

In short, you can use the PromptTemplate class to create a template for a string prompt.

from langchain.prompts import PromptTemplate


prompt_template = PromptTemplate.from_template(
    "Tell me a {adjective} joke about {content}."
)
prompt_template.format(adjective="sad", content="data scientists")

Output:

>>> 'Tell me a sad joke about data scientists.'

It is not necessary to have a variable. You can create a prompt using this class even if you do not have any variables.

from langchain.prompts import PromptTemplate
prompt_template = PromptTemplate.from_template("Tell me a joke")
prompt_template.format()

Output:

>>> 'Tell me a joke'

What if you are building a chat application where you need to save and send the message history to LLM every time a user sends a new message? Doing this manually is time-consuming.

For supported chat models, you can instead use ChatPromptTemplate:

from langchain.prompts import ChatPromptTemplate


chat_template = ChatPromptTemplate.from_messages(
    [
        ("system", "You are a helpful AI bot. Your name is {name}."),
        ("human", "Hello, how are you doing?"),
        ("ai", "I'm doing well, thanks!"),
        ("human", "{user_input}"),
    ]
)


messages = chat_template.format_messages(name="Bob", user_input="What is your name?")


print(messages)

Output:

>>> [SystemMessage(content='You are a helpful AI bot. Your name is Bob.'),

>>> HumanMessage(content='Hello, how are you doing?'),

>>> AIMessage(content="I'm doing well, thanks!"),

>>> HumanMessage(content='What is your name?')]

So you must be thinking, why do we need PromptTemplates? Why can’t we directly pass the string prompt to LLM? Here are some good reasons to use LangChain’s PromptTemplate:

• Prompt templates are reusable, allowing you to create a prompt once and use it in multiple situations, like summarizing different articles without rewriting the prompt each time.
• They separate the formatting of prompts from the actual model invocation, making the code more modular and allowing independent updates to the template or model.
• Using templates improves the readability of your code, as they encapsulate complex logic in a simpler format with clearer, named variables.
• Maintenance becomes easier with prompt templates, as changes to the prompt logic need only be made in one central place, rather than in every individual prompt.

LangChain Memory

In language model applications that involve chatting, it's important for the system to remember past conversations.

Just like in a normal chat, the system should be able to look back at what was said before.

At its simplest, this means the system can see a few of the previous messages. But a more advanced system will remember more details, like facts about different topics and how they connect.

This ability to remember past chats is called "memory." LangChain helps add memory to these chat systems.

A memory in LangChain does two main things:

1. Before sending the query to LLM, the system reads from its memory to enhance the initial input prompt it received from the user.
2. Once the response is returned by LLM, it writes it to memory for future interactions, before returning the final output to the user.

Memory in LangChain (Image Source)

To use the Memory feature in LangChain you can use the ConversationBufferMemory class.

from langchain.memory import ConversationBufferMemory
memory = ConversationBufferMemory(return_messages=True)
memory.chat_memory.add_user_message("hi!")
memory.chat_memory.add_ai_message("what's up?")
memory.chat_memory.add_user_message("how are you doing?")
memory.chat_memory.add_ai_message("I am fine, thank you and you?")


# to check the variables history
memory.load_memory_variables({})

Output:

>>> {'history': [HumanMessage(content='hi!'),

>>> AIMessage(content="what's up?"),

>>> HumanMessage(content='how are you doing?'),

>>> AIMessage(content='I am fine, thank you and you?')]}

It returns a dictionary where the key by default is history. This key leads to a list of messages, and these messages can be either from a human (HumanMessage) or from the LLM (AIMessage).

LangChain Chains

For simple tasks, using a single LLM (Large Language Model) works well. However, for more complex tasks, it usually requires chaining multiple steps and/or models.

The legacy way of using Chains in LangChain is through the Chain interface.

The newer and recommended method is the LangChain Expression Language (LCEL). Although LCEL is preferred for new projects, the Chain method is still useful and supported.

from langchain.llms import OpenAI
from langchain.chat_models import ChatOpenAI
from langchain.prompts import ChatPromptTemplate
from langchain.schema import StrOutputParser


model = ChatOpenAI(openai_api_key = “...”)


prompt = ChatPromptTemplate.from_messages(
    [
        (
            "system",
            "You're a expert data scientist and machine learning engineer who offer its expertise and responds to the point and accurately.",
        ),
        ("human", "{question}"),
    ]
)
runnable = prompt | model | StrOutputParser()


for chunk in runnable.stream({"question": "How is machine learning different than deep learning?"}):
    print(chunk, end="", flush=True)

Output:

>>> Machine learning and deep learning are subfields of artificial intelligence (AI) that involve training models to make predictions or decisions based on data. While there is an overlap between the two, they ……..

The way runnable is created is not pretty standard for Python, but this | is a pipe operator. Prompt | model | StrOutputParser() would mean that Prompt is passed through the model, and the output of that is then passed through StrOutputParser(). Each segment in the chain transforms the data in some way.

Want to learn more about interesting use cases and applications that can be built using LangChain? Check out our Engineering and Data Application use-cases blog.

LangChain Agents

The fundamental concept of agents involves utilizing a language model to select a series of actions.

Unlike chains, where a sequence of actions is pre-determined and embedded in the code, agents employ a language model as a decision-making tool to identify the appropriate actions and their sequence.

There are three key concepts in LangChain’s Agents:

1. Tools: Descriptions of available tools to LLM
2. User input: The high-level objective of the task
3. Intermediate steps: Any (action, tool output) pairs previously executed in order to achieve the user input

Tools represent functionalities that an agent can activate. Two crucial aspects of tool design include:

• Ensuring the agent has access to appropriate tools.
• Defining the tools in a manner that maximizes their usefulness to the agent.

For various routine activities, an agent requires a collection of interconnected tools. To address this, LangChain introduces the idea of toolkits. LangChain offers a broad range of toolkits to get started.

To learn more about Agents, check out this Official LangChain Guide.

Credit: Some examples in this blog have been generated using official LangChain documentation.

Conclusion

By offering a structured approach to designing prompts, LangChain not only simplifies the process of interacting with different language models but also maximizes the efficiency and relevance of their outputs. The features like PromptTemplate and Memory in LangChain demonstrates a thoughtful integration of flexibility and precision in prompt engineering, catering to a wide range of tasks from simple queries to complex conversational contexts.

The advanced features in LangChain's Chains and Agents furthers the potential of applications that can be built using LangChain in just a few lines of code with any underlying choice of LLM, and there are many to choose from.

Whether for simple tasks or intricate applications, LangChain's approach to prompt engineering is pretty simple, clear, and easy to adopt.

Want to learn more? Check out part 1 of How to Build LLM Applications with LangChain on DataCamp.