In November 2022, OpenAI took the world by storm when they released ChatGPT. In just 2 months, ChatGPT had gained over 100 million users, making it the fastest growing software product of the time. Since then, there have been explosive advancements in the field of Generative AI (GenAI), which, as the name suggests, is a type of artificial intelligence that can automatically create various types of new content, including text, images, audio, and video. A GenAI model can generate a lot of things — creative writing, travel plans, code, you name it.
That said, getting what you want from a GenAI tool is not as straightforward as it may seem. In order to harness the full potential of GenAI tools, one must know how to communicate with them effectively. You must know how to craft clear, unambiguous, questions and instructions for these GenAI tools. These should provide enough context and be targeted and specific. In other words, you must know how to engineer the inputs you provide to GenAI tools to get the most out of them.
This article offers a peek into the world of Prompt Engineering. Through it, you will learn how to communicate effectively with GenAI tools like ChatGPT, Bing Chat or Bard. It includes:
- An introduction to prompts — the what and the why.
- A glimpse into the internal working of GenAI tools.
- Guidelines for crafting prompts that enable you to communicate effectively with GenAI tools to make them do your bidding.
Let’s jump right in.
Prompt Engineering: The “What” and the “Why”
Simply speaking, a “prompt” is the text input that you provide to a GenAI tool: questions, instructions, anything. On receiving this input, the GenAI tool is prompted to start generating a response for it. Unfortunately, GenAI tools often misunderstand prompts. When prompts are not well written, the generated responses are, at best, irrelevant; and in the worst case, they are completely made up.
Subtle changes in prompts result in unexpected responses
Because of how GenAI tools work under the hood, even slight changes in a prompt can result in very different responses. Consider these two prompts: “Explain the concept of climate change” and “Describe the idea behind climate change”.
Here, the underlying topic asked about is the same — climate change. But subtle differences in wording (“Explain the concept of” vs. “Describe the idea behind”) lead to responses that emphasize different aspects of the topic. The first response focuses on defining climate change, while the second one focuses on its causes.
This is where Prompt Engineering comes in.
Prompt Engineering
Prompt Engineering is the practice of carefully crafting ideal prompts for GenAI tools to get the most out of them. The goal is to ensure that GenAI tools accurately interpret the prompt and generate the correct response we are looking for.
In addition to the accuracy and effectiveness of the response, there is another reason to get things right – and that is cost. While end user tools like ChatGPT are currently free, the OpenAI APIs have a pricing model that is roughly based on the prompt size as well as the response size. We will discuss more about this later in the article.
Prompt Engineering guidelines and techniques are agnostic of specific GenAI tools. While this post uses ChatGPT for all the examples, the underlying concepts can be applied to any GenAI tool (Google Bard, Bing Chat, etc.)
How GenAI tools work behind the scenes
To learn how to craft good prompts, let’s first peek under the hood to understand the internal working of GenAI tools — how they process prompts and generate responses. Specifically, let’s talk about GenAI tools that generate text content.
Text based GenAI tools internally use a Large Language Model (LLM). An LLM is an AI model designed to understand and generate natural human language (as opposed to machine language like binary). At a high level, it can be thought of as a function which accepts some input text (the prompt) and returns text that is most likely to be present after the prompt. Think of it not as a “question answerer”, but as a “sentence completer”.
LLMs are trained on massive amounts of text data so they can learn the patterns, structures, and semantics of the language, as well as the information present in the data. They can then use this knowledge to read the prompt and create new text that is most likely to be present after the prompt. How much data? GPT-3.5, the LLM that powers ChatGPT, was trained on 570GB of text data — about 300 billion words.
When an LLM receives a text prompt, it first converts this text into tokens. A token is the smallest unit of text that an LLM can understand. A more technical definition is: “A token is an instance of a sequence of characters in some particular document that are grouped together as a useful semantic unit for processing.”
How is this different from just words? Well, consider the word “everyday”. It can be broken down into two tokens: “every” and “day”. The same goes for the word “afternoon”. It can be broken down into two tokens: “after” and “noon”. That said, it’s not necessary that an LLM breaks down words into tokens the same way that humans would do. For example, a human would probably break down "upscale" as "up and "scale", but an LLM could break it down as "ups" and "cale". (OpenAI has a rough rule of thumb that a token is about 4 characters, but this is not absolute).
Basically, a word can be made up of one or more tokens. Each token is further converted into a token ID, which is a unique numerical ID assigned to each token in the LLM’s vocabulary. Same tokens share token IDs. This step — converting text into tokens is called “tokenization”. OpenAI has created an interactive Tokenizer tool which allows you to see how text is broken down into tokens.
At the ground level, LLMs operate on lists of token IDs. The prompt and the response are both lists of token IDs. LLMs use the tokens in the prompt along with their positioning to generate a new list of tokens based on patterns learnt from the dataset during training. The key takeaway here is that the model does not understand the prompt text literally in the same way that humans do. For the same reason, LLMs often mess up when asked basic math questions. Try asking ChatGPT: “What is (23 + 73 - 21) * 2 / 15?”
The correct answer is 10. The LLM is not really trying to solve the problem using the rules of math. It’s just trying to predict the tokens that are most likely to follow the prompt.
Types of prompts
Now that you have a better understanding of how things work under the hood, let’s discuss some different types of prompts which can be used for a number of use cases.
1. Questions and Answers (Q&A)
The prompt to the GenAI tool is a question in this case, and it responds with an answer for it. Pretty straightforward. The key thing to keep in mind here is that the question should be clear and should include all the context necessary for the GenAI tool to understand it.
Prompt:
On what date was the first Harry Potter book published?
Also, when was the movie based on it released?
2. Summarization
GenAI tools can be prompted to generate a summary of lengthy text — reports, documents, news articles, research papers, and more. Here, it’s important to explicitly and clearly state the expected length of the summary. The prompt can also optionally include characteristics of the text that the summary should focus on.
Prompt:
Please write a clear, concise summary of the following book review.
Limit it to strictly 50 words or less.
The summary should focus on feedback about the plot of the book.
Text:"""
"Harry Potter and the Enchanted Heir," penned by the beloved J.K.
Rowling, masterfully extends the enchanting wizarding world. Set years
after the original series, it follows Harry's determined son, Alaric,
as he faces his own destiny amidst shifting dynamics. Rowling's
unparalleled storytelling shines as she weaves intricate plots,
explores new magical realms, and delves into the complexities of legacy
and choice. The characters are richly developed, their struggles
relatable even within the fantastical setting. The familiar charm is
present, but the narrative matures, tackling themes of identity,
friendship, and the responsibilities that come with power. Magical
creatures and awe-inspiring spells abound, creating a captivating
atmosphere. While an engrossing read, some pacing issues and a surplus
of characters occasionally hinder immersion. Nonetheless, "Enchanted
Heir" is a triumphant return, rekindling the magic while offering fresh
perspectives, leaving fans both nostalgic and excited for the
wizardry's next generation.
"""
3. Language Translation
Prompts can be used to instruct GenAI tools to generate translations of text in different languages. LLMs have impressive translation capabilities, since they internally use modified versions of the transformer architecture, which is specifically designed for translation. Furthermore, they are trained on multilingual text data, which enables them to learn the structure, grammar, and patterns of various languages.
The prompt should clearly indicate the desired target language. If the source language is known, it should be included in the prompt as well. While LLMs can identify the source language on their own, specifying it helps in reducing confusions / mistakes.
Prompt:
Please translate the following text into English, and then also into
Spanish.
Text:"""
Il mondo è un libro, e coloro che non viaggiano leggono solo una
pagina.
"""
4. Inferring
GenAI tools can be instructed to draw logical deductions or conclusions from the provided text. We can ask them to extract information from the text, like sentiment, cause and effect, etc. As always, the prompt should clearly explain what is to be extracted / inferred.
Prompt:
Please extract some information from the product review below. In
particular, get the product name, its price, overall sentiment (one of
"Positive", "Negative", and "Neutral"), and the rating (if present).
Text:"""
The Apple Magic Mouse boasts seamless design and touch functionality,
enhancing Mac navigation. However, its charging setup can be
frustrating; the charging port's bottom placement renders it unusable
while charging. The absence of a pass-through option further compounds
the inconvenience. I'd rate it 6/10. Priced at $79, it's on the higher
side for a basic mouse.
"""
5. (Creative) Writing
GenAI tools can be used for writing essays, emails, stories, poetry, etc. Like summarization, it’s important to provide clear information about the length of the expected output. Also, prompts should include sufficient guiding information for the GenAI tool to ensure it doesn’t go off topic.
If you’re using the OpenAI Chat APIs, you can experiment with two parameters — temperature and top_p, which control the degree of randomness in the generated responses. Lower values of these parameters keep the output focused and predictable, whereas higher values make the output more random and surprising — or creative. The API guidelines are to change any one of these parameters, not both.
Prompt:
Write a short, formal email for a leave application of 2 days.
Clearly mention the start and end dates of the leave.
Strictly limit the email to 4 lines of text.
6. Code Generation
GenAI tools can translate requirements explained through natural language prompts into source code written in different programming languages. While it doesn’t provide end-to-end code for entire apps, it can help with writing short scripts, creating code examples, and with automatic code completion. GitHub Copilot is a great example that showcases the potential of GenAI in this domain.
It is critical that the prompt explains the requirements in detail here.
Prompt:
Write a C++ program that prints the sum of the first 20 elements of
the Fibonacci series. The program should not store all the Fibonacci
elements in an array.
LLMs also make reasonable assumptions and use decent defaults in some cases. In the following example, the LLM makes good assumptions about email servers.
Prompt:
I have a list of people's names and email addresses in a spreadsheet,
and a template email in a text file.
In the spreadsheet, column A contains email addresses and column B
contains names.
The template email contains a placeholder that is to be replaced with
the person's name. The placeholder is: {name}
Please write a short Python script which will populate the templates
and send them out as emails.
7. Text Completion
Recall that LLMs are fundamentally “sentence completers”. A portion of the prompt can be left incomplete, and the GenAI tool can be asked to complete it. This can also be used in finishing an incomplete story.
Prompt:
Please complete the text given below.
Text:"""
In a world where you can be anything,
"""
Guidelines for designing prompts
Next, let’s cover some guidelines and tips for crafting good prompts. We’ll start with some poorly written prompts and then improve them.
Note: An overarching guideline is that a prompt should follow the general rules of the language it is written in. It should avoid grammatical and spelling mistakes.
1. Be specific about the desired outcome
Prompts should mention details about the expected response clearly. The length and style of the output, the intended audience, necessary context, etc. should be a part of it. If the prompt is too short and general, it leaves too much to the imagination of the GenAI tool.
Rookie Prompt:
Can you write me a magical story?
Pro Prompt:
Write a short fantasy story for kids aged 9 and above. The story should
be approximately 600 words long. The plot of the story should revolve
around five friends who are chosen to learn magic from a wizard and how
they use their skills to save the world from interstellar forces.
2. Make prompts unambiguous
Prompts should provide enough context to disambiguate terms with multiple meanings. Consider the term “Java”. It could refer to the programming language, the type of coffee, or the Indonesian island.
Rookie Prompt:
In under 100 words, summarize the history of Java.
Pro Prompt:
In under 100 words, summarize the history of Java, the Indonesian island.
3. Use delimiters like """ or ```
Put instructions at the start of the prompt, followed by the data to process. Demarcate the start and end of the data with delimiters like """ or ```. This helps the GenAI tool avoid confusion about where the instructions end, and where the data begins.
Rookie Prompt:
{data_to_be_processed}
Please extract the product name and its price from the text above.
Pro Prompt:
Please extract the product name and its price from the text below.
Text: """
{data_to_be_processed}
"""
4. Clearly specify the format of the desired output
Don’t just tell, show as well. GenAI tools respond better when format requirements are shown through an example.
Rookie Prompt:
Extract post titles and author names from the following text and print
them out in JSON format.
Text: """
{data_to_be_processed}
"""
Pro Prompt:
Extract post titles and author names from the following text and
format them as a list of JSON objects.
Desired format:
[
{
"post_title": "...",
"author_name": "..."
},
...
]
Text: """
{data_to_be_processed}
"""
5. Provide multiple action items as steps
When there are several action items mentioned in the prompt, GenAI models might not get all the actions correct. To achieve good results, we should specify steps that may help models in completing the action items.
Rookie Prompt:
Summarize the following text in under 30 words, then translate the
summary into Spanish, then list out all the product names mentioned in
the text and print out all this information in an XML object containing
the keys "summary", "spanish_summary", and "product".
Text: """
{text}
"""
Pro Prompt:
Please perform the following tasks:
1. Summarize the following text in under 30 words.
2. Translate this summary into Spanish.
3. List out the product names mentioned in the text.
4. Print out the output of (1), (2), and (3) in the XML format with the
keys "summary", "spanish_summary", and "products".
Text:"""
{text}
"""
Prompt Engineering techniques
1. Zero-shot Prompting (Don’t be vague, provide enough context)
Most AI tools of today are built on the principles of Supervised Machine Learning — they learn patterns from labeled examples. However, LLMs are trained on such vast and diverse data that they can provide accurate responses for lots of prompts “zero-shot”; i.e., without any examples provided.
Prompt:
Please extract the sentiment (one of Positive/Negative/Neutral) from
the text given below:
Text:"""
I'm not enjoying this movie. I think it's quite convoluted and intense.
"""
We didn’t provide any examples of text and the classification here. The LLM already understands “sentiment”.
To harness the zero-shot capabilities of the LLM, the prompt should be as detailed as possible. Aim to follow the guidelines from the section above as far as possible.
2. Few-shot Prompting (Provide examples when needed)
While LLMs have impressive zero-shot capabilities, they fall short when generating responses for questions that they don’t have sufficient context about. This can be fixed by few-shot prompting — providing some examples in the prompt which the LLM can use to learn patterns and make deductions.
Separate examples with delimiters like ### or ---.
Prompt:
Text 1: Twilio provides easy-to-use APIs that developers can use to
integrate messaging into their websites and mobile applications.
Keywords 1: Twilio, messaging, APIs, developers, websites,
mobile applications
###
Text 2: Google provides APIs that developers can use to incorporate
features like maps and language translation into their applications
and websites.
Keywords 2: Google, APIs, developers, maps, language translation,
websites, applications
###
Text 3: GitHub provides APIs that developers can use to integrate
features like version control, issue tracking, and pull requests
into their software development workflows and applications.
Keywords 3:
With good examples, this can expand to some complex deduction problems as well.
3. Chain-of-thought Prompting (Force getting results in steps)
Recall that LLMs don’t understand the rules of math / logic. They make mistakes when asked simple arithmetic questions since they just try to predict the following tokens.
Prompt (without Chain-of-thought Prompting):
I went to the market and bought 10kg tomatoes and 4 apples. I gave 2
apples to my friend and 2kg tomatoes to the neighbor. Then, I bought
1kg more tomatoes and ate 1 apple. How many apples and tomatoes did I
have left?
In such cases, explicitly asking the GenAI tool to find the answer step-by-step helps in improving the accuracy of the response.
Prompt:
I went to the market and bought 10kg tomatoes and 4 apples. I gave 2
apples to my friend and 2kg tomatoes to the neighbor. Then, I bought
1kg more tomatoes and ate 1 apple. How many apples and tomatoes did I
have left?
Please go step-by-step.
4. Role Play Prompts (Ask for specific expertise)
This is an important and widely used technique. Through the prompt, the GenAI tool is instructed to take on a specific role and respond as the person in that role would. This allows simulating a conversation with a GenAI tool that behaves like a certain character / someone in a certain profession. It is a versatile way to interact with language models and can be used for various creative, educational, or entertaining purposes.
Prompt #1:
You are an AI research expert. Briefly describe your outlook on the
growth of GenAI.
Prompt #2:
You are now a salesman. Briefly describe your outlook on the growth of
GenAI.
Reducing Hallucinations
Sometimes, GenAI tools generate responses that are nonsensical or outright false. These are called “hallucinations” in the context of GenAI. When this happens, GenAI tools confidently provide fictional / made up answers.
Hallucinations happen because:
- LLMs are trained on vast amounts of data from the internet, which includes fake / spam data.
- LLMs try to generate contextually and probabilistically relevant answers. However, they don’t have true understanding of the source material or logical reasoning capabilities like humans do.
Currently, there is no guaranteed way to stop hallucinations. However, they can be reduced by using better prompts. The guidelines mentioned earlier are a good starting point to reduce hallucinations. Apart from this, prompts should not include incorrect information that could confuse LLMs into thinking creatively and making up information.
GenAI tools can also be instructed to not provide incorrect answers by explicitly telling them what to do when they don’t know the answer. Prompts can explicitly instruct the GenAI tool to either admit that it doesn’t know the answer or request for more context when needed.
Additionally, while using the OpenAI Chat APIs, the temperature and top_p parameters mentioned earlier can be used to reduce hallucinations as well. Lower values of these parameters result in more predictable results.
Jailbreak prompts
LLMs are trained to decline requests for dangerous, malicious, or inappropriate content, and they’re very good at it. However, through some carefully crafted prompts, they can be tricked into bypassing these ethical restrictions. Such prompts are called Jailbreak prompts, and this technique is called Prompt Injection. They use a combination of different techniques like role play, few-shot, chain-of-thought, etc. The image below is an example of how a tricked, or in other words, a “jailbroken” LLM would respond.
As LLMs get better, they will learn to either ignore or better respond to such jailbreak prompts, but theoretically, as long as they are fundamentally “sentence completers”, they can always be tricked.
Pricing
While some GenAI tools like ChatGPT, Google Bard and Bing Chat are free to end users, pricing comes into play when you want to integrate GenAI capabilities into your own custom apps and solutions. You would typically leverage the pre-trained LLMs through APIs provided by these vendors. OpenAI API’s pricing model is based on the length of the prompt and the response — or more precisely, it is based on the number of tokens used in your prompt and the generated response.
In general, longer prompts and responses cost more. Due to this, it is important to write efficient prompts that generate good responses. So, it is really important to pay attention to prompt engineering as you develop your custom GenAI app.
Wrapping up
Harnessing the full potential of GenAI tools requires effectively communicating with them. We hope you use the different tips and techniques discussed in this article to craft better prompts. Happy chatting and thanks for reading.