When I tell people I wrote an O'Reilly book on Prompt Engineering for Generative AI, their first question is always "won't it be out of date the moment it's printed?". It's something my co-author James Phoenix and I had to think about, a lot…
I’m sharing our approach to writing this book as it might help other potential authors think through how to approach a fast-moving topic in their own writing.
Principles of Prompting
When I started prompting back in 2020 with the GPT-3 beta, there were all sorts of tricks and hacks you had to use to get the model to return something useful. When GPT-4 was released two years later, a lot of those techniques were no longer necessary.
I didn’t want that to happen again when GPT-5 came out, so I spent some time thinking about what underlying principles would still be true no matter what happened with AI development. I looked for anything that was transferable across models and modalities, as we wanted the principles to still be useful whether you were using OpenAI, Google, Anthropic, or an open source model, to generate text or images.
Whenever I found two techniques that were kind of similar, I merged them and used a broader description that encapsulated both techniques. For example, ‘chain of thought’, the technique where you ask an LLM to first plan its answer before answering, is basically the same thing as chaining, splitting the task into multiple steps. The same underlying principle, division of labor, adequately describes both techniques.
Before working in AI I built a 50 person marketing team, so it struck me that the principles that were most timeless and transferrable were also things that worked for managing humans. My hypothesis became that management techniques that have worked for managing biological intelligence (humans) for hundreds of years, will likely continue to work even when the intelligence you’re managing is artificial. I took one final pass at the principles with this lens, which helped me finalize the list.
Here are the five principles of prompting I settled on:
1. *Give Direction*: Describe the desired style in detail, or reference a relevant persona.
2. *Specify Format*: Define what rules to follow, and the required structure of the
response.
3. *Provide Examples*: Insert a diverse set of test cases where the task was done
correctly.
4. *Evaluate Quality*: Identify errors and rate responses, testing what drives performance.
5. *Divide Labor*: Split tasks into multiple steps, chained together for complex goals.
The whole book is based on these principles, and we sprinkled them throughout every chapter whenever we gave an example of a technique being used, in order to reinforce them. Even if OpenAI dropped GPT-5 the week after the book was published, I would still be confident the content will remain relevant. It was a relief when OpenAI dropped their prompt engineering guide a year later, and my list lined up closely with theirs – a further sign that these principles were robust. We might not call it prompt engineering in five years, but I suspect these principles will still be useful for managing AIs far into the future.
Based on Real-World Experience
The principles were formed based on my actual experience automating my work and helping clients build AI applications, so it’s no surprise that many of those real-world examples made it into the book too. I had given 20-30 AI training workshops for companies looking to upskill their staff, so I had the advantage of seeing what problems came up again and again.
I am one of those obsessives that updates their presentation every time I give it – it’s never the same thing twice. I’m always testing new material, like a comedian trying out new jokes on a small crowd. When I see the audience’s eyes light up, I know that the example I just gave really made an impression. The very best examples would then become part of the book, and the rest of the material would be added to the popular Udemy course on prompt engineering we created. Books need far more condensed information than courses, as we had a page count limit due to printing costs, so that got the priority.
Basing your writing on real world scenarios is kind of a cheat code for making it useful to people, because if you have encountered the problem a bunch of times it’s likely other people have too. I find whenever I stray from writing about real-world scenarios, I end up straying too far into ‘trendy’ topics and those solutions are the ones that go out of date the fastest.
Historical Timeline Perspective
Everything seems to change every week within the AI industry, as new releases are happening all the time. We couldn’t easily make assertions like “Google Gemini can do X” or “the best model at doing Y is GPT-4”, because we knew that would be out of date quickly. Instead we focused on writing in almost a documentary style, telling what happened and when, rather than describing the state of the world today.
This was a little hard to get right, because it doesn’t come naturally to me. However, once I got it down, it made everything a lot easier to update. Some of the differences can be quite subtle. For example, I had first written something like “GPT-4 can’t do this task, as it only has an 8k token context window, whereas Anthropic’s Claude has 100k”. When GPT-4 increased the token window to 120k, my paragraph was instantly out of date. Instead I should have written “As of May 2023, Anthropic’s Claude 2 had a 100k context window, longer than the 8k token context GPT-4 had at the time.” Now if a reader looks at that statement and knows that GPT-4 can handle 120k now (and Google Gemini can handle 1 million), they can acknowledge that without finding anything incorrect about my statement.
Despite this way of writing getting us out of jail several times, we still were updating the book right up until the print deadline, which was coincidentally, the night OpenAI released GPT-4o. We only had to make minor changes that night, and nothing has been released since publish (so far) that makes the book dated. O’Reilly put us through multiple rounds of edits, with six different technical reviewers, as well as a couple of round of edits from our production editor, and another couple of rounds with a copy editor. Though this process took about a year, it did chip away at every line in the book, subjecting it to scrutiny. Everytime editors found something that needed to be updated, we rewrote it in a way that it wouldn’t need to be updated again any time soon. It meant getting to market slower than we would have liked, but ultimately it made the book more robust and sustainable over the long term focused.
The only major change that might cause us to publish a new version is GPT-5, however I don’t anticipate that would actually change much of the book’s content. We may need to add more examples of new things AI can do now if the capabilities expanded dramatically, but the existing principles, examples and problems would likely remain the same. At the end of the day you can only do so much to future-proof your book, and hope the readers still find it to be useful reference material far into the future.
