You can’t plan a battle without a map of the territory, and yet every day people launch courses on Udemy without understanding what works (or doesn’t) to rank highly in the Udemy marketplace.
My co-author James and I launched our prompt engineering course in March 2023, after a week of recording. We’re making a full time salary each from the course, working only one day per month to maintain it. We’ve taught ~125,000 people prompt engineering along the way.
Launching a course is a big undertaking, inherent with tonnes of risk. If you choose the wrong title, or teach the wrong topic, you could have wasted all that time recording. So we used memetic analysis before we launched to create a map of what works on Udemy, in order to guarantee as best we could that the course would be successful. Memetics is a method for de-risking all the creative decisions you have to make when launching a new product, like naming, writing marketing copy, and deciding what features or content to include.
I wanted to have one blog post I could point to on how we did it, because I’ve had a bunch of friends interested in launching their own courses (and other products). Hopefully you use this to avoid creative paralysis and finally launch that product you’ve been thinking of shipping.
What is memetics?
Memes aren’t just funny viral images on the internet, any recurring idea or pattern in our culture is a meme. Richard Dawkins coined the term as an analogy to biological genes, noting that ideas evolve too. If you look at the top Udemy courses in the Python category, and notice many of them use the word “bootcamp” in the title, congratulations – you just identified a meme. Some enterprising entrepreneur first used that term in their course title and did really well, enough for other course authors to notice and copy that term, replicating it over time.
This is actually how all creativity works – nothing is truly original, and we are all products of our environments. Some people will be explicitly copying the word “bootcamp” when they launch their course, and others copy it subconsciously because it “feels right” based on their experiences with online courses. The word bootcamp means something to the audience: it’s an analogy to military bootcamps, implying the course will be tough but get results.
We can see that the word bootcamp is successful in the marketplace of ideas being sold on Udemy, but success isn’t guaranteed. If a meme doesn’t get copied it dies out, just like the genes of unsuccessful species die out in the process of evolution. Eventually people might move on from using the word bootcamp if gets saturated by too many unsuccessful courses using it in the title. Understanding which memes to adopt or abandon plays a key role in any product’s success.
Scraping data for analysis
You can do memetic analysis completely manually, in fact it’s how almost all memetic analysis is done! Brand marketers and entrepreneurs often take a look at what’s working (or not) among their competitors, and identify different elements they can adopt in their own products or brands. The difference between what most people do and what I did was scale: I scraped a list of the top 100 Udemy courses in the Python category, and used Google Sheets to identify what memes were correlated with success or failure (check out the template here).
This isn’t a web scraping tutorial, as I’m not really the expert. All I did was inspect element, then copy and paste the HTML into ChatGPT, and ask it how to get a list of all the courses, titles, descriptions, reviews, and prices. It made a few mistakes which I then asked it to correct, and before you know it I had something to copy and paste into GSheets. If you can’t figure it out with ChatGPT, work with a developer you find on Upwork, or you can even outsource this kind of work to a virtual assistant in a low cost country, who is willing to do it manually.
Choosing a product category
Before we get deep into the memetic analysis, I want to zoom out a little bit and discuss how we decided to create a course on prompt engineering in the first place. We had been playing around with AI since 2020 with the GPT-3 beta, and were astounded by the results. We had been slowly replacing all of our work with AI, and when ChatGPT came out in November 2022, generative AI suddenly became mainstream. We saw demand for prompt engineering rising, and a blog post I wrote on the topic were getting traction: an early form of validation.
The only thing we really debated was whether to make the course technical or not. There were already a couple of early courses capitalizing on generative AI, marketed based on how many prompts you got access to. Looking in Udemy Markeplace Insights shows me these guys are making about 50% more than we are, but at the time it was much higher and we seriously debated whether to cash in. Ultimately we decided it would be unethical for us to sell a pack of prompts and a dream, when in our own experience you get better results knowing how to code and build systems that chain multiple prompts together.
How to name a product
The prompt engineering category didn’t have that many examples to go on, but we had already decided this would be a technical course, so we were competing in the Python category. Once we had the data, we could identify memes either by manual tagging (literally going through row by row and putting 1 or 0 in a column if a word, concept, or element was present), but that would be a pain for 100 courses. Instead I used regexmatch to see if a title contains a word, for example =if(regexmatch(B2,"(?i)complete"),1,0) returns 1 if the title contains the word complete.
I came up with ideas for what memes to tag by scanning the rows for patterns, and then formally tagged them if it seemed interesting. Once you’ve tagged or annotated all the different memes you care about, doing analysis is as simple as averaging that column: if 60% of the rows had a 1, and the rest were zeros, the result of averaging them would be 0.6.
Averages of labels told you something in terms of how prevalent a meme was, but you could also average the number of reviews and the ranking out of 100, which acted as a good proxy for how many sales the course had. I would do this in a pivot table so I could quickly see the share of 1s vs 0s, and averages of the review and ranking metrics.
Here are the insights I found through memetic analysis of the titles:
- The word ‘bootcamp’ in the title was worth 3x the number of reviews.
- Courses with either ‘advanced’ or ‘beginner’ in the title did terrible on average.
- Titles with the word ‘masterclass’ in them performed 30% worse.
- Having the word ‘learn’ in the title doubles the number of reviews.
- Almost every single course has the topic of the course in the title.
- Mentioning ‘AI’ or ‘machine learning’ in the title gets 4x the reviews.
- Having the current year in the title more than doubles reviews.
- It’s actually negative to include a specific framework or library in the title.
- There’s a small benefit from including the word ‘for’ in the title.
- Including ‘complete’ in the title gets 50% more reviews.
- Putting the number of ‘days’ in the title was uncommon, but extremely successful.
- Having ‘A-Z’ in the title did well, but it looked like a trademarked term.
- 40-60 characters in length for the title was the sweet spot, with 70% of all reviews.
Note: just because something is correlated with success, it doesn’t mean it actually caused that success. For example, if it just happened to be fashionable in Stanford university to call a course a ‘bootcamp’, you’d see a lot of successful courses with that in the name. It’s not because the word itself sells courses, but because all of the smart Stanford grads use that word, and going to Stanford might be what gives them the credibility and confidence to sell courses.
The title we went with, “The Complete Prompt Engineering for AI Bootcamp (2023)” used a number of these insights, but not all of them. Use the data as a guide of what to do if you have no opinion, because it’s often a better strategy to copy success even if you don’t know why something was successful. However, if you have high conviction in the other direction, zigging when the rest of the world is zagging can also be a fruitful strategy. As Rory Sutherland says, “in business, the opposite of a good idea, is often another good idea”.
How to write product page descriptions
I repeated the analysis for the product descriptions on the course pages, in exactly the same format as what I did for titles. This led to a number of interesting insights:
- Almost nobody used ‘fundamentals’, and those courses did 60% worse.
- The words ‘practical’ and ‘professional’ both seemed to matter.
- More than 90 characters in a description was good, more than 115 was slightly better.
- It also didn’t pay to include ‘beginner’ or ‘advanced’ in the description.
- The word ‘learn’ in the description also helpful, though not as much as in the title.
- Having the skill you’re learning in the description was a small boost in reviews.
- There was a disadvantage to having the topic or the framework in the description.
- Putting the library in the description conferred a huge advantage (small sample though).
- Listing the number of apps you’ll build in the description more than doubled the reviews.
- Adding primitives (for loops, recursion, etc) into the description doubled reviews.
The description as it stands is “Learn practical coding skills for working professionally with AI, including GPT-4, Stable Diffusion, and GitHub Copilot.” We initially had Midjourney in there instead of Stable Diffusion, and ChatGPT instead of GPT-4, but found early on that was attracting the wrong sort of customer. It’s useful to know what gets the most search traffic, because it often violates your assumptions (I had no idea Midjourney was so dominant), but popularity isn’t everything. More on that in the customer reviews section.
How to structure an online course
One of the hardest hurdles to jump when creating a new product is what features to build. With a course, that’s the table of contents: deciding what lessons or modules your course will cover, and how many of them there will be. This is all related to the price, which of course significantly dictates how much revenue you make, as well as the perceived value of the course. As you might expect, we did some memetic analysis on this too:
- The price should almost definitely be $59.99, as those courses get 13x more reviews.
- Even though they set it at $60, the actual sales price was around $16 on average.
- Almost all the top 10 courses have more than 100 lectures, though 50 was another bump.
For us it was clear: we price at $59.99 and opt into the Udemy dynamic pricing option, letting them promote us and discount us as they see fit. It can be painful to see the course priced so low (potentially it hurts our brand?), but you can’t argue that it doesn’t shift sales.
We also saw that it was important to get at least 50 lectures done, so that’s what we did. We launched with just over 50 (we did 25 each), and then kept adding more lectures every month. As we approached 100 we saw another huge boost to sales, validating this analysis.
How to improve customer reviews
The story doesn’t end there: memetics isn’t just for launch, it’s for life. With our successful launch we were almost earning enough to pay the mortgage, which in itself is life changing. However, 75% of our enrollments have come in the last 3 months, since we hit the top of the rankings when people search “prompt engineering” on Udemy.
We had to claw our way to the top, by using memetic analysis on the customer reviews we were getting. This was the first time I had ever launched a product to a cold audience – every thing else I had pushed to my social media followers or email lists, and it was a rude awakening to get feedback from people who don’t know or care about you.
After an initial bump from our own promotion, we started to get a number of 1 star reviews from people who came in from Udemy search and other promotion. The Udemy marketing team started advertising the course on social, and the negative reviews started bringing our average down. It got as low as 4.0 stars out of 6, which as every Uber driver knows, if you drop from there it might as well be zero.
We did some meta-analysis of the reviews to see what the general problems were, and noticed two main themes. The first issue was quality: our recording style was too casual, the text was too hard to read on the slides, and we didn’t do a good job at editing. The other issue was those that didn’t know how to code were super upset that much of the course required coding. It didn’t matter that even stripping out the coding parts they were getting a huge amount of value for their $15 – people hate paying for things they don’t use.
We re-recorded a lot of the worst lectures, and added fresh content. Every month we did a day of recording each, and have kept that up to this day. We also pivoted the course more formally to developers, removing ChatGPT and Midjourney from the description, and instead talking about GPT-4, Stable Diffusion, and GitHub Copilot more prominently – the words developers or more technical people were more likely to care about.
As we addressed these issues, our average review slowly rose towards 4.5, and we climbed up the rankings, until sales absolutely exploded when we reached the top. Returns are exponential online, and almost anything else you could be doing is worthless if you could do something that gets you into the number one spot on a marketplace or other platform. We toyed with the idea of releasing a non-technical course, but by then the field was getting crowded, and we also weren’t that personally motivated to work on it. So we doubled down on developers, and focused our new content on Langchain and Automatic1111, as well as other more technical topics.
Other tips for launching a course on Udemy
Of course, the success of our launch on Udemy wasn’t all down to memetics. It helped that the content was good, that’s a prerequisit. We were experts in a hot field, when almost nobody else had that experience. The demand for prompt engineering is leveling off somewhat, but you don’t actually need a huge market to make a comfortable lifestyle business if you find the right niche.
Opting into dynamic pricing was huge, because Udemy really does run a lot of experiments with the price. It’s different every time I look at it. The other big move we made was opting into Udemy Business. We don’t make as much from those users, because Udemy charges them less and 80% of them are from India, but they account for 90% of our revenue collectively. As a former large-scale Facebook and Google advertiser, I learned you should always find a way to do the thing the platform wants you to do, because they’ll promote you a lot more.
The other smart thing we did was repurpose the work we were doing each month to research and test different prompt engineering methods. Much of the initial content for the course came from my existing work and blog posts on the topic, as well as the courses we already had on Vexpower. We also had a book deal with O’Reilly in the works to write a book, so we could justify spending an inordinate amount of time playing around with AI, knowing it would be useful not just for the Udemy course, but also when we started writing.
How to use memetic analysis for your business
As you can see, the analysis we did wasn’t particularly complicated. Anyone could do it, but almost nobody does. An expert is just someone who has seen enough examples of success and failure in a domain to know what to watch out for. If you look at more examples than anyone else through memetic analysis, you can gain an edge through brute force, that normally takes decades of hard work to achieve naturally. The process is always the same:
First, expose yourself to more ideas than anyone else, by building a swipefile of samples from the space you want to own. Second, break samples down into their component parts, or memes, by tagging interesting or useful attributes and features. Finally, use memetic analysis to identify patterns that others missed.
Not only will you have data to prioritize what to test, but you’ll build your intuition for when you need to go with your gut. If you enjoyed this post and got value out of it, I hope you share it to spread the meme of memetics to more people.
.jpg)
