Another fact: there is a science to subject lines. And I did the research to prove it.

I looked at about 700m emails, plus billions of algorithmic simulations. And learned a lot. Now you’re about to as well.

Get ready for a load of statistics and analysis that will show you how to create sales-based email subject lines that sell. And a whole truckload of Simpsons references too, seasons 2-11 only.

Hi, I’m Parry Malm. You may remember me from such blog posts as “11 good or bad things that may or may not happen in digital marketing in 2015” and “Why Custom Audience targeting proves that email has won the internet.

Your subject lines set the tone for your emails… much like the chalkboard gags that begin every episode of The Simpsons. The two things have a lot more in common than meets the eye.

Encapsulating an entire joke in the width of a cartoon chalkboard is a comedic challenge. Encapsulating an entire email’s ethos in a subject line is a marketing challenge.

But, with a large enough set of subject line training data there are lots of things you can learn. I applied scientific rigour to subject lines and discovered loads.

And now, lucky reader, get ready for a bunch of stuff I learned from millions of emails.  Enjoy!

Here’s why subject line science is important

Some say email is boring. They’ve clearly never been to one of my parties.

Split testing email subject lines is profitable. That’s fun.

Most marketers want to split test their subject lines. Also, most marketers struggle with knowing what to test (something I’ve blogged about before here.)

Here’s there facts about subject line testing:

  1. It is a powerful way to increase your response rates.
  2. It is how we learn about the world around us.
  3. If anyone says you don’t need to split test: they are categorically and scientifically wrong.

Here’s the challenge – how do you know what to test? For any given subject line, there are hundreds of millions of ways to say the same thing. And, is what you’re testing the right thing to test? Plus, once the results come in, what can you learn from them?

Here’s the methodology of scientific subject line research

Make sure you read this post from start to finish. And here’s why:

You’re going to read about a bunch of statistics, such as open and click through rates. I looked at many other statistics, but want to keep this blog post under a million words. You can download the full results from the link at the end of this post.

The research is based on randomised, anonymised data that email marketers have entered into Phrasee, the email subject line generator.

It’s a sample of about 700m emails, all in the retail and ecommerce sectors, primarily from the USA and the UK, with a few Canucks, Ozzies and Kiwis thrown in for good measure.

It is all very recent data – from the last two months.

Great. You’ve seen this before in one of my previous blogs.

But wait: here’s where the science kicks in. Simply looking at past data is foolhardy. Just because it rained yesterday doesn’t mean it’s going to rain today.

So, we took the existing results, applied some human heuristics, and then ran millions of Markov Chain Monte Carlo simulations.  We then applied a predictive Bayesian inference model to expand the 700m sample to billions of plausible outcomes.


We treated every subject line test like a clinical trial. And predicted future results using advanced machine learning algorithms. This is the most scientific analysis of subject lines on the planet.

Split testing your subject lines is kind of important, eh?

Just because a given phrase has a high response rate doesn’t mean that you will magically become a millionaire. We are by no means saying, “Only use these phrases and forget the others.”  Any phrase can work well in a given context…

So, use this research as a brain storming tool, as an ideas shower, as a thought Jacuzzi.  And other metaphors as you see fit. You still need to test out your subject lines. Over and over and over… and over and over and over again.

If you take away one thing, let it be this: scientific split testing of subject lines is a fundamental skill for good email marketers.

Alright, enough waffle: now on to some of the results.

How great, awesome, or mediocre are various superlatives?

Some words, as our tragic hero Bart learned, are just not a good idea. Imagine if you sent out an email with the subject line:

“Hey stupid, buy this crappy product or I’ll club this baby seal!”

It probably wouldn’t work that well (or it may – for example if your audience loves clubbing baby seals – but I digress…)

One thing I looked at in my research was the use of superlatives.  These modifiers change how people interpret your subject lines. Some phrases affect response rates very positively. Take, for example, “Brand new,” “Latest,” and “Exciting,” which give open rate lifts of 37%, 24% and 19%, respectively.

But some, well, aren’t so great.  For example, “The perfect gift,” which imperfectly depresses open rates by -28%. Or the most mediocre of adequate adjectives, “good,” which reduces open rates by a not-so-good -20%.

Consider which superlative phrases add value to your subject lines.  Some may be “special” (+12%,) but nothing is “wonderful” (-2%) all the time.  The key is to keep testing.

How can you sell in subject lines without being a spamming b*st*rd?

For a long time, people said that you shouldn’t charge admission to your emails. “Don’t talk about sales in subject lines… because spam filters will be mean to you,” they’d say.

They are wrong.  In fact, talking about sales can increase your response rates.

Just imagine if you could offer a “buy one get one free” deal?  This stalwart of sales gets a shocking +89% on open rates.

Opens are great, of course, but we love clicks even more, right?

When people get emails about sales, they are much more likely to click through. Take, for example, “prices” (+246% CTR), “worth” (+134%), and “deal” (+91%.)

The key is to balance your hard-sales wording with not-so-hard-sales phrases.  If you say  “deal on now” or “sale has begun” over and over, your customers will get bored. And will stop responding to it.  Balance your messages over time and test out various phrases to see what resonates.

Can you ask questions in subject lines? Really? Are you sure?

Do questions work in subject lines? What types of questions work the best? Are you still reading this?

We’ve all gotten those emails with questions in the subject line.  They sometimes work, and sometimes just feel a bit odd. The statistics prove this. The open rate lift of a subject line with a question versus the average is 0% (average!)  But the spread ranges from -35% to +49%… so questions work some of the time, all the time.

Different question forms make a big difference.  For example, questions that start with “can’t” (+25%) or “won’t” (+20%) often do better than “will” (-27%) or “who” (-41%.)

When used in context they can work well. Context is key.

So, broken record time: test, and test some more, and so on, and so forth.

Subject lines matter. And that’s important.

Here’s the thing: your subject line can make or break your campaign. If you take away one thing from this blog, let it be this:

There is a science to email subject lines. That’s why we split test them. Treat every email like a clinical trial.

Split testing your subject lines is how you learn about your audience. If anyone tells you any differently, they’re wrong.

You don’t need to be Professor Frink to understand that.

To see the full list of keywords (211 common words were tested in addition to the ones above), you can download the report. (Registration required – Ed)