The killer metric for email subject lines that ‘experts’ don’t understand

In marketing, this tendency toward risk-aversion is even more acute. Often, marketers want a high degree of certainty when making any decision… hence our industry’s current obsession with applying KPIs to every single thing that we do.

“If we track every aspect of marketing performance right down to the smallest detail,” the ’experts’ tell you, “then we’ll be able to optimize our future marketing efforts and we’ll all be billionaires.”

Which is true… to a point. But this point is important – and it’s one that most ‘experts’ either fail to mention, or don’t understand (or both).

How consumers interact with marketing is a lot more uncertain than most realize. Trying to control uncertainty is a fool’s errand. In other words – you can’t boil down what works and what doesn’t in marketing to a short list of rules based upon universal truths. You just can’t.

For example: let’s take Anna. On Thursday night, she goes out after work, smashes back eight Jaegerbombs, and wakes up with a cold, coagulated kebab next to her, in a green-faced haze, ruing the impending doom of a crushing hangover. Ping! She get’s an email offering her a wondrous discount on new shoes. All she can think about is not dying. She’s not buying your shoes, dudes. 0% chance.

No marketer in the world could have predicted Anna’s decision. Why? Because her circumstance and all individual circumstances are uncertain and unpredictable. You can’t control what your audience is doing.

Another example: some marketers want to be able to apply some level of certainty to copywriting. This is understandable in principle, but wrought with error in practice. The uncertain, evolving nature of human tastes & preferences always thwarts such rudimentary efforts in the end.

Instead of aspiring for certainty in an uncertain world – a Sisyphean goal, to be sure – there’s a better way.

Here’s the problem with the concept of certainty: any given theory has underlying assumptions. But given the unpredictability of the Anna’s of this world, a list of concrete rules to follow simply doesn’t work.

Instead of spending time on a theory which will still end up being wrong most of the time… consider the scientifically robust approach: embrace uncertainty… and use it to your advantage.

Here’s what we’ve learned about uncertainty in copywriting (based upon actual science)

What some people want is a certain formula – “use phrase X, perhaps instead of phrase Y, and bingo, you’ve got the secret sauce!” And many an ‘expert’ has bilked an unwitting marketer with such fallacious promises.

The formula may work once, perhaps twice. But, over time, your results will decay. And that’s NOT the goal of marketing (unless all you care about is what happens today and you’re willing to forego tomorrow – if that’s the case, stop reading this article now!)

Instead of trying to control the uncertain world around you, the better strategy is to embrace uncertainty by using diverse language in your marketing. Experimenting with diverse combinations of the words, phrases, emojis, sentiments, semantics, and syntactic structures that comprise your brand’s message to the world is the statistically dominant method for long-run success.

The world will change around you faster than you think… so by relying upon something that was certain to have worked yesterday is a poor predictor of tomorrow’s uncertain reality.

It’s simple: the more diverse the marketing language you use, the greater the probability of success becomes.

Language diversity matters – and here’s the evidence

At Phrasee, we have a proprietary database of billions of consumer interactions with email subject lines.

Our data science department dusted off their pocket protectors, took their allergy pills, and got to work testing out this hypothesis: “Linguistic diversity in email subject lines is positively correlated with open rates.” (Note: our data science team is also available for children’s birthday parties.)

They found that split testing using a diverse set of subject line language will, on average, lead to an open rate uptick of 5-10% (Note: there are numerous other factors that affect subject line performance – but we can’t give away everything for free!)

Here’s how we reached this conclusion:

First, we anonymized all the data, and then normalized it, to allow for apples-to-apples comparisons.

Then, we developed a proprietary metric called ‘Diversity Score’. This is a score from 0-100, where 0 indicates very similar language, and 100 indicates very diverse language.

We didn’t just focus on how different are individual words. Instead, we focused on how different the sentiments are, the similarity or dissimilarity of word choices, the structures of the phrases, and the syntax used, not to mention things like inclusion of discounts, emojis and the like… and created a score to account for all these various properties and their intra-diversity.

More diverse language = better response rates.

In nerd terms, “we can confidently reject the null hypothesis.” Or, in actual human terms:

More diverse language means better response rates.

Note 1: to validate the transferability of the results, we isolated several industries across numerous territories. And lo and behold, the same result rang true.

Note 2: We used Open Rate as the measure of success. Some ‘experts’ would argue you should always use clicks, or even conversions. And these same ‘experts’ are wrong. Here’s the analysis of open rates we undertook to uncover this fun fact.

So you wanna write more diverse marketing language. Where do you start?

Few humans are good at writing linguistically diverse marketing messages at scale. With cognitive bias, risk-aversion, and preconceived notions about what will and won’t work, most of us will instinctively lean on language which has performed well in the past, things which sound good to our own ear, or what others have used. Those strategies suffer from short-termism and will not give you long-run value.

In today’s hyper-crowded advertising world, the old ways of working just don’t cut it anymore.

Our data scientists at Phrasee built a Language Diversity Calculator (note: registration required) that measures the linguistic diversity of email subject lines that you’re currently using.

The calculator measures language diversity based on several linguistic properties. It then extracts a small selection of these language properties, scores them, and plots them into a language diversity coverage graph.

What you’ll see is your own brand’s language diversity coverage visualized against the linguistic property combinations other brands in your industry are currently using… and you’ll be given a Diversity Score.

“Wait, what?” you say? It’s pretty simple, actually.

You put in a bunch of your email subject lines. Phrasee then analyses them and plots them against others.

You get a score.

The closer your score is to 100, the higher your statistical likelihood of embracing uncertainty and running a successful email marketing program. The closer your score is to 0, well, the opposite.

Stop trying to control the world around you.

There are some things you can be certain about – for example, death, taxes… and uncertainty.

You live in a world of uncertainty, and so do your customers. So forget trying to control it. Instead, embrace uncertainty… and use it to your advantage.