A couple of months ago, I wrote a blog post about how I believed email marketers don’t split test their subject lines very effectively.

To see if I was right, or if I was high, I ran a survey to discover more.

304 marketers answered the survey, and it seems that the majority of email marketers, according to the statistics, are split testing meth heads.

The kind that are caught in a vicious spiral of doing the same thing over and over while expecting different results… yet remaining extremely hyper in continuing to test the same things, over and over.

In this post, I’ll share with you some of the topline  results of the survey.  (Note: I did not write this while on crystal meth.  I’m always like this.)

1. People think subject lines are really important but don’t test them very much

Hello counter intuition! How’s it going?

Out of all the possible things you can do, the subject line is probably the most important, alongside quality of data and segment selection.  That is, according to our sample of 304 marketers.

And yet, about three-quarters of people either never test out their subject lines, or only test out a few of them!

Here’s the thing.  People spend countless hours and hundreds of thousands of dollars/pounds investing in a database. And have a data team in charge of picking segments. And pay hundreds or thousands having the email creative made and optimised for mobiles and whatnot.

And then spend a couple minutes going:

“Yeah, this subject line is probably OK, whatever.”

Erm…  surely the optimal strategy, considering the vast majority of email marketers think subject lines are more important than, for example, an email design, isn’t it time to spend more time on subject lines?

2.  Email marketers are split testing methheads.

People seem to test out emails for a quick, short-run uplift in response rates… and then forget about it and move on to the next one.

They go for that hit of meth, smoke it, get high from a marginally increased open rate, then go searching for the next hit.

This is a poor strategy, and here’s why: I read somewhere that crystal meth is bad for you. And also, if you’re constantly chasing your next hit (that is, the next minor uplift in subject lines) then you’re caught in a cycle of addiction.

The first uplift felt great.  Oh yeah, it felt great. And then, you do it again, going after the same hit.

But, what astute marketers should be doing instead is foregoing the downward spiral that is methamphetamine usage, and instead focusing on longitudinal gains.

What did you learn from the last split test?  And what can you do in your next email based upon the results?

The problem here is there is a huge, wide space of email subject lines you can consider, and learning from millions of linguistic permutations is difficult. And it’s hard to do with small data sets.

Further, marketers in general lack the tools to learn from such sets of data (for an intro to multivariate learning check out this post.)

3. But there’s no one you can trust

But this is the real problem – there’s no one for marketers to learn from!  Or at least, no one who they trust enough to learn from.

The “Net Promoter Score” for how likely email marketers are to recommend their ESP or email agency’s subject line advice is <-70%.

Yeah, that’s right, about -70%.

If this were a measurement of a brand, someone would be getting fired.

This is a real problem – marketers don’t trust their providers, and as a result don’t ever change anything.

But this is, of course, intuitive – if you ever buy meth from your local chemistry teacher, and ask them if it’s good, what are they going to say?

Then you wind up locked in your bathroom, digging ants out of your skin, wondering how you got there.

And if you google “Email subject line advice” or something like that, there’s very little out there. It’s all generic, Buzzfeedy style stuff like, “41 subject lines sure to get you opens” and so on.

I did that once, and, well, it generates clicks. But does it really help marketers? According to the survey… not really. Ho hum.

This is your email on drugs

I discovered much, much more in the full survey and have produced a report of the results, the link to download it is below.

In the meantime, if you think you have a subject line problem, consider your options: you can continue chasing that elusive uplift that gives you that high… or you can break the cycle, step back, and take stock of the information in front of you.  Learn from it.  Do something with it.  And get more than a quick hit on your response rates.

It’s up to you – but, spoiler alert – meth heads have a short lifespan, especially in email marketing.

PS – you can download the full report here – hope you enjoy it! (Editor’s note: registration required)