Newsletter Subject Line Testing
It’s that time of the month (no, not that time)…time to send my monthly newsletter. Iwas reviewing the results of my subject line test from February and found some interesting results. I tested whether including the word “Newsletter” in the subject of the email would have any effect on readership.
I was surprised to find that the email subject with the word “Newsletter” actually got a higher open rate, higher click through rate, and lower unsubscribe rate than the email that didn’t. I used Marketo to send the email using the 50/50 random sample function to over 30k emails which makes the test statistically significant.
Here are the results:
Email A - Subject does not include “Newsletter”
Open Rate: 12.28%
Click Through Rate (CTR): 1.83%
Unsubscribe Rate: 0.57%
Email B - Subject includes “Newsletter”
Open Rate: 13.86% (winner with 99.9% confidence)
Click Through Rate (CTR): 2.53% (winner with 99.9% confidence)
Unsubscribe Rate: 0.43% (winner with 95% confidence)
Conclusion
With all of the emails we get these days it’s nice to know exactly what you’re getting before you open it. Everyone on my list has opted in to receive our communications so clearly spelling out “hey, this is what you asked for” seems to work well. It’s a nice affirmation that keeping things honest with your customers pays off - even if in small ways.
A/B Testing stands for “Always Be Testing”
I’m sure my customers @HubSpot always felt like I was a thorn in their side when I’d ask - “did you test that?” They’d typically respond with “no, I just needed to get it out the door.” Now that I’m responsible for executing on the strategies I set @Codesion I have to eat my own dogfood. I’ve got a ton on my plate this week, but I managed to squeeze in 2 small A/B tests for an email campaign I’m working on.
Here are the results. I used Mixpanel’s handy A/B split testing calculator.
Register Now vs. Sign Up
Register Now - 0.40% conversion rate
Sign Up - 0.56% conversion rate
** Sign Up wins with 95% confidence
Text Email vs. HTML Email
Text Email - 0.34% conversion rate
HTML Email - 0.31% conversion rate
No clear winner yet
This is an interesting one that I’ll try again soon because the assumption is that our audience, typically developers, shy away from “marketing” messages so text emails should perform better. Stay tuned for the next test…
