Write On
Posts
The right way to A/B test

The right way to A/B test

Boost your open rates with better A/B testing.

Joe Portsmouth
June 21, 2023

Welcome to the 22nd issue of Write On!

The newsletter that forgot to include a poll 2 out of the last 3 weeks…

Inner monologue:

Estimated read time: 2 minutes 53 seconds

How would you start this presentation?

This Friday, I’ll be giving a 1-hour presentation on copywriting frameworks to members of Ship 30 for 30’s Captain's Table.

Dickie Bush and Nicolas Cole are paying me to give the presentation.

And the members I’ll be presenting to have paid thousands of dollars to be in this VIP program.

In other words… I better bring my A-game.

If you were in my position, how would you start this presentation?

Would you…

lay out the agenda for the call?
dive into a copywriting framework?
discuss the importance of frameworks?

No.

You’re smarter than that.

You’d start off by telling the story of who you are and why the hell you’re on the screen before them.

And that’s what I’ll do.

Because I need to sell them on me before they trust anything that comes out of my mouth.

180+ Write On readers are on the waitlist to become a better storyteller so they can sell more online. Consider joining them because you’ll get:

Early access to the course launch
Sneak peeks into the course
Exclusive pricing

Just click the button below 👇

Top Finds This Week:

📖 Storytelling: 5 storytelling frameworks to help you sell more online (even if you don’t know how or where to start): (link)

🧠 Psychology: 46 Psychological Marketing Examples for Smarter Marketing (link)

🖼 Framework: This company went from $5m to $100m in 18 months. Their secret was jaw-dropping entertaining ads. Steal the formula here: (link)

📜 Principles: 17 tips for great copywriting (with simple visuals highlighting good vs. bad examples): (link)

⚙️ Resource: 10 sources of copywriting and marketing inspiration to add to your bookmarks: (link)

🤖 AI: 3 in-depth ChatGPT prompts that'll create killer copy for you in seconds: (link)

✍️ ICYMI: I’ve shared 115+ links in this section of the newsletter over the last 5 months. These 10 links had the highest click rates: (link)

Always Be Testing

Sorry team. But you know nothing.

And I’m right there with ya:

Copywriting will humble you real quick.
75% of the time I run an A/B test, I feel good about which copy will win.
But I'm wrong more often than not.
Lesson: Assume you know nothing.
Let the data dictate what's right.
— Joe Portsmouth 🧲 (@joe_portsmouth)
6:55 PM • Jun 17, 2023

That’s why I run tests on every piece of copy that I write.

How I run A/B tests for this newsletter:

If you’ve been signed up for this newsletter for a few weeks, I’m sure you’ve realized that some newsletters arrive at 8:30am EST and some at 11:30am EST.

That’s because of A/B testing.

You see, I want to put my best foot forward every week.

So now that our community is just over 12,000 readers, I have a decent sample size to work with.

Each Wednesday at 8:30am EST, the A/B test goes out to a sample of about 4,000-5,000 of you.

Depending on initial open rates, beehiiv will automatically send the winning subject line variation to the rest of the list 3 hours later.

This has helped me raise my open rates from ~40% to ~45% over the past few months.

That’s 600 extra opens per week — just from A/B testing.

That’s meaningful.

Here’re some learnings I’ve uncovered 👇

Top 3 takeaways:

Make it clear
Make it interesting
Make it about the reader

You can read the full breakdown in more detail here:

Stop guessing. Start testing.
I run subject line A/B tests every week for my newsletter of ~12,000 readers.
Here're 10 recent test learnings you can apply to your subject lines:
— Joe Portsmouth 🧲 (@joe_portsmouth)
12:55 PM • Jun 18, 2023

A few words of caution:

A subject line test does not always tell the full story.

You must pay attention to a test's impact on secondary metrics.

e.g. I bet this subject line would win any A/B test that I run: “Your credit card has been stolen”

BUT, it would lead to higher unsubscribe rates and a lack of trust.

It’s a false positive.

The same idea can be applied to emails sent with the intention of generating sales.

More opens doesn’t always equal more sales.

So it’s important to calculate revenue per send when analyzing subject line tests for sales emails.

Because that’s your true north star — maximizing the revenue you get for every email sent.

💡 Bottom line: A/B testing is an effective way to boost your open rates, but make sure you pay attention to other metrics that could be impacted.

That’s all for this week. Thanks for reading.

See you next Wednesday!

Joe

P.S. Whenever you’re ready, here’re two ways I can help you:

Facts tell. Stories Sell. Use storytelling in your copywriting to grow your business. Get early access to my new course: The Storywriting Playbook (👉Add me to the waitlist!)
Keep your writing sharp all year round and enjoy free lifetime access to my collection of Write On copywriting guides (👉View all guides)