[36] Holiday Planning: 5 Pre-Email Holiday Tests You Should Be Running – Email Growth Society

[36] Holiday Planning: 5 Pre-Email Holiday Tests You Should Be Running Now

CWEG 36 | Pre Email Holiday Tests


With less than 35 days until Black Friday, it is important to KNOW what works ahead of sending out that email. The only way to know is to test. This episode is dedicated to providing the top 5 pre-holiday email marketing tests that you can be doing with the 2 weeks you have left ahead of the big rush.

One thing to note when listening to this episode: it’s important to execute true A/B whenever possible; introducing more than one variable makes it difficult to ascertain the source of a lift, muddying how actionable your insights will be. Since the options for testing are virtually endless, don’t feel like you have to do it all. Examine your existing program to identify low-performing campaigns that could use some attention, or choose to run tests in areas where you feel your decisions are being made by your gut rather than backed by data.

And don't forget about statistical significance. Tests must be at least 95% statistically significant in order to be deemed winners. You can leverage a statistical significance calculator like this one to measure. Just make sure your metrics are placed correctly. In the section that says "The number of visitors on this page was" should be your total sent. And in the section that says "The number of overall conversions was" should be the KPI you are measuring (i.e. unique clicks or opens).

Have questions? Email them to me at conversations@emailgrowthsociety.com.


Listen to the podcast here:

Holiday Planning: 5 Pre-Email Holiday Tests You Should Be Running Now

We have been so preoccupied with COVID that we totally forgot the common cold exists. I hope that your holiday planning is going well. As most of you have been in the show for a while, testing is an important part of email marketing and in any digital marketing. Testing allows us to know, be certain that we implement a winner in key moments, we will see success. Instead of guessing, we are going to know.

Remember, you have to test smart. I cannot tell you the number of times I asked new clients if they are testing and they say yes. I asked them what they have tested and learned, and they have no answer. That is because they quickly whipped up two versions of the subject line or are testing two images. They have not thought about what they can learn from making those changes.

Therefore, they have nothing to apply as it relates to knowing what a winner is or in the worst case ever, they have put too many variables in place. They have changed the subject line, the image, the call to action button, the pre-header text and therefore have no idea what influenced the lift. Testing does take some work. Those that have tested and learned ahead are going to outperform those that did not.

VIP Treatment

Good news for you, there is still time. You have a few weeks to get some tests in and I recommend that you do. Spend your time wisely, prioritizing a few of these ideas. That way, you know what winning elements to use and you can put your best foot forward. To continue our pre-planning, let’s talk about five pre-holiday tests that will serve you well if you get them going now. The first one is VIP treatment. I tend to always start with our very best customers. Why? They need to be treated the best like the VIPs they are.

CWEG 36 | Pre Email Holiday Tests


The good news, you don’t have to have a sophisticated VIP reward program to build up and capitalize on brand loyalty during the holidays. Create simple segmentation within your ESP to identify cohorts of existing and potential VIPs based on numbers of purchases, purchase frequency or recent total spend depending on the right indicators of loyalty for your brand. Once the audiences are created, test into the most optimal opportunities to give them special treatment to inform your holiday approach.

Some of the testing treatments you might deploy ahead of the holidays are going to be VIP messaging. This is testing tweaks to your existing creative that speaks to your VIP audience including call-outs in the subject line, banners or body copy. You can try VIP from the name. How about trying it custom from a name for your emails that denotes VIP status to determine lift on open rate or purchase intent? An example is VIPs only a brand name. What about early access to sales or launches? Determine if your VIP audience converts better when they get access to promotions or new products earlier than the rest of your database.

Promo Types

This is an easy one to do even for Halloween if you can get it going quick enough, a small discount to see. What about additional discounts? Try adding VIP-only deals to determine if a lift conversion outweighs the impact of your margins. This is a way you can test out your VIPs to see if there’s any there. That’s a big one for the holiday season. The next one is promo type. In email marketing, there are no take-backs or do-over, so nailing your cyber week offers can be a little tricky.

While running through competitive analysis, we will help you gauge how you offer stack-up against similar brands, it won’t necessarily tell you what types of promotions resonate with your audience. I know in our previous planning series, we talked about looking back at our holiday past promotions and our promotions over the years to see if we can plot any trends. Those might be the things you test against here. Get a read on the best motivators for your database by testing various offers to see what converts best, and then apply those learnings to your holiday promotional strategy.

You don't have to have sophisticated VIP reward program to build up and capitalize on brand loyalty.

Remember, you still want to reserve your steepest discount or hottest deals for Black Friday or Cyber Monday. Take a more conservative approach during the testing phase. The goal is to test out formulas. Here are some ideas. You can test dollar-off versus percentage-off. Those are going to be a very big difference in how we come across your audiences more interested in getting a dollar amount off versus percentage off. You can test shipping offers. Test free standard shipping or free flat rate upgrades on faster shipping methods. What about my favorite tiered offers?

Email Capture Site

Test a buy more save more approach or perhaps the first 50 people get X amount of percent, the next 100 people get this next percent, so on and so forth. Lastly, gift with purchase. Test a free gift on its own or in tandem with a discount. Try gifts with differing values to see what kind generates the most excitement amongst your audiences. This is another important one you all want to nail down.

Next, let’s go back to our pop-up box and do some testing around our email capture. Side traffic increases naturally over the holidays, so dialing in your email capture methods will allow you to significantly grow your list for re-targeting later during the busy shopping season and beyond. Black Friday and Cyber Monday are going to bring a bunch of folks but we still have December. The more we can capture, the better. Go ahead and try some of these tests during Cyber Week.

Trigger timing. Test triggering quickly versus after 30 or 45 seconds onsite. Maybe in your first test, the pop-up box pops away and then at the second test, we are waiting for 30, 45 or 50 seconds. You can test popping it up right away versus scrolling 50% or 60% on the way down. You can test creative type, try standard box versus a full-screen overlay versus a spin-to-win gamified capture method. You can test the incentive to test both the type and depth of discount to find the sweet spot.

CWEG 36 | Pre Email Holiday Tests


Triggered Messages

Remember that two tips of the discount will bring in low intent leads looking for a steal or freebie, so proceed with caution on this one. What about exit-intent? Try triggering a pop-up when a user intends to leave the site to prevent customers from abandoning without providing a way to keep in touch. You will want to focus on doing some testing on your email capture for sure. Testing type number four is your trigger messages. Similar to email capture, your triggers will see more action in increased volume during the holidays.

Particularly, your welcome series, cart and browse abandon messages, and any post-purchase emails. Take some time to test your automation as an even small lift on a core KPI. This can make a huge impact on revenue. Some things you can try are the timing test, the timing between the user’s behavior and the deployment of the first message, as well as the duration of time between the follow-up messages. You can test the number of touchpoints. If you do not see a huge drop-off in an engagement at the end of an existing series, consider adding in additional touch to drive more revenue.

Subject Lines

This is especially true if your triggers are not currently built out as a series. The order, review KPIs to determine, which message in the series is most effective at converting subscribers and move it up to capture revenue earlier on. You can test incentives. If you offer incentives in your triggers, test a variety of promos, and you can go back to the promo list type I have provided or the level of the discount to strike a balance between conversions and margin. Lastly, subject lines. It might sound like A/B testing 101 but changes to the subject line have been linked to a huge impact, not only on opens but purchase intent.

Test personalization, the inclusion of incentives, emojis, urgency statements, and sell-out risk messages will elevate the performance of your automated campaigns. If you are not focusing on your automated flows, you are missing out. Make sure you visit those. Lastly, trying to understand what drives AOV, which is your Average Order Value, is huge ahead of the holidays. In the peak of the discounting season, it can be tough to lift the AOV of your emails but this is the season for both gifting and self-gifting, one of them is going to be where your recipients are going to fit.

Keep a close eye on the relationship between conversion rate and average order value, ensuring that you're not losing out on volume in favor of lifting basket size.

Opportunities increase the number of products each shopper is ordering. There are more opportunities for AOV here. Keep a close eye on the relationship between conversion rate and average order value, ensuring that you are not losing out on volume in favor of lifting basket size. A few thoughts here, you can test the number of products. Test how many products are featured in a given message. Sometimes, less is more when it comes to driving clicks, especially with people that have not purchased from you before. You can test dynamic recommendations.

Add recommended products based on a customer’s past behavior, and test various display roles such as recently browsed, frequently purchased together, cross-sell. You can also test the threshold on incentives. Offer promotional pricing or free shipping only once a certain purchase threshold has been met or multiple products have been added to the cart. You will want to start to understand that average order value as you approach the holidays, so you are not thinking that metric. It’s an important one for eCommerce clients.

One more thought before we wrap up, it’s important to execute through A/B splits whenever possible. Introducing more than one variable makes it difficult to understand what one and what the source of the lift is, muddling how actionable your insights will be. Since the options for testing are virtually endless, don’t feel like you have to do them all.

I did list a few that can be done at the same time because we are running out of time, so you can be testing your pop-up box and your automation while trying to also see through email campaigns how you can be testing promos and AOV as well. Try to pick the five that can give you the biggest thing for your buck, knowing that we probably need to get this done in time. With that being said, feel free to send your questions to me at Conversations@EmailGrowthSociety.com. Until next time, happy holiday planning, everyone.

Important Links: