A/B Testing on push images, deeplinks, In-app messages

A/B Testing is one of the top three features demanded by our 400+ clients.

At Batch, we take this matter very seriously. In the past year alone:

• Our clients have x2 their use of A/B testing;
• 15% of the Batch Trigger Automations were A/B tested by our clients;
• 65% of our retail clients have A/B Tested their mobile campaigns;
• Batch was named one of 42matters’ top 3 A/B Testing SDKs category leaders for 2023.

Good news: our engineering and product teams have developed three significant features for A/B testing. These features will augment your content optimization routines and boost your CRM KPIs, including engagement and conversion rates:

• A/B Testing on push images (app and web);
• A/B Testing on push deeplinks/URLs;
• In-App Messaging A/B Testing.

What are the CRM benefits for you? What are the most effective use cases?

Let's have a read!

A/B Testing on push images

Historically, in the push channel, the fields that can be tested are [title] and [content], and the results of the different A/B test variants are available on the campaign analytics page.

However, these variations are not limited to textual optimisation (turn of phrase, wording, etc.): whether or not adding an image, or to test two different images, can create significant differences between the two variants.

The good news is that you can now test different visuals and compare the engagement and conversion generated by the two variants.

In terms of daily usages, numbers don't lie! Since the feature went live this summer:

• 60+ Batch customers have been experimenting images. This includes Decathlon, Bosch, Sky, Fnac Darty, La Redoute, Galeries Lafayette, TF1, VeryChic, Webedia, ManoMano, Sephora, WetterOnline, Showroomprivé.com, Oscaro.com, iVoox, etc.;
• The feature adoption has x3 in less than two months;
• 25% of A/B Testing experiments are image-based.

All types of customers, all industries, for different use cases (conversion, abandoned cart, onboarding, loyalty, etc.). Still, we have also noticed that many of our customers are using it for web push campaigns.

This is particularly the case in the retail sector, where over 57% of our web push clients are already experimenting with their campaigns.

A/B Testing on deeplinks & URLs

To conduct a thorough analysis of their A/B test experiments, our clients require data on their campaigns' impact beyond engagement KPIs, such as open and click rates.

This requires the integration of multiple tracking parameters to enable meticulous monitoring of the conversion rates for the different variants.

For example, they may A/B test two landing pages - one featuring a simplified form and the other a longer form - to determine which yields more leads or purchases.

With this new feature, they can now propose two different paths redirecting to different screens/URLs that can be tracked via point 1 (e.g. different product categories).

Since the feature went live this summer :

• Its adoption has x4 in less than two months;
• 25% of A/B Testing experiments are done on deeplinks/URLs;
• 60+ Batch customers have been experimenting this way. This includes Albert Heijn, BVG Jelbi, Bosch, Decathlon, Fnac Darty, L’Oréal, La Redoute, Louis Vuitton, ManoMano, Oscaro.com, Rewe, Rakuten, Sephora, or Sky.

A/B Testing on In-app messaging

Due to their contextual relevance and user mindset, In-app messages provide a more creative canvas than push notifications.

Thanks to their ability to be tailored to the user's current activity, they can include interactive elements, rich media, and storytelling. In addition, In-app messages provide design flexibility and personalised content.

When it comes to A/B Testing In-app messaging, many CRM managers need to create and experiment different dimensions: images, texts, CTAs, deeplinks, etc.

With this new A/B Test feature on In-app, our clients widen their room for both experimentation and analysis.

In terms of experimental capacity:

• Content A/B Test: any text fields (header, title, body, button (labels));
• Image A/B Test: different images;
• In-app Theme A/B Test: Full screen, Banner, Modal, Webview, or Images;
• CTA A/B Test: different types of deeplinks, action Dismiss, etc;
• Tracking ID A/B Test: different tracking parameters.

In terms of statistics:

• Click rate per version (click rate on main button identified);
• Total number of impressions per version;
• Total clicks per version;
• Total number of synced devices.
• Displayed information for the A/B Test statistics block: action name and button label.

To keep you informed, you can also access the A/B test information on the Review page.

FAQ

Who can use these A/B Testing features?

CRM Managers in charge of copywriting, content experiments and optimising engagement for automated lifecycle and one-off campaigns.

Plan & Pricing

• A/B Testing features on App Push and Web Push are available with no condition on any plan.
• A/B Testing on In-app is available with no condition.

How to activate theses features?

If the features are available on your plan, no activation is required.