A/B Test App Screenshots with Apple Search Ads Creative Sets
A/B testing gives you insight into what screenshots convert the best and is an important part of the conversion rate optimization process.
Creative Sets is a feature in Apple Search Ads that allows app marketers to showcase app screenshots and previews in their Search Ads that don’t normally show up. This helps you focus on conversion rate optimization for your Search Ads.
This helps marketers target keywords with different screenshots. For example, if you normally have a screenshot that is placed outside of your first 3, you can now move that screenshot’s order so they show up within your Search Ad.
But not all of us have unlimited budgets to throw at ads. So how can you use this ad product to increase your app’s organic conversion rate without breaking the bank?
You need new screenshots to test with
You may have seen apps that have redundant screenshots, with the first 3 and the last 3 having the same text captions and screens but different designs. This wasn’t a mistake. These apps are actively testing both these screenshot groups against each other with Apple Search Ads Creative Sets to see which one performs best.
You’re not limited to simple redesigning existing screenshots, either. If you have more than 3 screenshots, each with a different value prop or feature, you can run an A/B test for those against your first 3 as well.
Make sure your new Creative Sets are applying new ideas, templates, call to actions, color schemes, and more as you want to make sure they test new hypotheses and are different from your default 3 screenshots.
If you no longer have access to a designer or are dead tired of having to design screenshots yourself, there are a few options:
Design app screenshots on LaunchMatic. It’s as simple as uploading your app screens, choosing different device templates, adding captions, and choosing colors. The platform automatically designs for all supported devices so you don’t have to do it multiple times.
This includes all required iOS and iPad devices: iPhone 11 Pro Max, iPhone 8 Plus, and iPad Pro 3rd Gen, in addition to all relevant Android devices and tablets.
If you want more design control, take advantage of our App Screenshot Sketch Freebie and play around with the template yourself.
Setting up your Search Ad Creative Sets
Apple has provided a great step-by-step process on how to start your first Creative Set campaign.
Your screenshot ordering matters. Creative Sets limit how you showcase your screenshots based on the left to right ordering you’ve arranged them in. For example, you won’t be able to show your last screenshot in front of your first one.
If you have 10 screenshots and want to A/B test your first 3 against a combination of screenshots 3, 7, and 9 then they will display in the left to right order they were uploaded as.
How to select the right keywords to run your A/B on
Creative Sets are run against a specific or group of keywords, so knowing your target keywords and their search volume is important. There are lots of App Analytic tools available that can help you determine which keywords are winners for your app.
Keeping inline with budget limitations I recommend the App Store Optimization tool AppRadar. They have a free plan that lets you see your ranking for 20 keywords and offers a friendly pricing plan with a free trial if you want to see search volume scores for each.
If you’re running out of keyword space I would recommend AppTweak, another leading ASO tool that I’ve been using for nearly 4 years. Their 7 day free trial gives you 400 keywords for all your researching needs and includes search volume scores.
Tip: when searching for relevant keywords make sure to review other apps that appear in the top 10 for that query as this will quickly show how relevant it is
How much ad spend you need to determine a winner
This is influenced by many different factors, including how many other apps are paying for that keyword, how strong your CPA (cost per action) is, and how much search volume the keyword has.
From my experience a $50-75 USD test for each Creative Set you’re running, plus another $50-75 for your default screenshots, is just enough for you to get limited data on how users are interacting with your ad. If you’re running multiple Creative Sets and seeing poor performance for one, be sure to pause that Creative Set so your ad spend is funneled to the other ones.
Apple will also divvy up your spend towards the highest-impact Creative Sets, although they will likely eat up your small budget well before that.
How to determine a winning Creative Set
Once your Creative Sets are running you will see a wealth of user engagement data for each. Finding a winner is not quite as simple as picking the one with the best conversion rate, though.
For example, one Creative Set could have a phenomenally high conversion rate from a Tap to Install, but performs poorly when it comes to people initially tapping on the ad. Instead, performance is is best represented by your CPA (Cost Per Action).
I’d recommend going with the Creative Set with the lowest CPA as it essentially means that that Creative Set’s engagement will acquire more installs for you than the rest.
Here’s a quick glossary of the Apple Search Ad engagement data terms:
- Spend - How much you’ve spent on this particular Creative Set
- Avg CPA - The “Average Cost Per Action”, “Action” being an app install. Other networks call this a CPI (Cost Per Install)
- Avg CPT - The “Average Cost Per Tap”, “Tap” being a user tapping on your Creative Set. Other networks call this a CPC (Cost Per Click)
- Impressions - How many people have seen this Creative Set Ad
- Taps - The total amount of people who have tapped on this Creative Set
- Installs - The total amount of installs this Creative Set has acquired
- TTR - Your “Tap Through Ratio”, which is calculated by dividing your Impressions by Taps. Typically called CTA (Click Through Ratio)
- CR - Your Creative Set’s “Conversion Rate” from a Tap to an Install
Pushing your winning Creative Set live
Now that you’ve identified which Set performs best, the way to apply it to your store listing is to push a new build and add/edit your screenshots.
Once approved by your Apple rep (finger’s crossed they’re having a good day) your better performing screenshots are now live! Use AppStore Connect’s powerful analytic tools to calculate your new conversion rate, but make sure you have at least 1-2 weeks of data before further optimizing. StoreMaven has a good article on how to use AppStore Connect’s analytic tools to measure your conversion rates across all sources.
