From part 1, we’ve got our hypothesis, maybe that’s:
Will improve our conversion. Now we need to translate that into assets that can be tested (which we’ll talk more about in part 3!).
Today we’re chatting to Tim Jones, founder of LaunchMatic. He also worked for Keepsafe Software, creators of the popular iOS & Android app Photo Vault. At Keepsafe, Tim was put in charge of managing screenshot testing and localisation for Photo Vault.
That didn’t just mean making a few screenshots, a few variants and then putting them live. He was responsible for preparing 8 screenshots for:
1000 screenshots in total, and that’s before testing any variants!
Now he’s building LaunchMatic, an app that allows anyone to design App Store and Google Play screenshots without a designer.
First things first, as a follow up to our hypothesis post, do you have any thoughts about where you can get good ideas for tests from?
The single best method for finding screenshot design inspiration is from the App Store itself. While it’s good to know what your competitors are doing, I find it’s better to browse apps and categories unrelated to yours as it can help identify design practices and ideas that can set you apart from similar apps.
In our case at Keepsafe, we found that many of our competitors would adopt our screenshot designs (sometimes almost a direct 1-to-1 copy), so we had to think outside of the box to further distance ourselves from these copycats.
A hypothesis is one thing, but where do you go to find the look and feel you want for your screenshots, and do you ever try to validate a hypothesis before creating the test?
I will look at the app screens and features we want to promote before mocking up screenshot designs on my notebook. Keepsafe has a rich index of brand assets we can utilize in the screenshots, so these can help set the foundation before we dive into design.
Since Photo Vault has a sizeable paid install spend, any one change can have adverse impact on installs. We’ve been A/B testing these creative changes with Apple Search Ad’s Creative Sets, which allow you to run a paid campaign against new screenshots you’ve uploaded. While we are happy with results, it’s a tedious option as we need to delete some of our current screenshots to make room for the test creatives, while also costing us quite a large amount of money to run new paid campaigns.
What elements would you start testing with, and what process would you follow?
Your app icon is the first asset users look at when browsing, so it’s important you try to get it right. It’s also the easiest thing to test as there is only one image vs tens (or hundreds) of screenshots.
But your app icon should be static as it is the best (and only) representation of your brand to your users. Your app screenshots will forever be evolving as you build upon and grow your app, so it’s just as important to get the process right.
I focus on my screenshot layout first. I put myself in a user’s shoes and ask myself:
Typically it’s in how you showcase your app screens that will impact these questions. Your layout should be dictated by the information your app screens are showing.
How would you approach testing on iOS vs Google Play? Do your learnings carry over from one to another?
My advice is to focus on one platform first, but don’t expect the same outcome on both. Users may make the same searches between platforms but the way each platform showcases your app is different and will impact how users view and engage with your screenshots.
Apple showcases your screenshots directly from Apple Search, while Google Play only shows screenshots after the user taps on your app (unless it’s a brand name search). This difference may seem trivial, but can lead to very different user experiences between platforms.
So to summarize, focus your efforts on one platform before implementing your winning creatives on the other but be prepared for additional A/B tests if those creatives don’t take.
Would you run multiple icon tests one after the other, or run through the major test areas one after the other?
This will depend on your budget and organic traction. At Keepsafe we ran multiple app icon tests via Google Play’s dashboard, but found that it took 2-4 weeks to get definitive results. That could mean that, if an app icon has a lower conversion rate, it’s going to negatively impact your install growth over that month-long experiment.
What type of signals do you typically look for in your tests?
I primarily look at conversion rate from an Impression to Install to gauge performance.
Another issue is that unless you’re designing the assets yourself, you’re going to have to work closely with a designer to achieve this, and this may present some challenges:
a) They may not have the same vision of the A/B test that you have
b) They may not be experienced in creative optimisation for screenshots
So, Tim - how can creative optimisers work productively on screenshot designs with designers?
Come with visual examples of what screenshot designs you want. That can be a link to another app, paper sketches, an in-depth writeup, or all three.
Ask for them to create the layout first. Layout primarily consists of how & where your app screenshots and text captions will be placed. This is a great point to provide feedback as the time spent has been fairly low and edits to the layout post-design can be time-consuming and expensive.
How do you share your hypothesis with a designer so that both parties are satisfied with the outcome?
I always give my designers a deeper look into why we’re testing our screenshots, as it can help them formulate their own opinions on what we need to do. If you can, show them the conversion rate data you have and explain how you will be measuring its performance. Take time to review the post-experiment data with them so you can brainstorm together why your hypothesis did or didn’t take.
In your opinion, what are ‘well-designed’ screenshots?
If I can look at your screenshots and understand what your app does within 3 seconds, then I will consider them well designed! Not everyone needs to be flashy and (from my experience) sometimes simple is better.
In the modern app environment, the process of asset creation is getting more and more burdensome with developers having to optimise for: 5.5’, 6.5’, 12.9’ assets on iOS, and a full set of assets on Android too. How do you recommend navigating this complexity to achieve the best results?
It doesn’t matter if you’re an indie developer or an entrenched app company, screenshot designs will take a lot of resources and thinking to do. And that’s before you factor in the 40+ languages you can create screenshots for.
Running a test simultaneously between platforms can be overwhelming. Instead, I focus my time testing on a single platform before moving optimizations elsewhere.
For iOS apps, your most important screenshots will be your first 3 as they will be seen by users in Apple Search, even before they tap on your app listing. These first 3 screenshots are what you should spend most of your time thinking about.
The same applies for Android apps. Data shows that a majority of users do not view or engage with your 3rd & 4th screenshot. Your efforts should be focused on trying to explain and convert a user from your first 3 screenshots, which will help minimize the amount of creatives you need to design.
This is exactly why I started LaunchMatic: to automate the design process around the multiple platforms, devices, and languages. We’ve removed the redundancy around having to design your app screenshots for each device so you can focus on creating the best screenshots for your app. You can test our Launchmatic here.
