Prototype Testing in Mobile Games: CPI Tests and Retention Gates
Few industries validate ideas as ruthlessly as mobile games. Studios routinely test dozens of concepts with real advertising before deciding which one deserves full production. Even if you never make a game, the method is a masterclass in cheap, quantitative validation.
Step 1: the marketability test
The fastest test happens before a playable game exists. A team makes a short video or a very rough prototype showing the core mechanic, runs it as an ad on platforms such as Facebook, Instagram or TikTok in a chosen country, and measures how much it costs to get someone to click and install. This cost per install (CPI) is a proxy for how appealing the idea is at first glance. In the hyper-casual boom of the late 2010s, the leading publishers of simple arcade-style games became known for testing large numbers of prototypes this way and quickly dropping most of them.
Step 2: early retention gates
A cheap install is worthless if players leave after one session. Ideas that pass the CPI test are turned into minimal playable builds and measured on retention: the share of players who return the next day (day-1 retention), after a week (day-7) and so on, along with session length. Only concepts that clear the studio's gates move on to monetisation tests and soft launch in a small number of countries.
| Stage | Question | Main metric |
|---|---|---|
| Marketability | Will people click and install? | CPI, click-through rate |
| Prototype | Do they come back? | Day-1 and day-7 retention, playtime |
| Soft launch | Can it earn more than it costs to acquire players? | Lifetime value versus CPI |
What changed after 2021
Apple's App Tracking Transparency framework, introduced with iOS 14.5 in 2021, made it harder for advertisers to track users across apps and measure campaigns precisely. Studios that depended on cheap, precisely targeted ads had to adapt, and many shifted towards genres with stronger retention and in-app spending. The method survived, but the cost of each test rose, a reminder that a validation channel can change under you.
The social casino lesson
One genre has always had to take testing especially seriously: social casino games, the free-to-play slots, poker and bingo apps where players buy virtual chips that cannot be cashed out. Their revenue tends to be concentrated among a relatively small share of paying players, so acquiring the right players is expensive and every part of the experience, from slot themes and bonus wheels to store offers and the pacing of free coins, is a candidate for testing. Large operators in the genre have described analytics and live operations as central to how they run their games.
For founders, the takeaway is the pairing of two numbers: what a user costs to acquire and what that user is worth over time. If the second is not reliably larger than the first, nothing else matters. The same equation governs real-money sportsbooks and online casinos, where acquisition costs are notoriously high, as we describe in validating iGaming startup ideas. For how slot design itself is tested and tuned, see casino product design as a case study in testing.
Borrowing the method for any product
- Test the pitch before the product. A short video or mock-up ad, aimed at your segment, tells you whether the promise grabs attention.
- Gate on retention, not excitement. Do people come back without being nudged?
- Kill fast. Set gates in advance and drop concepts that miss them, even ones the team loves.
- Compare acquisition cost with value. Early on, estimates are fine; just make sure the gap is plausible.
The broader principle, many small, cheap tests and only a few big bets, is the portfolio approach we discuss in startups as bets.
