Sociaaal · B2C · AI-native app studio
Rerun old A/B tests that lost once the rest of the app has changed
Onboarding paywall removal lost several times, then won big
Workedexperimentation
What they did
Don't treat a losing test as permanently settled: results depend on the surrounding flow, which keeps changing. Periodically retest old assumptions, especially high-lift ones, in the current configuration. The Celebs paywall-removal test failed several times before winning.
What happened
Celebs onboarding paywall removal failed on earlier runs, then won.
In their words
The Superwall Podcast · Patrick Stuart-Constant & Pablo Sánchez - $16M AI CEOs: How to Make $1.3M/Month With AI Apps“For instance, that's a test that we have tried many times in the flow of evolution of the product. One day, it didn't work. Then, didn't work either. And one day, suddenly, it worked.”
“So it's often useful to revisit old A/B tests or old assumptions, especially on things which, you know, can have quite a big lift, and retesting them within the new configuration.”
Related topics
Get tactics like this every Monday
New tactics from the week's founder interviews, each linked to where it was said.
More from this episode
- Sociaaal: Buy apps that went viral and are now declining, then grow them againBought 6-year-old Celebs mid-decline; now bigger than ever, ~$300K+/month
- Sociaaal: Run the same style of test ads for apps you might build or buyStrong ad results count as a key data point in buy decisions; no numbers
- Celebs: Drop the paywall after onboarding; show the lookalike result fast and ask laterMore revenue per user; later paywalls converted the users who used to drop
- Sociaaal: Run about 100 A/B tests a month, starting small and scaling the winnersPortfolio ARPU roughly 3x in 18 months, 2x in 12; about 1 in 10 tests wins
- Sociaaal: Write down the reason behind every A/B test so AI agents can learn from it
- Sociaaal: Make about 4,000 AI video ads a month, with people coming up with the ideasProfitable at 3x LTV/CAC; creative team tripled in 6 months to keep up
- Sociaaal: Keep testing brand-new ad ideas instead of endless versions of one winnerCredited with staying market leader in their niches; no numbers
- Sociaaal: Use AI to find TikToks from other industries to inspire your ads
- WiseSam: Ask ad visitors many questions, reflect answers back, then offer a $1 first weekFounders say the long quiz works very well; no numbers yet
- WiseSam: Give new users one-tap task buttons so the AI assistant learns about them fastUsers get more engaged once the agent knows them; still needs more guidance