Life360 · B2C · family location sharing app
Test new ideas with a fake button shown to about 100 people
Unknownexperimentationvalidation
What they did
Size tests by absolute N, not percent of base: at ~100M users, 1% is ~1M people. For new or costly ideas, run a painted-door test (button that fails gracefully) on ~100 (even 10) users and accept ~70% certainty, then run the real test. Low-risk copy changes (headers, email headlines) can go to everyone to pick up 0.1% effects. Percent steps (1, 5, 10, 50%) are for rollouts, not tests.
- Stage
- scale
In their words
Sub Club by RevenueCat · Freemium Done Right: Lessons From a Multi-Billion-Dollar App — Chris Hulls, Life360“if you want to do like a painted door test, painted door for people that don't know, it's like it's basically fake. You click it and it breaks, or you've made some graceful failure, you should be able to do that on 100 people or 10 people.”
“But I'm kind of okay in a painted door that you get like 70% certainty.”
“like if you're doing a roll-out, we do do like 1, 5, 10, 50, cuz that's more server traffic and...”
Related topics
Get tactics like this every Monday
New tactics from the week's founder interviews, each linked to where it was said.
More from Life360
All 23 Life360 tactics- Keep 2 days of free location history even though cutting it doubled subscribersCutting to 6-12 hours doubled subs from that hook; not shipped to protect free users
- Show the top-price plan on the paywall only to people a model predicts will buy itDoubled new Platinum subscribers without losing a single Gold subscriber
- Announce a pet feature with a cute animated dog that asks to be pettedCalled a huge success; many users made pet profiles; no numbers
- Sell trackers at low margins to pull people into the app and keep them longerTile owners retain better than non-owners; Pet GPS requires a subscription
- Add in-app share buttons and gifted subscriptions to copy parents' word of mouthFailed across the board; parents don't share inside apps
- Offer a free Tile tracker to people who subscribeHelps conversion for certain users; no numbers
More from this episode
- Life360: Give daily location sharing away free and charge for safety featuresGrew to 80M+ active users and ~$400M revenue; about 1 in 8 families pay
- Life360: Don't charge anything until you reach 10 million active usersFounder says charging early would have killed them; no direct numbers
- Life360: Write down which free features can never be taken awayToo new to judge; founder says ask again in a year
- Life360: Show banner ads to free users and check harm with a holdout groupHoldout showed no harm; banners drive little performance and CEO wants them gone
- Life360: Offer a one-tap Uber ride when a user lands at an airportFounder says it adds value to users; no numbers
- Life360: Sell only grouped location data, not raw GPS feeds, to a data partnerAbout $20M+ a year from Placer.ai; still some bad press
- Life360: Match good drivers with cheaper car insurance using their driving dataPlanned; could double revenue at 1/8 of paid-user performance
- Life360: Block ads for young users and let staff report bad ads in SlackGuest says it's been okay; a few bad ads still slip in
- Life360: Chase every possible battery or location bug, even if most are false alarms80-90% of chases are false alarms; old battery bugs still hurt trust