Strava · B2C · running and cycling app
Report every test through one shared query and dashboard
Became the source of truth for nearly every experiment
Workedexperimentationanalytics-attribution
What they did
The product analyst turned a SQL query he re-ran for every test into an internal experiment reporting framework ('One Ring') with a Tableau dashboard; nearly every experiment reports its metrics there, run overnight. Teams keep a small set of measured (not modeled) outcomes. More recently they model how moving an output metric (often week-2 retention, e.g. for win-backs) changes WAU over 12 months with decay built in, and use long-term holdout groups to guard against models trained on their own outputs.
What happened
Source of truth for experiments; now so many metrics that overnight jobs are slow.
- Stage
- Scale (scaling)
In their words
Sub Club by RevenueCat · Lessons From Building a 70 Person Growth Team — Jason van der Merwe, Strava“my teammate, Dave, built, um, our internal reporting experiment reporting framework called One Ring.”
“typically, for us, it's week two retention in many cases, what is the impact on WAU over the course of a whole year, um, or whole 12 months?”
“I think things like holdout groups, um, long-term holdouts can help you with, you know, the source of truth.”
Related topics
Get tactics like this every Monday
New tactics from the week's founder interviews, each linked to where it was said.
More from Strava
All 23 Strava tacticsMore from this episode
- Strava: Put words like 'GO' on your icon-only record buttonsOne of the biggest wins ever for first uploads; new-user retention rose a lot
- Strava: Ask new users to add a profile photo during signupLater removed; it added too much friction
- Strava: Count a new user as active once they upload one workout and follow one person in 7 days
- Strava: Get more people to share workout links, then turn link visitors into signupsDrove new registrations, guest says; base of several hundred thousand shares a month
- Strava: Show 'Follow back' on anyone who follows you, wherever their profile appearsBiggest jump in follow rate Strava has ever seen, despite data predicting little gain
- Strava: Once a quarter, let every team member run any test they want on 10% of usersSurfaced ideas data had ruled out, including the biggest follow-rate win
- Strava: Split the big weekly test review into small team meetings once it passes ~70 peopleSmall meetings work great; cross-team learning still unsolved
- Strava: Build one tool for in-app tips, then test removing all tips once teams overuse itOnly growth used it for ~2 years; now so widespread a removal test is running
- Strava: Test which sport and gender appear in welcome-screen photos before signupRunners were put off by cyclist photos; women preferred seeing women
- Strava: Test the words on signup buttons, like 'Continue with' vs 'Sign up with'Copy changes had huge measurable impact; brand photos in product lowered conversion
- Strava: Keep crash-free sessions at 99.99% and ask happy users to rate the appGuest says more five-star ratings help ranking; no numbers
- Strava: Add videos to your store page that show clearly what the app doesVideos really helped, guest says; screenshot tests were hard to trust
- Strava: Test store page designs on a fake App Store web page firstTried; a lot of work, not kept
- Strava: Supply special artwork when Apple offers to feature you in a countryStrava benefits from these features; no numbers
- Strava: Start a growth team on new-user activation, then add teams where tests keep winningGrew from one team to five teams of 70+ people
- Strava: Give staff a switch to use the app as a free user
- Strava: Drive a van to local bike races and upload riders' GPS data for themHow Strava got its first cyclists; no numbers
- Strava: Launch a separate running app to win runners, then merge it with the main appHelped a lot in reaching runners, guest says
- Strava: Get pro athletes onto the app and support local running and cycling groupsMost growth is word of mouth, guest says; no numbers