Run about 100 A/B tests a month, starting small and scaling the winners

Joseph Choi… I sort of assumed automatically that you see retention as the long-term goal for some of these things, but it's not even about subscription retention. Sometimes it's just like you know that you see that that the market is bigger than than what other people think. So like for this one, it seems like the the lever that you pulled is like you acquire and then you figure out that you could just run ads better. Like was that the the lever for this?

Patrick Stuart-ConstantThat's part of it. It's, you know, it's sort of this virtuous cycle where we're very good at user acquisition, so we get a lot of users into the app. Pablo translates that into, you know, dozens of A/B tests a month for each app. Continuously lifts the uh average revenue per users. Like, you know, over the last 18 months, Pablo's probably 3x'd uh the ARPU on average on our apps, and uh certainly 2x'd it over the last 12 months.

Patrick Stuart-ConstantThere's this virtuous circle where we we're good at user acquisition, bringing more users, we monetize those users better, we bring in more users, etc. And so, you know, it's it's by playing on both parts of the LTV/CAC, of the LTV/CAC that we manage to yeah, continuously scale these over time.

Joseph ChoiIt seems like you mostly do weekly subscriptions on Celebs for like anywhere between $6.99 to like $14.99. You're you're basically just like running A/B tests on absolutely everything, like the onboarding, the pricing, all these data points. In general, are there certain data points that that have been highest-leverage for for increasing revenue per user?

Pablo SánchezI would say onboarding is an element very important. You know, all the apps have a funnel that is decreasing. The upper the interaction, normally the bigger the impact. But that's just one of the general rules. Depending on the phase you are, maybe you want to do things that really have big impact, like in the first screen. If you already have a 1 million users per month, we get more sophisticated and we try things all around the app, and we unlock value in every corner.

Pablo SánchezBut generally, adding or removing our onboarding screen could be something that makes a huge impact. But I don't have the key because, even in our apps, the decision that I take in Celebs or Litstick can be completely different. I just know that there is a lot of value there. But it will be testing. We run probably 20 A/B tests. 10 of them are neutral, 5 are terrible, 3 seem to be giving us something interesting, 2 are good. So we have a system of scaling A/B test: we run it at small scale, then bigger, and that way we are always adapting.

Joseph ChoiFor Celebs, could you give me one example of a pretty big test with that you ran like millions of users that had a big lift?