Rerun old A/B tests that lost once the rest of the app has changed

Patrick Stuart-ConstantRemoving a paywall at onboarding, funny enough, was a big win on Celebs. Generally, it's a good practice to have a paywall at the end of your onboarding. Here, the big win was removing it and actually just getting the users as quickly as possible to the moment of maximal intrigue, basically.

Pablo SánchezFor instance, that's a test that we have tried many times in the flow of evolution of the product. One day, it didn't work. Then, didn't work either. And one day, suddenly, it worked. It is all together is going to be taking part on that. But yeah, like Patrick said, removing the paywall in the onboarding against all the rules of the industry was a big win for us.

Patrick Stuart-ConstantThat was an interesting point from Pablo. It's I think a lot of teams A/B test something once, it doesn't work, and they won't ever A/B test it again. But the truth is, like, an A/B test works or doesn't work usually within a certain context, within a certain flow, etc. And, you know, as we're always testing different things, like parts of the apps are always moving and evolving. So it's often useful to revisit old A/B tests or old assumptions, especially on things which, you know, can have quite a big lift, and retesting them within the new configuration.

Joseph ChoiI see. Yeah, cuz there's so many different variables that you can run with the A/B test. It's it's not just about paywall during onboarding versus pay not no paywall during onboarding. It's like what answers they gave during the onboarding, gender, like there's just so many other A/B tests that you can run in different combinations that it makes sense that you would just you you can't just like run it once, you want to see like which variables …