Test clean, warm and social-proof sign-up screens; break ties with assisted conversions

HostYeah, let's look at the app now.

Yev MarusenkoUh this is more examples of variations testing and kind of like smaller differences versus bigger. Like one is like social proof. So, there's like promising metrics anytime you social proof, but what I'm trying to do is like rather than the test itself where it's three types of registration, one is more cleaner, one is warmer, one is there's like more social proof, tracking all of these metrics, same thing then on paywall side where it's just trying different variations, whether it's comparison, free versus a paid plan, more detail versus less detail. These more like bigger variations, but then within that, it's finding like micro variations.

Yev MarusenkoSo, it's an aspect of social proof where I think the interesting part is certain metrics go up, you have like more engagement, but less purchases in in certain situations. It's like, "Oh, crap. We're like on to something, but the social proof is not in the right place." So, you have to take these learnings and then put them into micro experiments. So, the goal isn't specifically like to do these tests, but is to learn from it as fast as possible …

Yev Marusenko… during onboarding, and then at the end, you have more or less purchases or more or less engagement, but sometimes there's a screen where people don't convert on that screen, but it's like an assisted conversion. So, making sure tracking that, and you find these patterns that you don't see if you're not tracking assisted conversions. So, I don't know like how common it is for others, but making sure that you're setting up your analytics.

Yev MarusenkoThat way, you kind of have like a second opinion or like a tiebreaker. If purchases are neutral in your test, look at assisted conversions. That will help deciding some of these screens helping or not. So, that was the example here where like social proof have mixed findings. Purchases were up, but in certain types of demographics or pathways, but when you look at it as a assisted conversion, then it was like neutral, meaning like there wasn't the negative effect that I thought it was if you're just looking at the final conversion metric across all users, all countries.

Yev MarusenkoBack to like the organic and ad side, when you're running those different experiments from the different creatives, they might lead to different dynamics on like is social proof relevant for them or not. But when you're looking at the assisted conversions, it kind of adds this tiebreaker. That way, you don't like make a decision based on just the final data point. You have to take this first principles, making sure that you're measuring …