Flo · B2C · period and ovulation tracker

Remove a step that just won to check the win was real

Removing the 23%-winning step lost, confirming the uplift

Workedexperimentationonboarding

What they did

Common practice at Flo: after a win, run a blackout test that removes the new element. For the perimenopause phase step (a 23% win that seemed too good to be true), a fourth iteration removed the new section; it lost, proving the uplift. Blackouts are also used to test removing existing onboarding steps, since funnels tend to keep accumulating steps.

What happened

The blackout variant lost, confirming the perimenopause step's uplift.

Stage: scale

In their words

Sub Club by RevenueCat · What 80M users taught Flo about A/B testing — Mike Charniatsou

Play from 32:14

32:14

“So it was too good to be true. We did the fourth iteration. It was like a blackout test. We removed that new section, and the test lost, which proved that there was an actual uplift, you know.”

Mike Charniatsou

33:21

“it's a pretty common practice at Flo to do blackout tests, like for multiple reasons, you know. One of them is to validate that there was an actual uplift. And the next one is, you know, sometimes you just keep adding things to your onboarding funnels, and sometimes a good thing to do is to remove a step, you know.”

Mike Charniatsou

Get tactics like this every Monday

New tactics from the week's founder interviews, each linked to where it was said.

More from Flo

All 21 Flo tactics