A/B testing tactics

When analytics show content-market fit, source more creators from the marketplace or revisit passed-over applicants, do a 5-10 minute call explaining how the campaign is going and why it is scaling, assign them and add them to the group chat. About 90% of the creator cohort posts the winning format; ~10% (usually 1-2 creators) keep testing new formats so a replacement is ready if the winner saturates. Creators on view bonuses put in extra effort once they see the format work.

A/B tested the sign-in button copy 'Sign up with [Apple/Google]' vs 'Continue with [Apple/Google]'.

Posts the same video with different hooks as Instagram Trial Reels, which show only to non-followers. Lets them run a few hours, then promotes the best-performing version to the main feed, and keeps the losing version up in trial reels because it keeps reaching non-followers. Hook drives watch-through. Winning hook opened a question ('Raise VC money. You can make $50 million and still be broke') while the loser answered it ('I teach people how to fundraise, but I wouldn't raise a single dollar today').

Runs creative testing campaigns on Android, where events arrive immediately (cost per install, registration, checkout). Winners go into a single long-lived Android scaling campaign that builds purchase history; new winners are tested against the current champion and switched off if they lose. Only ads that hold up over time are moved into a single iOS SKAN campaign, because SKAN data arrives late and testing there would need ~$1,000/day. Started at $50-100/day, scaled to $150/day once purchases came.

A/B tested a priming screen right before the paywall with the mascot bee: 'We want you to try Sunflower for free. We'll remind you before your trial ends,' then shows the paywall. Addresses trial anxiety. Tested a couple of versions of the screen.

Under 300 views: account problem (shadow-flagged or banned, setup wrong), fix the account. 300-1,000 views: content problem, try new formats and hooks. 1,000-5,000 views: keep the same format and type, change the copy slightly until it crosses 100K views.

Content that doesn't rank organically gets ads instead, so one piece of content serves both. Bids on the same high-intent commercial keywords targeted for SEO; each ad group has its own landing page tailored to the keyword and intent (e.g. an Instagram page for 'Instagram API'). Uses Google Ads automated bidding with a $120 target CPA (their average CAC threshold). Ad copy sells value, not features, e.g. 'scale 30 days of social media in 1 hour' or 'reach 100K followers fast with our API'. Continuously A/B tests headlines, descriptions and landing pages, tracked in PostHog. Playbook version: 3-5 copy variations per ad group; pause poor keywords and double down on high-ROI ones; raise budget month over month while ROAS holds; track CAC, ROAS, revenue and traffic per channel in a dashboard (Google Sheets, Data Studio).

Iterated purchase button copy in A/B tests: 'Subscribe' (baseline) -> 'Start my free trial' (big uplift) -> 'Try for free' (better) -> 'Try for $0.00' (best; believed live). Part of ~400 concurrent A/B tests, many on the purchase flow.

A/B test, half of users each: in-app purchase only vs. web checkout only. Iteration 1: paywall button sent users to the web. Iteration 2: in-app paywall with package selection (monthly/annual, via RevenueCat) and a 'Start Today' button; the tap opens Safari to a stripped-down Stripe page showing only the chosen plan, styled like Apple's in-app purchase sheet, then auto-redirects back into the app. No extra discount for paying on the web, to protect LTV. Next ideas: let users choose web or in-app, and route only high-intent placements (e.g. after the welcome class) to web.

Make many organic creator videos (multiple creators, multiple hook styles), then put the platform minimum behind each one rather than judging by organic views, since good videos sometimes flop on timing or the algorithm. TikTok: ~20 creatives at the $20 minimum each, can switch off after ~$5; Meta: 30-40 creatives at ~$1 each since it is easier to duplicate. Judge within hours on top-of-funnel clicks only: most get 0-2 clicks, one gets 5-20. Keep spending on that one to see whether it was a fluke, then track it down to purchases and make variations (change the first or last second). Mid-virality videos (top 25%) often convert better than the most viral, clickbaity ones. For a $5K starting budget he'd spread it over as many hypotheses as possible, aiming for 100 variations, and compress '500 videos' of learning into a week. Test the same creative across Google, YouTube, Meta and TikTok; one platform can give 2x results.

Instead of asking users at onboarding when they want to be reminded (morning, evening, a set time), schedule the next day's practice reminder for 23.5 hours after the user's first practice of the current day, so it tracks when they actually practiced the day before. Tested against many other scheduling strategies; user-set times lose because people are rarely free at the same time forever.

Never upload untested creatives into the scaled campaign: any change resets the learning phase and causes swings. Run a separate testing campaign that mirrors the scale setup: same optimization event (trial, not install) and one of your live core markets (the cheapest of them), not cheap unrelated markets like Thailand or Mexico where nobody knows the brand. Move only proven winners into the scale campaign. Fewer, more deliberate creative bets than pre-ATT.