Stop Celebrating Conversion Wins Before Checking Renewals – Sara Grana, Yousician

When starting in a revenue role (Babbel, then Yousician), split subscription revenue into four buckets: new subscribers, upgrades between plans, renewals, and win-backs (churned users who return). Track how each bucket evolves over the years in Excel, find the big jumps or drops (e.g. new-subscriber revenue jumping from a given month), then ask what happened and map it to the decision (e.g. launching a lifetime subscription). Link or embed the documentation of each decision in the same spreadsheet and turn it into a short presentation to share. Prefer this over reading every entry of a long experiment log, since many experiments don't matter.

After rolling out a winning experiment, put a recurring 'experiment recheck' in Google Calendar with the data analyst. Prioritize anything touching price or discounts (not layout/copy changes, unless the lift was surprisingly large or top-line growth isn't matching the stacked test wins). Recheck the cohort at three and six months depending on what is sold, or up to a year. Keep the cohort set up so the recheck is just a quick pull.

For paywall tests, look at revenue per user but always also check refunds and chargebacks, waiting a couple of weeks since that effect appears quickly. Pay special attention when introducing a new payment method.

When a user cancels auto-renewal, send them a discount offer to win them back. The headline numbers looked strong, but many users switched off auto-renew right at the start, requested a refund, and then took the offer, making the campaign net negative. Check refunds and chargebacks alongside win-back conversion.

When testing a new plan such as lifetime against a subscription, compare the lifetime price to the expected lifetime value of the subscription (e.g. 300 euros over its life, not the 100 euro first payment), because lifetime front-loads value and otherwise always wins.

Cut every cohort by which plan was bought, discounted or not, and where (web checkout vs in-app), plus marketing channel where it produces different users. Blended renewal rates move mostly because the share of these cohorts changes (e.g. more web buyers or more 1-month plans), not because the product improved. Discounts can bring revenue but low renewals; price increases can shrink the pool of users to upgrade.

When most test users buy a 12-month plan, don't wait a year: when the test result lands, model scenarios of renewal rates (e.g. 10% lower or 5% higher) for each plan, and watch how the 1-month plan renews as a proxy. Set a threshold (e.g. if it drops below 10) that signals the 'winner' is actually the loser.