Duolingo · B2C · language learning app
Let a learning system pick which reminder wording each user gets
Practice reminder copy under test for 6 years; no lift numbers shared
Unknownexperimentationnotifications
What they did
Machine-learning-driven system chooses the notification copy (push and email) for each user and optimizes on its own; PMs just add copy variants to the pool (soon AI may write them). Never treat a notification as 'done': the practice reminder has been under continuous experimentation for 6 years. Judge variants on open and click-through rates, which reach significance fast at scale. Lapsed users deserve the most experimentation because you know least about what they need.
In their words
Sub Club by RevenueCat · How to Time Reactivation Campaigns for Maximum Impact — Jackson Shuttleworth, Duolingo“we have machine learning-driven systems that choose the right notification copy for the right user. They optimize on their own.”
“our practice reminder notification, we've been experimenting on that notification for 6 years.”
Related topics
Get tactics like this every Monday
New tactics from the week's founder interviews, each linked to where it was said.
More from Duolingo
All 61 Duolingo tactics- Cut the free trial from 14 days to 7More conversions overall and experiment results twice as fast
- Sell a one-time ad-free month at $3, $5 or $10 to test demandUptake far above expectations; led to the subscription
- Tell users on the paywall you'll remind them on day 5 of a 7-day trialWon A/B tests by easing fear of subscription traps
- Change the buy button to say 'Try for $0.00'Beat 'Try for free' on both trial starts and trial-to-paid
- Let an algorithm pick from 250 reminder messages, and A/B test adding new onesNotifications called a top retention driver; no numbers shared
- Keep reminders to about three words for newer users; say more to loyal onesThree-word messages 'worked really well'; no numbers shared
More from this episode
- Duolingo: Send one practice reminder for each missed day; stop after day 7Reminder efficacy drops sharply by day 7; users gone past day 3-4 rarely return
- Duolingo: Send tomorrow's reminder 23.5 hours after today's lesson, not at a time users pickBeat every other scheduling strategy tested, including user-chosen times
- Duolingo: Email lapsed users a weekly progress report comparing this week to lastGuest says simple activity stats get most of the value; no numbers
- Duolingo: Send even long-gone users a fun, personal year-in-review of their statsGuest says personal hooks perform much better than product updates; no numbers
- Duolingo: Aim winback messages at back-to-school and New Year's resolution seasonGuest calls these the most important times of year; no numbers
- Duolingo: Ask active users to bring back friends who stopped using the appOne of the best-performing winbacks; beats app-sent messages, even from leaderboard strangers