Practitioner pattern · Thomas Petit (Sells consulting services and runs a paid signal-engineering workshop at App Growth Annual; works with Voyantis)
Compare the value you report with real revenue by country, not just in total
What they did
In their words
Signal Engineering: Strategic Data Filtering for Better Ad Performance — Thomas Petit, Independent Consultant
Thomas Petit… and the amount is one. The amount is very critical, is something because over time, maybe the best engineering you can do today is very different from the one you can do in 6 months, just because your budget has moved from 100k to 500k enables you to do a very different kind of filtering. Also because your cohort has grown, so before, even with the teams that have the most sophisticated model, we keep revisiting the the value that we're giving,
Thomas Petitbecause plans change, because we make monetization change, because the mix of of channel is different as well, and the whole user maybe because there's a crisis, maybe because there's this, like if you're based on an aggregation of 5 years of data, here is also a big trade-off about how recent or how much data do I have? And like sort of, I've got this rule, for example, in in one case, which is, if we've got enough users in the last 2 weeks to make a decent prediction, we use this,
Thomas Petitbut if we don't, then we expand to 3 months, and if we don't, then we're going to loosen up one criteria. A criteria could be a could be a country, could be an onboarding question, and so on, because otherwise, and I made this mistake several times, like, you create useless fluctuation on on very small samples. So, obviously, the more data you have, the easier it gets. It's not a fixed thing, so the advice here, the TL;DR is, revisit your assumption every now and then, every every 6 months or whatever,
Thomas Petitor at least control that gap between what you're sending and what is actually happening is not too big. And the thing about this gap is the trap is to look at total. "Oh, we only have a 2% deviation." I'm like, "Yeah, but there's 10% of people who are responsible for 80% of this deviation." So, this is the one So, don't look at this gap only in total. Do monitor that it doesn't go sideways too much, and country is a big one here, like country and platform. Maybe not every single criteria is needed,
Thomas Petitbut at least monitoring the value of the events that you filter are fairly reliable at country level over time is is kind of a a good idea.
Get tactics like this every Monday
New tactics from the week's founder interviews, each linked to where it was said.
More from this episode
- Check that Meta and Google receive the same event counts as your own analyticsGaps of 30-50% are common; fix anything above 10% before touching ads
- Send trial starts, direct purchases, trial conversions and renewals as separate eventsDefault SDK mapping often lumps them all into one event; guest says fix it first
- Send ad networks only the trials likely to pay, as their own eventGuest's go-to first step when paid trial quality lags; no numbers given
- Tell Meta to value some users more or less instead of excluding themKeeps cheap under-25 reach instead of cutting it; no numbers given
- Report each user's expected value at month 13, not what they paid on day oneGuest's main advanced lever; no numbers given
- Look up each conversion's value from a server so you can change it without a releaseAvoids app releases for every price or currency change; says it makes a big difference
- Send your conversion signal within a day, waiting a few hours to see early useVoyantis showed timing matters; guest says signals after 24 hours are nearly useless
- Keep at least 10 conversion events per campaign per day before filteringUnder ~10/day optimization breaks; well above it, fewer better events win
- Report low-value users as worth less and high-value users as worth morePushes delivery toward top users but hides true ROAS; has backfired when overdone
- Send every event to every ad network, then pick a different signal for each
- Send Google only the trials that look likely to payGoogle trials converted at ~15% vs ~40% elsewhere; the fix moved the needle
- Count a business user's yearly plan as worth 5 weekly consumer plansWorked at first; went stale after 18 months of price changes and needed fixing
- Change which events you send to ad networks, even as a young appBig uplift per the guest; details saved for a blog post
- Put all spend behind one channel and one optimization signalGrew from 0 to eight figures in two years, then had no fallback when it broke
- Let a signal specialist filter and send your ad events, and test itOne of two tests looks good, the other doesn't; not conclusive
- Tell ad networks which users reported high anxiety, since they pay far more
- Treat users who rush a 30-screen onboarding in under a minute as low valueHeld for one app; another app found its fastest finishers were its best users
- Change what you report as user value when cash matters more than long-term LTV
- Sell $5-10 top-up credits on top of subscription tiers
- Send buyers to web checkout once the store fee jumps from 15% to 30%Works for them, per the guest; no numbers
- Report a $1 value for non-payers who look likely to stick around
- Report your biggest spenders as mid-value so Meta doesn't stop early
Tags: signal-engineering, monitoring, model-refresh, sample-size