Ask people why they want Premium, then show plans and content that fit

Ora Levit… members best as fast as we can. We also know that members have different needs. If you're a member who's coming from Italy and you're in the food industry, you have different desires and needs from someone who works in accounting in New York or someone who is a small business owner in Brazil. And so we want to make sure that our experiences are customized and are localized and are relevant for what you do and what you're looking from us to do.

Ora LevitEssentially if you're job seeking, it's very different and you're trying to grow your business, so we don't want to bombard you with things that are irrelevant for you. And so we try to tailor and customize and personalize the experience the best we can for what we think would be valuable for you. We haven't found downsides so far to it. Usually, the downside is from not doing enough personalization. Every time we tailor the experience more, we make it more relevant. Our members find more value in it and so that's an ever-growing effort.

Jacob EitingHow do you even just manage a thousand tests per year because that's like three plus a day? Is there a single air traffic controller that's making sure these aren't conflicting or do you just send it and just you'll hear back if it did something bad between ... I'm talking to cross or maybe you have them all isolated, because you have so many users, maybe you can keep it all isolated. What is that process like as a growth product team?

David BarnardI think you mentioned personalization and I know one of the things you do personalize is which plans you offer to a user. Yeah. I'd love for you to dig more into how that personalization plays out in the product itself, but then also in those premium experiences and the plans you offer and things like that.

Ora LevitYeah, so when you come to our fold saying you're interested in Premium, we ask if you're looking for a job or if you're looking to grow your business or to hire or something else. Based on your answer, we then decide what type of content to show you. Oftentimes, we also know because we saw that you just searched for a job and so you're looking for that job. And so we try to do the best that we can to match what you're looking for by your actions and the input that you provided us to the experience that we show you, so that it's relevant, because ultimately, if it's not, you'll be like, "Okay, you guys totally don't get me and what I'm looking to do here. This is not for me," and you'll be right to do so. And so making sure that it's relevant is a key for us.

David BarnardI would imagine a lot of that's driven by machine learning, but I'd love to hear, because it's such a hot topic these days, but have generative LLMs been useful at all in that or is it primarily driven by more machine learning model, algorithms on the backend?