When an ad wins, change one thing at a time to learn why before making more

Alper Taner… output in a way, like good or bad, it's all about the learning. Okay, this creative for whatever reason performed better and other creative performed for whatever reason worse. Let's understand why. What is the hook rate? How is the thumbnail? What is the audio? What is the background and this and that. So, once we understand and then we start changing one variable at a time, then it becomes a lot more clearer on how we can engineer the success.

Alper TanerThat's the whole point of, that's how you create winners from the winners because you have a winning concept and thanks to the interpretation of the output, you're able to create more winners because you look at the relative metrics, that's a guidance for what you should be testing and what you should be not testing anymore because whatever you do in that particular case doesn't work. However, I have seen from the same concept, like the third variation worked, but the first and second failed. It's the same concept. It's just slightly different messaging and such. So, that's why you would always test with few variations just to avoid such cases where you don't just jump to conclusion of a whole concept and so on. I think that's also important. So, there could be, of course, false positives and so on like nothing is perfect, but also, I'm not also saying like, "Oh, you should trust algorithm blindly and stuff like that."

Alper TanerWhat I'm just saying is we should be able to interpret the results and then drive conclusions to shape the next creative iterations based on that. So, that's when you need to interpret the output. And then what matters at the end of the day is your success rate and not the amount of creatives because I think LinkedIn nowadays is full of, "Oh, you know what? We just tested 500 creatives." "Oh, no, we tested 700. How much did you test?" Kind of …

David BarnardHow do you think about forming a hypothesis around that? Because it seems like that's really the important step, is not being overly confident in why you think something won, but instead just allowing it to help you form a hypothesis for other things to test based on a multitude of reasons why that creative could have won.

Alper TanerYeah. You should never just produce 200 variations of just from a one signal. I think it's always in phases. So, let's take the dog example. Okay, let's have the same dog, let's have a puppy, let's have another breed, let's have another animal. So, just to test and see if it's the animal or not. So, what we want to understand is what makes the difference? Is it the dog? Is it an animal? Is it something else? And once we figured it out, then we would slowly roll out the production and so on. So, it's never about, "Oh, the dog won. Now, we need 200 radiations of the dog." It's more about, "Okay, is it a dog?" We want to validate that. So, in every step, we want to do that and we want to do in both directions. That's why we have both iterative concepts and the radical concepts.

Alper TanerIterative would be, "Oh, let's put another breed, let's put two dogs maybe, and it's even more effective." But in the radical one, you can have completely different hypotheses than what doesn't work based on what works today, but also based on other sources. So, when it comes to what creative to test, I think you should utilize both the first party data and the third party data. It's not always about the first party data. So, first party data, …