Heather is a strategic advisor to women-led startups including The Luupe, a platform for creators; Pluralytics, an AI-powered platform that customizes language to audience; and GOLD, an online comedy platform. She is also the president of the Harvard Business School Women’s Association.
Prior to founding Spark, Heather led corporate strategy and development for Scholastic, a leader in K-12 educational technology and children’s book publishing. Before joining Scholastic, Heather managed the $12 billion merger of Polygram and Universal and created a digital music division for the Universal Music Group; she also led strategy and business development efforts for Universal Studios’ television, motion picture, theme park, and music businesses. Heather holds a BA in English from Dartmouth College and an MBA from Harvard Business School.
In this Masterclass, Heather teaches how to use digital ads as market research to test ideas, build new audiences, and de-risk complicated business decisions.
Does it seem odd that we're still inviting a bunch of people into a bland conference room to ask them questions about their behavior and attitudes? It does to me. It's kind of like watching a lion eat her dinner at the zoo and thinking you understand how she takes down prey on the savanna. The old way ways of doing market research are getting a little tired. The surveys, for example, may have run their course. So think about it. How many people give honest feedback on an employee survey? Very, very few, right? You find the truth on Glassdoor or when employees are on their way out.
I'm not going to tell you never to use a survey but I'm going to suggest that traditional market research has some serious limitations.
Focus groups, surveys, conjoint analysis, panels, pretty much all of the traditional market research tools take place in an environment that's different from reality. Even people who have spent their entire careers in market research are starting to focus more on behavioral research. After 100 years basically of market research, it still rarely answers the big question, which is, what's the right growth strategy? And how will you know before you launch?
Market research is not a terrible way to understand attitudes and opinions about behavior. But it is not a great way to predict behavior. There are many reasons for this, but I'll call out my favorite. Traditional market research doesn't reflect the rich environment in which people make decisions. It's an environment full of competition for your attention. Market research doesn't really tell you how to break through the clutter. And there is clutter, and you really have to embrace the clutter, right? Only about 20% of retail dollars are spent online but almost three quarters of ad spend happens there. That means a lot of people are discovering new products on a screen somewhere and those screens are cluttered. Your screen is probably cluttered right now. So doing research in a tidy environment is really artificial, and it leads to artificial results.
So what's the right growth strategy? That's our big question, right? And the way that I'm going to propose to answer that big question is to use digital advertising as a medium for research. Why? Well, first of all, it's realistic. It takes place in a realistic environment with lots of real life clutter. It's behavioral. If someone clicks on an ad, they're showing genuine interest, not hypothetical interest. It's measurable. A click, a like, a comment, and email sign up. Digital Advertising makes it easy to compare responses to different variables.
It's a way to use digital advertising, not just to test ad creative or targeting. It's really a way to test innovation and strategy. And I love strategy. I spent a good chunk of my career creating strategies that didn't always get implemented because there was no proof that they would succeed, which is how I ended up heat testing.
So how does this work? Heat testing uses ads that represent strategies. So, here's an example of an ad that we use to gather data about a new product. This is an ad that might appear on your social media feed on Instagram. In this case, again, it's one way to gather data. If someone sees this ad, it's because they're in the target audience group. And if they engage or click, they generate data about whether this particular product position and ad creative resonates with them. But the response to one ad doesn't tell you very much, right. That's where heat testing really shows its power.
Heat testing allows for testing multiple variables all at the same time. That means that you can test three product concepts, four different ways, with six audiences. The result? A heat map that tells you exactly how to target each audience.
I'm going to share one example of heat testing. Say you work at a beverage company and every product you make is delivered in a bottle or a can. Your customers are looking for more sustainable products, and your packaging is becoming an issue. Even worse, governments are starting to eye landfills overflowing with bottles and cans, charging beverage makers some sort of tax is not out of the question, and this is starting to happen in Europe. So you would love to identify a product that consumers like that's bottle free, but you don't want to invest too much until you're sure there's demand.
You decide to heat test. If it works, the business opportunity is huge, right? Eliminating bottles and cans could radically change your cost structure, but your entire company DNA is wrapped up in bottles and cans. There’s some cultural stuff going on here. You're going to need some hard evidence to convince anyone that bottle free is a thing. Here's the product concept. It's a dissolvable tablet that creates an instant drink. And yes, such a thing exists, but it's usually marketed as a sports beverage or a vitamin drink. Also, it's usually not very good. The innovation here is incorporating artisanal flavors and upscale branding. It's a higher quality product that actually tastes pretty good. The working title for this concept is Palette, which suggests that you have one. You've been checking out the landscape and looking for unmet needs, you and your boss agree that there's an opportunity for a tablet that is not just about hydration or phytochemicals, it could be more about the form of the product. And your boss thinks the opportunity is in positioning the product as a sustainable solution. You think there's more to it than that. You think the product might need to have some kind of like aspirational factor if it's really going to have legs in the long run. Heat testing is a great way to test hypotheses.
You decide to create multiple versions of the product. The first, of course, is a nod to your boss. It's all about benefiting the planet, it's called Palette Planet, very in your face on that point. The second version brings to light your hypothesis about something that's a little more aspirational, in this case, a luxury product with exotic flavors, and then you also have an eye on Gen Z. They're pretty responsible about the planet, but they're also young so you're thinking maybe a party beverage could work here. That's your concept number three. You can see these all have their own identity, but are really just different versions of the same core product. The nice thing about heat testing is that you can test more than one thing at a time and you learn a lot more by testing in parallel. Heat testing is a way to identify product market fit for a new product.
We've talked a little bit about the product. Now let's talk about the market. Who's the initial audience for palate. In many cases, people like to test things with a so-called Gen pop audience. Everybody, right? And well, that may tell you that there's someone out there who likes your product, but doesn't tell you much about them. It also puts a ton of power in the hands of your ad platform, which is an increasing trend these days. They end up knowing everything about who likes your product, but you don't. Heat testing gives you a way to test hypotheses and define an audience who will be your early evangelists. In this case, you decide to test with four good size audiences, each of them is five or six million. That's going to help you really identify where the sweet spot is here. And you can see the four audiences. One, sustainable people who care about composting and stuff like that. Another audience, sober patrons who still like to get out at night, but do not drink alcoholic beverages. Another audience, beverage enthusiasts are kind of the foodie set, they're really interested in the latest new foods and beverages. And then last, holistic hipsters, probably with some roots in Brooklyn I'm guessing, but very interested in healthy lifestyles, might be drinking apple cider vinegar, or kombucha or something like that.
Okay, now the fun part. Let's make some ads. When you heat test, you never want ad creative to be the reason that concept didn't work. It's always useful to test more than one ad for each concept. We tend to be really rigorous about this. And we will often use the same creative styles across all three concepts, again, kind of identifying another variable. In this case, the ads are a little looser creatively, because we wanted to explore the concept itself a little bit more. For example, the first Palette Planet ad is very direct about the value proposition. It's about cutting waste. Second ad a little more subtle, right? The flowers and leaves suggest the planet but the main message is really about change. So when people click on ads, they go to an associated landing page where they can sign up to stay up to date with news about the new product. Signups are really an important indication of interest. As you all know, giving an email address is currency. And most consumers know that too. Getting a lot of signups is an indication of real interest. It's better than a click.
We've talked about audiences. We've talked about the ads and landing pages that represent each concept. And now it's time to test. So what ends up happening for testing is that you're essentially building ad campaigns to create side by side funnels. You're testing a whole bunch of different combinations at once. We call it a test matrix. And you can visualize it a little bit here. Each of our four audiences is going to see all of the ads, so that means several 1000 people from each audience will see each ad just enough to get a statistically valid sample of each combination. And you are setting up ad campaigns, in this case on Instagram, but it could be on LinkedIn, it could be on TikTok, it could be on YouTube, it really depends on the product.
If it's genuinely new, no one's looking for it on Google. Google tends to be a little tricky for testing things that are genuinely innovative, but of course, much better for when people know about them and are ready to buy.
So which Palette concept were people most attracted to? I'm going to show you the heat map for Palette. The heat map is based in this case on how many people clicked on an ad and viewed the landing page. Oftentimes, people will click on something and never make it to the landing page. We want to have a pretty strong metric, they have to be interested enough to actually get to the landing page. Orange in this case is hot, blue is not, yellow is in the middle. Where did we see a strong match between audience and product concept? Well, Palette Planet attracted the audience, it seems pretty obvious. The sustainable minds people right. But also really resonated with the sober patrons audience, which is kind of surprising. In fact, Palette Planet did pretty well overall, right? There's a fair amount of orange and yellow there. Maybe your boss was right. There's something else going on here. If we dig a little deeper into the data, we can look at who actually signed up to learn more, because remember, we said email is currency. We don't expect to get lots of signups in a first wave of testing. I mean, really, you're throwing spaghetti against the wall, right? But here we have the three combinations of ads and audiences with the highest landing page view rates. Not a single person signed up to learn more about Palette Planet. But the sober patrons audience signed up for both of the other two concepts. And in fact, for the luxury concept, they signed up at a high enough rate that the cost per lead was actually not bad.
So what's the right answer? Palette Planet or the luxury version of Palette? Let's see. Couple of takeaways here. What you learned in one way of testing is that sustainability does in fact build appeal. It's a way to attract attention to draw people in and absolutely relates to the question about that awareness, right? What's the thing that might build awareness for this tablet based product. But what generates a genuine interest is something a little more aspirational, right? Two takeaways from one wave of testing. Not bad. There's something else really interesting. Check out that sober patrons audience. They clicked a lot on almost everything. Are they underserved? This feels like a huge strategic finding that you weren't expecting, and maybe an opportunity for your company, right.
The secret sauce of heat testing is iterating. Imagine you keep heat taste testing your beverage concept, you can leave behind the things that aren't working, and continue to experiment with different product features, positioning, targeting for the things that are working. You get smarter and smarter about what people really want, and the ability to iterate quickly. Because this is a very fast process is a huge benefit for this type of testing.
Here's an example. And this is real data that we've taken the labels off of. Three waves of testing for a home fragrance company and in this case, a small assortment of product was actually available for purchase. So there was revenue. And you can see, as you look across the users, there's three lumps of users, which are the three waves of testing. And you can see the correlation of revenue. In wave one, not so much revenue. Wave two, things started to pick up, and we were figuring out where product market fit was for this particular brand. And by wave three, which in fact, was three tiny waves, we had nailed it. And there was this steady drumbeat of revenue, which really validated this.
As I said, I've spent my career doing strategy and being very frustrated that you can't really test it. But other benefits are that it really changes the conversations you have about innovation and strategy, because now you have proof about what's working, or at least what's starting to work. And just as important, you have proof about what doesn't work, because the last thing you want to do is go throw a lot of money at something that's just not going to resonate.
If you were to heat test your next big idea, you would have empirical evidence of whether it works or not based on actual customer behavior. And because you can test more than one version of your idea at a time, you're going to know exactly which approach is going to attract customers. And you'll have product mocks, ads, landing pages to show other people exactly what that winning approach looks like. This is fast. The first insights usually roll in within a few weeks.
Last point is when should you use heat testing. The four areas that we tend to work on with our clients over and over again are developing new products, like Palette, identifying new customers, so we work a lot with legacy brands who have a very established customer base but they'd like to reach a younger demographic, and we help them identify new ways to position their brand to find that product market fit with younger consumers. Rebranding, which can be really risky but also a great way to use heat testing to de-risk a big move of changing your brand. And then lastly, prioritizing a product pipeline. If you are an innovation group and you have 10 ideas, you're not sure what should be at the top of the pile for development, fast heat testing is a good way to get a sense of where the market opportunity is.
