Generating Insights: A Framework & Perspectives

Deciphering performance data is a core responsibility for anyone that has ever worked in paid media. If you work in this field, I’m sure at one point you’ve had to write insights for a report that was shared with your team, or sent over to a client. These insights are how we make decisions that evolve the marketing programs that we support.

Over the course of my agency career I’ve seen a handful of guides and training materials delivered on the topic, but many of these resources focus more on concepts than application. This can make it difficult to understand this process, and apply to day-to-day work, regardless of experience.

That’s why my goal with this post is to outline a hopefully tangible framework paired with a set of perspectives that will empower you and your team to start generating stronger insights and improving communication. As always, I’ll be throwing in a few hypothetical examples to paint a clearer picture. Keep in mind, this isn’t the only (or even best) solution, but ideally a simple one that can be adopted quickly.

First, we’ll start with a framework that will shape how we can approach insight generation. This framework will serve as our foundation for the process, and it consists of three parts: Observation, Hypothesis, and Optimization.

I certainly didn’t invent anything here, as this general framework goes by many names depending on where you work. I’ll break each of the components down from a conceptual perspective below, then dive into examples later in this post to tie everything together.

• Observation -  What happened? What are we seeing in the data? This usually takes the form of a comparison between channels, campaigns, creative, bidding strategies, audiences, etc… or comparing trends over time. Regardless, an observation is always a comparison. Working in marketing, you naturally want to constantly iterate and improve performance, so you need a benchmark to understand what constitutes good or bad results.

• Hypothesis - What do we believe to be the cause of what we’re calling out in our observation? There’s a big difference between a hypothesis and an explanation. You might feel pressure (particularly in the agency world) to come up with a definitive explanation behind a change in performance that highlights your industry expertise, but that’s not actually possible many times. You don’t need all of the answers, right away in any scenario. However, if you have a hypothesis you can start to develop further tests to prove or disprove that hypothesis, and get closer to that genuine explanation you’re looking for.

• Optimization - What are we going to do next? Conceptually, this component is pretty straightforward. We outline what changes to our campaign(s) we plan to make to test our hypothesis. The cornerstone of a strong optimization statement is the perspective through which it’s written.

As mentioned, the three framework components listed above are conceptual, but before diving into examples I want to share some perspectives that you can adopt while crafting the statements behind each of these components so that they can be more impactful. These perspectives are a mindset shift in how you approach the analysis and eventual development of your insights.

• Observation - If you owned the business you’re writing these insights for, what data points would you be paying attention to? I’ve seen countless observation statements focus on differences/changes in CTR, CPC, CPM, etc…, but many times these miss the mark. I’m not saying to ignore these metrics, as they are often great leading indicators for changes in core KPIs. However, what I am arguing is that if you’re trying to profitably capture more customers, do you really care about your CTR, or are you focused on your cost per customer acquisition? There’s a good chance if a campaign is already live you understand the business goals behind it, but if you’re new to a team or just catching up, getting familiar with these goals is a great first step.

• Hypothesis - Why do you think the platform behaved in the way it did? In the world of automation we now live in, developing hypotheses can be tough. It can be easy to chalk up changes in performance to the algorithm because that’s simply what Facebook’s black box wanted to do. In reality, it is our responsibility as marketers to attempt to understand the inner workings of these platform algorithms. Not at the code level, but if we can isolate a few variables within the comparison group we’re analyzing, we can make an educated guess around what we think the driving force behind differences in performance might be.

• Optimization - If you actually had the ad platform open in front of you, what specific actions would you be taking? I’ve seen insights that contain filler words like “push” or “monitor” that usually lead to follow up questions because they’re not clear on what exactly is being done. If you’re going to “push” a specific campaign because it’s performing well, what specifically are you going to do? Are you going to increase the budget, if so, by how much? Are you going to expand keywords or audiences? If so, share that keyword list/audience composition. If you’re going to “monitor” performance, what specific thresholds need to be crossed in order for you to take action? As you can see, specificity in the optimization statement is important. It proactively answers follow up questions, and doesn’t leave any room for confusion regarding next steps.

Ok, those were a lot of words, and I’m sure at this point the information I’ve provided isn’t much more useful than some of the resources I mentioned at the beginning of this post. With that, time to dive into some examples.

Disclaimer: these are hypothetical scenarios with example data, and I’m in no way affiliated with Airbnb.

Example One

Context: Airbnb is running a Facebook campaign trying to acquire new hosts.

Observation

Statement

• Ad creatives A and B have similar CPAs, with creative C having a CPA almost 3x that of its counterparts despite a higher CTR.

Breakdown

• You might notice that creative C has a CTR roughly double that of A and B, however it’s important to note that the business goal of this campaign is to drive new Airbnb customers in the form of hosts. Therefore we’ll want to focus on CPA (or CAC or any other acronym you prefer for customer acquisition cost).

Hypothesis

Statement

• Ads that call out what someone can do with their home while they’re away will resonate more with potential hosts than an ad that highlights vacation destinations or amenities.

Breakdown

• Given that ad creative A and B have similar CPAs, we’ll want to analyze what A and B have in common that might be different from C. The ad copy in A and B call out how your home can still be useful, even when you’re not there. On the other hand, the copy in creative C simply highlights infinity pools. Therefore we can make a strong educated guess that the copy (and imagery) in A and B resonate better with potential hosts.

Optimization

Statement

• Pause ad creative C, and develop one additional static ad that highlights how a host can make money off of their home while they’re away with Airbnb.

Breakdown

• Our next steps are concise and specific. We call out which ad variation we want to pause, and how many additional ads we want to replace it. Not only that, we’re providing recommendations on the type of ad, and what angle the copy and creative can take.

• Pro Tip: You can actually beef up your hypothesis section to include more than one hypothesis that helps to generate one strong optimization.

Example Two

Context: Airbnb has been running state specific campaigns over time to drive booked trips in specific regions. Airbnb is comfortable with a CPA below $70.

Observation

Statement

• The CPA for the campaign advertising Airbnbs in Florida improved as we got deeper into the year, but the CPA became less efficient over the same time period for the campaign promoting destinations in Alaska.

Breakdown

• Similarly to the first example, we know from the business context that we want to focus on the efficiency of bookings, so CPA is the main metric we want to highlight in our observation statement.

Hypothesis

Statement

• As the northern hemisphere approaches winter, and temperatures begin to cool down, consumers prefer to travel to warmer destinations.

• Based on historical data, seasonal changes in booking typically stabilize around November. (Separate hypothetical data you might have access to if you worked with Airbnb.)

Breakdown

• Not all changes in performance are the result of in-platform optimizations. Sometimes macroeconomic (or weather) related factors need to be considered. When looking at seasonality in particular, referencing any historical data can always be helpful to provide additional layers of context.

Optimization

Statement

• Adjust the monthly budget for the Destination - Alaska Alaska campaign to be $70,000 until April 1st. If November booking totals remain consistent through the winter, an estimated 1,000 bookings for Alaska a month at a $70,000 budget would yield a $70 CPA.

• Re-allocate the roughly $30,000 excess budget from the Destination - Alaska to the Destination - Florida campaign from December through April 1st.

Breakdown

• The first optimization statement outlines exactly how much budget to move, and the context behind why it should be changed by that specific amount.

• To create an even more robust second bullet in the optimization statement, we could include a project CPA resulting from the budget increase to the Florida campaign.

That wraps up the tools that I use, and have shared with my teams over the past, when it comes to generating insights. While these examples are fairly straightforward, I hope that when paired with the concepts behind the framework and supporting perspectives, you and your team can now more easily develop impactful insights that empower clear communication and improvements in performance.

Have questions, considerations, or critiques? I’d love to hear them! If you’re reading this via email, just hit respond. Otherwise, you can find me on LinkedIn and X (Twitter).