Ad testing has long been a staple of market research. But as the media landscape has transformed in the digital era, ad testing methodologies have transformed along with it, incorporating more and more tools from a growing ecosystem of digital media and advertising software and services. As a result, today’s popular digital media platforms include features explicitly designed for optimization and testing, enabling advanced marketing teams to conduct some of their own testing.
As consumer insights professionals partner with marketing teams to create effective campaigns, it’s vital that they speak the same language. Here, our digital analytics team offers 27 important terms to know about ad testing, with many terms extending past digital to the broader advertising space.
(Terms listed alphabetically and in groups of ad testing categories)
A/B Testing | Ad Unit | Ad Variations | Addressable TV | Advertising Assets | Average Completion Rate | Average View Duration | Audience Segments | Call to Action | Clickthrough Rate | Conversion | Cross-Device Tracking | CPA | CPC | CPM | Engagement | Hypothesis | Impressions | Multivariate Testing | Pre-Roll Ads | Sample Size | Statistical Significance | Skip Rate | Target Audience | TrueView | Unique Visitors | Video Drop-Off Rate
Content in digital form created by a brand.
In order to promote products on their site, a brand may need to create assets in the form of logos, photos, and videos.
Refers to a specific digital ad format, with the size and location of an ad defined.
A campaign with digital elements will have to determine how to activate across many different ad units. For example, a banner ad unit is a rectangular display ad.
Multiple versions of ad creative to be tested.
Ad variations may differ by headline copy, images, whether the ad is still or moving, calls to action, and more. The key thing is to focus on potential changes, and why they might impact the business outcome.
Group of people to whom advertising is directed.
An automotive ad may have a general audience of people watching a live football game, and a specific audience they wish to target, such as men 18-24 who like trucks. They may further split up their target into two test audiences that each receive different creative on digital channels.
Subgroups of the population defined by distinct attributes such as behaviors or demographics.
Digital ad testing commonly involves analyzing performance by segments, to understand if their performance is different from the average. The ability to communicate differently to different audiences is key for digital marketing to drive growth, and the foundation of targeting and personalization.
The text intended to persuade users to perform a specific act from a digital asset, such as clicking to learn more.
Learn More, Subscribe Now, Sign Up, Get Started, etc., are all common CTAs. CTA engagement could be a success metric for a digital ad test.
The rate at which users clicked, out of everyone who saw the ad.
CTR = clicks / impressions
An ad that receives 50 clicks out of 1000 impressions has a 5% CTR.
When a person completes an action valuable to the business, such as signing up for emails, or completing a purchase online.
Conversion is typically the most important metric when evaluating ad performance. While ad engagement may be desirable, conversion directly ties the ad to business outcomes.
Tracking and measurement of ad performance across the different devices a single user may own, such as mobile, tablet, smart TV, laptop.
The ability to reconcile one user’s data from different sources is technically difficult, but an important goal to pursue. One solution is to require logging in, so you can identify activity linked to one account.
Cost-per-acquisition – measure of aggregate cost of customer taking an action that leads to conversion.
Cost-per-click – measure of cost when an ad is clicked.
Cost per mille – measure of cost when an ad is shown 1000 times.
CPM, CPC, and CPA are three primary ways digital media is charged to advertisers, and which one is used depends on the platform and campaign goals. CPM is generally for brand building, when you want to get in front of a lot of people. CPC is a more efficient, performance-based metric. CPA is a more robust metric that determines ROI, but is rarely a pricing option.
An overarching term that includes most trackable behaviors related to an ad, social platform, or website.
Boils down to how and how much an audience is interacting with your ad content. It could mean clicks on an ecommerce website that demonstrate shopping actions, or it could mean likes and shares on a social platform.
The number of times an ad has been served, regardless of whether the user has consciously interacted with the ad.
Typically, in a digital testing environment, test variants are split up so that each has the same number of impressions.
Refers to the number of distinct individuals, most commonly identified with a system or device ID, that have visited a website or other property.
To better understand the impact of an ad variation, and to more accurately track test results, it’s recommended to design tests to the unique visitor-level, rather than the impression-level. This prevents a user from being served multiple test variations, among other complications.
A randomized experiment where two or more variants of a user experience are shown and a winner is determined using statistical analysis of the performance of each.
An e-commerce site could test two versions of a product page to understand which drives more online sales. Google, Facebook, and Amazon feature testing functionality in their ad platforms, and encourage its use to improve ROI.
A prediction created prior to running a test that states what is being changed, what you believe the outcome will be, and why.
A strong hypothesis is just as important as significance in determining and understanding the winner in a test. Many creative tests make the mistake of testing wildly different variants, which prevent conclusive results.
Designing a test with multiple variations, typically requiring more complexity in the hypothesis generation and test set-up.
Many testing programs advance to testing many MVT tests at a time, once they develop robust test roadmaps, strengthen their testing processes, and learn to manage sampling to maximize their traffic.
The number of people from a population which will be included in the test.
The concept of sampling from a larger population to determine how that population behaves or thinks is familiar territory for consumer insights, but now the purview of digital advertising as platforms enable A/B testing. Determining an adequate sample size that will reach significance, and in a relatively short amount of time, is a key decision in an ad test.
The probability that the difference in performance between test variants isn’t due to random chance.
Ad testing is typically run until data reaches 90%-95% significance. This helps ensure the test doesn’t have sampling errors, and that the winner will perform with a wider audience.
Technology that enables advertisers to select audiences to target for TV ads, rather than buying based on programming.
Addressable TV permits an advertiser to show different ads to different households during the same program, based on targeting data. Showing a dog food ad to a dog owner, and a cat food ad to a cat owner, for example.
The percent of the video watched by your audience, divided by the number of people who watched.
This measures your video’s ability to hold attention.
The amount of time a video has been watched divided by the number of times the video has been played.
Indicates the attention span users have for your content, and can be helpful for guiding ad creative. For example, if videos see an average view duration of 30 seconds, a 30-second ad would be best for this topic.
An online video ad that plays before a video the user selected to watch.
These videos are commonly 15-, 30-, or 60-seconds long, and run before the user’s desired content, meaning the audience is likely still engaged. Like most other digital media, these ads can be tested and optimized for performance. Relatedly, Mid-Roll ads run in the middle of a video.
The percent of skips for skippable video ads.
The most effective pre-roll ads capture the user’s attention immediately, as many video ad formats offer them the chance to skip the ad entirely and continue to their desired video.
YouTube’s advertising platform. Enables targeted ads, where advertisers only pay when someone watches the ad for its entire duration.
TrueView ads are the skippable ads that play before a video or during a longer video. They also appear in the search results and side-bar. People are now watching a billion hours of YouTube every day, making this a go-to platform for video ads.
The rate of people who navigate away or close a video as it progresses in time.
Shows which moments in the video people tend to lose interest and leave. Pay particular attention if you’re seeing a high drop-off in the first 10-15 seconds in the ad, as it’s an indication of whether you have an immediate hook.
You must be logged in to post a comment.
