Last year, I sat on a panel at the 4A’s Data Summit with peers from the MRC, Google, and TAG (Trustworthy Accountability Group) to discuss the topic of data quality. As I prepared for the discussion, I was struggling with definitional issues. When someone talks about data quality, what are they really talking about?
What Is Data Quality?
I believe that there are two primary axes to consider in defining data quality: performance and accuracy. Performance answers the question: Does the data affect marketing outcomes in a positive way? Accuracy answers the question of whether the data is what it says it is.
Data doesn’t necessarily need to be both high accuracy and high performance for it to be high quality, however. A data set could be devoid of any description and still perform spectacularly in terms of marketing efficiency. Meanwhile, a completely accurate data set may defy expectation and perform poorly for a given campaign. How is this possible? Well, data is a tool, after all, and a great tool can still be used poorly and lead to bad outcomes.
But I have found this dual-axis framework to be a helpful way to clarify discussions around data quality with both buyers and sellers of data.
(The goal is to avoid the gray square.)
Measuring Performance Correctly
Of the two, performance is the harder data quality metric to measure. You need a keen eye on not only the performance of the campaign, but also on whether the data has been correctly applied and the goals appropriately measured. If a dataset is used to deliver a branding message with no call to action, there would be a mismatch in measuring the effectiveness of the dataset by looking at the clicks or click-through rate. Likewise, if I buy a spoon and find that it does not cut a ribeye steak very well, then I shouldn’t write a review claiming the spoon does not work.
Gathering Accurate Data
Accuracy should be the more straightforward data quality metric; an objective truth—at least theoretically. We may not always be able to verify what that truth is, but we know that a truth exists. The person is or is not a certain age, is or is not the owner of a Ford, is or is not a buyer of toothpaste, is or is not in-market for a toaster (or at least is or isn’t yet). Nielsen Digital Ad Ratings (DAR) and comScore validated Campaign Essentials (vCE) are tools to assess the accuracy of core demographic data points, but are far from a means to assess the accuracy of all potential data.
In the absence of other means to establish the accuracy of a data set, you can develop a reasonable level of confidence by documenting what you know about the data set and its travels on the way to reach you. How was it created? Has it been modeled? At what level of precision was the data onboarded or matched to other identifiers? What happens when the data is sent to different channels and platforms?
To answer these questions simply requires transparency. It’s not perfect, but understanding the provenance of the data you’re using will give you a level of confidence in its accuracy.
Transparency Leads to Accuracy
As an industry, we’re moving toward greater transparency. Shifts like standardizing definitions and expanding metadata bring a higher level of transparency and clarity.
When evaluating different data sources, there are a few organizations doing this work. The Interactive Advertising Bureau (IAB) and the Coalition for Innovative Media Measurement (CIMM) are both working on standards for data with the goal of providing an additional level of confidence.
Until we reach peak transparency, there are other ways to get the information necessary to make an informed decision. Simply put: ask and you shall receive. Ask your data providers for more information to fill in any gaps in the accuracy of your data.
Putting Data to the Test
There is no way to guarantee that accurate data will be high-performing data. But the really great thing about data is that it’s pretty easy to test it. Before going all-in on data that you don’t know will be high-performing, it’s easy to compare or A/B test different data sources to see what works for you.
We live in a very data-driven world, and that’s why it’s more important than ever to have a firm grasp on your customer journey. The only way to ensure performance is to test, test, test. So, test away, and you’ll find what data performs best for your brand, all the while establishing what makes you stand out.
For more about data-driven marketing, read our blog on how to ensure internal alignment on data initiatives.