Whitepaper: The Evolution Of Measurement

Embracing Change

We often get questions about how we built our unified measurement platform, Polaris. We thought that it would be helpful to share our thinking in this blog post.

The mobile advertising industry has always been a dynamic one that has attracted brilliant minds to embrace its constant change and bring to life businesses that wrestled with the messiness of advertising data to deliver increasing value as they evolved.  Now is no different.  So we think it is important that as we approach the privacy apocalypse and its measurement challenges, that we commit once again to center ourselves, continue to embrace change and innovate.

Let’s begin by taking stock of the positives.  Firstly, this major industry shift offered increased user privacy.  This is in alignment with our core values as end users ourselves and as a mobile ad tech company.  Secondly, it allows the industry to ditch the issues with last click attribution and forge a new chapter of measurement innovation.

Thirdly, our unified measurement platform will ensure that you have granular visibility in order to continue to successfully optimize and execute. You will see more detail about this lower down in this post.

Our Quest For The Future Of Measurement

The IDFA restrictions, that rolled out in 2021, introduced a huge but welcome change to the mobile advertising ecosystem. After an Apple user upgraded to iOS 14, any iOS 14-compatible app that wants to access their device’s persistent identifier (IDFA), most often for the purposes of retargeting, user profile linking, or measurement, needs to ask for their permission (as shown in the example below).

IDFA User Consent Messages

This presented an interesting challenge for the industry since so many players in the ecosystem rely on the IDFA to varying degrees. At MetricWorks, we’ve focused on how measurement data will flow in the post-IDFA world. Back in 2020, based on the spirit of Apple’s new terms, we decided to evaluate this challenge with the assumption that MMPs will no longer be able to send events with attributed channel, campaign, country, and publisher app information. In its current state, SKAdNetwork, Apple’s on-device measurement offering, does not seem to provide enough granularity, nor does it handle post-install events well, making retention and LTV prediction impossible.

Early in our search for a suitable measurement solution, we identified incrementality as an important piece of the puzzle.  For those not as familiar with the concept – incrementality refers to the incremental value directly caused by each advertising touchpoint in the user journey. This is impossible to measure in the last touch attribution model (Fig.1 below), which credits the final touchpoint with 100% of the value for the user without consideration to the possibility that the user could have been acquired with fewer, more valuable touchpoints or even zero advertising as an organic install.

Last Touch Attribution Model. MetricWorks

A methodology known as incrementality testing is an ideal solution to prove the causal relationship between advertising touchpoints (ad buys) and uplift.  It prescribes a rigorous scientific process similar to randomized clinical trials used by pharmaceutical companies where a population of users is randomly split into a test group that is delivered an ad and a control group that receives a “placebo” (often an unrelated public service announcement or PSA). However, it can be costly since, if you’re displaying PSAs, you still have to pay for those impressions.  There is an even bigger problem with the advent of iOS 14. Most forms of incrementality testing require a large list of device IDs so that the audience can be split. iOS 14 will make this difficult to accomplish.

Another powerful tool in the measurement toolkit that also considers incrementality is media mix modeling (MMM). This technique uses regression models to find correlations between ad spend and business value. As a “top-down” technique (Fig. 2), versus a “bottom-up”(Fig. 3) approach like last touch attribution that works at the device level, it eschews device IDs in favor of aggregated data, and is therefore naturally aligned with user privacy. As you can see in Fig. 2 and Fig. 3 below, both top-down and bottom-up measurement techniques attempt to allocate the same users and app activity data (installs, opens, revenue) to the same four campaign/publisher app combinations in the proper proportions, but come up with different answers.

Top-Down Measurement Process By MetricWorks
Bottom-Up Measurement Process By MetricWorks

Key Idea: Based on the measurement output of the two different methodologies, we can see that they somewhat agree on the value of ironSource campaign B, publisher C (red). However, ironSource campaign A, publisher A (purple) didn’t get credited much in the bottom-up last touch methodology (just 1 install with little revenue) while top-down shows that, even though it might not be getting the last touch, it is providing significant incremental value. On the flip side, bottom-up attribution gives Vungle campaign A, publisher A (orange) a solid amount of credit for last touches, but statistically, it is providing almost no incremental value. We would have acquired those users either through other campaigns or organically anyway.

Coming back to the top-down media mix modeling (MMM) technique (Fig.2), let’s look at its advantages:

  1. Requires few inputs – mostly just a time series of spend and target outcome measurements.
  2. Robust to incongruities among ad channels, both online and offline, in terms of functionality and data availability.
  3. Can be used to predict the change in outcomes as a function of different spend inputs which is quite handy for planning purposes including what-if analysis.
  4. Can also be used in conjunction with other algorithms to optimize towards a given goal, which can be used for budget allocation optimization.

MMM has its own problems though, which also eliminated it from contention in our quest for the future of measurement. When we attempted to apply it to mobile app advertising, some major issues became apparent including:

  1. Requires several years of data at a minimum due to aggregation at the week or month granularity.
  2. Not used for quick decision making since it takes weeks or months to update a model with new data.
  3. Usually custom built for an individual advertiser by very expensive specialist consultants.
  4. Can only prove correlation, not causation (eating seafood may be highly correlated with personal wealth, but that doesn’t mean eating a lot of seafood will make someone more wealthy).

While MMM wasn’t a perfect fit, we recognized early on that the MMM concepts held a lot of potential. The key is that we found that most of the downsides could be mitigated through a combination of creative feature engineering and automated model validation through constant backtesting and live experimentation.

Keep in mind that the entire measurement process including modeling and validation must be completely automated in order to be scalable. No need to worry though because our existing automation technology in UACC is being enhanced to make this a powerful reality for you.

Our Solution For Mobile Measurement

Thank you for bearing with us and making it this far.  Let us now look at the high level overview of our solution:

  1. We believe that measurement of campaigns at the country and publisher app level (when applicable) remains critical for UA decision-making.
  2. Compatibility with the current iOS14 technical specifications and alignment with the spirit of the new rules are equally crucial for any future measurement solution  including ours.
  3. Regression models similar to those called for by media mix modeling can be augmented with techniques that address problems unique to mobile app advertising in order to finally deliver a measurement solution that considers incrementality.
  4. Using daily data helps solve the data volume issue inherent to MMM and allows the model to be updated more quickly so it can inform the types of fast decision-making required by modern UA teams.
  5. Backtesting allows us to automatically evaluate a wide array of approaches and select only the most accurate models based on their ability to predict known historical outcomes.
  6. Automated controlled experimentation enables live testing of incremental value predictions in the real world so that models can be validated and improved by rejecting poor models and feeding results back into the next model.
  7. Our unified measurement platform, Polaris, (based on our analysis outlined above) ties aggregated cohort activity with the channel, campaign, country, and publisher app dimensions, instead of the MMP.
  8. Our measurement module will be provided as an option at the app level so that advertisers can rely solely on MMP measurement for their Android apps, should they wish.

We hope that the above was helpful and invite you to connect with us about any questions so that you have a chance to shape our solution.


The MetricWorks Team






MMP 2.0 Launch At MAU

Following our VentureBeat launch by Dean Takahashi of MMP 2.0 (Polaris), we got to launch Polaris at one of the best mobile industry shows, MAU Vegas on May 23rd!  The reception was fantastic as mobile marketers are desperate for a measurement solution that closes the three value gaps (poor accuracy, lack of privacy-safety, and limited measurement scope) in last touch MMPs (MMP 1.0) to deliver superior performance.

In summary, we had a fantastic MAU! Thank you to the MAU Vegas team, our partners, clients, attendees and everyone who took the time to learn more about MetricWorks‘ MMP 2.0. If you missed our MAU presentation by our CEO, Brian Krebs, then do sign up for our FREE TIER to start enjoying the benefits of MMP 2.0. We’ve had a ton of signups. Make sure you don’t miss out on the solution that will help you demand more from your data.

If you would like to learn more, submit a demo request or visit our Polaris product page.

How MMP 2.0 Drives The Mobile Business


Reliable measurement is at the heart of decision making for the mobile business. As you can see from the diagram above, MMP 2.0 leverages multiple measurement methods (including geo lift testing and MMM), blending them to produce an output (single source of truth) that drives the business processes and delivers superior performance.  Our team has worked tirelessly to continue innovating so that we can deliver a solution that:

1. Looks like a MMP so that it is easy to use

2. Fits into your existing UA process to avoid any disruptions or productivity problems

3. Merges the strengths of multiple measurement methods to deliver far superior accuracy than individual methods and MMP 1.0

4. Offers sophisticated algorithms that removes the guesswork and signal conflict by outputting a single source of truth the plugs into your existing business processes

We hope you like Polaris and find it valuable in your quest to achieve higher performance while preserving your budget. 


The MetricWorks Team 

Vibe.co offers radical transparency in CTV reporting thanks to new partnership with MetricWorks

Vibe.co and MetricWorks are teaming up to make it easy for marketers to launch CTV campaigns with accurate incrementality measurement.  Recent years have seen a dramatic increase in global viewers moving from linear TV to CTV streaming services.  This trend has created a fantastic opportunity for marketers to leverage new  programmatic platforms like CTV DSP Vibe.co. 

Launched just over a year ago by adtech industry veterans, Vibe.co aims to radically democratize access to television advertising by developing the first self-serve platform in the industry; becoming, in essence, the “Google Ads of CTV.”  Businesses big and small can now launch hyper targeted CTV campaigns in mere minutes with CPMs at or below other social media or Google prices, and 3rd party, real-time reporting. 

Fig. 1: Vibe + MetricWorks allows advertiser to make big moves in CTV

Vibe.co is singularly focused on developing the most powerful, simple, and transparent CTV tool in the industry. Nothing more, nothing less. 

“We obviously believe in the power of CTV, and so do our investors who just funded a $6.35M seed round a mere three months after we released our self-serve feature, but advertisers are still wary of this new technology, and I don’t blame them,” says Vibe.co co-founder Arthur Querou. “For years, ad tech solutions were plagued with bloated account management teams, unnecessary features, and confusing reporting. But here’s what I want them to know: the CTV of 2022 is incomparable to its early days. Measurement tools and standards, especially, have developed in ways we couldn’t have dreamed of, even just a year ago. Vibe.co has kept its promise to our clients: launching the simplest, most transparent, most powerful platform. And now, thanks to MetricWorks, we finally have the opportunity to prove the true power and scalability of this exciting new channel,” he concludes. 

MetricWorks is the leading provider of MMM-based incrementality measurement which closes the gaps in last touch measurement. MetricWorks’ incrementality MMP, Polaris, in particular, is a low effort, easy, drop-in addition that doesn’t require any extra SDKs, additional skills, device IDs or heavy migration lift.  Polaris offers an easy, turnkey “MMP experience” so that you can get started in 24 hours. It also integrates directly with your last touch MMP to provide a single repository for side-by-side comparisons of your last touch and incrementality data. 

“We are excited to partner with Vibe.co to deliver a powerful solution that will make it easy for marketers to try out CTV with the assurance of comprehensive and granular incremental measurement that is far superior to last touch measurement,” said Brian Krebs, CEO and co-founder at MetricWorks. “Our partnership will enable marketers to leverage media mix modeling (MMM) to ensure that they achieve performance marketing success on CTV which has been an undervalued medium.” 

If you are looking for an edge over your competitors that will elevate your marketing effectiveness, please contact either Dmitri or Chris (info below) to see how easy it can be:

Dmitri Souffan
Head of Mobile at  Vibe.co

Chris Hoyt
Chief Growth Officer at MetricWorks

tvScientific partners with MetricWorks to boost ROI on CTV

The cord-cutter revolution has seen over 128M+ US viewers transition to watching TV via streaming services, representing 63% of all US TV viewing time. Because CTVs are by definition connected to the internet, this astonishing shift in consumer behavior has created a massive opportunity to democratize access to TV’s awesome power as an advertising medium, complete with self-serve capabilities and digital-like performance measurement. The tvScientific platform will play a major role in enabling millions of digital-first advertisers to participate in CTV advertising for the first time.

tvScientific has developed the most sophisticated CTV performance advertising platform, allowing businesses to prove out the power of TV advertising.  The platform simplifies and automates CTV buying and optimization, leveraging massive data to prove the actual value of TV advertising.  “Value is proved with ROI and this is why measurement down to the granular level continues to be the center of conversations when it comes to CTV,” said Jason Fairchild, CEO and co-founder of tvScientific.

With this partnership, tvScientific continues to enhance marketers’ ability to accurately gauge campaign effectiveness through incrementality measurement. Now, with MetricWorks, tvScientific can give marketers vastly improved performance visibility to CTV. This is due to the fact that the performance is incremental rather than last touch. MetricWorks has quite a bit of data about how wrong last touch can be when comparing performance across various traffic types, including tvScientific’s platform. Our partnership now rectifies this last touch issue and leads to improved ROI/ROAS for marketers.  “With the 108M streaming households market presence of CTV, digital marketers should be looking at CTV as another attractive performance channel in their daily marketing mix,” said Jason Fairchild, CEO and co-founder of tvScientific. “tvScientific’s partnership with MetricWorks will provide performance marketers with deep and comprehensive insight into campaign performance across platforms while proving that CTV has truly been undervalued due to limitations in last-touch attribution.”

MetricWorks is the leading provider of MMM-based incrementality measurement which closes the gaps in last touch measurement. MetricWorks’ incrementality MMP, Polaris, in particular, is a turnkey, drop-in addition that doesn’t require any extra SDKs, device IDs or heavy migration lift.  

With the significant market presence of CTVs, marketers should be looking at CTV as another attractive channel to boost their marketing ROI. This partnership between MetricWorks and tvScientific, makes it far easier than before for marketers to take full advantage of the CTV channel opportunity.  You get the best of both worlds with tvScientific’s self-managed solution coupled with MetricWorks’ fully automated, granular, daily incrementality measurement outputs that are tactically engineered for successful daily marketing effectiveness.

MetricWorks’ incrementality MMP, Polaris automates every step from designing experiments and calculating ground truth to training econometric models and deriving incrementality results. Incrementality is delivered in the form of the KPIs you rely on today including ROAS so that your UA or marketing decision-making processes don’t need to change. Polaris is also designed to offer an easy, turnkey “MMP experience” so that you can get started in 24 hours. No IDFAs or GAIDs. No migrations, no extra SDKs, and no heavy implementation lift. Polaris integrates directly with your last touch MMP to provide a single repository for side-by-side comparisons of your last touch and incrementality data. “Today’s digital marketer needs to have a holistic understanding of how each platform is performing in a marketing mix and MetricWorks is tactically engineered for successful daily marketing effectiveness,” said Brian Krebs, CEO and co-founder at MetricWorks. “We’ve seen CTV significantly undervalued by last-touch across our customers, often because the bulk of its impact is felt higher in the funnel. The partnership with tvScientific will enable marketers to apply performance marketing strategies to advanced television and leverage media mix modeling to drive ROI on an undervalued channel.” 

The full press release is available here.  If you are looking for an edge over your competitors that will elevate your marketing effectiveness, contact us today to see how easy it can be:

Meet Us At MAU Vegas

Well, it is finally the week of MAU and we are excited to meet you in Vegas! MetricWorks is proud to be a Gold Sponsor of MAU Vegas. There are many opportunities to meet our team and learn how you can close the gaps in your measurement stack with our MMM-based incrementality MMP, Polaris.

For a full listing of what MetricWorks is up to at MAU, visit MetricWorks At MAU22. Here are the quick highlights.

1) Don’t miss our MAU Speaking Session with our CEO, Brian Krebs on Wed, June 8th @ 11:40am in Terrace Ballroom 152. Brian will describe “Why It Is Time To Close The Gaps In Your Measurement Stack With MMM-Based Incrementality Measurement”

2) Drop by our Booth 644 to win fantastic daily prizes! Book a time to grab a drink with us.

3) Be among the first 50 to get an exclusive peek on our webinar at the very first Incrementality Industry Report. Sign up now for this exclusive webinar and the report that will follow.

4) Get our FREE GUIDE on How Marketing Mix Modeling Can Fill the Gaps of Attribution.

Hit us up with any questions or for a demo of Polaris at demo@metric.works. See you soon at MAU! 

How does incrementality enhance UA prediction and optimization?

When COVID-19 hit the world, digitalization skyrocketed. People had to find new ways to communicate, be entertained, and live their life from home. Brands also had to find new ways to survive, grow and interact with their customers. Just like that, the number of apps boomed and so did the competition. It won’t slow down any time soon. That’s why apps need to advertise successfully. Otherwise, it is difficult to differentiate from the competition and acquire new users.

So here’s the secret sauce for successful advertising: innovative UA driven by innovative measurement. Innovative UA provides the ability to make smart, and fast decisions today that outperform the competition tomorrow. This new type of decision-making is powered by prediction. Prediction itself is fueled by innovative measurement that is accurate, unbiased, granular and grounded in scientific methods that have been honed over many years. This type of measurement is known as incrementality.

What is incrementality?

You may have heard about incrementality and wondered what all the fuss is about.  Well, in the post-IDFA world, last touch is certainly a flawed and unreliable form of marketing measurement.  Fortunately, incrementality does not depend on IDFAs or GAIDs, or arbitrary attribution window. It is far more accurate than last touch or SKAN in powering your UA prediction and optimization.


Incrementality is the lift, in terms of any KPI (such as ROAS), over all other media spend plus organic demand. It’s really all that matters. If certain media spend is cannibalizing organic lift or overlapping with other media, true value can be significantly impacted.  

MetricWorks’ incrementality MMP, Polaris automates every step from designing experiments and calculating ground truth to training econometric models and deriving incrementality results. Incrementality is delivered in the form of the KPIs you rely on today including ROAS so that your UA or marketing decision-making processes don’t need to change. Polaris is also designed to offer an easy, turnkey “MMP experience” so that you can get started in 24 hours. No IDFAs or GAIDs. No migrations, no extra SDKs, and no heavy implementation lift. Polaris integrates directly with your last touch MMP to provide a single repository for side-by-side comparisons of your last touch and incrementality data.

Why are traditional MMPs and SKADNetwork not enough anymore?

The general issues with traditional MMPs and SKADNetwork is that they are built on the simplified yet flawed view of the world (see Fig. 2a below) and they ignore the fact that conversions are the product of a cohesive media mix (Fig. 2b below) rather than a single ad.

Last touch attribution ignores this complex reality, producing random results that are skewed toward self-attributing networks and dependent on arbitrary attribution windows. This has major impacts on business value. Due to this, UA optimization and prediction are also affected. 

In simpler terms, an inferior measurement signal results in inferior optimization and prediction. Tempr. is a fantastic prediction and automated optimization solution. However, to get the most out of your investment in Tempr., it is critical that you feed it a superior accurate signal tied to business value, namely incrementality. This ensures that Tempr. will make the best predictions and optimizations for you based on the best measurement inputs from MetricWorks Polaris.

Prediction: today’s decisions, with tomorrow’s data

The focus on the protection of personal information increases, but marketers still need to understand how to run successful ad campaigns. And we’ve great news: with prediction, marketers can get an accurate estimate of how their campaigns will perform, before wasting any budget or time. Even with limited data.

Where does prediction come from?

To be fair, prediction is no novelty for big app studios. Their teams usually design their own internal BIs. Most of them have data scientists working on predictions, because key decisions for the future should never be made blindfolded.

But here’s the thing. Any app studio, no matter the size, the language, the revenues, the structure, the MMP, etc., should have affordable and easy access to predictions. Not just the big guys. That’s why Tempr. – a predictive tool that adapts to each app vertical, and their KPIs – was created. Picture it: one big team helping UA Managers make today’s decisions, based on tomorrow’s data.

How does prediction work?

Prediction gives clarity on what’s going to happen next. The core principle is to use the past to predict the future

This marketing approach uses data science to forecast the strategies that are the most likely to succeed. In a simple way, here’s how it works:

1: Historical data mapping
The past is what we call historical data. On the graph below, each black dot represents the revenues made on a specific date.

2: Historical data understanding
The aim is to find trends, seasonality, and any other metrics that affected the past revenues – taking into account bids, budgets, traffic, app verticals, KPIs, etc.

3: Calculation
With machine learning, a mathematical formula that reflects the trends and seasonability of the graph in one curve is calculated. The objective is not to precisely follow each dot but come to a global result like the blue curve on the graph.

4: Data enrichment
The source’s data is enriched with data from MMPs such as MetricWorks for more accurate results.

5: Prediction
The prediction model powered by machine learning create thousands of scenarios with changing parameters and suggests the best bid-, budget-, and creative combination that will achieve the highest results (ROAS or CPE) in the future.

Fig. 3: How predictions work

Fig. 3: How predictions work

Benefits of prediction in UA
Now that the privacy era is on, Attribution and Measurement are more and more limited with SKAdNetwork. To make competitive decisions, mobile marketers need specific data. Not only yesterday or today’s data, but also tomorrow’s. 

UA Managers have to make assumptions on their users journey all the time. How active they will be, what type of events they like, and how much revenue they will generate. And since they don’t exactly know what to expect, they use A/B testing. Experimenting, testing, retesting, deleting. And wait to see what works, and what doesn’t. Which can be very costly.

However, when marketers associate predictions and A/B testing, they limit as much as possible the risks, the time-consuming guesswork and the loss of money along the road. Because prediction shows the successful path to follow, and adapts to the environment every day. And most importantly, Tempr.’s predictions give actionable insights.


More than just prediction: actionable insights

To predict the future is good. To recommend exactly what to do in order to achieve greater results? Even better.

Tempr. recommends campaign optimizations based on the predictions. The algorithms predict the combinations of bid, budget, and creative that will have the highest returns or that will decrease costs to achieve a certain event (first listening, download, etc.).

Marketers can finally drop the doubts and let predictions run the thousands of scenarios for them instead.

Fig. 4: Basics of Tempr.

Key takeaways

With prediction, apps increase their competitiveness, their efficiency, and ad campaign profitability. Mobile marketers can focus on what’s important: design better strategies, expand into new markets, and develop new features and apps. As predictive marketing fills in the information gaps left wide open by privacy rules, it’s now a vital practice to adopt.

How incrementality and prediction can work together

As we mentioned above, inferior measurement signal results in inferior optimization and prediction.  Incrementality’s superiority lies in its ability to model complex reality where media sources do interact, both beneficially and detrimentally, due to high audience overlap. Marketing can also both boost and cannibalize organic demand. Therefore feeding Tempr. with a superior incrementality measurement signal from MetricWorks ensures that your predictions and optimizations are far better than those derived from the SKAdNetwork signal.

In particular, as described in Fig. 5 above, with MetricWorks’ incrementality serving as the measurement signal, Tempr. will be able to more accurately identify situations where you might be underspending on a good media source or overspending on a bad media source.

Fig. 6: MetricWorks x Tempr.

The perfect partnership between Tempr. and MetricWorks

This recent partnership between Tempr. and MetricWorks offers mobile UA teams the best of both worlds with superior prediction and optimization by Tempr. being driven by superior incrementality by MetricWorks. “We’ve seen first hand the impact of poor mobile measurement. Therefore, our goal with MetricWorks Polaris is to provide true-north measurement that innovative UA-focused solutions like Tempr. can count on to deliver excellence to their clients in the post-IDFA era,” said Brian Krebs, CEO of MetricWorks.

To move towards a powerful predictive-data-driven decision model, it is crucial for marketers to have tools that provide an accurate understanding of how users interact with their apps. This partnership provides just that, giving marketers the ability to not only understand their app user behavior but to also take action on that information to improve their efficiency and maximize their ad revenues. Prediction-driven UA is undoubtedly linked with reliable data to start with. Therefore we are very happy about our partnership with MetricWorks that offers a very promising, and complementary technology to the well-known attribution platforms, helping mobile marketers to take the right decision,said Cloé Dana, CEO of Tempr.

Together, MetricWorks and Tempr. create the most efficient mobile UA solution on the market today. If you are looking for an edge over your competitors that will elevate your UA performance, contact us today to see how easy it can be:

Vincent Schmiedhausler
Sales Director at Tempr.

Chris Hoyt
Chief Growth Officer at MetricWorks