Login

The Dark Sides of Digital Marketing

  1. Home
  2. Publications
  3. The Dark Sides of Digital Marketing
  4. Metrics Gone Wrong: What Managers Can Learn from the 2016 US Presidential Election

Metrics Gone Wrong: What Managers Can Learn from the 2016 US Presidential Election

Raoul Kübler and Koen Pauwels

Keywords

Metrics, Dashboards, Decision Making, Polls, Probabilitic Models, User Generated Data

download pdf

The Age of Data – Bone or Bane?
In the last decade, we witnessed an explosion of data availability. Humankind creates more data each day than we did all together in the last 20,000 years. Despite all this data, it’s not its size, but what you do with it that matters. Marketers often start from the available data to brainstorm potential uses, instead of asking the right questions, and then dig deep in: how do we find the adequate answer, which type of data do we need, where do we get this data from and how do we access, process and combine this data with our existing insights? This leads to important issues such as “Which source is reliable?” or “Which data is richer in information?” Don’t stop once a data source confirms your preconceptions or indicates your strategy is working great – check alternative data sources to ensure your conclusions are valid. Our analysis of the 2016 US presidential illustrates what could happen if you don’t. 

 

Let the user speak - the power of alternative data sources
Could any campaign manager have known better? Given that campaign managers – alike marketing managers – often put their decision making on the base of a limited set of performance metrics, finding the right polls or marketing metrics becomes essential. How about looking at other data sources and variables that – alike polls – show voter engagement and preferences? In the political arena, donations, media coverage, social media followership, engagement and sentiment may similarly indicate how well a candidate is doing. In addition, most of these variables are available for free and can easily be “harvested”.

 

While the majority of traditional polls painted an overly optimistic picture for Hillary Clinton, the majority of user generated data clearly indicated that the predicted landslide win of the democratic candidate was in jeopardy. Donations may still have comforted the Democratic campaign managers, as Clinton received much more donations than her rival throughout the whole campaign. The amount of news media coverage in the 4 months prior to the election, however, showed a different picture. To obtain this information, we conducted a text mining and topic analysis of the tweets of the major 56 US news outlets prior to the election. This data clearly shows that Trump dominated the media and received much free publicity from center and left-of-center media. Also, the left and lean-left media focused more on the inner-party rivalary between Clinton and Sanders, while the right and lean-right media focused more on highlighting Trump’s strengths and the weaknesses of his democratic opponent.

Social media deliver an even clearer picture. Throughout the whole campaign Trump shows substantially more followership and higher growth than Clinton, again indicating that Trump enjoyed more momentum than indicated by the majority of the traditional polls (Figure 4).

 

Blinded by the wrong numbers: A jeopardy for sound decision making

Validating the bigger picture with alternative data sources is not limited to politics. The latest marketing research shows that online consumer behavior metrics can enrich, and sometimes replace traditional funnel metrics. Trusting a single ‘silver bullet’ metric does not just lead to surprises, it can also mislead your decision making. Econometric models can help disentangled a complex web of dynamic interactions and show immediate and lagged effects of marketing or political events. Our model for the 2016 election clearly showed the impact of external events, user generated content, campaign actions, and media coverage for both candidates, explaining the poll gap between the two candidates at different points of time.

Don’t be fooled by data – Lessons for marketers

  • Assess numbers and forecasts ciritcally
    A key lesson from the 2016 elections for marketers is to stay as much critical as possible of numbers and forecasts. Therefore, we recommend a healthy dose of skepticism when assessing insights presented to you. One way to do so in the avalanche of data is to verify existing insights and to ensure validity with alternative data sources. Combining different data that is linked to a similar outcome – in our scenario data on voter preference – helps to predict the actual outcome, to explain it and to drive it with appropriate action. As pointed out in our example, such data may be gained directly online from users -e.g. reviews, social media comments and posts, or online forums, or other sources such as e.g. statistical databases.

Trusting a single ‘silver bullet’ metric does not just lead to surprises, it can also mislead your decision making.

  • Rely on marketing theory to evluate suspicious or contradicting “evidence”
    Another key challenge arises, if the gathered data does not give a unanimous vote. In this case, management’s key responsibility is to use its expert judgment. A first step into the right direction is to check for face validity. Does the sign of the estimated effect ring true to you? As humans, we have the uncanny ability to integrate many different signals, from anecdotes and feelings to current data and the interpretation of past events. While an econometric analysis is typically better at pinpointing the magnitude and the duration of an effect, managers can easily tell whether it should be positive or negative. In many cases simple marketing theory can be tremendously helpful here. E.g., if you find that your intended marketing performance variable increases, when you increase price, you may be sceptical about having the right measure at hand. Similarly, simple correlation analyses may help you to understand how variables work together and behave together. Again, a first face validity control may be helpful to screen out suspicious effects that go against marketing theory or your own experience.
  • Use dashboard based on econometric modelling
    Finally, marketers are well adviced to develop company or brand specific dashboards, which should be based on econometric models. Relying on established procedures and the help of econometric methods such as e.g. vector autoregressive models, may help managers to not only identify and track key performance variables, but may also be helpful to understand which data sources bring meaningful information to a decision maker’s table, as suggested by Pauwels in his 2014 book “It’s not the size of your data, but what you do with it.”

Using such approaches to continuously monitoring your company’s data environment and controlling the reliability and valditiy of available data for decision making, will finally enable you to not be blinded or overwhelmed by the richness of data available to you. Or in other words: to avoid being lured into the dark side of decision making, shed some light onto your data and critically think about its utility. Then your marketing will be great again! Seriously great!

Authors

Raoul Kübler, Professor of Marketing, Marketing Center Münster, Germany, raoul.kuebler@uni-muenster.de

Koen Pauwels, Distinguished Professor of Marketing,Northeastern University, Boston, MA, USA, k.pauwels@northeastern.edu

Original Publication

Kübler, Raoul and Pauwels, Koen. "Metrics Gone Wrong: What Managers Can Learn from the 2016 US Presidential Election" NIM Marketing Intelligence Review, vol.13, no.1, 2021, pp.30-35. https://doi.org/10.2478/nimmir-2021-0005

Further Reading

Kübler, R. V.; Colicev, A.; & Pauwels, K. H. (2020): “Social Media's Impact on the Consumer Mindset: When to Use Which Sentiment Extraction Tool?”, Journal of Interactive Marketing, Vol. 50, 136-155.

Kübler, R.; Pauwels, K.; & Manke, K. (2020): “How Social Media drove the 2016 US Presidential Election: a longitudinal topic and platform analysis”. 
https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3661846

Pauwels, K. (2014): It's Not the Size of the Data--it's how You Use it: Smarter Marketing with Analytics and Dashboards. Amacom.

Inhalt