download pdf

The rise of the dark side
Our relationship to technology is deeply paradoxical. On the one hand, we buy and constantly use more technological devices and apps, leaving our traces in the digital space. On the other hand, the dark sides of how these digital traces can be used and abused are increasingly evident and concerning to many. Unregulated fake news, fueled by algorithms that constantly present users more of the same, spread without much restriction on social media and have ultimately facilitated the storming of the Capitol in Washington, D.C., by fierce supporters of the former US president. Even the most ridiculous conspiracy theories get amplified and make fighting the current pandemic less effective. But fake news and conspiracies are only two of the many problems that inspire and challenge researchers, fiction, movies, regulators and – well, yes – even Big Tech, the obvious beneficiaries of the world´s digitalization. Other problems are data privacy, hate speech or the question of free choice. Are humans still in control of their actions, or are we becoming puppets on the strings of global players with motives we do not even know?

This state of affairs and potentially dystopian future developments were not intended
Sir Tim Berners-Lee, the inventor of the World Wide Web, built it on the utopian promise of giving all people access to the best information at any time. Social media were supposed to connect the world and enable community between long-lost friends and strangers alike. User-generated content would equalize the information differential between traditional content producers and consumers. These new technologies would enable companies to achieve true customization and build authentic individual relationships with their many customers.
In many ways, this utopian vision has actually been achieved: Wikipedia is the world’s largest, freely accessible, user-generated knowledge resource; Facebook connects almost three billion people; and even small companies can reach out to customers all over the world in a more targeted way than ever before. Yet, we are also increasingly grappling with the unintended consequences of these technological advances.

Why unintended consequences arise
To help us think about unintended consequences, it is useful to ask why and how they arise in the first place. In his classic essay in the American Sociological Review in 1936, the sociologist Robert Merton describes four main causes of the emergence of unintended consequences of social action, which are still relevant today (see Figure 1).

• Inadequate knowledge
  Being able to develop some sort of “foreknowledge” to anticipate unintended consequences requires a detailed understanding of all potential effects of an action, and in particular, of the interplay between these effects and other forces. The marketing industry’s steep adoption of sophisticated advertising and marketing tech during the past decade has created increasingly complicated decision environments for marketers. For example, automated digital advertising markets or artificially intelligent products that interact with networks of other products make it almost impossible to have the knowledge required to fully understand and predict all potential outcomes. Even worse: As Kozinets and Gretzel point out in a recent commentary in the Journal of Marketing, most marketers are not machine learning or data analytics experts but mere users of complex technologies and artificial intelligence (AI). Therefore, they are only able to observe and interpret outputs, often not understanding how they were produced. As a result, marketers are unable to learn from them. If we already struggle to understand some of the intended outcomes of our marketing actions, how can we expect to predict the unexpected ones?
   
• Error
  A second source of unintended consequences is error, which Merton discusses in the sense of bias and logical fallacies. One of the most paradoxical features of the digital marketplace is that while numbers and data abound, insight often does not. For example, observational rather than experimental data are too often used to make causal claims for advertising and other marketing effects. But what true uplift is attributable to a campaign if we cannot compare it to a control group? As Blake and his colleagues demonstrate in a well-known study published in 2015, eBay found out that the return of advertising spent on their Google search advertising was in fact negative after designing a quasi-experiment in which they halted search advertising in some geographical areas but not in others. This came as a surprise to eBay’s executives, who believed, based on prior observational data, that search advertising was effective in driving traffic to their marketplace. Confusing correlation and causation can lead to potentially dangerous inferences about why we observe certain phenomena. For example, think about the ongoing debate on vaccines and adverse health effects: Are they caused by the vaccine, or did the two events merely co-occur? The result of misattribution of causality is not only suboptimal decision making but also incorrect prediction. If we falsely attribute the causes, we cannot accurately predict the consequences – intended or otherwise.
   
• Imperious immediacy of interest
  A third source of unintended consequences is a sole focus on the intended immediate consequences of an action at the expense of considering long-term potential consequences. For the longest time, Facebook’s relentless focus on growth and disruption was captured by their infamous internal motto “Move fast and break things.” By focusing on this immediate strategic imperative, Facebook neglected many other consequences of the platform they were building, such as data protection and consumer privacy, the potential to manipulate opinion, consumer mental health, and so on. These consequences were possibly not important enough to stifle growth. As another example, YouTube’s recommendation algorithm is designed to optimize a user’s time spent on the platform. The longer a user stays, the more YouTube learns about their behavior and the better they can monetize their platform. That is the consequence of immediate interest. But an unintended consequence could be the creation of so-called “rabbit holes”: The recommendation algorithm may suggest increasingly extreme content to keep a user interested. It is noteworthy that the consequence of immediate interest often relates to a commercial objective, whereas the unintended consequences often affect wider societal issues. In contrast to inadequate knowledge and error, which make it difficult to predict unintended consequences, the immediacy of interest makes it unimportant or uninteresting to do so. It is a choice.
   
• Uncritical focus on fundamental values
  The fourth source of unintended consequences is in some way also the result of choice. In this case, further consequences might not be considered when an action seems to be a logical and mandatory consequence of fundamental values. The reluctance of many social media platforms to regulate content is a good example here. Freedom of speech is an important fundamental value in democratic countries, and in particular in the US. US-based social media companies are extremely uncomfortable with the idea of accepting any sort of editorial responsibility over the content shared by their users. Yet, allowing anybody to say anything can result in incredible distortions to our sense of reality and our ability to judge what is “true.” This can have dangerous consequences, as we are seeing with the proliferation of conspiracy theories that are causing untold damage to our societies – ranging from undermining the vaccine rollout against COVID-19 to Trump supporters storming the Capitol. Of course, having private tech companies become the regulator of free speech can also have dangerous consequences. The problem with fundamental values is that they are rarely questioned – because they are so fundamental. If that is the starting position, there is indeed no room to even consider the unintended consequences that might arise as a result.

Some current battlegrounds of digital marketing
Let´s now have a closer look at some of the complex battlegrounds of digital marketing and how inadequate knowledge, errors, shortsighted choices and an uncritical focus on fundamental values can produce outcomes we do not want.

• Algorithms: Friends or foes?
  We increasingly rely on algorithms to either make automated decisions for us or assist our decision making. Because these algorithms are often black boxes, we basically entrust many decisions to mechanisms we mostly do not understand. This comes in handy if we can save time and effort to reach certain goals, but this convenience also comes at a price: loss of autonomy. Buder and his colleagues argue that algorithms fulfill various organizational objectives that users may not be aware of and that may not be in their best interest. We cannot be sure if algorithms truly optimize the benefits of their users or rather the return on investment of a company. The options an algorithm suggests are only a subset of all possible choices; yet, we will never know what these other possible choices are. In these settings, free choice is a mere illusion. Even worse, narrowing down options can open the door to discrimination or manipulation.
  Examples of algorithmic racial or gender discrimination abound, but even if an algorithm itself is non-discriminatory, market forces can lead to biased outcomes: Lambrecht and Tucker  found discriminatory effects of Facebook advertising. In their study, women received information about STEM careers less often than men, even if they were targeted equally: a problem that seemed fairly simple but turned out to be almost impossible to fix. This is a typical example of inadequate knowledge leading to unintended consequences.The world of interconnected algorithms has become so complex that acting in consumers` best interest is tricky, even with the best motives.

• Data privacy: The price of personal data
  Consumers are used to accessing free and very convenient digital services. Free email and messaging, free social media, free apps, free search and information, and customized offers are integral parts of our daily routines. We chat with friends, post our pictures, measure our performance, navigate to desired locations and buy the interesting products that make it to our screens as if by magic. But there is catch. Free isn´t really free: We pay with the traces and data we leave behind online, often without being aware of it. In his article, Wertenbroch reports results of a study showing that consumers underestimate the monetary value of the personal data they provide. This is an example of error leading to unintended consequences. Companies in the data business can take advantage of this underpricing and accumulate profits at the expense of consumers. Regulators such as the European Union try to protect consumer privacy with legislation like the GDPR, with limited success. Regulation is necessary but can also undermine competition for data and hence prevent a fair price for data.
   
• The power of metrics
  In our data-driven world, everything comes down to seemingly undisputable numbers, metrics and benchmarks. In this issue, Kuebler and Pauwels take a closer look at the 2016 US presidential election and analyzed why democratic election managers trusted in the wrong metrics (indicating a comfortable lead for Hillary Clinton) and hence made devastating mistakes in their campaign. We often have multiple data sources to choose from, and finding the right mix of data and metrics for sound decision making can be challenging. As the famous saying goes, “garbage in, garbage out.” Managers should therefore be critical of the metrics that guide their decision making and use common sense and alternative data sources and metrics to counter check results.
  Another problem is the agenda-setting power of metrics. Once a metric has been defined as relevant, a lot of focus and effort concentrate on improving on this metric. In our interview, Douglas Rushkoff points out that a greater part of humanity is working on making our social media feeds more persuasive than on making clean water more accessible. This is a striking example of unintended consequences caused by the imperious immediacy of interest. But even if we agree on what is important, numbers and metrics can be misleading. In his recent book, Tim Harford observes that data “may be a pretty decent proxy for something that really matters.” If what matters is complex, the proxy might miss out on relevant aspects, leading to critical gaps between what we’re able to measure and what we actually want. For example, if marketers decide that high engagement with content is important, the focus will be on improving metrics such as the number of clicks or shares. These objectives become an incentive to produce content that is attention-grabbing and evokes strong emotions – leading to an environment in which facts and cool-headed information have less chance to spread. Is this really the world we want to create? This brings Albert Einstein’s famous quote to mind: “Not everything that can be counted counts, and not everything that counts can be counted.” In the age of algorithms, everything needs to be broken down into numbers; therefore, the problem of unintended consequences due to simplified, incomplete or simply wrong metrics is more striking than ever before.