download pdf

What is the big deal about big data?
Big data are taking center stage for decision-making in many organizations, especially retailers. The McKinsey Global Institute has predicted that retailers embracing big data can increase their operating margin by more than 60 %. There is an explosion in data availability and collection. Business data double every 1.2 years. Specifically, customer-related data are growing by leaps and bounds. These data include online browsing data, social media data, mobile usage data, purchase data, customer satisfaction data and the like. For example, a retailer like Walmart collects data on about 1 million transactions per hour, contributing to 2.5 terabytes of data. Furthermore, with the runaway growth of the Internet of Things (IoT), more data are continuously collected from sensors in multiple devices such as smart watches, smart speakers and other wearables that are connected to the Internet. These data require massively parallel software running on thousands of computer servers often in a cloud-based environment. By some estimates, in 2020, one third of the data will be processed through the cloud, yielding 35 zettabytes (35 x 10²¹ bytes) of data, which may be spread across about half a million data centers across the world.

How can retailers better understand and leverage big data?
Retailers like Amazon are constantly collecting, curating, and analyzing data, and making critical decisions. Their decisions, in turn, fuel customer interactions with retailers with more data which are again recorded, processed and analyzed for further decisions. Many such decisions are made in real time. Thus, the cycle of constant data collection, analysis, decision, and further data collection keeps escalating with larger volumes of data. By some estimates, the market for big data is expected to be $56 billion in 2020. The framework in Figure 1 can help to understand big data analytics and its impact on retailing.

In this framework, customer data on attitudes and behavior across channels, touchpoints, devices and platforms are constantly recorded and collected. These data are integrated from multiple sources and stored or warehoused, often in a cloud-based environment. Statistical, econometric and data science models are developed for enabling appropriate decisions. Computer algorithms and programs are created for these models. A class of models, called machine learning-based models, are particularly useful for learning from the data and making predictive decisions. Many decisions, especially continuous and real-time decisions, are automated. These machine learning models form the backbone for the generation and development of AI-assisted decisions. In many cases, such decisions are automated using systems such as chatbots and robots. For example, chatbots assist in customer service and robots help in warehouse and retail store automation.

Hindsight, insight and foresight from data analysis
Data science models are at the heart of this framework. These models can be classified as descriptive, predictive and prescriptive models. Descriptive models primarily capture past behavior and data. The outputs from these models can be viewed as hindsight. Predictive models predominantly offer forecasts of focal outcomes. These models typically offer insight for retailers for decisions. Prescriptive models focus on providing normative decision recommendations. These models can be thought of as offering foresight. They incorporate optimization of focal decision variables. In the retail pricing context, a descriptive model could be a demand model of how customers responded to past price changes. A predictive model could be one that predicts future sales response to price changes. A prescriptive model is one that offers optimal price recommendations to retail managers. Because a large retailer deals with several thousands of items with millions of customers and perhaps billions of transactions, such a pricing problem is truly a big data problem.

How retailers can benefit from big data
Of special interest are issues such as omnichannel shopping behavior, resource allocation across channels, the effects of the mobile channel and mobile apps on shopper behavior, retailer pricing (in particular, dynamic pricing), data privacy and security. Research on these issues reveals several interesting insights on which retailers can build.

Omnichannel shopping behavior
Multichannel and omnichannel shoppers are typically more valuable than single channel shoppers. However, for certain categories, single channel shoppers may be more valuable than multichannel shoppers. A mobile channel typically enhances overall purchases. In particular, mobile apps have interesting effects on shopping behavior. They lead to greater frequency, quantity and monetary value of purchases in both online and offline channels, but also result in greater product returns. However, overall, mobile apps lead to greater monetary value of purchases net of returns. By the same token, a failure in a mobile app can lead to decreases in the frequency, quantity, and monetary value of purchases in offline channels. Also, marketing efforts in one channel tend to have cross-channel effects. Therefore, retailers need to carefully analyze big data and leverage the findings for improved decisions.   

Personalized recommendations and offers
Amazon, for example, uses big data about its customers, including over 100 million Amazon Prime customers, mainly for predicting customer purchases, making personalized recommendations of offerings and optimizing supply chains. Retailers such as Kroger and Safeway use big data to offer weekly promotions. Many of these retailers’ models are based on machine learning. The larger the customer base and the number of interactions and transactions, the bigger the training data. The more the training data, the better the algorithms’ learning from the data, which in turn translates into more accurate predictions of future customer behavior.  

Customer relationship management
An integrated database is critical to business success, so many retailers are investing in creating such databases. Retailers are also focused on using analytics to identify new sources of revenues to improve topline growth, as well as identify and implement profitable customer relationship management (CRM) strategies. To fully leverage big data in today’s retailing environment, CRM strategies must be location specific, time specific and channel specific in addition to being customer specific.