From E-Commerce to Virtual Commerce: The Exciting Opportunities of Virtual Shopping

Developments in e-commerce have revolutionized the way consumer experiences can be designed. Social media, influencers and streaming platforms create a social presence, and sophisticated recommendation systems create the desire for even niche products. Personalized interfaces coupled with highly personalized advertising have raised expectations that the shopping environment is tailored to one’s needs. Mobile devices and the proliferation of smartphones have made e-commerce even more accessible and available 24/7. In addition, conversational agents make consumers feel that not only technology but also human(like) contact persons can assist them with their decisions. What comes next?

In July 2021, Mark Zuckerberg revealed that his empire Facebook would rename itself Meta, and the term “metaverse” has become a buzzword. Yet, similar ideas had been on the table decades ago, as early as 1984 with William Gibson’s novel Neuromancer or 1992 with Neil Stephenson’s book Snow Crash. An idea formerly held only by visionary authors now seems about to become real due to technological progress, namely, the creation of a highly immersive, embodied form of the Internet in which users can experience a sense of presence in a virtual place with other people rather than being mere spectators. The metaverse should be interoperable, allowing us to take our identities and digital possessions with us to different virtual spaces. The metaverse should be further decentralized to ensure that users still own their virtual goods when a platform no longer exists. Moreover, changes created by users should be visible to others and persist permanently. Among all these features of the envisioned metaverse, one feature is the most central: a 3D space where you can move around. Consumers enter via x-reality (XR) technologies – applications to experience virtual reality (VR), augmented reality (AR) and mixed reality (MR) – such as head-mounted displays. The vision of extended reality goes beyond what we can imagine today and could have major implications for socializing, sports, work, education and other activities and experiences. It could certainly change the way we shop: will we soon have virtual commerce instead of e-commerce?

VR stores already exist but have yet to make a major breakthrough. Back in 2017, virtual SATURN, an established electronics retailer, went live, making it one of the first VR stores in German-speaking countries. Interested customers could go shopping in virtual reality in a loft or on the planet Saturn. However, it remained a demo app. Also, not much has been heard since Taobao’s 2016 attempt to let consumers shop in a virtual version of Macy’s New York store. One of our studies using 2018 technology, the HTC Vive, points to technical hurdles, such as the HTC Vive’s resolution of “only” 2160 x 1200 pixels, which was insufficient to accurately read product details on packaging. In our study, these limitations meant that with some headsets, products could not be evaluated as well in VR as in e-commerce. In addition, problems such as cybersickness surfaced (see Box 2). Technological advances are rapidly reducing limitations such as small display resolution, heavy headsets and cybersickness. Newer generations of headsets come with such high resolutions that every detail can be seen very clearly, even in VR, as with the Apple Vision Pro, Valve Index, Oculus Quest Pro and Varjo VR 3. In addition, image delays and tracking errors have become less frequent. The latest headsets, such as the PlayStation VR2, are particularly efficient because they only show the focal point of attention in high resolution while the periphery of the field of view is rendered in a reduced and thus more efficient way using “foveated rendering.” This requires the integration of an eye tracker. In Figure 1, the yellow dot shows the consumer’s gaze point, which is determined by the eye-tracker. In addition to hardware-specific advances, Web VR also removes software-specific hurdles. Whereas previously, a separate app had to be created for each device, there is now a single interface for many. It is similar to the availability of many applications in a web browser, such as Zoom or Microsoft 365, which can be used independently of the type of hardware.

The technological challenges are increasingly being solved, but what are the benefits of a virtual commerce shopping experience for consumers and businesses?

Higher telepresence means that consumers are more immersed in the VR store than in an e-commerce store and forget about the world around them more easily. This leads primarily to hedonistic effects, such as experiencing more fun while shopping online. We were able to confirm such effects in a study that compared VR shopping to shopping on a desktop. The additional fun results, in particular, from the interaction possibilities with 3D products and from the inclusive field of view through the head-mounted display, which hides the physical world.

Perhaps you’ve tried to assess a camping tent online or purchase your new car on a digital platform. If so, you may have noticed that it’s difficult to correctly assess sizes and positions on the limited space of a 2D screen. Visiting a physical store is often necessary to make accurate judgments. Now, a virtual walk into a tent or a driving simulation in a virtual car cockpit can make visiting a physical store obsolete. The usefulness of these technologies is clearly demonstrated by the proliferation of virtual 3D tours in the real estate market and the availability of VR and AR apps in the furniture industry for furnishing one’s own home. The benefits of making certain products tangible and experienceable in real size are enormous, and the XR apps from Ikea and car manufacturers are pioneers in this field. The imagination gap of consumers can thus be reduced by XR technologies, as Tim Hilken and his co-authors discuss more deeply in their article (p. 30).

Current studies show further interesting benefits for suppliers, such as a positive influence of the virtual commerce experience on brand recall, which in turn leads to a higher willingness to buy. Our studies with virtual supermarket shelves further show lower price sensitivity for fast-moving consumer goods and a wider product variety in the shopping cart in comparison with buying via desktop. However, the wider product variety could also lead to lower brand affinity. The effects have not yet been sufficiently explored, as few study participants can be described as experienced VR users. Therefore, the novelty of the shopping environment could also lead to greater curiosity. Similar research exists for AR environments. Philipp Rauschnabel discusses in his article that AR also inspires consumers more and that this inspiration has a positive effect on brand attitude (p. 24).

The shopping atmosphere – smell, ceiling heights, lighting, music, etc. – has a major impact on customer behavior. While virtual realities can only inadequately address the senses of smell and taste at present, as Thies Pfeiffer creatively demonstrates in his article on the metaverse homunculus (p.36), the other factors can be varied quite easily in virtual realities. Ceiling heights in physical shopping stores, for example, are very difficult to change. In virtual reality, in contrast, only a slight adjustment in the representation of the 3D environment is required. Rearranging products on a shelf is also very easy to do virtually, as the 3D models can be rearranged by hand or algorithm without much effort, and there is enough space even for large assortments. In addition, the VR environment can be personalized very easily. For example, in VR, one could reduce the cognitive workload of consumers by adjusting the store layout, the information presentation or the virtual shopping atmosphere. If the cognitive load is high, the retailer could play soothing music, for example.

In our ongoing research, we examine virtual scenarios in which a salesperson shows up when consumers needed assistance. Two exciting options for assistance emerge. On the one hand, it is possible to assess the cognitive load of shoppers based on eye movements and take this as an indicator of when help is needed. This is done automatically by machine learning, without consumers having to activate the shopping assistance themselves or a human observer needing to access it for them. Figure 2 shows an example of an assistance system from our research that offers a product comparison matrix at the time of high cognitive load. In this, ingredients can be easily compared with each other. The table “floats” in space so that it is easily accessible and can be filled with products without restricting the field of view. On the other hand, sales consultants could also be connected to a kind of dashboard that displays the information they need to provide high-quality advice based on the consumer’s gaze and interaction data. Such a salesperson-dashboard could contain customer-specific information on the viewing time of individual products or attention to price tags and advertising messages. It would enable salespeople to better assess customer preferences, even if they do not continuously observe the consumption situation. If we take this scenario further, with progress in intelligent assistance systems and generative language models such as ChatGPT, a human-controlled salesperson avatar could even be replaced by an algorithmically controlled avatar. Customers would still perceive the consultation situation as human-like, which we know from numerous research findings on “computers as social actors” and anthropomorphism research. However, the choice of the situation in which a digital assistant appears should be planned carefully: In one VR study, virtual subjects were naked in a dressing room while observed by an avatar. This setup generated emotions, but they were negative: shame and discomfort.

The example of personalized assistance systems is just one idea of how social interaction can be designed in virtual environments. In fact, many studies in human-machine interaction deal with the design of avatars that can recognize and show emotions, as well as enable a particularly human-like interaction. In VR, avatars can be designed to be particularly human-like, for example, by being able to make and hold eye contact. The topic of virtual eye contact with robots is addressed in more detail in our article by Carolin Kaiser and her co-authors (p. 48). Further, the shopping experience with your best friend 500 miles away could soon feel very human and close, almost like shopping together in a pedestrian zone. And the next gift you select with your friend for another friend or your sweetheart might then be a chic and unique digital asset, as described by Yogesh Dwivedi and his co-author in their article on marketing opportunities in the metaverse (p. 18).

All of these insights into consumer behavior in virtual consumption situations and the personalization opportunities of shopping experiences are based on the abundance of data that can and, in some cases, must be collected in VR. Many of the sensors in head-mounted displays are required technically to create an immersive experience. Tracking motions and head positions, for example, is necessary to adapt the virtual world to user actions. Some available headsets already include eye trackers due to the “foveated rendering” mentioned above. Other recent models include biometric measurements such as electroencephalography (EEG) and electromyography (EMG). In this issue, Enrique Bigné (p. 42) looks at how these technologies can be used to elicit consumer responses, such as current cognitive load or stress. Although such sensors are more of an add-on than a necessity, and synchronizing different sensors is challenging, their data can reveal user preferences based on, for example, distances between shoppers and products and interaction patterns with products and their components. For example, in one of our studies, we were able to use eye-tracking data and machine learning to estimate whether a purchase was exploratory or goal-directed after only 10 to 15 seconds in a shopping situation. In another study, we were able to predict brand and taste preferences quite accurately from 20 seconds of data.

Of course, this very “El Dorado” also harbors risks, especially relating to data protection and privacy, but also issues of potential discrimination, insults and fakes. From a legal and political perspective, the challenge is VR regulation and the implementation of the GDPR in VR. Ethically questionable matters should be prevented while innovation should still be possible and encouraged. The same issues need to be handled in the current debates about the EU AI Act, which regulates the use of artificial intelligence algorithms. Another fascinating field is technical approaches in the area of “privacy computing.” For example, data can be intentionally distorted in order to protect privacy, provided, of course, that the gain in knowledge is only marginally diminished by the distortion. Another example is machine learning, where some of the collected data can be processed locally, i.e., directly at the consumer, without sending it to central servers. Of course, the enormous data streams and their complex processing, especially for graphical representation, require resources and energy, and these negative consequences should be considered during implementation. Retailers should be aware of these sensitive issues and plan their approaches very carefully so that the potential of virtual shopping experiences can be used in beneficial ways for our society.

According to our interview partner and XR expert Julian Weiss (interview p. 54), AR applications are already well established in marketing and retail. But will virtual commerce also soon become more established? The answer to this question is still open. Figure 3 summarizes what speaks in favor of and against the rapid spread of virtual commerce. Ultimately, the answer to the question will heavily depend on the visions and investments of the tech giants, which have also driven past major developments such as e-commerce platforms (Amazon), mobile communication (Apple), Internet search and personalized advertising (Google) and social networks (Facebook). But it’s not just the tech giants that play a crucial role in the future of the metaverse and other developments in the virtual realm. Research and startups are also making significant contributions to further development. It will be up to all of us to shape the future of virtual worlds and their commercialization opportunities in a responsible and beneficial way.

Meißner, M., Pfeiffer, J., Peukert, C., Dietrich, H., & Pfeiffer, T. (2020). How virtual reality affects consumer choice. Journal of Business Research, 117, 219–231.

Dzardanova, E., Kasapakis, V., & Gavalas, D. (2018). On the effect of social context in virtual reality: An examination of the determinants of human behavior in shared immersive virtual environments. IEEE Consumer Electronics Magazine, 7(4), 44–52.

Pfeiffer, J., Pfeiffer, T., Meißner, M., & Weiß, E. (2020). Eye-tracking-based classification of information search behavior using machine learning: Evidence from experiments in physical shops and virtual reality shopping environments. Information Systems Research (ISR), 31(3), 675–691.

White, T., & Pfeiffer, J. (2023). Consumer decisions in virtual commerce: Good help-timing and its prediction based on cognitive load. Proceedings of the NeuroPsychoEconomics Conference, 19(1), Granada, Spain.

White, T., Merkl, L., & Pfeiffer, J. (2023). Customer decision-making processes revisited: Insights from an eye tracking and ECG study using a hidden Markov Model. Proceedings of NeuroIS 2023. Vienna, Austria. Cham: Springer International Publishing