They have already been seen on roads and highways many a time: Prototype cars with their bumpers taped up, covered in wildly patterned film and protective guards. These camouflaged and not-yet-released car prototypes make for a sought-after photo opportunity for journalists. Car manufacturers use these visual aspects in order to shield their latest models from the public's curious gaze and (most importantly) from the competition as they undergo test drives under real conditions. However, the prototype is shrouded in secrecy long before leaving the workshop for its debut spin. But how can you find out what potential customers think when the car is still in the development stage? The answer lies with car clinics in which potential customers evaluate the design, features and comfort of the prototypes, therefore playing an important role in driving forwards the vehicle’s optimization during the development stage. In theory, digitalization has already made virtual car clinics a possibility, saving time and expense, but how well do new visualization technologies perform against traditional "live” evaluations? It is really possible to judge a car from a virtual model alone? The GfK Verein has sought to shed light on these questions as part of a comprehensive study in cooperation with automotive market research from GfK SE.
Competition is fierce on the automotive market: scores of developers battle for the various manufacturers and for potential customers – always with new ideas, technical innovations and new designs. Market research companies are commissioned to run car clinics in order to find out how appealing the planned new model is to customers whilst it is still at the development stage. A selection of potential customers are sworn to the utmost secrecy before being shown prototypes of the new cars. In an interview, they then rate the cars in comparison with well-known competitor vehicles. This process is expensive and time-consuming; not only must real prototypes of the new models be built, competing vehicles must also be hired and taken to the showroom – all under extremely strict security measures. There must also be staff on-site to make sure that none of the testers snap a forbidden photo on their cell phones. In the age of globalization, manufacturers have to appeal to the tastes of an international clientèle, meaning that the cost is multiplied many times over. And every real car clinic brings with it the risk of information on the new model being leaked out or finding its way into the hands of competitors.
This is where virtual car clinics can help, where the participants simply evaluate digitally depicted 3D variants instead of real cars. This saves time and production costs as well as minimizing the risk of information being made public too soon. But how effective is this method for participants? Would it be possible to create a similar visual impression just by using pictures? Or does VR in fact perform better? And finally: Do the virtual car clinics produce equally valid results as the traditional real-life test scenarios? In order to answer all of these questions, the GfK Verein, together with the Automotive Research Team at GfK SE, carried out a comprehensive study in May 2017 on the various car clinic possibilities.
Two car models were used for the study: A known model and an unknown model, with only a few cars licensed in Germany. This lesser-known model was used in order to create the most realistic simulation as possible for the real-life test of the prototype. The participants rated aspects such as overall impression, external details, interior, amount of space and proportions. Four test scenarios were carried out, each with approximately 100 participants. The first method was the traditional car clinic, where the participants viewed real-life vehicles in person. In the second method, participants simply viewed photos of cars on large screens. The two remaining methods used virtual reality (in the form of an immersion experience using high-resolution VR headsets). In order to find out which test variant the participants thought was best, the groups not only went through one whole scenario, such as the pictorial evaluation; they also went through one of the other variants in part. In so doing, they were able to make a direct comparison between two methods and as a result, they could state their preference. To complete the tests, participants required around one hour – about half the duration of a traditional car clinic, where the prototypes are compared with not just one, but several competing models. Because of this, all participants were asked if they would be prepared to carry out an evaluation using "their” method for another hour. Regarding the VR varieties, this aspect is not to be underestimated: Some of the participants, who had already worn VR headsets in the early days of the VR trend for computer games, for example, had experienced symptoms such as dizziness, nausea or headaches. In particular, the time lag between when the participant moves his/her head and when this is aligned with the visual impression in the virtual world can lead to problems for those sensitive to these symptoms. (Examles: www.techbook.de and vr-world.com). However, these negative effects did not arise during the study.
So how exactly is VR technology used in the car clinics? In order to use this digital advancement for automotive market research, real vehicles must be replaced by computer-generated virtual models. A depiction of the cars is created using CAD data (computer aided design) which must be provided by the car manufacturer. The advantage of this data is that every detail of the car is portrayed completely realistically. Sensors in the room which capture the participants’ head and body movements in real-time and align them with the image depicted in the glasses enable the participants to view the vehicle from an array of perspectives as soon as they put on their data glasses. The only real element that the GfK Verein used in the study was a car seat so that the VR test participants had the opportunity to evaluate the inside of the car from a driver's perspective as well as the exterior. Two versions of the virtual car clinic were tested: A cable and a wireless variant. In the cable VR test, the necessary data was transmitted to the headset via cable. In the wireless version, the data came from a laptop which the participant carried in a rucksack. Whilst in the first version, the radius of the participants was restricted (due to the cable connection, they were only able to move around within an area of around three square meters), the laptop-carriers had significantly greater freedom of movement and were even able to walk around the virtual car.
One of the results of the study is that virtual reality is a technology which was widely approved of among respondents aged 21-69. The vast majority of participants (94%) experienced no problems whatsoever when using the VR glasses. In addition, almost three quarters of the VR group would have definitely been prepared to carry out a further test. As a result, it is clear that if this technology was implemented in automobile market research, the participants would have very few qualms over the duration of the test. However, there is one condition: The "old” VR teething problems such as a time lag after the participant has moved his/her head and before the image is displayed must be eliminated. Naturally, the participants would then be able to manage the additional two-hour studies. Nevertheless, the virtual variants were no match to the real vehicles (yet). Overall, the respondents preferred a real car to a digital depiction. However, the VR varieties performed far better when compared with the photo varieties: Only around one in five would rather evaluate a new vehicle model from a photo on a screen than in virtual reality. The remaining 79% preferred the VR variants.
Therefore, the VR varieties definitely have a chance when it comes to tester approval. However, for the car manufacturers, an important issue is also how useful the findings are compared with those of the traditional car clinics. So, how consistent are the ratings provided by the participants? How valid are the results from the car clinics which used VR technology? Do they deviate significantly from those of the traditional car clinics? The study shows that, overall, VR methods for making evaluations are very much in the same league as "real-life'” methods. In order to verify this, the participants were asked to rate both the known and unknown model on specific criteria. In terms of overall impression from all perspectives, all of the methods tested were close: There were no significant differences in the evaluations using traditional car clinics, those using the photo-based method and those using the virtual version. The unknown (prototype) vehicle also received similar ratings as the known (comparison) model in all four test scenarios. When awarding points on a scale from 1 (completely dislike) to 10 (like very much) from a wide variety of perspectives, the known model was awarded between 7 and around 8 points in each scenario. Overall, the unknown model was favored slightly less by the respondents, no matter which test method was used.
Whilst there were almost no differences between the individual test methods regarding overall impression, when it came to rating the external details, huge discrepancies arose in at least one area. However, it appears that, for the time being, it is already possible to realistically evaluate many aspects of a vehicle using VR methods today: Radiator bonnets and grills, headlights as well as bumpers and the area below them were all awarded a very similar number of points in all of the tests – regardless of whether the participant was looking at a known or unknown car model. However, when it came to the wing mirrors, the testing groups had differing opinions: Whilst all those who evaluated the wing mirrors using photos or with the two VR variants awarded around 7 points, those testing the real vehicle gave a significantly worse rating. Indeed, the mirrors differed in one significant aspect: The real mirror was highly reflective, but this was not taken into account in the programing of the virtual varieties and therefore was not visible. The reflection was also not visible in the photographs – it appears that the testers preferred this in both cases. The unknown model had the opposite effect: Testers preferred the wing mirrors on the real vehicle much more than those conducting their test on the basis of VR data or images. This is because, whilst the real-life wing mirrors were made in a high quality chrome finish, those on the photographic and virtual reality varieties came across in a dull silver because they were lacking the reflective elements, making them appear less attractive to potential customers. This result shows that the reflecting elements in the 3D model must be designed in a particularly meticulous fashion in order to give a realistic depiction of the vehicle.
Regarding the interior, another weakness of the VR and pictorial methods is made clear; one which lies in the very nature of these methods: Participants lack the haptic impression which they get in real-life by touching the vehicle. This was particularly evident in the ratings on the comfort of the front seat in the known model. While the real car seat was awarded seven points on the comfort scale, respondents gave a worse rating (below seven points) to the (non-authentic) “replacement” seat used to furnish the virtual showroom, as was expected. Of course, respondents can hardly rate the material in the front seat without a real sitting experience, particularly when rating an unknown car. Those testing the VR and photographic models gave significantly lower ratings for this aspect than those who could actually sit in the original car upholstery. Nevertheless, this issue could be remedied in the future if the VR car clinics use the seats from the real prototypes and comparison cars – or at least provide textile samples. However, by the same token, the results of the pilot study indicate that, for other aspects of the interior, the use of original components may not be necessary; both the VR and real-life cars were awarded similar ratings for the visual appearance of the front seat as well as the central and ground consoles and steering wheel alone.
One likes it large and sleek, the other small and compact - when it comes to a car's proportions, tastes are sure to be different. The VR variants surpassed the purely photographic versions – particularly for prototypes of the unknown model. At 67%, the clear majority of participants rated the real-life unknown vehicle as well-proportioned; 32% rated it as disproportioned and 1% were unable to judge. The VR headsets (both cable and wireless) gave almost exactly the same results. However, amongst the testers who only viewed the prototypes in photographs, uncertainty prevailed: Around half found the proportions consistent; the other half did not. Why were the virtual portrayals more successful than the photographs? Using the VR headsets, particularly the wireless varieties, the testers were able to walk around the car, viewing it from all possible perspectives. However, when it came to the photographic representation, only selected perspectives were depicted. In addition, the participants were unable to judge how the size of the car compared with their own height; what use is it to judge a car from the perspective of someone who is 1.80m tall when the tester is 20cm shorter? In such cases, the proportions of the car were inaccurately perceived, making evaluation difficult. The effect was not so pronounced when assessing a known make of car. In this instance, photographs were able to compete well with the other methods. This may be due to the fact that the relative proportions of the vehicle photographed in an everyday streetscape were familiar and could therefore be more realistically assessed using memory. However, this was not the case in the car clinics where new vehicles were evaluated.
The advantages of VR in comparison with the photographs are made even more apparent when it comes to the amount of space. Proportions are also important here: How much space do I personally have in the front of the car? How much leg room does the back seat offer? Am I only able to transport a medium-sized suitcase in the car or would I be able to fit in a weekly shop for a family of six? And how easy is it to load and unload the trunk? In order to shed light on these aspects as realistically as possible, various aids were used in the pilot study, such as a life-sized dummy on the back seat and a water tank in the trunk; these could be seen both in the photos and in virtual reality. Nevertheless, the photographs achieved the worst results for realistic assessment of interior proportions. The participants who were only able to make an evaluation using photos gave the vehicles worse ratings than all other participants for both the amount of space in the front of the car and leg room. Whilst most of the participants in the "photo” test overestimated trunk capacity, they also indicated that it would be more difficult to load and unload the trunk. By contrast, the VR test groups rated all aspects of the interior proportions very similarly to the testers who viewed real vehicles – in this context, VR performed much better than a simple on-screen presentation. This is due to the limited possibilities of a photographic depiction; fixed perspectives also played a role here, dictating from which angle the model appeared. By contrast, VR headsets enabled the respondents to look around inside the vehicle. They were able to turn around in the front seat and kneel down and inspect the trunk in greater detail. In so doing, they got the same visual impressions based on their individual height as they would have received in real life. All of this made judging the proportions just as realistic as it would have been in a real car.
So, are we still a long way off from virtual car clinics for now or do they in fact represent a credible alternative for car manufacturers today? The study shows that in many aspects, virtual portrayals can match real vehicles – particularly when it came to the solely visual aspects, most assessments turned out just as realistically as in "analog reality". In order to achieve a valid evaluation of haptic aspects such as the quality of surfaces, materials and seating comfort, real elements can and should be used in the virtual experience. In any case, it would be interesting to continue to consider this idea and to develop it further - and not just in the automotive industry: VR makes it possible to not just save a lot of time and expense but also gain realistic feedback on a huge variety of design alternatives, even early on in the production process.
Data source: method test ‘Virtual Car Clinics’, May 2017 ;
Responsible for the method test ‘Virtual Car Clinics’, May 2017: Anja Dieckmann and Joerg Leissner, GfK SE
Responsible for the article and contact person for queries about Compact: Claudia Gaspar. (e-mail to hello@nim.org)