The new shining beacon for retail

Kevin Prösel

Kevin Prösel

Managing Director Saint Elmo’s Berlin

“Mixed Reality” experiences and the holographic future of retail

Imagine if “talking” products existed in the retail trade. Imagine if every retail surface and every product could act as a stage for brand and product presentation. Your favourite celebrity chef could pop up in the supermarket as a hologram and offer you cooking tips for the products on display on the shelf. When looking for the right golf club, a 3D light figure of your favourite sports star could appear in the room and talk you through the best products available in the sales area. Does that sound like something from the future? Well, it actually already exists today: Mixed Reality makes experiences like this possible.

MR, the sister of VR and AR, is a combination of the two and yet completely different to them both

As with VR, the user accesses Mixed Reality by using smart glasses. However, it works in a different way. The room is continually scanned by sensors attached to the glasses that are worn by the user and this creates a virtual 3D image of the room. Spatial computing not only detects all the surfaces and objects in the room, but also takes the position of the wearer in the room into account. Any location, any surface and even any object can become a projection surface for holographic content, which appears vivid and surprisingly palpable due to its stable positioning. A HoloLens is transparent like normal glasses so you can see the real room in front of you, but then virtual objects are projected into the user’s field of vision using a special lens. The highlight is that you can move around these projected objects so you can see them from all sides and interact with them. These three-dimensional holograms, or “shining beacons”, show us the possibilities for the future world of advertising and retail.

The use of holograms turns the point of sale into a point of experience

Once you understand how the technology can be applied, you’ll be quick to see where and how it can be deployed in retail as a special form of communication. Through the combination of real retail space and virtual superimpositions or enhancements, we can create unique experiences and outstanding experiential content. Depending on the product and customer group, the following scenarios have come forward as the most viable plans for the technology:

Product staging
Mixed Reality means that products that do not actually exist in reality yet or that cannot be transported, can be displayed as though they are physically present. For example, special limited editions and concept cars in the automotive sector. A specific example: It was possible to exhibit the new BMW X2 in BMW showrooms as a hologram before the vehicle was actually physically available worldwide for display purposes. This meant that visitors were able to inspect the new vehicle from all sides by walking around the 3D projection and could also interact with the projection and become familiar with the various features. As well as providing visitors with the opportunity to experience this new technology at first hand, it also gave them the chance to see themselves next to the virtual vehicle as a hologram, which added another dimension to the experience.

    Product demonstration
    Using holographic staging, products can be demonstrated in the environment for which they are intended without leaving the sales room. The example we used at the start for promoting golf clubs indicates how a tangible experience can work in this scenario: The sales space is enhanced by a holographic projection. After putting on the MR glasses, the customer’s view is extended to include a projected landscape of a gold course alongside the display shelves for the golf clubs. A famous golfer appears as a hologram to greet them. He will guide them through the different golf club models with his expert advice and they will be able to see him from all angles.

    This is not dissimilar to the previous scenario, but it focuses on the appeal of celebrities. Why not have your customers encounter a hologram of a charismatic Hollywood actor in the coffee isle at the supermarket so that they can enjoy an espresso with him, what else? And who wouldn’t want to meet their music idol, maybe even a rockstar who died in their prime because of their rock-and-roll lifestyle? Mixed Reality makes it possible to bring them back as a hologram and have them greet customers – all of this can become a clever part of your product staging strategy: Jamming with Jimmy Hendrix on a new Les Paul. Mixed Reality will help to make the experience more exciting, more surprising and more realistic.

    The possibilities of virtual projections in real spaces don’t end with placing virtual objects into the user’s field of vision, the projections can also take over entire wall surfaces and provide insights into different locations or worlds. Almost like a portal to Narnia – a whole world is on the other side of the wardrobe. This form of staging is primarily of interest to tourism companies: We can show users in one location what to expect in another location. With a few additional haptic effects, such as temperature, smell and sound, it’s possible to create a special experiential scenography.

    Mixed Reality closes the gap between the digital and physical world

    It seems that we have reached a milestone: The real world can be combined with virtual content – and not just visually. This is a godsend for advertisers and retailers alike as it represents a new manifestation of connected retail, combining the online and offline world. But changing patterns in thinking and action also present challenges. This is particularly the case when it comes to using the technology correctly. Not only does it need to be embedded in omni-channel communication, but it must also offer relevant added value for the user at the right moment. Here it is important to find the right forms of staging for the relevant scenography and to present this to the user as a seamlessly connected experience.

    The Wrong-Shui of Mixed Reality

    Less is more. Just because this technology offers us lots of new possibilities doesn’t mean that we should use them excessively. Our experiences show that users get most excited about the small things during a holographic experience: A visual effect that suggests a certain depth in the space, some staging which includes the real space while introducing the placement of virtual objects. It is important to give the user time to become familiar with the world of mixed realities and to explore it playfully, while offering them clear guidance throughout the experience.

    The technology is still not entirely self-explanatory. It requires clear instructions for use and even then, interaction with virtual objects is not necessarily intuitive for everyone. It is recommended to create applications that avoid the need for complicated gestures or voice controls entirely, or to design applications where there is a host there alongside the user to act as a kind of co-pilot.

    At the same time, it is essential to eliminate any disruptive factors that could influence the technology or the sensor system. Lighting conditions and a noisy environment with lots of moving guests can present an issue and influence the outcome of the experience. It is therefore important to design the complete scenography alongside the pure Mixed Reality application. This is a wonderful challenge that combines the creation of experiential content with the planning of retail situations.

    Where to go from here

    At the moment we are experiencing the first generation of Mixed Reality devices. Despite impressive possible uses, there are still certain limits (field of vision, performance, etc.). It can be assumed that this subject will experience a huge evolution. With all the big blue chip companies investing in technology like this, MR glasses will become smaller while their performance will be maximised. The price of the devices will also decrease, which will result in a wider market reach. The topic of AR is already experiencing extensive market penetration within the context of smartphones, so a large number of users will be suitably prepared for MR. In combination with technological developments, this will result in a high level of acceptance and relevance for users, customers and advertisers. One thing is certain – there are good times ahead for Connected Retail.

    This page is available in DE