In Minority Report, do you remember when Tom Cruise’s character fast-walks through a futuristic retail store, with brands calling out to him, offering him deals based on all the data they have on him? Well, that future is almost here. As store beacons are becoming a reality, those who have opted to receive notifications will soon have retailers zero-in on your physical location and sending customised offers to your smartphone. But why should they stop at passive texting/emailing when they can speak directly in your ear, tempting you with offers tailored especially for you. And that’s where the possibilities of contextual sound come into play. Imagine relevant information being delivered to you, privately, when you need it without you having to lift a finger. This could be made possible by AuR (Audio Augmented Reality), the newest addition to the family of XRs, the other two being VR and AR.
VR, short for Virtual Reality, is a term that describes Immersive Virtuality. Some applications of VR are in the arenas of entertainment, therapeutic role-playing, 3D data visualisation, etc. But the thing is, VR has a significant limitation: it is physically restrictive: you can’t take it “to go.” You may be trekking a Moroccan desert in VR, but in real reality, you had better stay within the confines of a small room because, as a consequence of wearing a head-mounted display, you are navigating blind. In short, VR is not for everywhere.
A more portable sibling of VR is AR, short for Augmented Reality. In AR, virtual objects are superimposed over reality. The best-known example of AR is probably the Pokemon Go mobile game. No doubt there is a fun element to AR, but its applications can range from the frivolous to the life-saving: a surgeon superimposing a VR diagram over the site of surgery for instance. But as with VR, there can be a safety concern in using AR: if you are on the road, and your eyes are on the virtual object, you are distracted.
Enter Audio Augmented Reality (let’s call it AuR for short.) Just like AR, it adds to our visual reality by incorporating virtual add-ons. AuR also enhances our reality by curating or selectively manipulating what we hear. It offers an immersive experience by adding an audio layer on top of real reality, and without the safety concerns of VR and AR. It is the safest way of layering on a combined sensory experience because the hearing has a lower cognitive load than staring at screens because “seeing” takes up a lot of our focus. Speech, on the other hand, allows us to continue using our other faculties. In the longer term, short of all us inhabiting virtual worlds 24/7, the audio input will hold an edge because of its safety-related and multi-tasking advantages.
Hearables or smart audio devices such as Apple’s Airpod and Bragi’s The Dash have already established themselves in the consumer technology space. AuR adds yet another dimension to their smarts. Peripherii’s Smart Earrings are the newest entrant in this arena. Their advantage over other devices is that being earrings, they are already at the wearer’s ear when needed, unlike earbuds and earphones which are stored away until needed. From the point of delivering contextual information, smart earrings are, therefore, a boon to retailers and wearers alike. Also because earrings are not considered a social faux pas unlike, say, AirPods, they can be “always on” making them precisely right to tap into the many strengths of AuR in any environment from work to an evening out.
“Earrings are not considered a social faux pas unlike, say, AirPods, they can be “always on” making them precisely right to tap into the many strengths of AuR in any environment from work to an evening out.”
Revisiting the scenario from Minority Report and factoring in Peripherii’s Smart Earrings, now imagine yourself being a customer who walks into a retail store, and a virtual assistant welcomes you by speaking directly into your hearable device. From then on, you can either decline further help with the shake of your head, or you can choose to have the assistant with you through your entire shopping experience. You can even ask conversationally, “Are there any special offers in the perfume section?”
This kind of convenience will allow you to put your phone away in your bag and shop hands-free because, thanks to the hearable, you are no longer anxious about missing that vital phone call/text. As you wander through the store, or you linger in a particular section, the store’s virtual assistant can choose to whisper information about a specific designer or a product or a display directly into your ear (privately delivered of course).
As you start to walk away from an item that you seemed interested in, the store offers you a sweetener – extra loyalty points for instance. To add to your personalised experience, a store could run a highly individualised an impromptu pop-up sale or a trivia contest delivered right to the shopper’s ear via her earbud/smart earring.
The great thing about a hearable device is that it could provide an intuitive way of locating a product in a store. Instead of a customer having to walk through a store looking for a sales assistant, they can request sales help via the earring. Store personnel can reach you quickly thanks to indoor positioning systems. For retail stores that have an app with digitally indexed products, a customer can ask her earphone. In fact, this product finder feature could be used by the store’s sales assistants as well. In place of headsets, they could wear store-branded earrings. A customer’s shopping experience can be further customised by offering her a choice of music. For stores that sell music, this can lead to an in-app purchase as well. At the very least, it will make for a much more pleasurable shopping experience.
“The great thing about a hearable device is that it could provide an intuitive way of locating a product in a store.”
For retailers, the benefits of AuR can’t be emphasised enough. Should a potential customer walk through a store and stop to look at a mannequin, based on their in-store location and proximity to the mannequin, the retail store can deliver an enticing discount offer right in your ear. This is more than just passive information delivery; it is a tailored call to action. Similarly imagine that you are in a museum, looking at a painting. Based on your physical position and the orientation of your earbud/earring, AuR will enable the delivery of information about the painting right in your ear. In place of renting ungainly headphones, your wireless hearable will do the job seamlessly.
Likewise, when you are travelling and find yourself in an unfamiliar neighbourhood, you might stop before a store window debating whether you should go in. Your AuR hearable can give you a quick summary of the store’s merchandise categories, customer service ratings, etc. You can then choose to go in (or not!) In addition to the assisting functionality, the realm of AuR also includes smart sound filtering. Thanks to the microphones included in its electronics, an AuR device can read the ambient noise level and then modulate how much of it you hear by allowing a range of options for pass-through audio. From that range of options, either you or your smart hearable can choose to let through more or less sound.
Real-time translation, digital coaching, navigation aids for the vision-impaired, etc. are some other reasons why AuR is a compelling space. Add to this the easy utility of smart voice assistants, or the ability to smart-soundtrack your daily routine, and it is easy to see the vast potential for Audio Augmented Reality.
At first glance, one might perceive AuR to be a lesser relative of AR and VR in that it does not have a visual component. But that is precisely its strength: it not only allows you an immersive experience but it does so without hijacking your eyes and your attention. Many new technologies find themselves in limbo for years for want of meaningful use cases. That is definitely not true for AuR; it’s an easy sell both from a consumer’s point of view and a retailer’s as well.