Though the term “augmented reality” is most commonly associated with overlaying digital imagery and information on top of the real world, researchers have continued to explore potential audio applications of AR, generally using headphones or audio-backed glasses to weave data into a wearer’s exploration of spaces. On October 20, Foursquare will jump into the audio AR field with Marsbot for AirPods, a “lightweight virtual assistant” that whispers location-specific guidance in your ear as you walk through unfamiliar places.
Digital assistants have evolved from pipe dreams to mainstream over the past decade, as Google’s Assistant leveraged cloud servers to proactively organize users’ emails and calendars, Amazon’s Alexa facilitated frictionless voice-controlled shopping, and Apple’s Siri promoted on-device processing to reduce user privacy concerns. But for better and worse, these assistants have been passive — evoked only on request — and non-persistent in the sense that they otherwise stay silent. Inspired by Scarlett Johansson’s human-like AI assistant in the movie Her, Foursquare is exploring how a more active and location-aware virtual aide could work if it was always able to communicate with you, using audio cues rather than on-screen notifications as flags. The AirPods project builds upon the company’s prior Marsbot, which uses text messages to offer location-aware tips.
Walking through New York City with Marsbot for AirPods, users will hear tips automatically surfaced for specific nearby locations, attractions, or people, ranging from the rating of a restaurant to the presence of a Foursquare contact. Some of the tips are provided directly by Foursquare, including useful facts that might draw users into businesses, but there’s another layer: Users will be able to record their own audio clips to share with the public, and Foursquare’s samples — “get the burger!” and “don’t get the burger!” — barely scratch the surface of what individual- or business-generated recordings might contain. Anticipating some concerns, the company says it “will be proactively monitoring uploaded clips for abusive or offensive content,” and let users flag offensive content for human review.
Foursquare bills Marsbot for AirPods as an “experiment” that’s “meant to demonstrate what is possible” with location-aware audio, and plans to update the app with additional features based on the results of user engagement. It could also wind up paving the road towards more useful versions of Google Assistant and Siri — or kick off an era of even more annoying advertising, as McDonalds tries to steer you away from a nearby small restaurant, or five different stores compete for your attention as you walk through a neighborhood. For now, however, the app is meant to be light, delivering “no more than 10 spoken syllables” per tip, and typically only a few tips per day.
In its initial “version 1” release, Marsbot will work “anywhere in the world” when walking, running, biking, or driving, but has “the most pre-loaded content” in major cities such as Chicago, Los Angeles, New York City, San Francisco, and Seattle, and tip timing that’s optimized for walking outdoors. Unlike audio AR experiences designed for art galleries or museums, Marsbot isn’t meant to provide walking tours of specific spaces, but rather to offer tips and facts as you walk around, automatically lowering music and podcast volume levels to offer guidance. Foursquare notes that the app won’t interrupt voice or video calls.
Marsbot for AirPods will be available at 8:00 a.m. Pacific Time on October 20 as a free download from the iPhone’s App Store, and despite its name will actually work with any headphones. Foursquare has also set up a waitlist for an as-yet-to-be-named Android version of the app, which should be available in the near future.