Playlines Logo

Concept Album Comes to Life With the Help of Augmented Reality

From the booming role-playing video game industry to the growth of escape rooms, the public is enjoying a massive shift toward “lean-in entertainment.” As technology advances, such entertainment will only become more elaborate. Award-winning Immersive Augmented Reality Studio Playlines, in partnership with executive producer Muki International and record-breaking British rapper Harry Shotta, has designed and rolled out CONSEQUENCES, a live choose-your-own-adventure audio-AR game in London. The experience builds upon Bose AR with Frames to allow interaction with both real and game worlds simultaneously.

 

 

Described by Playlines’ Creative Director Rob Morgan as “part silent disco, part immersive theater experience,” CONSEQUENCES marries the richness of a live world populated by actors and infuses it with the music of Harry Shotta. Using the spatial audio capabilities of Bose AR, players become a character in an exclusive Grime / Drum-n-Bass concept album where they choose their path and discover one of six possible endings.

*Players prepare to experience interactive merged reality experience CONSEQUENCES enhanced with Bose AR

“For most people, the phrase 'Augmented Reality' raises certain expectations. They tend to picture visual feed superimposition like the Ikea app or Pokémon Go,” says Rob Morgan, a veteran experience designer, and game writer. “Our goal is always to add to reality, rather than compete with or overwrite a player’s experience of reality.” For this reason, the experience begins with the player being handed a Playlines device that already has the game pre-installed on it. This offers us better quality control and the player a friction-free UX from the very start.

“Once the AR Frames are connected, the player’s phone goes in their pocket and Bluetooth iBeacons allow the app to track and respond to the player’s location,” says Morgan. As the player moves, the mobile location allows the app to track their decisions with an old school, choose-your-own-adventure video game engine. Players make decisions and progress the plot with their feet and their imagination. The beacons communicate with the app and trigger content as the player steps into a new space.

 

 

For example, at one point an actor will ask the player to follow them, which represents a decision point. The player’s location will signal which path they have chosen. Then, as in a video game, this decision will influence the rest of the narrative going forward. 

How Playlines Uses Bose AR

While Playlines has been developing beacons and 4D narratives for several years, the possibilities for them increased massively when it was integrated with Bose’s AR hardware, which offers a unique means of interacting with players.

“There’s a novelty to the Frames that isn’t gimmicky,” says Muki Kulhan, Executive Producer of CONSEQUENCES. “They’re accessible, they don’t give you motion sickness, and they remove the barrier and myth that AR has to involve a clunky headset. That really democratizes access to the experience, which is a huge priority for us.”

Another benefit of the Frames is the open ear form factor. The player can be fed audio without being shut out from sounds and interactions in the real world. This allows actors and genuine atmospheric effects to create a form of immersion which is not separate from reality. 

“Our augmented reality is not a trick. It’s not achieved by making a player forget where they are. It’s achieved by making them interested in the story that’s being told to them where they are.” Rob Morgan, Creative Director, Playlines AR

The spatial audio feature is perhaps most valuable to the Playlines team. “When we’re creating virtual characters who are in the environment with you, the impact of spatial audio cannot be overstated,” says Morgan. “It allows us to give a real sense of spatial identity to characters who are talking to you and that allows us to create drama.”

Playlines narratives pull players into the story, with a mix of sound, music, and real-world activity that makes CONSEQUENCES players feel intimately involved. “You’re not actually experiencing everything in the story,” Morgan says, “but it is a much more effective, visceral experience because of spatial audio.”

The Tech Behind CONSEQUENCES

“Being a developer is sometimes like being on The Great British Bake Off. You’re given a set of ingredients and told to make the best cake you can make. But where before we were using basic ingredients grown in our backyard, Bose has come in and supplied us with Michelin-star ingredients.”  
Muki Kulhan, Executive Producer of CONSEQUENCES

The team was particularly pleased with the SDK, which improved the development process overnight. While in the early days of Playlines’ mission, prototype testing involved Morgan carrying a backpack full of beacons to different public areas, we can prototype location-based stories on a desktop computer.

“Since the official release of the SDK, it has all been plain sailing,” says Jerry Carpenter, Lead Developer of CONSEQUENCES. “The bolt-on components from the basic and advanced demos now allow you to recreate the functionality of the Frames without having to understand the complex math under the hood which could previously be distracting. This aspect means you can integrate the Frames at a much later stage of the project.”

With Frames delivering the gaming experience, Playlines uses a relatively simple game engine to track the player’s path. A local server stores JSON files to identify the current stage:

 

{

   "Room": "Car Park",

   "UID": "consq_queue",

   "TipText": "The password is \"Shotta\"; you need to say it to the bouncer to get in.\nBut shh! Keep it to yourself!",

   "StorySoFarText": "After a bit of argy-bargy with the bouncer, Barry gave you the password to get into the underground club: SHOTTA.",

   "PresetTimer": 15,

   "Beacons": "0:32118:4932,0:62997:58966,0:49499:17214",

   "PathList": "",

   "People": "BC",

   "Sequence Name": "QUEUE"

 },

In the future, Playlines have their eye on creating more stories that merge multiple realities and bring in more features of the Bose AR technology to more precisely interpret a player’s input cues. They will also look to incorporate more of the spatial audio factor, a rousing success from CONSEQUENCES, into all their other experiences. This will evolve as well, as they intend to leverage the spatial audio for uses such as giving clues to the locations of physical objects like in an escape room or treasure hunt. “Each game is trying something new that acts as a prototype to be perfected by the next one,” says Morgan. 

The team is eager for other developers to join them and explore the potential of storytelling which is enhanced by spatial audio and the development of which is fueled by the Bose AR SDK. “If you have a story you imagine telling someone as you walk through an interesting location, you can start thinking about location-based augmented storytelling,” says Morgan. “A lot of people will find they have a story to tell.”

“Think of who you are making the experience for,” Kulhan offers up to other developers. “What story do you want to tell them and how will they experience it? Find a partner like Bose to help get it to them.” 

Learn more about Playlines / CONSEQUENCES