May, 2014

What is Augmented Reality?

, animator at Animat Habitat

Augmented reality (AR) adds a layer of digital information on top of the physical world around us, and we are already using this technology in many different ways. One of the most common ways is with our smart phones, which have a few features that make AR possible and relatively easy to implement. Key components include a camera as well as sensors like a gyroscope, accelerometer, GPS receiver and compass that enables the smart phone to determine its orientation, the direction that it is pointed in, its general location, and then – with the camera – it can use recognition software to figure out exactly where it is and what it is looking at. Once you know where you are, you can start overlaying accurate and designed information on top of the world around you.

AR is changing the way we view the world, and blurring the line between natural and digital. Everyone from tourists to teachers, to someone looking to share information about the world around them can now benefit from computer-generated graphics, audio and other sensory information superimposed in a real-world environment in real time. In this article, we look at where augmented reality is now and where it may be headed.

An Added Perspective

For better insight into AR technology we turn to a developer's perspective. For the past five years, Mai El-Awini has worked at an advertising studio in Halifax, creating augmented reality applications that have boosted marketing campaigns into an interactive space. In between studio projects, Mai has helped to create the companion app for White Elephant.

First, may you describe your role as lead developer and what about it you find compelling?

Mai — My main responsibility is of course developing. I do as much coding as all the other developers in the company. I also provide support for the junior devs, both technical and practical. Being a lead dev then also adds a few managerial tasks to my plate. With every new project I sit with project managers to provide time estimates, plan out milestones, decide which devs on the team are best suited for certain tasks… things like that. I also lead a weekly dev huddle where the dev team has a chance to catch up, share knowledge, or bring up any issues.

Like, what are we not doing that we should? And, what are we doing that we should not?

Mai continues — I think the most fun part of my job is whenever I get a project I have no idea how to do. It's definitely scary, especially with our timelines always being so tight. But I think any developer would agree that solving a problem for the first time is an amazing feeling. And to add to the fun, there is usually some shiny new piece of tech that we get to experiment on.

Will your upcoming adventures continue to explore the skills you've built in the AR space?

Mai — AR and VR are becoming more and more mainstream, so I feel like that is something I already have an edge on and will definitely continue to keep on top of. I've gained invaluable knowledge working with 3D artists in-house. It's taught me a lot about that field and how their work integrates with mine. Because our work was usually 3D heavy, we used the gaming engine Unity a lot, so I didn't do as much native iOS and android development as I would have liked. So that's something I'd like to take further in terms of skill sets. Outside the technical world I've learned a lot about client interaction and project management, and in general how a studio runs. One of the perks of being a small studio is you get exposed to everything.

After five years working professionally in AR, can you succinctly explain augmented reality?

Mai — It's an experience in which our real world interactions are enhanced by virtual content — but most people just have to see it.

What is your sense of the role that AR plays in our world today?

Mai — AR is certainly contributing to a more plugged-in society. Whether that's good or bad can be argued both ways I suppose. With mobile devices or computers or video games, there is still that divide that lets you step back into the non-digital world. With AR, we're bringing that digital content into real, physical space and allowing them to coexist simultaneously. There's no longer a divide. I think that has potential to dramatically change the way people interact and the way society evolves.

Can you offer any predictions for the future of AR?

Mai — I think AR is moving past its gimmicky stage. It was new to people at first, so the wow effect was enough of an experience. Now, I think we'll start seeing it put to more practical use. I can see AR working its way into our day-to-day life very soon. Whether it's assisting us with mundane tasks or providing us with information quicker, I think there are more useful applications of AR coming.

What thoughts can you share on the state of wearable technology in 2015?

Mai — Wearables are becoming one more platform that studios want to port their products to. If you're making applications, you want to support smart watches. If it's a health app, tying it to a fitness bracelet is almost expected now by users. At Current [Labs] we've recently started working with different types of headsets to create immersive VR experiences. We've experimented with a couple of AR glasses, but the technology is not quite there yet, in my opinion. The experience isn't user-friendly enough and it feels like it's more of a hassle to use them. Until the hardware is less intrusive and the interaction is more seamless, I don't expect people to start jumping on AR glasses the way they did on other (non-AR) wearables.

How is AR distinguished from Virtual Reality (VR) and do you think the advantages of one outweigh the other — now, or in the future?

Mai — AR enhances physical space with digital content, whereas VR replaces it completely. So, in terms of any advantage, it really depends on what you're trying to achieve. For example, in a lot of gaming applications, VR might make more sense. For other cases where the content is more tied to the user's environment (let's say a maps app), then AR is the better choice. Personally, I find both experiences nauseating when either the software or hardware isn't well designed.

For young developers starting out in AR, what are some great resources that we may direct them to?

Mai — I would definitely point them to Qualcomm's Vuforia toolkit. It's a great AR software development kit (SDK) and very popular. I would also point them to Unity3D if they're looking to tie 3D content to their AR experience. Both can be downloaded by anyone and are free to use (with minor limitations). And the great part is that they're both professionally used in the industry, so you're not wasting your time learning with them.

If you have more questions about AR app development, Mai has more info on her website:, and she's on twitter if you want to say hi.

A Look Forward

AR displays may eventually take the form of traditional glasses and contact lenses, with informative graphics that appear in your field of view and react to the movements of your head, in real time. While similar devices and applications already exist, particularly on smartphones like the iPhone, AR screen space may soon be be everywhere. The world is more than ever a playground — soon to include a digital layer of interaction with nearby restaurants, tourist attractions, historical sites, etc. It's up to creatives at the forefront of the technology to imagine the possibilities and design the world we want to live in.

There is such a thing as too much information.

Just as the “CrackBerry” phenomenon and internet addiction are concerns, an over-reliance on augmented reality could mean that people are missing out on what's right in front of them. […] There are also privacy concerns. Image-recognition software coupled with AR will, quite soon, allow us to point our phones at people, even strangers, and instantly see information from their Facebook, Twitter, Amazon, LinkedIn or other online profiles. With most of these services people willingly put information about themselves online, but it may be an unwelcome shock to meet someone, only to have him instantly know so much about your life and background.” — Kevin Bonsor. (2001-2014) How Augmented Reality Works. How Stuff Works.

A Look at Wildlife

Scene one at Animat Habitat is an interactive journey with an elephant, with the goal of creating a connection with our natural world through a gizmo of our media-centric culture. With developer Mai El-Awini and her expertise in AR technology, White Elephant — comic book and companion app is made a more immersive experience than was traditionally possible in print or in animation alone.

Behind the scenes:

Register for membership behind the scenes of White Elephant.