Adam BroitmanCaution: eye-poppingly cool links ahead.

If you haven’t yet heard of augmented reality, or its applications in marketing, it’s time to start working on some serious mind-expansion. AR may be brand new cutting edge technology, but it’s already started creeping into marketing campaigns from organizations ranging from GE to Esquire magazine…even the US Postal Service.

Adam Broitman, partner and ringleader of the New York based agency, has been preaching the AR gospel — and working on some groundbreaking campaigns. We caught up with him to ask what augmented reality is all about, and what marketers should being doing to prepare for this next wave of radical technological possibilities.

Q: So let’s start at the top. What’s augmented reality, and how does it apply to marketing?

Adam Broitman:  The way augmented reality is being used today is not a proper definition of what it’s going to be. It’s overlaying digital information on a live digital feed. So in a football game you’d see the first down line on a football field when the players are playing. That’s not actually there. They’re overlaying information on a live video feed to augment the experience.

Other example of that are war movies when you see additional information that helps them navigate the world, for example. It’s overlaying of data on top of the real world.

Q: So let’s talk actual marketing applications.

The stage that we’re in right now is what my business partner refers to as through-the-looking-glass. You’re seeing the GE example. If you look ecoimagination ( it was done about a year ago. That kind of got everyone buzzing. It wasn’t the first implementation, but it was the first example the mainstream saw. You held up a marker and out came this landscape. It looked like a real landscape, very 3D, very whiz-bang. The reason it became popular was the open source FLARToolKit. That technology allowed for anyone for anyone with Flash and 3D skills to create augmented reality. Marketers are still trying to figure it out while 15 year-olds are in their basements doing this stuff already.

There were a number of examples after that. Papa John’s did an augmentation on a pizza box. My agency looked at that and asked, “Well, how do we take this tech and do something that you couldn’t just do on-screen? What if that piece of paper was the input device for a game, and as you moved the paper around you moved the puzzle piece around, with gravity, you could program gravity into it.”

So it became a game and was a great success for A+E. For the series “The Five Lives of Criss Angel” you were able to move this thing around and have it look like you were doing it with your hand.

Since then everyone started doing it. Always, the tampon company, did it. That’s when you know something like this is going mainstream – hold the marker up and see something happen! We did something for Carl Jr. When you held the marker up Kim Kardashian did something – we took a video feed and turned it into augmented reality. It is fun, it’s engaging, but I like to ask, what does this mean? These forms are not going to last.

There’s no way people will be doing this into next year. The novelty will wear off. Around mid last year mobile augmented reality began gaining traction. The adoption of Android and increasing adoption of its open source architecture allows you to program within the environment. Apple doesn’t give you deep enough access. So a couple of companies came out, one is Layar. Essentially what they did was create an augmented reality browser whereby you’d hold up your phone, it would take the coordinates of where you are, and you could look around and point it on various homes or a specific building and get an overlay: is this on the market? How much is it going for? And all kinds of other information about that market.

In five years we’ll look back and we’ll laugh that we used to type words into Google. Cut out the steps. If all you have to do is hold your camera up and you get all the info you need, why wouldn’t you do that? People are still talking about the buzz and the cool factor. What this really means is we’re moving toward a new model. QR codes are simply a stepping stone to an augmented reality future. But we seem to skip steps. If augmented reality technology takes off, why do you need a QR code?

It’s not augmented reality, but Tesco has a wine app in which they’ve built a database. You still need to take a picture of a bottle of wine, send it, and have it return the information because it recognizes hat image. Imagine this is information overlaid on a live video stream, so take out the steps of a picture and a server call and cut right to the chase: this is what it is, how much it costs, where to get it, etc. It’s pretty profound when you think about it.

People say it seems like a gimmick. It’s not a gimmick. And so what if it is? Since when was it wrong to have some gimmicks in marketing? Rich media banner ads – what were they?

Q: In the future, what are the real possibilities for marketers with augmented reality?

AB: The through-the-looking-glass phase is still not done. There are also brand utility phase applications. Say you make eyeglasses and you’re selling them online. Wouldn’t it help if you could see yourself in those eyeglasses? Rather than taking a picture of yourself and uploading it, you could just do an overlay on a webcam image.

Q: Ditto makeup, hair color, fashion. We’ve seen a lot of apps that do this, but nothing live. They’ve always been much more cumbersome.

AB: That will be the coup here, the ability to make it simple. We’ve seen a lot of cool technology but consumer adoption is so low because no one wants to do the work to get it done. We’re trying to make it real simple. There’s an implicit simplicity in this technology if I say all you have to do is buy this lipstick or this hat is look into your webcam. We’re not there yet, but a year from now the conversion rates online will go up tremendously.

I don’t shop for clothing online now. With this you won’t know how things fit, but you’ll know how they look.

Q: What other applications do you see this working in?

Fashion and beauty. Entertainment and gaming are a no-brainer. The ability to make offline or traditional media extendable, like Esquire’s efforts (video demo here). Ultimately I don’t think it’s different from saying, “go to the website,” though I applaud them for the effort. It does have a certain appeal that would work for children and learning – they can hold a book up to the webcam and get a reveal.

Mobile and local are really where it will become important. When you think about the information Google already has on the world, and Google maps, Google earth, these are getting exponentially and further tagged with deeper information. Imagine putting on a pair of glasses as you walk down the street and getting information from the world as you move through it. It’s almost like SEO for the whole world. It’s just like search when, if there’s an instance where they want to buy something it there, it will be that way with augmented reality. It’s the hand raises, and you have to be there.

Five years from now, this is what will be the next generation of search engines.

Q: What are the practical steps for getting started?

AB: Learning at its core what augmented reality is. If you’re looking to building buzz there are always creative ways to do cool things. We’re working on a project now I can’t tell you too much about, but imagine you’re going car shopping and you were able to put on glasses that would take you on a virtual tour, kind of exploding the engine or enabling you to look at various elements. It would help sell the car and make the selling more interesting. BMW did a digital overlay of an engine to help an engineer when he’s working on an engine that says things like remove screws here. For marketers there’s an utility in being able to add information to the sales process. I don’t have the answers for every vertical yet. In entertainment there are story telling elements. For CPG it’s a little harder. I have yet to pick up the exact application for holding your phone up to a box of Cheerios. Beyond the wow factor, where is the utility? I challenge other marketers to think about this. Creativity is still at play, but so is familiarizing yourself with the toolkits and what they can and cannot do.

We’ve done some of it and we’re going to be doing a lot more. What can and cannot do. The difference between marker and markerless augmented reality is important. The next project we’re working on will not require a marker. You need a huge payoff to get somebody to print out a sheet and hold it up to a webcam.

So those are a number of example of things marketers should be thinking of today. Every marketer should have an information in everything strategy. All forward thinking marketers should be preparing to think about how their product can be represented anywhere. Think about the product in the outernet, if you will. People can overlay digital graffiti on the real world. Things like that will become a reality.

Q: Back to CPG for a moment. Since you were saying there aren’t good AR solutions out there now, what did Always do with tampons, anyway?

AB: All they really did was you held up the piece of paper and the commercial came up. It was a 3D banner. Adidas just came up with a campaign where the shoes were augmented and this amazing world came out of your shoes. The IAB hasn’t made standards of what you’re allowed to do with this.

The US Postal Service created a campaign in which what you did was hold up the marker and you can get different size boxes so you know how big that box is relative to your person. It was at least a clever application.

Q: Who are the players in the space?

Aside from the creative shops there are software companies. Total Immersion and Metaio. Those are the companies that are leading the charge in terms of software. There are still skills a lot of agencies don’t keep in-house. Not everyone has 3D designers and developers who know how to use these platforms.

I would realistically say 10 years when the idea of typing into Google will seem ridiculous. Everyone right now is talking about the real-time Web, but this is the real-time information layer on top of the world.

Adam will be discussion AR at aNew York event on Jan. 21.