Today Metaio is holding Inside AR in Munich, Germany.   Metaio (the picture above shows Metaio co-founders Thomas Alt and Peter Meier), is behind some of the best known commercial and industrial AR experiences of recent years.  But as important as the many AR projects they have executed are the AR tools that Metaio has made available to developers.  Metaio’s AR products and tools have played an important role in bringing AR to a wider public, and given many developers the opportunity to explore AR.

Inside AR is a great opportunity to see what these AR pioneers  will be up to in the coming months.   I could not make it to Munich this year.  But, fortunately, I had the opportunity to talk with Thomas Alt, recently.   In this conversation – see below, I got a chance to discuss what was going on inside AR with Metaio.

The Fall season is always jam packed with great events, and I wish I could be in two places at once.  But this week, I will be in my home town, NYC, attending Web 2.0 Expo which, reflecting the heat in the NYC tech community, is a sold out event with a very exciting schedule this year (more on some of the presentations that I will be attending later in this post).  If you missed out on tickets to Web 2.0 Expo, all Keynotes will be Streamed Live: TUES 9/28 to THURS 9/30, and keep your eye on @w2e and #w2e on twitter.

Meanwhile, I am missing Inside AR, which had some great speakers lined up, including fellow New Yorker, John Swords, partner and Ringleader at Circ.us.  Hopefully, Swords will share his experiences at next month’s ARNY Meetup which will be “joining forces with another vibrant community – NY Gaming – for an unforgettable night of Augmented Reality Games” on Tuesday, Oct 19th, 6:30 PM at AOL Ventures in New York, NY.

At the most recent ARNY @swords gave a brilliant talk on the possibilities for AR Game development on the newly available opensource Parrot ARDrone platform.  It was great to hear from social game guru @swords on his plans for Parrot ARDrone games, and more.  The picture below of an ARDrone camera view is from John Swords Flickr set.  Swords was flying it inside his garage because the winds outside were too strong.

Also, I kicked off what will hopefully be an ongoing discussion on, “Story Telling with AR and the Big ARNY a collaborative AR Game for NY,” with a few slides.  I have opened up the presentation document for collaboration, so please ping me if you would like to be added as a contributor/editor, and are interested in getting involved.

Ori Inbar, CEO and co-founder of Ogmento, Games Alfresco, ARNY and my co-chair on Augmented Reality Event 2010, suggested The Big ARNY – A Collaborative AR Game Development Project modelled after A Swarm of Angels last year at the First ARNY Meetup – so let’s make it happen!  I will be catching up with Ori in October about what Ogmento has been up to since they became the first VC backed AR Game company!

“Games allow us to see each other, for a moment, in a way that living in a city prevents” – Kevin Slavin

I believe that, AR, to get beyond the stage of “interface du jour” needs to offer us new ways to relate to each other and the world around us so that we can actually improve  and deepen our engagement with reality not just create experiences that are primarily optical  (see James Turner’s interview with Kevin Slavin “Reality has a gaming layer” on not letting “the pleasure of a game and the meaning of a game and the experience of a game rest primarily in the optics.“  And see my recent post, Urban Augmented Realities and Social Augmentations that Matter: Talking with Bruce Sterling, Part 2).

Two of the most inspired creators of urban games,  Kevin Slavin and Kati London of Area/Code will be speaking at Web 2.0. Expo tomorrow.   And you can be sure I will be at both these sessions. Loitering on the Motherboard, Kevin Slavin, is 2:35pm Tuesday, 09/28/2010, and Games that Know Where you Live, Kati London – is a keynote that will also be live streamed – 4:55pm Tuesday, 09/28/2010

Recently Kevin Slavin was interviewed by James Turner, on O’Reilly Radar.   This, Reality has a gaming layar, is a must read piece about a “world where games shape life and life shapes games”  (see @timoreilly). 

Interview with Thomas Alt

Tish Shute: Perhaps you could just start with your background Thomas because I think there’s a lot of newcomers to AR but you are really one of the first movers in commercial AR. How long you’ve been involved in this?

Thomas Alt: Actually I’m an ex-researcher in augmented reality. I started with me actually after getting my master’s work in engineering from the Technical University of Munich working for a big company called Volkswagen. And at that time,1999, we got a research grant for researching how augmented reality could change manufacturing processes in the automobile industry.

And from the research work there, I basically went back to school, did my PhD about augmented reality. And while speaking at a conference, I met Peter Meier who is the co-founder of the company who was also a master’s student writing his thesis about augmented reality. That was in 2002.

And so it really was in the very early days of augmented reality. And both Peter and myself we got really excited about the technology; we saw endless possibilities. We said, ‘OK. Let’s just found a company. We actually founded the company in early 2003 with virtually no money. As a matter of fact the founding capital of the company was 25,000 Euros and this 25,000 Euros were won in a case competition in Germany – a business plan competition.

Tish Shute: So you won 25,000 Euros on this case competition and that’s where Metaio started….

Thomas Alt: Exactly. And to legally found a company in Germany it takes exactly 25,000 Euros so that was the founding capital. We started pretty much like good old SAP started.  It wasn’t in a garage though it was a very small office and we basically built up the business through work, so we don’t have any investors or whatever. Right now we are 66 people located in Munich where our headquarters have been for five years. We have some presence in the US, and we have a venture company in Seoul, South Korea.

Tish Shute: Awesome. I just noticed how fast you’ve been growing. So right now, I’m going to ask a couple of questions about where you see the technology and the emerging industry going.

First, what are the platform of choice for Mobile Augmented Reality at the moment?

Thomas Alt: Obviously in the cellphone hardware space there’s a fierce competition going on. It’s yet to be defined what will be the prevailing platform right now, obviously it’s the iPhone is big now, right? But Android is catching on very, very fast.

Tish Shute: You have pioneered bringing a cross platform SDK for vision assisted AR to a wide community of developers with Junaio and with your partnership with Kooaba – a visual search company from Europe.

Thomas Alt: Yes, yes, and this is how we would, also in the future like to position ourselves with Junaio.  Junaio will be a platform, a technology platform, which will allow users to do whatever they want to do in augmented reality. The API of Junaio is huge in the sense you can do anything from outdoor gaming, to visual search, to normal, uh, lay out style, you know, find the next burger king a mile away kind of super impositions.

Tish Shute: The only licensing you pay is for unifeye right? When you want to use your tool kit right?

Thomas Alt: Yes and this is how we’re distinguishing it. Junaio is our consumer brand targeting newbie AR developers, with limited programming skills, while the Unifeye platform is really our B to B platform where B to B customers can create their individual AR experiences.

Tish Shute: Yes which is what my friend Patrick O’Shaughnessey, Patched Reality, did for the Ben and Jerry’s app he created using Unifeye.

Thomas Alt: exactly…

Tish Shute: It is a lot of work developing for so different mobile platforms isn’t it. Junaio is on Android and iphone but you havent moved Junaio to Symbian?

Thomas Alt: To be honest with you right now its a matter of priorities we have other things we want to do first. And from analyzing the user base, iphone was a big step Android was a big step and now we are pretty much seeing what is happening next. As you know Nokia going into different directions as far as their smart phone operating system goes, and so on and so forth.  There are also capacity constraints. And right now obviously the most – potentially not the most possible users, but the users most inclined to do AR on a day to day basis are the ones using the iphone and android devices. But obviously there are a lot bigger cellphone manufacturers out there.  But just you know even the mobile web users aren’t as strong as the users on the iphone and android devices?

Tish Shute: So what do you think the iphone 4 has that brought to the AR picture?

Thomas Alt: Very fast camera access, very good for marker recognition. If you go to the Metaio site you’ll find a movie where we show on the iphone 4 app for a real augmented reality Leggo peice – this is something which is very nice

Tish Shute: Yes I see that, yes that is nice, yes, yes very nice. The Unifeye SDK is really putting markerless AR into the mainstream.

Thomas Alt: Yesterday we launched the first, a err very nice shopping… shopping solution for , but that’s completely external.

Tish Shute: Oh yes – the augmented reality shopping for seventeen.com, i was going to ask you about that, because it is the first augmented reality online shopping using natural feature tracking.

Also I am very excited to see the gestural interface, awesome!

The seventeen augmented reality shopping app is a PC experience but are you working on developing gestural interfaces for mobile AR?

Thomas Alt: We are continually pushing the envelope of what’s possible with AR. Gestural interfaces for mobile AR is certainly the next step in taking what we’ve done on the PC and making it more portable by using the mobile platform. One thing to keep in mind here is the limitations of mobile platforms and size of the screen needs to fit and make sense for the user experience.

Tish Shute: I know you started off as an AR researcher (although as you mentioned earlier you have been working in commercial AR and building Metaio for a long while now.

So in addition to how we are progressing towards gestural interfaces for augmented reality, I would like to ask some questions about AR eyewear. We won’t really have hands free AR without eyewear. What is your projection on when we will see consumer AR eyewear? And, Do you have a any comments on those speculating that we will not see AR eyewear go mainstream for 20years?! What is Metaio doing to move eyewear technology along?

Thomas Alt: Well as you know, it’s always, you know on the technological roadmap, and we’re still doing research projects, in our AR research department. We have worked on things like calibrating eyewear for augmented reality. There is some nice development there.

But really, commercially, the whole thing with eyewear has never caught on on a level which would make it a valuable avenue, business avenue, at least for Metaio. So, I guess as an ex-researcher, it’s still a very interesting, a very good technology. And it would definitely change the marketplace radically when available. But as per right now, there are very few commercial applications.

Tish Shute: Are the obstacles to AR eyewear technological obstacles or is it just a question of a a business model. I mean is it realistic to see eyewear in the next 3 to 5 years at a price point affordable to consumers, where you really, truly can have eye tracking? You know, the whole problem there was with virtual reality and eyewear giving people a headache. How far have we come in terms of the technology?

Thomas Alt: Well, it’s not so much technological factors ’cause all fundamental problems are solved. It’s more a rather large corporation, I guess, would have to step up to the plate and say okay, do, let’s get all the state of the art in electronics and develop just a perfect HMD.

Tish Shute: Something Yohan Baillot’s company Simulation 3D is doing at is looking at is hooking up eyewear to smart phones.

Thomas Alt: Yes exactly that would be even better. Metaio has done a strategic move into this HMD space for augmented reality about a year ago by acquiring a bankrupt company. I mean, we had considerable IP around it from our research base but in the long term we still believe in it and we did a move about a year ago in buying what was left over from a bankrupt company including a lot of IP, which basically goes into the direction of mobile augmented reality but also mobile augmented reality in connection with head mounted displays.

There’s actually a press release about this but that’s about a year ago…

I know that the whole HMD thing… I mean, I’ve seen companies come and go. Metaio has worked previously, very closely, with Microvision of Seattle. We have worked with a German company, doing HMDs and we have worked with Vuzix.  We are still working with Vuzix, so we’re still consider it very valuable.  But right now, I mean, it’s just not a big part of our commercial pipeline, to put it that way.

Tish Shute: It was interesting what Bruce Sterling said in his keynote at ARE 2010.  He actually made a strong case for why smart phone augmented reality may be more interesting because it’s less immersive. I mean, he raised the question of the fact that if you really truly had AR eyewear and HMDs you’d re-enter the world of virtual reality or as he called it AR’s Gothic step sister VR would rise from the grave….

Thomas Alt: Yeah, well, that’s a cultural or even a philosophical question and we have discussed it a lot, especially in the industrial domain. Also will the deployment of HMDs come about from end consumers using it in their spare time, or from professional users using the idea in their work time?

Tish Shute: Do you think it surprised people who have been working in augmented reality research how much people have engaged with the idea of smart phones as the mediating device for AR, and that rather than having the always on experience that eyewear would give us, we use smart phones as a magic lens of a smart phone when we need to or want to. Some people were skeptical that anyone would want to hold up a little window to look at augmentations of the world – a magic lens. I mean, it wasn’t self-evident that that would be an experience people enjoyed, and it turns out that it was.

Thomas Alt: That’s actually a very good analogy. And also in my view, I mean, certain behaviors just change also, right? I mean, this is exactly what Apple’s trying with the iPad, right? You’re taking the iPad, and all of the sudden you’re not constrained to a laptop or whatever. And it’s truly a companion of the couch, in-bed Web, in the kitchen, and so on and so forth. So digital usage with the iPad, which is a different market, and I’m aware of that but as an example, the iPad has changed our behavior. And obviously, the augmented reality guys are hoping that something similar happens with AR.

Tish Shute: Which of course brings up the question, I’m assuming that some of the next generation of slates/ipads are going to have front and back cameras, GPS, and compass, right?

Thomas Alt: Actually we know that.

Tish Shute: You know that, yes. I assume that you know that, because are you working on some prototypes, and have you got some plans?

Thomas Alt: You have to understand that I cannot talk as I’ve talked as a researcher. It’s the rules, so I have to be a little bit careful about what I say. We very much think that a webpad, or whatever pad, you would want to call it is on some occasions very good device for AR.

Tish Shute: Yeah. But it is an interesting point with holding up a larger device, because you hands aren’t free, but the neat thing about the phone for augmented reality is that you really can do a lot with your thumb, as we’ve found and just the position of the phone. But, how will this work it with the two hands on the larger device?

Thomas Alt: Keep in mind, everyone’s talking about mobile augmented reality, but really where the case for augmented reality, at this point, is the strongest is in the installation business, it’s in the web business… Not necessarily only commercially, but also use case-wise. There are tons of museums out there which are using our augmented reality system in an installation fashion, and to communicate products better, and more efficiently, and so on, and so forth.

So, I know that the hype is clearly on the mobile augmented reality side, but there are many examples augmented reality experiences where holding up a larger device is not a big obstacle.

Tish Shute: Well this brings me to some questions about the future of mobile AR. My interview with Jay Wright focused on how we are now in a new period for AR bringing together computer vision, visual search into a mobile stack that is really optimized for AR. What do you see emerging in mobile AR as we move beyond compass, GPS, camera, accelerometer based AR into markless image-based AR. What will the new use cases and where will we see mainstream users getting in AR. Will AR games be the first mainstream AR experiences?

Thomas Alt: My partner is actually, first of all, one of my best friends, second of all, very emotional, third of all, very intelligent, and he said the other day something I think very valuable in this area. He said, basically think about Mobile Augmented Reality, Thomas. There’s really a very limited number of use cases which you can do if you look at these classical Point and Find applications, ok? But there are almost unlimited number of use cases when Augmented Reality becomes a day to day companion, ok? So what he meant is, ok, I’m looking at my normal day… I’m looking at the city, I’m walking throughout this, I’m coming home, I’m having dinner basically. I can deploy Augmented Reality in a pure POI search fashion perhaps not even once. Ok when I’m travelling it’s a different story, but in an ordinary day I might use a POI search not even once.

But where this ultimately leads, you know, is even in the 15 minutes I’m having breakfast, I’m using AR – looking at the cereal box with my cell phone, I’m taking part in a sweepstakes or whatever. So from that we draw the conclusion that as a general strategy for Junaio, we should basically throw as much technology as possible into Junaio, make it halfway self-explanatory, and just give people the possibility to come up with ideas on how to deploy augmented reality continuously.

We have actually got a creative team from an Art School working on that, and just, you know, with very little programming skills coming up with things you can do with Augmented Reality on a day-to-day level. And it could be a scavenger hunt game, in the city, with monsters flying around, it could be the normal POI routine, it could be marketing purposes, and so on and so forth. And I think that’s really the roadmap, and this is a little bit similar on a more technical level, to what Qualcomm is doing, ’cause they’re floating out possibilities or capabilities I want to call them, and Metaio is doing that, but on a higher level [re the tools] meaning on a Junaio level.

Junaio is a capability platform. It is also a way for Metaio to demonstrate the capabilities of our technology. We will offer all the possibilities for AR and more that we have already demonstrated on PC augmented reality.

Tish Shute: What is the business model for Junaio? Are you encouraging developers to develop business on your platform ?

Thomas Alt: Junaio is our end consumer platform and our business model is similar to the way Google structures its business model. We work with OEMs, content partners, brands, and developers to offer free content to our end users. Where we do charge is on the advertising side, more specifically contextual and location based advertising. At the current stage, we are focused on building the content base, fostering our developer community, but in the near future, we will be introducing advertising channels.

First of all you have to have very good use cases in the platform basically. And then to put a business model on top of that from a technology stand point is not hard – its a pay channel. Its all prepared for this.

Tish Shute: You have quite a broad base as a company don’t you – you do everything from industrial AR, marketing to technology licensing and more?

Thomas Alt: Basically, there’s a lot of things people don’t see us. There is also the Unifeye PC SDK and we have a client base and partners who are sourcing software from us, and we are doing great pieces. I mean the hype of augmented reality is really coming to a peak. There are lots of pieces that are not even talked about any more. China’s GQ magazine launched with AR from Metaio, the biggest AR campaign anywhere – there are a lot of potential readers in China. And um, so that’s our business model… we have our IP, our patents and so on. And on this we can move on onto the mobile platform whenever it’s advisable or feasible.

Tish Shute: Right. Yeah. I mean uh, you’re very fortunate to have this base built on uh, years of developing IP. What are the most important areas of AR that Metaio holds IP and patents in, in your view?

Thomas Alt: there’s sleepless nights in that too…

So far we’re extremely excited about what’s going on with Junaio, it’s one of our big, big success stories. But we are sensible and trying to experiment because, you know, analogies from the past won’t really work in my view for Augmented Reality in a sense, that, you know, you better bring for a new system to fly, for a new technology to fly, you better bring a very concrete use case to the table, ok? And a well-defined use case. And we are, right now, with Junaio, in a state where we are checking out what could be such a very defined use case.

Tish Shute: So how many users does Junaio have now?

Thomas Alt: Let’s put it that way, we are, especially in the last 2 months, we are very satisfied. But we are not disclosing that, because users, and we’re seeing that from the competitive landscape, always needs 1 page of description what exactly a user is.

You understand what I’m saying? So this is why, ’cause we don’t want to up or downplay things, we are very careful saying, with users. Because I mean we have people who are actually also commercially very interested in Junaio…  We go through with them and discuss what exactly a user is. Cause there’s more then…a download is not a user. An app or something on your phone is not a user, basically, in my opinion.

Tish Shute: I am still waiting to see someone do something with AR and the Four Square API, or now the Facebook Places API. Do you see an interesting potential in the marriage of the rapidly emerging location based social networking space and augmented reality?

Thomas Alt: Definitely. Augmented reality offers a way for users to find information around them easily. Adding in the social networking component such as geo-tagging, rating, commenting can enhance the user experience and create engagement beyond just viewing the information. For example, within junaio, an average user can create their own personal channel and geo-tag photos or leave text messages at different locations. They can create a virtual tour of San Francisco and share it with their friends. By connecting the social side with good content, the augmented reality experience becomes more fun and interactive.

Tish Shute: And, of course, there’s the Junaio API. Are you beginning to see developers use that?

Thomas Alt: Yeah exactly, I mean if you go to Junaio.com, you can get a login, you have an API description. And the way it works is, that you bascially set up a call, which contains the information you would like to have in your individual channel. You submit it to us, it will get checked for profanity and other things, basically. And then we admit it to Junaio basically. The API is huge.   You can also use Junaio indoors.

This is very relevant. And there’s a tool chain for that, and so on and so forth. You can do mission-based search with Junaio. It’s in there, it’s called Junaio Glue. And there will be another very interesting feature coming up in a couple of weeks. And you can just do it, you can do a scavenger hunt, a game, normal POI search, and so on and so forth. And it’s all active. And that’s, what’s sometimes difficult for us to communicate, is it’s really a capabilities platform, but on the other hand it’s obviously very good to developers. And I mean on the developers side there’s huge interest.

Recently, I talked to Ori Inbar (above), formerly senior vice- president at SAP.  Ori is on a mission to make augmented reality commercially successful not in 5, 10, or 15 years, but now. Ori is the founder of Pookatak Games – a video game company, “with a vision to upgrade the way people experience the world.” Ori will be participating May 20th, in O’Reilly’s Where 2.0 panel, “Mobile Reality” -  an event not to be missed IMO.

The taste for computing anywhere anytime has entered human culture via the iphone and is spreading like chocolate cake and pizza at a preschool party (see why the iPhone changed everything).  And while the full flowering of the next step is yet to come – computing anywhere, anytime by anyone and anything (“the internet of things”), our love for these first devices capable of being mediating artifacts for ubiquitous computing (Adam Greenfield) is a vital first step to free us from our tethers to computer screens, and fulfill the promise of augmented reality.

If you need more convincing on the pivotal role augmented reality will play as the web moves into the world, check out Tim O’Reilly’s recent comments in this video clip posted on Augmented Times and here early last year.

From another perspective, the gloomy specter of economic and environmental catastrophe  is driving a movement to “infuse intelligence into the way the world work’s.” But the challenge for a smart planet is not just about making environments smart, it is about using smart environments to enable people to act smarter (see my interview with Adam Greenfield).

We need a rapid upgrade in both the way the world works, and the way we experience the world.

((Note:  It is time to read (if you haven’t already) Bruce Sterling’s Caryatids (Cory Doctorow’s book of the year for 2009) “as a software design manual” (see Julian Bleeker) because Caryatids reveals the Gordian knots of human folly, greed, compassion and desire entwined in near future designs for technologies to save the world.))

Ori Inbar, worked with Shai Agassi (Shai is now leading the world changing Better Place ) driving Netweaver from a mere concept to a “major, major business for SAP.” So Ori has already been through the cycle of working in a very small startup and growing it into a billion dollar business.  He has both the experience and the passion to realize his vision for augmented reality.

At Pookatak, he explains :

“We design “reality experiences” that make users’ immediate environments more significant to them. We wish to free young and old from getting lost in front of the screen. By delivering the world’s information to people’s field of view, and by weaving real world objects into interactive narratives, we help people rediscover the real world.”

Pookatak will release their first game this summer. Currently it is under wraps. But Ori gives us some glimpses of what is to come in the interview below.

In addition to founding Pookatak, Ori is involved in a broader effort to move augmented reality forward. On his blog, Games Alfresco – he recently welcomed a new partner, Rouli Nir, Ori has focused his eye of wisdom on every significant recent advance in Augmented Reality (check out this essence of Ori’s thinking in a fast paced video presentation for WARM ‘09).

Also Ori is one of the organizers of the interactive media track at ISMAR 2009.  At ISMAR this year, Ori explained, “we are trying to bring in people that develop interactive experiences for consumers, beyond the traditional attendees coming from a research perspective.”

In the interview below, Ori explains much of his thinking on how augmented reality will become commercially successful.  Enjoy it, think about it, and share it. And most importantly, if you can, get involved with ISMAR 2009.

Ori  has inspired me to participate in ISMAR this year.  Ori pointed out:

The call for papers is on, and this year it targets well beyond the typical research papers audience and into interactive media and art folks.

There are plenty of opportunities such as:

Art Gallery

Demonstrations

Tutorial

Workshops

It’s a huge opportunity to shape the emergence of augmented reality.

Interview With Ori Inbar

Making Augmented Reality Commercially Successful

Tish Shute: You are considered a key trail blazer in AR and you have the go to blog for augmented reality!  What are the most important lessons you have learned researching, writing, and developing AR in the last couple of years?

Ori Inbar: You need to have a vision. You need to know where this is going to go in ten or fifteen or twenty years. But you’ve got to start with something really simple that makes use of the technology you have on hand. And do something that is practical, that people will like, and something they would actually want to buy. Its as simple as that. I’m currently looking at what we could do with existing technology. First of all, you have to put it in front of people. Right now most people have never heard about the term augmented reality. Go into the street, and ask 100 people about it, maybe 2 would know about it. So you need to put it in front of people because most people think it’s still science fiction or a special effect you see in movies, not something you can experience in real life.

Tish: It seems to me to that for augmented reality applications to become popular with existing technology the key breakthrough would be getting people to hold up their phones. What are the obstacles to getting people to use their mobile devices like this?

Ori: There’s a really nice cartoon by Tonchidot (below) – the Japanese company behind the Sekai Camera. It’s an illustration showing the evolution of man, from ape to man (holding a cell phone looking down), to the developed man holding a device like a camera – in front of its eyes.

Which is exactly what you’re talking about. People ask, “are people going to walk with this like that all day long?” Probably not. I mean you have to build it in a way that doesn’t require them to hold it like that all the time. People are used to this gesture with the ubiquitous digital cameras. I tested one of my prototypes on a two and a half year old girl. She had no problem holding it just like she holds a camera.

Tish: Blair MacIntyre mentioned, “The problem with the mobile phone as a AR device is a problem of awareness,” i.e., you have to have a way of letting people know when there’s something interesting wherever they are. One of the issues regarding this is if you get too many alerts, then you tune them out.

Ori: First of all Blair is one of the people in academia that get it. Because he looks at it from an experience perspective. Not just as an interesting technical problem to solve. Let’s start with getting people to enjoy this new experience. The AR demos so far were mostly eye candies, and mostly for advertising – the GE AR ad created a lot of buzz; but you look at it for 10 seconds and you forget about it.  You need to build something that people would want to experience over time and would be willing to pay for. I think that’s the big test, right?

Now in terms of having a ubiquitous experience where you’re continously connected, it doesn’t have to be an overwhelming experience. Just like some of the social media tools we’re using today, we decide when to connect, and we filter out the trash. You could get alerts only for things that really matter to you, not for everything that happens in your immediate environment.

There will be many layers of information, and it’ll be up to you to pick the ones you want to experience. The real benefit is that you get the information in your own field of view and in context of where you are or what you do.

Tish: So what are you working on these days?

Ori: We are working on a little app that targets a very different audience than what you’d expect: pre schoolers. We think we can encourage them to get away from a PC or TV screen and learn something while playing – in the real world. You’ll hear more about it as soon as this summer. Nuff said.

But, it is a small application that will run on the iPhone. People ask how many pre-schoolers own iPhones? Well, their parents do.

Tish: Yes there are certainly many New York kids with iPhones – my kid now has my old iphone.  He has pretty much switched from playing games on his DS to the iPhone. I noticed in your WARM video you place a big emphasis on AR as something that will get kids away from screens and engaged with reality.  This is something parents will approve of!

Ori: Yes I saw something really interesting at my kids’ party one day; they were all sitting around the room – looking down at their own DS screens.  You could play the DS anywhere, but kids would usually play it on the sofa, looking at the screen, isolated from the world. With an iPhone and a camera, and the application we’re producing, reality becomes part of the game. Yes that makes it all of a sudden much more interesting for parents. Because kids are spending so much time in front of the screen, all of a sudden they’re something that will encourage them to interact with real objects, real things. Every parent I’ve talked to loves that idea.

Tish: Yes that is what is cool about the work of Kati London – I think I saw someone say this on Twitter, “Kati puts the computer in the game not the game in the computer.”

Ori: Yes, kids are spending more time in front of games and the computer because it’s more interesting. It captivates them with “game pleasures ” that tap into their brain’s dopamine circuitry – constantly seeking reward and satisfaction. So you’re not going to be able to tell them to go back to playing in reality without these pleasures. We have to study these mechanics from games and bring them into reality. It’s about programming real life; and augmented reality helps you achieve that.

Here’s an example: cause and effect; in a game when you do something you always get an immediate effect. You’re good, you get a reward. You’re not good, you get a cue to improve. In real life you do things and you could wait 2 or 3 years until you actually get feedback (if you’re lucky). Augmented Reality allows you to bring these mechanics into the real world. I think that’s going to help kids rediscover reality, in a new sense, which is what every parent is dreaming about.

Tish: I don’t know how much you can say about your app. But in regard to doing augmented reality on the iPhone.. there’s no compass. Is this a limitation?

Ori: True, no compass yet. But the camera gives you a lot of information that you can interact with. When you run the application, you see the world in front of you, and if the app can recognize real life objects – it can put virtual elements on top of it.

Tish: But not with any accuracy unless you’re using markers. Are you using markers?

Ori: We’re using natural feature recognition. It doesn’t have to be an ugly looking marker. It can be any image.

Tish: So you’re using image recognition. Are you working with one of these image recognition startup companies (list here )?

Ori: We’re working with one of those. What’s unique about it is it runs very nicely on any cell phone, and on the iPhone it works the best. For this first app, it doesn’t really matter where you are physically; the geolocation is not part of the experience.

Tish: For a truly engaging AR experience we will need more of a backend than is currently available?

Ori: I call the backend the cloud, where you have all this information and ways to access it from anywhere. Actually I think it’s become pretty mature today. If you look at the different elements required to enable an augmented reality experience to work, you have – first – the user whose always in the center. Then you have the lens. The lens can be an iPhone, or glasses, even a projector. The lens allows you to watch, sense and track information in the real world: people, places, things. Then in the backend you have the cloud where you store and retrieve information.

So if you look at the maturity of these different elements, I think the cloud is in pretty good shape. Because there’s so much information we’re collecting and storing. Anything from Google, Wikipedia, Facebook, all that kind of stuff, it’s a lot of useful information you can access from anywhere using APIs. And a lot of it is also starting to include geolocation information. Take Loopt or Google’s friends service that allows you to see where your friends are and what they’re doing. There’s tons of information out there and it’s pretty easy to access it. Now what do you do with it is the question?

Wikitude is such a simple and brilliant application and nobody thought about doing it until this guy from Salzburg did. It doesn’t have any sophisticated visual tracking. It knows your position and it’s simply looking at the angle you’re pointing to. Based on these parameters it brings information from Wikipedia that pertains to your field of view. So most of it was already there. It’s just a matter of connecting the pieces in an experience that is valuable for people.

Tish: It is the uptake of even a very simple technology that puts the magic in it.

Ori:  Yes, take Twitter. If you go to its homepage it looks like a very simple boring app but it is something that is both enjoyable and very useful to people.

Why you should participate in ISMAR 2009

Tish: I know that you are involved in organizing  ISMAR (picture above from Ori’s post on “ISMAR 2009: The World’s Best Augmented Reality Event…,“) and there is a call out for papers and for volunteers, can you tell me more about it?

Ori: Yes, we hope to have the first ISMAR where we practice what we have just discussed: let’s build on all the research invested so far and instead of thinking only about 5-10 years from now, let’s see what we can do today. So we are bringing people in from other disciplines – artists, interactive media developers and people from the entertainment industry.  The goal is to use the technology to make something interesting for people – again, something that people would buy, and making it commercially successful.  Many people either don’t know about ISMAR because in the past it was a pure engineering-orientated event and people  from a commercial perspective of AR weren’t attracted to it.  The Chair of the Event this year is based in Florida and he is going to bring in a lot of people from the entertainment industry such as Disney. I think this will transform this event into something more like SIGGRAPH – more of an industry event.  As one of the organizers of the interactive media track we are trying to bring in people that want to build applications for consumers.

Tish: In terms of AR applications what are the flagships today?

Ori: There are very few because it’s just the beginning. There’s one tiny studio in France called Int 13 . They’ve created maybe the first commercial game running on a mobile device using AR technology. It’s called Kweekies. It was one of the contenders for the Nokia Mobile innovation awards. They were one of the ten finalists, but they didn’t win it. It’s looks really cool. It’s somethng that runs on your desk, with a marker. Many AR folks say markers are the past, markers are ugly. But it’s still a cool experience. I think people will go for it.

Tish: Yes I think we will have to look to small companies that are free to think creatively to lead the way.  It seems many games companies are tied up pulling off huge big budget projects and enterprise is still catching up on how to use social media!

Ori: Yes, last year I was in the game development conference (GDC); there was no mention of augmented reality – not on the exhibition floor, none of the sessions, nobody talked about it. I was stunned. Then this year, there was a little a change. There were like three demos on the exhibition floor, Metaio, Vuzix and a Dutch company called Beyond Reality.  And then there was Blair’s talk, which was very very cool. The room was packed with people. And after the talk there were dozens of people lining up to talk with him about the topic. There was definitely interest, but still on the very edge. The video game industry is still a hit driven business and publishers spend upward of 20-30 million dollar to create the best AAA game possible. They just can’t take the risk. So it’s going to come from smaller companies, from outsiders coming in with a vision and understanding on how to put the AR pieces together to create a totally new experience.

Tish: But the basic tool set is there isn’t it?

Ori: I talked to some folks at the games developer conference, many folks with MMO background, and they have great ideas about AR. It’s great to see different people with different views on what’s needed first. “Joe the Programmer” had this idea of creating a small piece of hardware that you can put in every house and provide accurate geospatial information in your home. That could  open up many opportunities for AR experiences in homes.

Tish: Don’t you think we have enormous resources in terms of image databases that provide a great basis for augmented reality.  I was talking to Aaron Cope at ETech about The Shape of Alpha – Flickr’s vernacular mapping project using all the geotagged photos in Flickr. That is such cool project. Aaron will be speaking at Where 2.0 also.

Ori: Think of Google Earth. Google Earth leveraged communities to basically map all the major cities around the world into 3D models. And that is an essential step to be able to do augmented reality outdoors. Because if you had to model everything from scratch, it wouldn’t be realistic.

Augmented Reality and Becoming Greener.

Tish: I am really interested in how AR interfaces might be useful to some of the emerging energy identity/metering projects like AMEE and WATTZON because I think it is very important that people have very intuitive, immediate, and enjoyable ways to relate to energy data so they can make greener choices.

Ori: Back in the day I had an idea to build an Augmented Reality application to become greener. You look at things around your home with the camera and it  recognizes its green gas footprint and makes recommendations to reduce it.  I guess it was a bit too early to do that based on visual recognition alone…you’d need  additional sensors that would provide related information about what you are looking at.

Tish: Well as there is more interest in Green technology do you think we may see VC interest in some green AR projects now?

Ori: I talked to some of the investment folks, Angels as well as VC’s about AR and they had no clue what it is. There’s a need for a whole lot of education. And there are no proof points (as in successful investments in this domain), and counter to popular belief – they don’t like risk so much…

Tish: And consumer adoption must lead the way, right?

Ori: Just like with every emerging technology in history, people never bought the technology, they bought the content, the apps, the benefits that came on top of the technology. Whether it was VHS winning over Beta Max, or BluRay winning over HD. It’s always because of more/better content. Look at the video game console war: Xbox, and Nintendo did better than Sony just because they had more and better games. Even Windows was a success thanks to its applications. People bought it for the applications not the OS. The content is the first to drive demand.

Tish: One of the challenges to giving people new ways to relate to their energy consumption is that you can just have them looking at graphs of how bad they have been in the past you – that may make them feel bad but that doesn’t necessarily give them ways or motivation to change. There perhaps needs to be more immediate relationship to the data to facilitate change. I think the mantra for optimization of anything from energy usage to supply chains is timely, actionable data?

Ori: There are a lot of ideas about measuring information and displaying it to people. For example, the Prius hybrid car, one of its interesting features – which is kind of game like – is a constant display of your current fuel consumption. That alone changes how people drive because they try to beat the “Score” and as a result conserve more fuel. That model can be applied to our homes…

Tish: Yes that is something I am very interested in. I have been following several projects in this area – one of my favorites is the Arduino, Current Cost/Tweetawatt, Pachube integrations I saw at Homecamp.

You joined a start up with Shai Agassi which was bought out by SAP right? He has a brilliant approach with Better Place.

Ori:  I think what’s really unique about Better Place’s approach is that he doesn’t require people to change their behavior. People are still going to have their own cars. They’ll be able to drive as far as they want, and for the same (or lower cost). Its not necessarily about a new technology, electric cars have been around for a long time but there was no way people were going to be limited by the 50 or 70 mile range and Better Place is solving that problem. With its infrastructure of charging spots and battery switching stations, drivers are going to be able to drive anywhere. And it’ll be similar to having to stop once in a while to refuel your car. The price maybe even lower than what you pay today for your transportation needs – and you’ll stop generating green gas. It’s a clever way of taking technology to a whole new level without changing the behavior of people.

Tish: Better Place is a classic example of things as a service isn’t it?  It is basically a utility company.

Ori: It is similar to a phone carrier model.  You pay for a membership that gives you access to the car (equivalent to the phone) and electricity (equivalent to the phone line) for the same price of fuel cost today. And as bonus you get to save the world.

How the iphone changed the game for AR – and the iphone versus Android

Picture from Ori’s post, “GDC 2009: Why the iphone changed everything”

Ori: And back to AR, you have to take the same approach, because nobody’s wants to don those huge head mounted displays or backpacks. You have to take advantage of people’s current behavior: they already carry their iPhones or similar devices.

Tish: As we discussed, you just have to get people raising up their phones and looking through them when that is a useful thing to do. Both Wikitude and Nathan Freitas’s graffiti app were enough to get me interested in the evolutionary step of raising my phone! Nathan’s graffiti app is nice. You leave a marker for your graffiti so other people can find view/add their own – a nice primal experience like pissing on the lamp post to let your pack know where you’ve been.  Also the graffiti app taps into a long history of  NYC street culture around tagging and graffiti art (see my interview, “Is it OMG finally for Augmented Reality?”).

Ori: The app store has fundamentally changed the mobile gaming industry. Last year they were in shambles. There was no growth. Everybody was complaining, “we can’t handle it, there’s a million phones, and you have to test it on each phone. And carriers suck, they don’t care about sharing and promoting your content. Everything was bad. This year mobile gaming is the hottest thing. And it’s all because of the iPhone. It changed the game.

Tish: How do you think Android is going to get traction against the iphone?

Ori: Well the number one thing is the form factor – the iPhone is just much cooler than the G1. Its OK but it doesn’t have the same feel. People thought it was going to be easy to clone the iPhone but none of the attempts succeeded so far.

Tish: How much does it matter for AR not being able to runs things persistently in the background on the iphone?

Ori: Actually they have add a such a capability in OS 3.  You can now make use of a background service.

Tish: OS 3 will open up new possibilities for AR?

Ori: The access to the video API is still not public.  But there is a new Microsoft application – Microsoft Tag that makes use of that API which means it is probably OK to use it.

Tish: (I ask Ori for his card and he shows me how to read it with my iphone.) Oh nice you have an AR card, of course!

In Search of Pong for Augmented Reality

Tish: So how will AR begin to, as Blair’s friend put’s it, “facilitate a killer existence,” particularly as we are probably looking at some new and perhaps pricey hardware?

Ori: You could take the Better Place approach. We’re going to give you a great experience and we’ll include the devices as part of that experience for the same price. Let’s say you subscribe to an AR experience  which offers access to multiuser, support, and all the information you need wherever you go – exactly according to the vision. You pay for a subscription on a monthly basis and included in that cost we give you a better device that offers a  better AR experience. It’s following the phone carrier approach, but in a good way.

But first of all we do need our Pong! I was sitting with a couple of AR game enthusiasts at the GDC and we were asking ourselves, “how do we create the first pong for AR?”

Was Pong a multiplayer game? Not necessarily! Did it connect to the network? No! We have to create the first dot in a long line of dots that will bring us to our destination.

Tish: You haven’t seen a Pong yet have you?

Ori: Not yet. I mean there’s maybe a handful of games and apps out there, but I don’t think any of them is a Pong yet. Still, it’s getting closer.

Tish: Kati London is doing some very interesting work on bringing games into reality, isn’t she?

Ori: Yes, she works with Frank Lanz at Area/Code. He teaches at NYU and has designed games for the “Come Out and Play” festival here in Manhattan. And a lot of these games are actually low tech.

Tish: Yes I have a big alternate reality game blog brewing that I haven’t had time to write yet!

Ori: The city is the gameboard is their slogan. It’s going to be a great playground for AR games. The city becomes a theme park. The city could become an even bigger touristic attraction. People will come to the city to be part of these games. So you’re having thousands of people running around the city playing all sorts of games from laser-tag style to history adventures, to treasure hunts.

Composing Reality

Tish: So why haven’t you focused on one of these kinds of games with your company?

Ori: We have a couple of scenarios along these lines that we’re planning for 2010-11. But first focus on what’s possible today.

Tish: And what’s stopping you from doing those kind of games today?

Ori: Many things. The devices are not there yet, location services are not accurate enough, ubiquitous sensors are not  there yet.

Tish: You think alternate reality gaming needs more “ubiquity” than is currently available?

Ori: Not necessarily. People are doing alternate reality games with no “ubiquity” at all. But my interest is to add the visual aspect. I believe humans are mostly driven visually.

Jane McGonigal said in a talk at GDC, that AR would allow us to program reality, which is exactly how I look at it. Once you can recognize things, some of it with WiFi and RFID and all sorts of sensors. But visual sensors is always going to be the ultimate way to recognize things. And once you recognize things and know what they are, and can pull information about those things (or people and places) from the internet, you can program it (visually). You could program it to be fictional, like in a video game, or it could be programmed as non-fictional, like a documentary. And that allows you to do things that before were unimaginable.

Tish: But you can’t forget the visual, it is primary the connection to peoples’ primary sensory relationships.

Ori: Yes, it’s like you go to a grocery store and you pick your vegetables, a lot of it is by sight and by touch. And what if you could also see just by looking at it that it’s from a local store, and that it’s organic?

Tish: It goes beyond overlays really?

Ori: By the way, I don’t like the term ‘overlay’. I know that’s how it looks: you either overlay or superimpose, but I’m still searching for a better term. A term I prefer to use is “composing reality”. Just like painters, they use brushstrokes and colors and compose a painting. We need to take the real element and the virtual element and compose them into something new. It’s not just about slapping one on top of the other.

Tish: yes I think the idea of dashboards is not so appealing.

Pookatak Games

Tish: Do you want to explain the evolution of your company? You have an interesting history of success with high end enterprise applications.

Ori: Since I was a kid I wanted to invent and create things. When I discovered software, that was a really cool way of actually creating things from nothing. From thin air; and you can do it very quickly. That’s what brought me into software. But I was always looking for the intersection between technology and art. Looking for ways to bring these things together. In the early nineties virtual reality was doing it. It had the appeal of cutting edge technology that can be combined with art. But then, as we all know, it crashed. So I joined Shai Agassi’s startup (who is now doing Better Place) back in the early nineties. I was one of the first employees in his startup which was developing multimedia products. I was leading the development of one of its flagship product. At some point we realized the technology could be great for an enterprise environment.

It was a really great experience. First going through this cycle from a very small startup and growing into this multi billion dollar business. I was responsible for defining and marketing SAP’s platform, which was called Netweaver. It was just an idea when we joined SAP and by the time I left it was a major, major business for SAP. I learned about the challenges of building a platform. No matter what purpose you’re building it for, it typically has similar rules. It’s definitely not just about the technology; the content that comes with it is really key to making a platform successful.

The third part of this platform trifecta is the community. If you don’t build a community, you won’t get the critical mass required for adoption. It may be your own platform but it’s not necessarily the people’s platform. That experience is very key to what we’re doing today. Now, a new industry is being born on the basis of a remarkable technology. But to drive adoption, first we’ll need good content. The content will be created using today’s technology with internal tools developed to simplify the process. Next step would be to make the tools used internally – available to other developers. Help scale the industry, enable innovation on a larger scale. That way we have a chance to create a platform. So it isn’t really just about my company. I’m so passionate about augmented reality, I want to it to become a healthy and successful industry for the next 5, 10, 15 years.

Tish: Yes I am so ready to be liberated from the sitting behind a computing screen! And I know that all this hardware is murdering the environment.

Ori: There’s ‘s the book by Rolf Hainich which is called “The End Of Hardware. ” It’s about hardware for augmented-reality. Once you use goggles or other AR interfaces you eliminate the need for screens, laptops, etc. It’s going to be great for the environment. You have read Rainbow’s End, right? According to the book in few years there will barely be any (visible) hardware. At least it’ll have a much smaller footprint for the environment. And it’ll touch every aspect of life, everything you do. It’ll change the way you interact with the world.

The Illusive Eyewear for Immersive AR.

Friend of Ori’s in San Francisco wearing retro AR goggles (from Games Alfresco, Ori’s roundup of GDC 2009)

Tish:OK lets talk about goggles.

Ori: Goggles are going to happen, we want to be hands free.

It’s going to happen because it’s just a more intuitive way to use this technology. But above all it has to look cool. Because if it’s not, if it’s a big headset, then maybe a small percent of the population might use it, but most people won’t. It has to look like an accessory, like new cool eyeglasses that you just must wear.

I recently talked to a friend, who runs an industrial design firm, and has experience in designing such glasses for companies like Microvision and Lumux. He says that when you try to bring the images so close to our eyes – there are some really hard problems to solve. Otherwise it can become really annoying and cause dizzyness.

But I’m optimistic. I believe it’s going to happen 3 to 5 years from now. It’s already starting now: Vuzix announced goggles that will be available this year. Some AR apps that are going to take advantage of next year. Initially only a fraction of the population will use it. And that’s going to help advance it and make it better and better. But it’s going to take time until it reaches the mass market.

Tish: In virtual worlds we have seen, I think, a lot of mistakes in terms of reinventing the wheel and producing too many proprietary versions of the same thing and not enough concerted effort on standards and open platforms that could create a vibrant ecosystem.  How can augmented reality not make the same mistakes?

Ori: There are some early AR open source efforts ARTookit, ARtag but it is not a movement yet.  One of the things we’re trying to do at ISMAR this year is to put together  discussions around key industry issues, such as standards. Some people say it’s too early, you have to have a defacto standard to start from. But pretty soon it’s going to be too late. Just like with virtual worlds, all of a sudden you have all these islands that don’t talk to each other. Why get to that point if we can plan to avoid it? Let’s start thinking about it right now. On the other front there are devices. There are pockets of people working on adapting devices for AR, second guessing the hardware companies. Why not get them together with the Intels and Nvidias of the world, and discuss what this device should be able to do. And then compete to make it happen.

Tish: How much luck are you having with this discussion part?

Ori: People are very interested in doing this. We proposed these panels for ISMAR. And I’ve got some key people already on board. They have tons of input, they want to get involved. We’ll see how much we can actually get out of it.

Tish: In virtual worlds it was a while before vibrant opensource communities developed.  OpenSim has I think been the breakthrough community in this regard.

Ori: You have to think about the elements up front. The dream job is to architect the industry. Say we agree on the required pieces. Then we could help the right companies succeed in delivering the pieces. Next, we have to collaborate so that these pieces talk to each other. And eventually these communication methods will become defacto standards and most developers will adopt it.

Tish: So I’m going to put you in the role. You’ve got your dream job. You’re going to architect this community. So what are the key pieces and where would you like to see the open source communities take hold first?

Ori: Open source will not be exclusive. It’s going to live side by side with proprietary technology.

The key pieces? You have the user at the center. And the user interacts with a lens. The lens includes both the hardware and the software. And then the lens senses and interacts with the world, which includes people, things and places. And these people-things-places emit information – about who they are, where they are, what they’re doing, etc  – which is then stored in the cloud.

And then you have the content providers, the people and companies, composers who weave AR experiences through the pieces we mentioned before. These composers need a platform that glues these pieces together. Pieces of the platform will be on the lens, and in the world, and in the cloud. If you manage to remove the frictions, and connect these pieces into an experience that people like – then you have a platform. What the platform does it reduces the overhead and accelerates innovation.

Tish: Another problem virtual worlds faced in their development was their isolation from the world wide web.  Will augmented reality avoid this plight?

Ori:  Yes, I believe the key, like you said before, is not to reinvent the wheel. The cloud is already there.  Take Wikitude for example, all Mobilizy had to do is build  a relatively simple client app, connected to wikipedia, and all of a sudden it offered a wealth of information in your field of view.

I think we can learn a lot from web 2.0. For example, in order to have a ubiquitous experience like Robert Rice and others are striving for, you’ll need to 3d map the world. Google earth like apps are going to help but it is not going to be sufficient. So let’s leverage people. Google became successful in part by making people work with them.  Each time you create a link from your blog to my blog their search engines learn from it.  So let’s find ways to make people create information that can be used for AR.

Ori Inbar directed Wiki Mouse – a WIKI Film co-created by a swarm of movie makers around the world.