Today Metaio is holding Inside AR in Munich, Germany.   Metaio (the picture above shows Metaio co-founders Thomas Alt and Peter Meier), is behind some of the best known commercial and industrial AR experiences of recent years.  But as important as the many AR projects they have executed are the AR tools that Metaio has made available to developers.  Metaio’s AR products and tools have played an important role in bringing AR to a wider public, and given many developers the opportunity to explore AR.

Inside AR is a great opportunity to see what these AR pioneers  will be up to in the coming months.   I could not make it to Munich this year.  But, fortunately, I had the opportunity to talk with Thomas Alt, recently.   In this conversation – see below, I got a chance to discuss what was going on inside AR with Metaio.

The Fall season is always jam packed with great events, and I wish I could be in two places at once.  But this week, I will be in my home town, NYC, attending Web 2.0 Expo which, reflecting the heat in the NYC tech community, is a sold out event with a very exciting schedule this year (more on some of the presentations that I will be attending later in this post).  If you missed out on tickets to Web 2.0 Expo, all Keynotes will be Streamed Live: TUES 9/28 to THURS 9/30, and keep your eye on @w2e and #w2e on twitter.

Meanwhile, I am missing Inside AR, which had some great speakers lined up, including fellow New Yorker, John Swords, partner and Ringleader at Circ.us.  Hopefully, Swords will share his experiences at next month’s ARNY Meetup which will be “joining forces with another vibrant community – NY Gaming – for an unforgettable night of Augmented Reality Games” on Tuesday, Oct 19th, 6:30 PM at AOL Ventures in New York, NY.

At the most recent ARNY @swords gave a brilliant talk on the possibilities for AR Game development on the newly available opensource Parrot ARDrone platform.  It was great to hear from social game guru @swords on his plans for Parrot ARDrone games, and more.  The picture below of an ARDrone camera view is from John Swords Flickr set.  Swords was flying it inside his garage because the winds outside were too strong.

Also, I kicked off what will hopefully be an ongoing discussion on, “Story Telling with AR and the Big ARNY a collaborative AR Game for NY,” with a few slides.  I have opened up the presentation document for collaboration, so please ping me if you would like to be added as a contributor/editor, and are interested in getting involved.

Ori Inbar, CEO and co-founder of Ogmento, Games Alfresco, ARNY and my co-chair on Augmented Reality Event 2010, suggested The Big ARNY – A Collaborative AR Game Development Project modelled after A Swarm of Angels last year at the First ARNY Meetup – so let’s make it happen!  I will be catching up with Ori in October about what Ogmento has been up to since they became the first VC backed AR Game company!

“Games allow us to see each other, for a moment, in a way that living in a city prevents” – Kevin Slavin

I believe that, AR, to get beyond the stage of “interface du jour” needs to offer us new ways to relate to each other and the world around us so that we can actually improve  and deepen our engagement with reality not just create experiences that are primarily optical  (see James Turner’s interview with Kevin Slavin “Reality has a gaming layer” on not letting “the pleasure of a game and the meaning of a game and the experience of a game rest primarily in the optics.“  And see my recent post, Urban Augmented Realities and Social Augmentations that Matter: Talking with Bruce Sterling, Part 2).

Two of the most inspired creators of urban games,  Kevin Slavin and Kati London of Area/Code will be speaking at Web 2.0. Expo tomorrow.   And you can be sure I will be at both these sessions. Loitering on the Motherboard, Kevin Slavin, is 2:35pm Tuesday, 09/28/2010, and Games that Know Where you Live, Kati London – is a keynote that will also be live streamed – 4:55pm Tuesday, 09/28/2010

Recently Kevin Slavin was interviewed by James Turner, on O’Reilly Radar.   This, Reality has a gaming layar, is a must read piece about a “world where games shape life and life shapes games”  (see @timoreilly). 

Interview with Thomas Alt

Tish Shute: Perhaps you could just start with your background Thomas because I think there’s a lot of newcomers to AR but you are really one of the first movers in commercial AR. How long you’ve been involved in this?

Thomas Alt: Actually I’m an ex-researcher in augmented reality. I started with me actually after getting my master’s work in engineering from the Technical University of Munich working for a big company called Volkswagen. And at that time,1999, we got a research grant for researching how augmented reality could change manufacturing processes in the automobile industry.

And from the research work there, I basically went back to school, did my PhD about augmented reality. And while speaking at a conference, I met Peter Meier who is the co-founder of the company who was also a master’s student writing his thesis about augmented reality. That was in 2002.

And so it really was in the very early days of augmented reality. And both Peter and myself we got really excited about the technology; we saw endless possibilities. We said, ‘OK. Let’s just found a company. We actually founded the company in early 2003 with virtually no money. As a matter of fact the founding capital of the company was 25,000 Euros and this 25,000 Euros were won in a case competition in Germany – a business plan competition.

Tish Shute: So you won 25,000 Euros on this case competition and that’s where Metaio started….

Thomas Alt: Exactly. And to legally found a company in Germany it takes exactly 25,000 Euros so that was the founding capital. We started pretty much like good old SAP started.  It wasn’t in a garage though it was a very small office and we basically built up the business through work, so we don’t have any investors or whatever. Right now we are 66 people located in Munich where our headquarters have been for five years. We have some presence in the US, and we have a venture company in Seoul, South Korea.

Tish Shute: Awesome. I just noticed how fast you’ve been growing. So right now, I’m going to ask a couple of questions about where you see the technology and the emerging industry going.

First, what are the platform of choice for Mobile Augmented Reality at the moment?

Thomas Alt: Obviously in the cellphone hardware space there’s a fierce competition going on. It’s yet to be defined what will be the prevailing platform right now, obviously it’s the iPhone is big now, right? But Android is catching on very, very fast.

Tish Shute: You have pioneered bringing a cross platform SDK for vision assisted AR to a wide community of developers with Junaio and with your partnership with Kooaba – a visual search company from Europe.

Thomas Alt: Yes, yes, and this is how we would, also in the future like to position ourselves with Junaio.  Junaio will be a platform, a technology platform, which will allow users to do whatever they want to do in augmented reality. The API of Junaio is huge in the sense you can do anything from outdoor gaming, to visual search, to normal, uh, lay out style, you know, find the next burger king a mile away kind of super impositions.

Tish Shute: The only licensing you pay is for unifeye right? When you want to use your tool kit right?

Thomas Alt: Yes and this is how we’re distinguishing it. Junaio is our consumer brand targeting newbie AR developers, with limited programming skills, while the Unifeye platform is really our B to B platform where B to B customers can create their individual AR experiences.

Tish Shute: Yes which is what my friend Patrick O’Shaughnessey, Patched Reality, did for the Ben and Jerry’s app he created using Unifeye.

Thomas Alt: exactly…

Tish Shute: It is a lot of work developing for so different mobile platforms isn’t it. Junaio is on Android and iphone but you havent moved Junaio to Symbian?

Thomas Alt: To be honest with you right now its a matter of priorities we have other things we want to do first. And from analyzing the user base, iphone was a big step Android was a big step and now we are pretty much seeing what is happening next. As you know Nokia going into different directions as far as their smart phone operating system goes, and so on and so forth.  There are also capacity constraints. And right now obviously the most – potentially not the most possible users, but the users most inclined to do AR on a day to day basis are the ones using the iphone and android devices. But obviously there are a lot bigger cellphone manufacturers out there.  But just you know even the mobile web users aren’t as strong as the users on the iphone and android devices?

Tish Shute: So what do you think the iphone 4 has that brought to the AR picture?

Thomas Alt: Very fast camera access, very good for marker recognition. If you go to the Metaio site you’ll find a movie where we show on the iphone 4 app for a real augmented reality Leggo peice – this is something which is very nice

Tish Shute: Yes I see that, yes that is nice, yes, yes very nice. The Unifeye SDK is really putting markerless AR into the mainstream.

Thomas Alt: Yesterday we launched the first, a err very nice shopping… shopping solution for , but that’s completely external.

Tish Shute: Oh yes – the augmented reality shopping for seventeen.com, i was going to ask you about that, because it is the first augmented reality online shopping using natural feature tracking.

Also I am very excited to see the gestural interface, awesome!

The seventeen augmented reality shopping app is a PC experience but are you working on developing gestural interfaces for mobile AR?

Thomas Alt: We are continually pushing the envelope of what’s possible with AR. Gestural interfaces for mobile AR is certainly the next step in taking what we’ve done on the PC and making it more portable by using the mobile platform. One thing to keep in mind here is the limitations of mobile platforms and size of the screen needs to fit and make sense for the user experience.

Tish Shute: I know you started off as an AR researcher (although as you mentioned earlier you have been working in commercial AR and building Metaio for a long while now.

So in addition to how we are progressing towards gestural interfaces for augmented reality, I would like to ask some questions about AR eyewear. We won’t really have hands free AR without eyewear. What is your projection on when we will see consumer AR eyewear? And, Do you have a any comments on those speculating that we will not see AR eyewear go mainstream for 20years?! What is Metaio doing to move eyewear technology along?

Thomas Alt: Well as you know, it’s always, you know on the technological roadmap, and we’re still doing research projects, in our AR research department. We have worked on things like calibrating eyewear for augmented reality. There is some nice development there.

But really, commercially, the whole thing with eyewear has never caught on on a level which would make it a valuable avenue, business avenue, at least for Metaio. So, I guess as an ex-researcher, it’s still a very interesting, a very good technology. And it would definitely change the marketplace radically when available. But as per right now, there are very few commercial applications.

Tish Shute: Are the obstacles to AR eyewear technological obstacles or is it just a question of a a business model. I mean is it realistic to see eyewear in the next 3 to 5 years at a price point affordable to consumers, where you really, truly can have eye tracking? You know, the whole problem there was with virtual reality and eyewear giving people a headache. How far have we come in terms of the technology?

Thomas Alt: Well, it’s not so much technological factors ’cause all fundamental problems are solved. It’s more a rather large corporation, I guess, would have to step up to the plate and say okay, do, let’s get all the state of the art in electronics and develop just a perfect HMD.

Tish Shute: Something Yohan Baillot’s company Simulation 3D is doing at is looking at is hooking up eyewear to smart phones.

Thomas Alt: Yes exactly that would be even better. Metaio has done a strategic move into this HMD space for augmented reality about a year ago by acquiring a bankrupt company. I mean, we had considerable IP around it from our research base but in the long term we still believe in it and we did a move about a year ago in buying what was left over from a bankrupt company including a lot of IP, which basically goes into the direction of mobile augmented reality but also mobile augmented reality in connection with head mounted displays.

There’s actually a press release about this but that’s about a year ago…

I know that the whole HMD thing… I mean, I’ve seen companies come and go. Metaio has worked previously, very closely, with Microvision of Seattle. We have worked with a German company, doing HMDs and we have worked with Vuzix.  We are still working with Vuzix, so we’re still consider it very valuable.  But right now, I mean, it’s just not a big part of our commercial pipeline, to put it that way.

Tish Shute: It was interesting what Bruce Sterling said in his keynote at ARE 2010.  He actually made a strong case for why smart phone augmented reality may be more interesting because it’s less immersive. I mean, he raised the question of the fact that if you really truly had AR eyewear and HMDs you’d re-enter the world of virtual reality or as he called it AR’s Gothic step sister VR would rise from the grave….

Thomas Alt: Yeah, well, that’s a cultural or even a philosophical question and we have discussed it a lot, especially in the industrial domain. Also will the deployment of HMDs come about from end consumers using it in their spare time, or from professional users using the idea in their work time?

Tish Shute: Do you think it surprised people who have been working in augmented reality research how much people have engaged with the idea of smart phones as the mediating device for AR, and that rather than having the always on experience that eyewear would give us, we use smart phones as a magic lens of a smart phone when we need to or want to. Some people were skeptical that anyone would want to hold up a little window to look at augmentations of the world – a magic lens. I mean, it wasn’t self-evident that that would be an experience people enjoyed, and it turns out that it was.

Thomas Alt: That’s actually a very good analogy. And also in my view, I mean, certain behaviors just change also, right? I mean, this is exactly what Apple’s trying with the iPad, right? You’re taking the iPad, and all of the sudden you’re not constrained to a laptop or whatever. And it’s truly a companion of the couch, in-bed Web, in the kitchen, and so on and so forth. So digital usage with the iPad, which is a different market, and I’m aware of that but as an example, the iPad has changed our behavior. And obviously, the augmented reality guys are hoping that something similar happens with AR.

Tish Shute: Which of course brings up the question, I’m assuming that some of the next generation of slates/ipads are going to have front and back cameras, GPS, and compass, right?

Thomas Alt: Actually we know that.

Tish Shute: You know that, yes. I assume that you know that, because are you working on some prototypes, and have you got some plans?

Thomas Alt: You have to understand that I cannot talk as I’ve talked as a researcher. It’s the rules, so I have to be a little bit careful about what I say. We very much think that a webpad, or whatever pad, you would want to call it is on some occasions very good device for AR.

Tish Shute: Yeah. But it is an interesting point with holding up a larger device, because you hands aren’t free, but the neat thing about the phone for augmented reality is that you really can do a lot with your thumb, as we’ve found and just the position of the phone. But, how will this work it with the two hands on the larger device?

Thomas Alt: Keep in mind, everyone’s talking about mobile augmented reality, but really where the case for augmented reality, at this point, is the strongest is in the installation business, it’s in the web business… Not necessarily only commercially, but also use case-wise. There are tons of museums out there which are using our augmented reality system in an installation fashion, and to communicate products better, and more efficiently, and so on, and so forth.

So, I know that the hype is clearly on the mobile augmented reality side, but there are many examples augmented reality experiences where holding up a larger device is not a big obstacle.

Tish Shute: Well this brings me to some questions about the future of mobile AR. My interview with Jay Wright focused on how we are now in a new period for AR bringing together computer vision, visual search into a mobile stack that is really optimized for AR. What do you see emerging in mobile AR as we move beyond compass, GPS, camera, accelerometer based AR into markless image-based AR. What will the new use cases and where will we see mainstream users getting in AR. Will AR games be the first mainstream AR experiences?

Thomas Alt: My partner is actually, first of all, one of my best friends, second of all, very emotional, third of all, very intelligent, and he said the other day something I think very valuable in this area. He said, basically think about Mobile Augmented Reality, Thomas. There’s really a very limited number of use cases which you can do if you look at these classical Point and Find applications, ok? But there are almost unlimited number of use cases when Augmented Reality becomes a day to day companion, ok? So what he meant is, ok, I’m looking at my normal day… I’m looking at the city, I’m walking throughout this, I’m coming home, I’m having dinner basically. I can deploy Augmented Reality in a pure POI search fashion perhaps not even once. Ok when I’m travelling it’s a different story, but in an ordinary day I might use a POI search not even once.

But where this ultimately leads, you know, is even in the 15 minutes I’m having breakfast, I’m using AR – looking at the cereal box with my cell phone, I’m taking part in a sweepstakes or whatever. So from that we draw the conclusion that as a general strategy for Junaio, we should basically throw as much technology as possible into Junaio, make it halfway self-explanatory, and just give people the possibility to come up with ideas on how to deploy augmented reality continuously.

We have actually got a creative team from an Art School working on that, and just, you know, with very little programming skills coming up with things you can do with Augmented Reality on a day-to-day level. And it could be a scavenger hunt game, in the city, with monsters flying around, it could be the normal POI routine, it could be marketing purposes, and so on and so forth. And I think that’s really the roadmap, and this is a little bit similar on a more technical level, to what Qualcomm is doing, ’cause they’re floating out possibilities or capabilities I want to call them, and Metaio is doing that, but on a higher level [re the tools] meaning on a Junaio level.

Junaio is a capability platform. It is also a way for Metaio to demonstrate the capabilities of our technology. We will offer all the possibilities for AR and more that we have already demonstrated on PC augmented reality.

Tish Shute: What is the business model for Junaio? Are you encouraging developers to develop business on your platform ?

Thomas Alt: Junaio is our end consumer platform and our business model is similar to the way Google structures its business model. We work with OEMs, content partners, brands, and developers to offer free content to our end users. Where we do charge is on the advertising side, more specifically contextual and location based advertising. At the current stage, we are focused on building the content base, fostering our developer community, but in the near future, we will be introducing advertising channels.

First of all you have to have very good use cases in the platform basically. And then to put a business model on top of that from a technology stand point is not hard – its a pay channel. Its all prepared for this.

Tish Shute: You have quite a broad base as a company don’t you – you do everything from industrial AR, marketing to technology licensing and more?

Thomas Alt: Basically, there’s a lot of things people don’t see us. There is also the Unifeye PC SDK and we have a client base and partners who are sourcing software from us, and we are doing great pieces. I mean the hype of augmented reality is really coming to a peak. There are lots of pieces that are not even talked about any more. China’s GQ magazine launched with AR from Metaio, the biggest AR campaign anywhere – there are a lot of potential readers in China. And um, so that’s our business model… we have our IP, our patents and so on. And on this we can move on onto the mobile platform whenever it’s advisable or feasible.

Tish Shute: Right. Yeah. I mean uh, you’re very fortunate to have this base built on uh, years of developing IP. What are the most important areas of AR that Metaio holds IP and patents in, in your view?

Thomas Alt: there’s sleepless nights in that too…

So far we’re extremely excited about what’s going on with Junaio, it’s one of our big, big success stories. But we are sensible and trying to experiment because, you know, analogies from the past won’t really work in my view for Augmented Reality in a sense, that, you know, you better bring for a new system to fly, for a new technology to fly, you better bring a very concrete use case to the table, ok? And a well-defined use case. And we are, right now, with Junaio, in a state where we are checking out what could be such a very defined use case.

Tish Shute: So how many users does Junaio have now?

Thomas Alt: Let’s put it that way, we are, especially in the last 2 months, we are very satisfied. But we are not disclosing that, because users, and we’re seeing that from the competitive landscape, always needs 1 page of description what exactly a user is.

You understand what I’m saying? So this is why, ’cause we don’t want to up or downplay things, we are very careful saying, with users. Because I mean we have people who are actually also commercially very interested in Junaio…  We go through with them and discuss what exactly a user is. Cause there’s more then…a download is not a user. An app or something on your phone is not a user, basically, in my opinion.

Tish Shute: I am still waiting to see someone do something with AR and the Four Square API, or now the Facebook Places API. Do you see an interesting potential in the marriage of the rapidly emerging location based social networking space and augmented reality?

Thomas Alt: Definitely. Augmented reality offers a way for users to find information around them easily. Adding in the social networking component such as geo-tagging, rating, commenting can enhance the user experience and create engagement beyond just viewing the information. For example, within junaio, an average user can create their own personal channel and geo-tag photos or leave text messages at different locations. They can create a virtual tour of San Francisco and share it with their friends. By connecting the social side with good content, the augmented reality experience becomes more fun and interactive.

Tish Shute: And, of course, there’s the Junaio API. Are you beginning to see developers use that?

Thomas Alt: Yeah exactly, I mean if you go to Junaio.com, you can get a login, you have an API description. And the way it works is, that you bascially set up a call, which contains the information you would like to have in your individual channel. You submit it to us, it will get checked for profanity and other things, basically. And then we admit it to Junaio basically. The API is huge.   You can also use Junaio indoors.

This is very relevant. And there’s a tool chain for that, and so on and so forth. You can do mission-based search with Junaio. It’s in there, it’s called Junaio Glue. And there will be another very interesting feature coming up in a couple of weeks. And you can just do it, you can do a scavenger hunt, a game, normal POI search, and so on and so forth. And it’s all active. And that’s, what’s sometimes difficult for us to communicate, is it’s really a capabilities platform, but on the other hand it’s obviously very good to developers. And I mean on the developers side there’s huge interest.

I am kicking off what will be a series of  talks with key players in the rapidly emerging augmented reality industry with an interview with Ori Inbar, co-founder of Ogmento. For Ori’s full bio see here.  Ori not only has a passion for interactive entertainment and a commitment to developing augmented reality to, “free young and old from getting lost in front of the screen.” Ori also, as I noted in the intro to my first conversation with Ori in May, brings invaluable experience to the young augmented reality industry because he has already gone through the cycle of working in a very small start-up and growing it into a billion dollar business. Ori worked with Shai Agassi (Shai is now leading the world changing Better Place ) driving Netweaver from a mere concept to a “major, major business for SAP.”

I have been tracking developments in Augmented Reality pretty carefully since my post, “Is it OMG finally for Augmented Reality?: Interview with Robert Rice.” And I talked at length to Blair McIntyre on Mobile Augmented Reality and Mirror Worlds recently. These interviews, and my first conversation with Ori, are long in depth conversations. But, if you haven’t already read them and you want the full story, they may be a good place to start.

As Ori notes, the Google trend on Augmented Reality is really growing exponentially at this point, and recently there have been two high profile round ups in the New York Times here,  and one in Venture Beat here.

Ori comments (for more see full interview below):

“What was unique about the article in Venture Beat is that it was probably the first roundup of Augmented Reality companies. It wasn’t very comprehensive or detailed, but it might be a sign that VCs are getting interested in AR companies.”

I feel that now is an excellent time for a detailed and comprehensive series of interviews on the state of play for augmented reality. I hope to speak with all eight founders of the AR Consortium, Int13, Metaio, Mobilizy, Neogence Enterprises, Ogmento, SPRXmobile, Tonchidot, and Total Immersion. The recent founding of the AR Consortium focused on facilitation of, “faster market penetration, robust technical standards, and a strong focus on the end-user’s experience,” is an important development for augmented reality.

Ogmento co-founder, Ori Inbar, created augmented reality’s trail blazing blog, Games Alfresco, and Pookatak Games – now Ogmento.  Ogmento will bring out their first augmented reality game for the iphone later this summer.  But Ogmento also brings two important new facets to the rapidly growing augmented reality field. One is that Ogmento brings leadership from veterans of the entertainment industry into augmented reality development  – Brian Selzer and Brad Foxhoven from Blockade have partnered with Ori on Ogmento.  The other, a very exciting announcement from Ogmento, is that they will be acting as publishers for a fast growing cohort of augmented reality application developers. Ori explains, Ogmento will be helping, “AR development teams out there bring their concepts to the market.”

The emergence of a savvy publisher for augmented reality developers is a vital step forward for this emerging industry.  As Ori notes in the interview below:

“recognizing elements on product packaging, posters, games will kick off a frenzy of new consumer experiences before the end of the year mobile AR will take the center stage. Next year will be huge for these experiences.”

The first commercial and practical toolsets, SDKs, and APIs developers wishing to create AR experiences have emerged including AR engines with key development tools for markerless image recognition like Imagination and SRengine Lite (for more see these posts on The Future Digital Life and Games Alfresco).  Also the pioneering and the most used AR engine out there, ARTookit, now has a version for the iPhone. And Metaio has produced a powerful  modular development tool kit, Unifeye.  Wikitude (Mobilizy) and Layar (SPRXmobile) have caused a lot of excitement recently with applications that unlock the potential for a wide range of augmented reality browsing experiences. Notably, they have both opened up developer APIs for their mobile augmented reality platforms in the last couple of weeks (although Layar is only providing “50 developer keys to interested companies across the globe,” whereas the Wikitude World Browser APIs are open to all). Maarten Lens-FitzGerald explained:

“There will be more keys distributed by Layar.  We are starting with a small amount to make sure that the servers will be able to handle it. Biggest difference is that the Layar API is aimed at Business to Business market: Companies and or developers with major brands or other content and services which are relevant for the new AR world. They are able to get their own branded experience in Layar”

Wikitude, however, has introduced the vital component of user generated content. With Wikitude.me you can add your own tags now.

I just added my first tag to Wikitude.me and twittered from TweetDeck:

“Signed up for Wikitude.me today, very cool the world as a wikipedia platform.”

click on image above to watch the video

I have interviewed both Mark A. M. Kramer, Mobilizy and Maarten Lens-FitzGerald, Sprxmobile, for this series so I will be posting more on Wikitude and Layar soon. See Wikitude’s “World Browser” press release and video for more, and the video here for read here more about Layar.   I was very chuffed to hear from Maarten Lens-FitzGerald of Sprxmobile the other day that my Interview with Robert Rice, in January of this year, was a key inspiration for SPRXmobile to get started on the development of Layar.  I hope this series of interviews and the arrival of a world class augmented reality publishing team, Ogmento, will be the inspiration for many more game changing augmented reality projects.

Robert Rice (who founded the AR Consortium with Ori) is a key innovator in the field who is tackling some of the really hard problems of AR development. While we will have to wait until ISMAR in October to see demos of Robert’s AR platform, Robert explained in my interview with him (to be posted soon):

“I guess you could say that we are trying to build a platform for platforms, or more accurately the infrastructure for the global augmented reality network. If Neogence does its job right, anything you create using ARtoolkit, Unifeye, or Imagination would be applications you could link to, integrate with, or deploy on what we are building, and not be tied to a specific set of hardware, browser, or walled garden.”

Augmented Reality and the Next Generation of Compelling Interactive Entertainment Experiences

Ogmento and the partnership of  Brian Selzer and Brad Foxhoven from Blockade with Ori Inbar is a very significant development for Augmented Reality.  I am very excited to be discussing Augmented Reality with Entertainment Industry veterans (interviews with Brian and Brad upcoming soon). Here is an excerpt from the beginning of my conversation with Brian.

“My first conversation with Ori was actually about my interest in Location Based RPG concepts. We had a long conversation about the possibilities with AR, and it was clear that we shared similar interests, but we’re coming from different complementary backgrounds. The idea of collaboration was exciting, so we just kept talking until the timing felt right. Now, with Ogmento we bring a unique blend of AR development experience with a deep backgrounds in AR technology, animation, video games, entertainment, social media, etc. I think this is a powerful mix that will allow us to do some great things.

It’s still so early, and things are just getting started in AR. There are only so many webcam magic tricks you can enjoy before you are ready for something else.

The location-based apps have the most potential in my opinion, which is why we are really focused on mobile AR. We have some board-game type projects, which do not instantly scream location-based gaming, but if you look at  something like the ARhrrr board game [for more see my interview with Blair McIntyre here], you can see how much more compelling it can be when the game invites the player to be actively moving around during the experience.”

Talking with Ori Inbar

Tish Shute: First I really want to catch up with you on your new venture – Ogmento.

Ori Inbar: It’s basically an evolutionary step. A little bit about Ogmento, I partnered with a couple of very strong veterans of the entertainment industry from LA, and we want to do two key things. One is develop augmented reality applications and games; and two, which is becoming really interesting lately, is to help AR development teams out there bring their concepts to the market.

Tish Shute: Your new partners are Brian Selzer and Brad Foxhoven. Are they both from Blockade?

Ori Inbar: Brad and Brian are both from Blockade and are leveraging their contacts and deep knowledge of entertainment companies and big brands they worked with over the years.

Tish: Sounds great to have a team like this getting into AR! How will you work with developers? Will you help them market their AR applications?

Ori: Exactly. In parallel to my blog becoming popular, it seems there’s a lot of activity picking up in the AR space. People are reaching out to us and are asking asking for help. So we started actually making that a part of our business. We help connect them with the right technologies if they need it or connect them with the right brands or companies and strategize with them on how to go to market and help publish their applications or games. So that’s becoming an exciting part of what we do.

Tish: It seems that great content rather than applications is going to be what drives the early AR market. What is your direction/vision for content development and what technologies are you focusing on: In my recent discussion with Robert Rice he asked the question: “The iphone…future or failure? With a seemingly anti-developer stance regarding augmented reality, and only a sliver of the global market share, are we letting the short term glitz of Apple and the iPhone fad pull us in the wrong direction? Shouldn’t we be focusing on Symbian devices that have the lion’s share of the market? or should we be looking more at either other OSs (winmobile, android) or not at all and trying to create a new platform that is more MID and less smart phone with a hardware partner?”

Ori: I can tell you that we’re now being inundated with requests for projects from all angles. Either from big brands, independent entertainment companies, all and everything in between. In terms of the hype it’s curving up. More and more people are hearing about it. In a sense we’re moving from a very strong push to a little pull. We’re starting to see some pull from people asking for development of AR projects.

In terms of technologies, what platforms to use? At this point we’re very open to picking the right technology, the right platform for each project. For example: Wikitude, or Layar, could be good for certain types of experiences, but they do not visually recognize elements in real life. So for that purpose you’ll need technologies from other folks like ARtoolkit or Imagination. We are basically picking the right tool for each job.

Tish: What distinguishes Imagination from ARtoolkit?

Ori: Imagination have been doing virtual reality and augmented reality for years now. They probably have the best augmented reality engine for mobile devices. It’s the only engine built from the ground up for mobile.

Tish: What about this whole issue of most of the world being Symbian and the U.S. being iPhone and possibly moving somewhat towards Android?

Ori: That’s right. We’re currently focusing on the iPhone because it’s the coolest out there and fits our target audiences perfectly, and there’s a pretty good market here in the U.S. which is currently our main target market. But you’re right, it could change tomorrow, when another major platform becomes popular. Another thing is that is attractive about the iPhone is that all of the iPhones are the same (although it’s now changing with the 3G-S). It’s very easy to develop an application and distribute it to 40 million people. But if you’re going into the Symbian market or Windows Mobile, it’s a nightmare. It’s not just the technical aspect of testing the app on all these devices but also the distribution challenge. Apparently there’s just no simple channel to distribute it across these various versions of these platforms.

Tish: That’s what stopped mobile gaming taking off until the iPhone wasn’t it.

Ori: Absolutely.

Tish: So this Austrian AR engine, Imagination, it must be for Symbian too, is it?

Ori: They have an augmented reality engine that works on P.C. webcam, Symbian, Windows Mobile, and now in the iPhone. You write the application once and theoretically it runs on all these platforms.

Tish: Any interesting development showing up on Imagination yet, or is it too early?

Ori: Our first game that we discussed [see my previous interview with Ori here] is actually going to be based on that engine.

Tish: Your game for pre-schoolers that we discussed before? What’s the release date for that?

Ori: We said it’s going to be released this summer and that’s still the plan, but it depends a little bit on Apple. You’ll be among the first to know when it’s in the app store.

Tish: It seems at the moment the two main forms of AR are mainly tethered or marker experiences or kind of stepping stone apps like Wikitude and Layar that are markerless but mainly pull info from web to world rather than truly aligning graphics in a tight relationship to the “real” world. Although what differentiates Wikitude from Layar is that you can actually create content with Wikitude.me and add your own tags now. When are we going to see something that goes beyond the tethered experience and the “browsing” experience and get to the magic of AR in terms of tightly aligning media/graphics with real world objects?

Ori: That’s right. That’s one difference between Layar and Wikitude. Another is that Wikitude is actually being used across the world by what is it now 160,000 or 200,000 people and Layar only works in the Netherlands for the moment. That’s a big difference. But things are changing rapidly.

But there are a lot of new AR concepts being developed out there (and we are fortunate to be working with some of them).

We’ll still see many webcam campaigns for another year or so, but recognizing elements on product packaging, posters, games will kick off a frenzy of new consumer experiences before the end of the year mobile AR will take the center stage. Next year will be huge for these experiences.

But to address your question, you’re going to see two different types of apps. One is the so called browsers like Wikitude and Layar, which actually doesn’t comply with the scientific definition of augmented reality in the sense that you have to align graphics with real life objects (Azuma).

Tish: Wikitude and Layar are not the purist idea of AR and technically should they really claim the title “browser”?

Ori: Right. But I don’t think it really matters to users that find these apps useful. The second type of apps you’re going to see in parallel are those which recognize markers or natural images and soon any real life object – and overlay on top of computer graphics. I believe we’ll see more and more of that towards the end of the year.

Tish: And you think Imagination really does give people an engine that allows them to do image recognition more easily?

Ori: Yes absolutely. They do. We’re using it. Right now it’s not an official SDK, but pretty soon they’ll open it up and more and more people will be able to take advantage of it.

Tish: When do you think they’ll open it up?

Ori: You’ll have to ask them But there’s also the ARtoolkit – the most used AR engine out there, which now has a version for the iPhone. They also have a version that can track images, but it’s still not running on the iPhone. It’s going to take some time until these, as well as the products from Metaio, and other folks are going to catch up with them.

Tish: I want to ask you about the Cartoon Network AR campaign? I didn’t get a chance to look at it before I spoke to you. It’s still a tethered experience I think but there is a “next generation AR” claim?

Ori: I think in terms of the experience it’s trying to take it a bit further than what we’ve seen so far by creating an actual game. But you are still tethered to a PC screen which is not significantly different than playing a 3D game on a PC.

Tish: Then the other thing is did you get a chance to play with the iPhone video API? Is it accessible now? I noticed your blog on this.

Ori: It’s under NDA with Apple so we cannot really talk about the details. But it seems like it’s doing only half of what we need for AR. It’ll be useful for applications like Wikitude but not when you want to align graphics with live video.

But I’m pretty sure it’s going to happen soon and there’s no reason why it shouldn’t… It’s just that Apple’s policy is to never reveals their plans. We don’t know when it’s going to happen.

Tish: You basically think we’re going to have to be very flexible because it seems as a developer the iPhone is obviously the only place to get easily to a market. But in terms of developer you do have to make some serious decisions and there are some interesting MIDs and new Android platforms coming out aren’t there?

Ori: Last year I thought MIDs would make a big difference in the AR world but they didn’t. Not in terms of adoption and not in terms of the number of MIDs that we expected to be released this year. Some companies are trying to resurrect it for next year… or later this year. I think the high price is going to be a barrier for adoption.

Tish: The economic climate hasn’t been good for it. Which bring up another question, now that people are really catching on to AR, should there be a partnership with the hardware people to get some of the hardware that really will make AR rock n roll moving e.g. accurate gps (Robert mentioned to me the idea of creating pseudolites = ground based differential GPS with <10cm accuracy hotspot say in partnership with Starbucks, for example).  And it seems to be that there definitely needs to be an eyeware push now we have seen close but not yet perfect prototypes from companies like Vuzix and Lumus. Will Ogmento,’cos you are going to be involved in lots of cool projects focusing on content, partner with the hardware people and get it moving along in that area?

Ori: That’s absolutely right. For ISMAR this year we’re planning some panel discussions with industry leaders as well as hardware companies to define the ideal mobile  device for augmented reality. These discussions are already happening. But I don’t think companies like Intel and others are paying enough attention to it just because they don’t see the demand yet. I believe that with more content and apps in the market, they’ll realize they can’t wait any longer and will accelerate their decision process and act.

Tish: But it does seem to me – one would have to be an Ostrich not to have noticed the excitement that relatively trivial Augmented Reality apps have stirred up. It’s something that people get is a cool and engaging experience and right away they like it. Although we haven’t seen a popular game yet have we?

Ori: Exactly. It’s gaining mindshare but it’s still very low on most people’s radar and there’s no market right now. You can’t even size the market. Before they invest a lot in this concept they’ll want to see some ROI.

Tish: So that’s the plan then. You’re basically hoping at Ogmento to make some popular iPhone apps? That’s really the first step?

Ori: Yes. That’s true. That’s what we think is going to trigger the other parts of the industry to contribute and to invest.

Tish: Right. Because ultimately a partnership between content and hardware – each enables investment and ROI in the other, its a synergistic combo. You need the big content to push the hardware companies, and you need the hardware to get the really dynamic content.

Ori: You are so right. It’s multiple elements in the industry that have to come together. You have the technology companies like Imagination and Metaio, then you have content companies like what we’re trying to do; the hardware vendors and the large content providers. Those brands that we’re trying to go after and educate them about the potential of AR. All these pieces need to come together for this market to ignite.

Tish: So is Ogmento talking to major content franchise owners or are they coming to you?

Ori: It’s both ways actually. Mostly we’re still educating. That’s one piece of the puzzle, the other piece is connecting them with various AR developers out there who have great ceoncepts, but don’t have the expertise or connections to market it.

Tish: So Ogmento acts as an agency for them or how would you describe it?

Ori: It’s more of a publisher-developer relationship. Similar to the gaming industry where you have game developers and publishers.

Tish: At the minute are you working with any particular team or are you still in the early stages with that?

Ori: We have several projects in the early stages. Follow us on Ogmento.com or Games Alfresco to see the progress with these projects in the future.

Tish: Have you had seen anyone coming up with good ideas for a Green Tech AR application? Seems that visualizing emissions and carbon footprint for everything in our lives would be a big step forward in taking action to make the changes we need to avert a climate catastrophe – seeing is believing!!

Ori: Not really yet. I was kind of playing with this idea (see my previous interview with Ori here) about that but the technology wasn’t really ready for this kind of application. I think that when it happens this will be a very important area.

click the image above to see video

Tish: I was thinking if we could organize a prize, for development of really good Green Tech AR apps. that would help.  It is early for Green Tech AR because it really involves a level of instrumentation and visualization/alignment of media with nearby objects  that is hard to do at the moment (although the necessary instrumentation is on its way to becoming ubiquitous – see Google’s Android to invade homes and Touch Revolution household Android devices coming this year). And smart energy monitoring like Current Cost is already widely available in the UK.  AMEE – the world’s energy meter is integrated with Pachube and can be used to calculate the carbon footprint of a Pachube feed that is monitoring some kind of energy consumption.  And many people are interested (see Tom Raftery’s Green Monk post here) in this kind of application that will really advance the usability of much Green Tech. So we have some ground work for a competition already!  Particularly, I think, if the competition focuses on a target instrumented environment.

[Note: Pachube has produced a really nice augmented reality demo (see video above) where 3D Pachube visualisation data is overlayed in realtime ‘on top’ of Arduino sensor boxes around their office. The demo  “features the Pachube office carbon footprint wall with chalk-drawn augmented reality marker, for Dennou-Coil-style space-hacking.” The code is written in c++ using openFrameworks, ArToolKit and the ofxPachube library and is being cleaned up for public release soon.  As the Pachube team notes, this “could provide an easy way to inspect rooms (or streets) full of sensor and environment data via a camera (e.g. iPhone)… or even AR goggles!]

Ori: Yes a prize is a good idea!

Tish: What do you think would be a reasonable sum to get the right kind of developers into that?

Ori: I think fifty thousand is the minimum. Or a publishing deal.

Tish: $50,000 is the minimum. And how would you offer it?

Ori: A cash prize for the first place and a little bit less for the second and third.

Tish: I’ll start talking to people. Maybe Ogmento could help adjudicate it, if we can find a way to raise the prize money, would you be willing to help run a competition?

Ori: We would love to. Just like we discussed before, it’s a matter of understanding the  current technical limitations, and then designing something that works within those limitations and delivering an added value.

Tish: What have been the most exciting things that you’ve seen since we’ve last talked. What are the things that got you going wow this is moving forward?

Ori: First If you just looked at the Google trend on Augmented Reality, it has the right curve. It’s really growing exponentially at this point. Also I really like a couple of articles just last week, one in the New York Times and one in Venture Beat. What was unique about the article in Venture Beat is that it was probably the first roundup of Augmented Reality companies. It wasn’t very comprehensive or detailed, but it might be a sign that VCs are getting interested in AR companies.

Tish: Yes.

And, in a way AR is perfect because it’s suited to the kind of level of investment that people are interested in right now – relatively small scale investment. And what’s good about particularly say iPhone apps, you can do a lot with relatively little can’t you?

Ori: Yeah, I agree. But to answer your question, I think the most exciting thing that is happening is the (small) swarm of AR developers popping up in various corners of the world and are looking to bring their ideas and products to the market. I think that’s unique and we haven’t seen it until this year.

Tish: Why are we seeing most things coming out of Europe like Layer, Wikitude, Imagination?

Ori: It’s a true statement. You know what? I have no idea why. I really don’t know.

Tish: Is there anything special coming out of Silicon Valley that’s normally leading the field in these kind of things?

Ori: I can’t say, but at least…there’s Ogmento!

Tish: Yes! There is Ogmento. Right. But it is interesting isn’t it? I think I know the answer to my own question. It’s because mobile culture is very well developed in Europe and this is springing out of mobile culture.

Ori: There is another important initiative from the US – the AR consortium which Robert Rice and I have recently launched…

Tish: Oh yes I must ask you about that. And Robert’s been really working hard on the big problems of full on AR- massively multiplayer, 3D, virtual goods market, tight alignment of media/ graphics with real world objects, partnerships to move the hardware forward – eyeware etc.  I know he has some exciting demos planned for ISMAR 2009,  it’s just that he’s still under the radar.

Ori: Robert is going after the hard problems, which is good.

Tish: It is great. Actually  I just spoke to him today and it seems like he’s a few months off being able to show us something. He is working hard to push the hardware forward.

Ori: I think he is and he’s probably targeting existing platforms.

Tish: Which is were we get back to the snake biting the tail – great content need good hardware and hardware investment comes from seeing great content.  What does the current crop of AR browsers have to do to take it to the next level?

Ori: I think what they’re doing right now is a very good first step. Which is opening it up just like Wikitude and Layar. It’s kind of what made a difference for Facebook versus other social networks and for Twitter as well. Their approach as a platform is what made it so huge. The other step which Wikitude is doing with wikitude.me is allowing people everwhere to tag the world.

Tish: They’re the first who are doing that so far.

Ori: There were other attempts to provide ways to tag the world, but I think Wikitude is the first that is actually available on a global scale.

Tish: Have you had a chance to try out Google Wave yet – do you see some potential for some interesting AR integrations with Wave? I think Wave may be a big game changer in real time communication, if it gets mass adoption. It basically makes the web like a videogame – bringing a real time many participant shared interaction to the web. I have been exploring in the sandbox and there are some interesting possibilities for role playing games. It’s completely open so it could be integrated into an AR project.

Ori: I am going to have to look into Wave in the next few days.

Tish: So what should we look out for from Ogmento in the next few months? What do you want to develop next?

Ori: I have given some hints above – you’ll have to stay tuned to discover…