Picture on the left Mirrorshades, picture on the right a Metroid Hud.

“Augmented Reality is like a Philip K Dick novel torn off its paperback rack and blasted out of iPhones,” Bruce Sterling in Beyond the Beyond

“a techno visionary dream come true – those are rare, really rare, you have to be patient,  it’s super cyberpunk”… Bruce Sterling, “At the Dawn of the Augmented Reality Industry.”

The Dawn of the Augmented Reality Industry continues to brighten, and now we have two augmented reality companies, Total Immersion and Ogmento, firmly established in Hollywood – the dream mother of so many of our augmented realities.

Ogmento is the most recent of these two pioneering augmented reality companies to set up shop in LA.  Total Immersion’s CEO Bruno Uzzan moved to LA from France two years ago, although he still has a fifty person RandD team in France.   Total Immersion began 10 years ago in the quiet, lonely, hours before the dawn of an AR industry.  But Total Immersion’s AR toys for Mattel, and augmented reality for Topps baseball cards, fired CNet writer Daniel Terdiman up enough to say, “I have seen the future of toys, and it is augmented reality” (see full post here on CNet).

Recently, I talked with  Ori Inbar, one of the founders of Ogmento and  the premier augmented reality blog Games Alfresco about his new venture in Hollywood. Bruce Sterling, @bruces, had some fun with my invention of brand new augmented reality trade jargon here!  Ori pointed out Ogmento brings two important new facets to the rapidly growing augmented reality field: firstly they are bringing leadership from veterans of the entertainment industry into augmented reality development. Brad Foxhoven and Brian Selzer from Blockade have partnered with Ori on Ogmento.  And, in an another important step forward for a young industry, Ogmento announced they will be acting as publishers for a fast growing cohort of augmented reality application developers and helping AR development teams out there bring their concepts to the market.

So I was very happy also to have the opportunity to talk with Brian Selzer.  Bruce Sterling pointed out in his seminal sermon from the flatlands at the Layar Developer Conference, AR is kind of a “Hollywood scene.” We have seen the web early adopter/developer/blogger community  embrace augmented reality browser experiences in recent weeks in an awesome wave of enthusiasm. Are Hollywood creatives equally smitten? For the answers see the full interview with Brian Selzer below.

Brian Selzer (www.brianselzer.com and twitter – brianse7en ) has an extensive involvement with emerging platforms:

“from launching dot com entertainment sites in the late 90′s to creating early versions of social gaming platforms, or bringing big brands like Spider-Man and X-Men into the mobile space for the first time.  Last year I was focused on bringing video game characters and worlds into the online space as UGC [user generated content] projects (mashade.com, instafilms.com).”

I began my own career in Hollywood doing motion control photography and creating software that bridged the language of robotics and servo motors with the visions of  film directors. Eventually our little company, NPlus1, moved on to 3D vision systems and image recognition stuff.  So yes, I have been really, really patient waiting for this particular techno visionary dream.  And, while I have been waiting for augmented reality to manifest, I have grown to love the internet.  But now, how awesome, It is OMG finally for mobile AR!

Augmented reality is busting out all over – through our laptops, our phones, on the streets, toys, baseball cards, art installations, sticky light calligraphy and more.

Many of my questions to Brian were directed at at how and when we will see augmented realities with near field object recognition, image recognition and tracking and, of course, the illusive eyewear.  As Bruce Sterling points out we are just at the very, very beginning – the dawn of an industry.  I created the photomontage below on the right to compliment Tonchidot’s illustration suggesting the evolutionary inevitability of holding our phones up (below on the left).  The Evolutionary Reality of AR will not end there.  It is just a step into eyewear, hummingbirds or Nano Air Vehicles, and more…….

The Evolutionary Reality of AR

Cartoon on the left by Tonchidot on the right a collage of a stock photo and the Pentagon’s Robot Humming Birds – “Nano Air Vehicles.”

While we finally we have, an affordable mediating device with the horse power, mindshare and business model to bring AR mainstream with the iphone.  The much anticipated Apple 3.1 Beta SDK to be released in September will not, I am sure, open up the Video API at the levels that augmented realities with near field object recognition and tracking require (I would love to be proved wrong though). But the magic wand to deliver even tightly registered AR graphics/media (that require a lot of CPU and GPU) to a wide audience is in our hands, so full access to may not be far off. And others, of course, can/will/might knock the iphone off its current pedestal.  AR made it’s mobile phone debut on the Android after all.

Like everyone else who loves AR, I wish that Apple would open up faster (and I wish Android would manifest on some rocking hardware). But we will see enough of the iphone Video API open for the next generation of mobile augmented reality games and applications to emerge in the coming months.

One of these will be Ogmento’s. Although Ogmento is in stealth mode, they have released a teaser for their first game, “Put A Spell,” developed by ARBalloon – screenshot below.  Ori did reveal to me in this interview that they are doing image recognition and using the Imagination AR engine.

As Brian notes, Hollywood has had the AR bug for a long time. AR has been everywhere in Science Fiction Movies and video games. Nintendo’s SPD3 head Kensuke Tanabe, “effectively the man in charge of overseeing all the Metroid franchise underneath original co-creator Yoshio Sakamoto,” explains the story of Metroid to Brandon Boyer of Offworld here (an image of a Metroid Hud on the right opening this post) :

“the idea of the different visors you use in the Prime games to interact with the world: the scan visor, for instance, set the game apart from other first person shooters in that the player was using it to proactively collect information from the world, rather than having the story come to them passively, in the form of cut-scenes or narration. “Prime could have adventure elements with the introduction of this visor,” says Tanabe, “That’s how we came up with the genre — first person adventure, instead of shooter.”

But as Brian points out:

“the light bulb has been lit and Hollywood is seeing that the software and hardware are here today to deliver these types of AR experiences in real life (to a lesser extent of course, but the path is getting clear).”

Talking with Brian Selzer

Tish Shute: Bruce Sterling’s sermon at the Layar Developer conference, “At the Dawn of the Augmented Reality Industry,” was absolutely awesome. He spread the future feast/orgy of augmented reality before us  – and described many of the dishes we will tasting both delectable and diabolical.  One of the many things he points out is, AR is kind of a “Hollywood scene.” And, as Ogmento is one of only two augmented reality companies in Hollywood, I am interested to hear how it looks from your neck of the woods. We have seen the web early adopter/developer/blogger community  embrace augmented reality browser in recent weeks in an awesome wave of enthusiam – are Hollywood creatives catching the buzz?

Brian Selzer:  It was a thrill to hear Bruce Sterling mention Ogmento. I devoured all of his Cyberpunk books back in the 80′s, along with writers like Gibson, Rucker, Shirley… To me, sci-fi writers are the visionaries who define and influence our technological paths into the future. They make science and tech sexy enough to want to manifest those experiences in the real world. Clearly Bruce sees the AR industry as being sexy. I love that he called it “a techno-visionary dream come true… and super-cyberpunk.”  And yes, kind of a Hollywood scene.

Hollywood creatives caught the AR bug before they knew what AR was.  Look at science fiction movies and video games to see AR everywhere. Terminator, The Matrix, Minority Report, Iron Man.. the list goes on.  Look at any video game with an integrated heads-up display.  It’s clear Hollywood loves AR.  It’s only been in the past few months though that the light bulb has been lit and Hollywood is seeing that the software and hardware are here today to deliver these types of AR experiences in real life (to a lesser extent of course, but the path is getting clear). So yes, the buzz is here and it’s strong.  With that, we all have to be prepared for the good, the bad and the ugly as AR goes mainstream.

It certainly goes to show how young this industry is when Ogmento and Total Immersion are currently the only AR companies based in Los Angeles. It’s very exciting to be the only company right now demonstrating a natural feature tracking (markerless) iPhone experience in Hollywood. We are in talks to bring some very big brand and properties to the mobile AR space. The goal is to deliver experiences that create added engagement and value to the consumer.

Tish Shute: Also in his landmark sermon Bruce Sterling noted that augmented reality has been around for 17 yrs and now at last we are seeing the dawning of  an augmented reality industry. What inspired you to take up the challenge of launching an augmented reality company in Hollywood?  Oh congrats that Bruce Sterling name checked Ogmento in his list of companies that prove that this really is the dawn of an industry!

Brian Selzer: I’ve always been involved in emerging platforms… from launching dot com entertainment sites in the late 90′s to creating early versions of social gaming platforms, or bringing big brands like Spider-Man and X-Men into the mobile space for the first time.  Last year I was focused on bringing video game characters and worlds into the online space as UGC projects (mashade.com, instafilms.com). Working with all these great CG game assets, I continued to think about what’s next, and that’s when I started to follow AR very closely and started engaging with those who were pioneering in the space.

I remember swapping instant messages with Robert Rice (@robertrice) right after the 2008 Super Bowl.  We were not chatting about the football game, but rather about some of the commercials that aired during the event as a sign that AR was making its way into the mainstream.  A lot of people became aware of AR for the first time when the GE SmartGrid commercial aired.  There were all these YouTube videos popping up of people blowing on holographic wind turbines.

The commercial that really got me excited though was the Coke Avatar commercial.  In that commercial people in the city were sporadically being portrayed as their digital persona’s, avatars, gaming characters, etc..  For me that spot did a great job showing how many of us already have these ‘alter egos” that live in cyberspace, and how the line between these worlds can sometimes be blurred. I remember watching that commercial and thinking that is exactly the type of experience I’d like to create with mobile AR.  I want to overlap the virtual world into our every-day reality. Why cant I bring my World of Warcraft or Second Life persona with me into the real world?

I am big on the notion of “Games and Goals.” I believe that games have the power to motivate people in a very powerful way. By challenging ourselves while playing a game we can climb mountains.  Augmented Reality is the perfect platform to bring gaming into the real world.  By mixing the virtual world with the physical world, this added layer of perception provides a very powerful experience for something like a role-playing game.

One of my earlier social-gaming projects was a website called Superdudes.  This was a “Be Your Own Superhero” concept that celebrated and motivated kids to create superhero avatar/persona’s online, and we gave members all sorts of games, challenges, and rewards, some of which carried into the real world. The site recognized members for teamwork, creativity, volunteer work and things like that. So the Superdudes were often involved in charity events and benefits to help children.  Everybody called each other by their Superhero names, and the line between fantasy and reality were being blurred.  This project really got me thinking about what happens when you take positive role-playing like this and mix it into the real world.  I started to work on a plan for location-based activist missions for points and rewards, but never got to complete that. So I have some unfinished business here.

I think it would be fantastic to be able to show up to some type of fun event with friends, and everybody could see each others alter ego personas standing before them. When you can turn the world into a playground, and use the power of gaming to make a positive impact on the planet… well, I don’t think there is anything better than that.  These are the types of projects that drive me, and I think AR is the best platform to support these types of social gaming experiences.

Tish: Does Ogmento have any RPGs under development?  I noticed in the Google Wave on RPG someone has been working on doing something with the Dungeons&Dragons API.  I am interested in exploring the web of protocols underlying Wave as a transport mechanism for multi-person, mobile, AR experiences (not requiring downloads), on an open global outdoor AR network. If not Wave, what do you see as the potential infrastrucure and protocols we could harness for an open augmented reality network?

Brian:  Ogmento has a deep background in video games and we interact regularly with most of the major game publishers. As a company we are not so much developing our own RPGs right now, but rather exploring what mobile AR extensions make sense for existing brands.  There are many limitations to location-based gaming, but a global AR network is exactly along the lines we are thinking.  Lots of discussions are taking place on protocols, platforms, API’s, and there are numerous ways to approach this.  We need to be able to use what’s available now and continue to refine and customize for AR’s specific needs and issues as we progress.

In general though, Ogmento is focused on what types of experiences can be had today and over the next couple of years. I still think we are several years out from a truly open augmented reality network.  We are certainly looking at launching our own “Ogmented Network” which would support some fun treasure hunt type experiences, or add an entertainment layer on top of traditional outdoor marketing campaigns.

Tish: I don’t know whether you have read Thomas Wrobel’s ideas for an open augmented reality network that I just published here on Ugotrade.  The principals he talks about are very important for augmented reality to become a major part of our lives – .  Considering the difficulty open networks can pose for emerging business models how can we fund the development of an open framework for augmented reality?

“a future AR Network, I mean one as universal and as standard as the internet. One where people can connect from any number of devices, and without additional downloads, experience the majority of the content.
Where people can just point their phone, webcam, or pair of AR glasses anywhere were a virtual object should be, and they will see it. The user experience is seamless, AR comes to them without them needing to “prepare” their device for it.”

Brian: I think funding for these types of projects will definitely come from Venture Capital groups in the near future.  It’s early in AR, but the VC’s are watching and deciding which horses to bet on.  Until that time, it’s about service work, and developing AR experiences for others with what is possible today. That work will help fund internal development of original AR products, and platform development.

Tish: How did you get started with Ogmento?

Brian: My first conversation with Ori was actually about my interest in Location Based RPG concepts.   We had a long conversation about the possibilities with AR, and it was clear that we shared similar interests, but were coming from different complimentary backgrounds. The idea of collaboration was exciting, so we just kept talking until the timing felt right. Now, with Ogmento we bring a unique blend of AR development experience with a deep backgrounds in AR technology, animation, video games, entertainment, social media, etc.   I think this is a powerful mix that will allow us to do some great things.

It’s still so early, and things are just getting started in AR. There are only so many webcam magic tricks you can enjoy before you are ready for something else.  The location-based apps have the most potential in my opinion, which is why we are really focused on mobile AR.   We have some board-game type projects, which do not instantly scream location-based gaming, but if you look at something like the ARhrrr board game, you can see how much more compelling it can be when the game invites the player to be actively moving around during the experience.

Tish: I am interested in your perspective on how we can create the kind AR experiences that really embody what has always been so exciting about AR – the tight alignment of graphics and media with real world objects and ultimately a rich immersive 3D experience, so I am going to hit you with a bunch of those, “Is this really eyewear or vaporware?” questions.  The real deal eyewear changes everything!

While eyeware is a big challenge technically and aesthetically,  I am pretty sure that there are several outfits out there that can pull off the optics and projection. 
Will the entertainment industry get excited enough to put a major push into delivering the eyewear in short order instead of the 5 to 10 year project that some people still think it is?    The business development challenge is bigger perhaps than the technical obstacles perhaps? What is your view on this?

And, perhaps, the eyewear is a clear example of a need for partnerships. For example, we have seen efforts from companies like Vuzix and Lumus, and recently a Japanese Company, Masunaga.

I have no reports from people who have tried the Maunaga eyewear yet.  But,  limited by small field of view, and tethered, currently eyewear offerings, available at a reasonable price point, are not workable solutions for augmented reality experiences. But the problems are not insurmountable. What will facilitate the real deal?  


It seems that it is critical to start creating hardware relationships now. The industry is costly and slow moving and as Robert Rice put it to me in a recent conversation, “once the software cat is out of the bag, its going to go wild and if the hardware isnt there, its going to stutter.”

As Ori notes some of the hardware companies like Intel and others don’t seem to be paying enough attention to AR.  Ori points out they don’t see the demand yet.  But in order to create an awesome AR experience and demand from a mass audience, don’t we need to work in conjunction with hardware designers?

Brian: It’s fun to think about who will eventually deliver a great hardware solution for AR glasses. It will happen. It would be cool to see somebody like an Oakley or Nike partnered up with a company like Vuzix to deliver something people actually might wear in public.  Perhaps a hardware manufacturer like Apple or Nokia will bring us something like the iSight or the NGaze down the line.  I’d love to see a set of glasses designed by Ideo.  Microsoft or Sony are already playing with technologies like Project Natale and the EyeToy, so I think its only a matter of time before they deliver an eyewear solution. I would even look to the toy companies to eventually make an investment here.

Gamers will be the early adopters, and in a few years we may start to see people running around in the park wearing glasses with headsets, but it will be acceptable because it’s clear they are using them for a game.  It’s going to take a very sexy and stylish piece of hardware for everyday people to be willing to wear AR glasses in public while going about their everyday business.   It’s like the recent cover of Wired magazine where Brad Pitt is wearing a mobile headset in his ear, and the editors point out that even he can’t pull that look off, so why do you think you can.  When AR glasses come in designer frames, and you can’t tell them from non-AR glasses, to me that’s when things get really interesting from a mass-adoption perspective.   Compare how many people were carrying around a mobile phone in the 80s to now.  I think it will be the same thing with glasses.

I was in an AR pitch meeting the other week at a very significant media company, and brought up the point that today’s handheld Smartphones will eventually evolve into tomorrows Smartglasses. My comment was quickly shrugged off as sort of a sci-fi notion that was irrelevant to the business at hand.  Probably true, but I think it is important to understand where digital media and entertainment is going, so you can adapt quickly, and evolve into those spaces more naturally.  The more we see people walking around with their Smartphones in front of their face (like a camera), the sooner it will be that we make the jump to eyeglasses as a key hardware device for AR experiences.

At Ogmento, we definitely are working on AR experiences with the hardware and software available today.  We will get some product out this year, and 2010 will be a banner year for markerless mobile AR in general.  I think the entire AR community is looking forward to bringing this technology to the mainstream in the form of games, marketing campaigns, virtual docent apps, and much more.  It might not be the full experience we are all dreaming about for some time, but we can see the path and the true potential, and it’s pretty spectacular.

You mention the tight alignment of graphics and media with real world objects.  That is really our focus. A lot of well-deserved attention is going to the browser overlay “post-it” approach right now, which uses compass and GPS.  We are focused on markerless natural feature tracking, so once you identify something that is AR enhanced in your environment, you can interact with that integrated experience.  On an iPhone that can be as simple as using your touch screen to interact.  When you are wearing glasses, it becomes more about visual tracking. There are lots of smart people thinking through these issues. Many of which you have interviewed. It is my hope that there are exciting collaborative efforts to be had in the coming months to get us all there together and faster.

Tish: Bruce touched on some of the hard problems that have to be solved for augmented reality – and he noted for instance security needs to be tackled in the early stages. Robert made a nice list, “privacy, media persistence, spam, creating UI conventions, security, tagging and annotation standards, contextual search, intelligent agents, seamless integration and access of external sensors or data sources, telecom fragmentation, privilege and trust systems, and a variety of others.” Will Ogmento be leading the way in solving some of these hard problems?

And, won’t trying to solve these hard problems for networked AR in walled garden scenarios one company at a time lead to a lot of reinventing the wheel wasted energy?

Brian: These are all important issues, and again there are a lot of smart people thinking about solutions to these problems on a daily basis.  Ogmento is interested in partnering with developers and supporting their efforts as a publisher of mobile AR experiences.  While we intend to roll up our sleeves in these areas, we are currently more focused on taking AR mainstream with the hardware and software available today.  As the industry evolves, so will Ogmento. As the opportunities evolve, our ability to make a greater impact tackling these issues will be realized.

Tish: Another area of development that could really kick AR into high gear might be creating augmented reality hotspots  where we use can deliver the kind of location accuracy/instrumentation necessary to create interesting AR experiences (partnership with Starbucks, perhaps ?!).  Augmented reality hots spots, could deliver the kind of high quality AR experience that isn’t possible ubiquitously at the moment, and may be a real way to get people really exploring the potential of AR now, rather than later?

Brian:  Agreed. I see a great opportunity here with this approach.

Tish: Although there are many obstacles to Green AR – the energy hogging servers at the backend for starters! Last week I had a conversation with Gavin Starks, AMEE, and Robert Rice and Jim Purbrick about how to work with AMEE and the technology available and encourage Green Tech AR development (see an early exploration of green tech AR from Pachube here).

We came up with the idea of holding a competition perhaps centered around a targeted instrumented space. But I would really love to hear your thoughts on the topic of Green Tech AR (the energy hogging servers at the back end being the first cloud on the horizon!.)  Cool GreenTech AR imaginings, social gaming ideas, RPGs, not even necessarily even tied to the immediately practical, would be like rain in a drought!

Brian:  I go back to “Games and Goals”… If you make environmental and other activist efforts fun and rewarding, more are likely to be motivated and participate.  Can you imagine having a personal “carbon footprint stat” floating over your self at all times? Or over your home or factory?  How would that change your behavior?  We all love stats. Look at how the Nike+ campaign has used technology and gaming to motivate people to run.  I think there is a lot that can be done to make being green fun. It starts with the individual, and spreads from there.  Keep me posted on that one!

Tish: I would also like to explore further the RRW suggestion that ambient intelligence is both the Holy Grail of AR and possibly snake oil:

“The holy grail of the mobile AR industry is to find a way to deliver the right information to a user before the user needs it, and without the user having to search for it. This holy grail is likely in a ditch somewhere beside a well-traveled road in the district of the semantic Web, ambient intelligence and the Internet of things. Be wary of any hyped-up invitation to invest in a company that claims to have gotten the opportunity right. What we’ve seen in the commercial industry to date is a rather complex version of a keyboard, mouse, and monitor.”

So Holy Grail, Snake Oil, or a ditch somewhere….?

Brian:  I instantly think of Minority Report, where Tom Cruise’s character is being bombarded with holographic ads personalized with his name and to his current situation.  In the future, Spam is a nightmare, especially when it knows who you are.  I think the key thing here is delivering “the right information”, and we still dont have that down. I do see a day where we can truly customize what comes to us, how we want it, when we want it.  My future vision of ambient intelligence is the ability to “turn everything off” if I want to… block out the stimuli and replace it with images of nature, or natural surroundings, etc.  Where I live in Los Angeles, we have those digital billboards everywhere, so it’s like advertising overload wherever you look (hints of Blade Runner).  I personally don’t mind them, but I know there is great debate on there being simply too many billboards everywhere. So AR would only add to the noise of life by adding yet another digital overlay of information, right?

Perhaps the holy grail is to use technology to filter things out. AR might become a solution to leading a simpler life, or a perfectly customized life if you want that. Ultimately the control needs to be with the individual.  I guess I am talking about something like TiVo taken to the extreme.

Tish: And then that other biggy – augmented reality search! I am asking this next question of  Wikitude and Sekai camera too and now I must also ask Acrossair and several others I guess! Obviously a huge area of opportunity in this broader landscape that uses location-awareness, barcode scanners, image recognition and augmented reality is to harness the collective intelligence – a whole new field of search. There is the beginning of a discussion on this in the comments here.

What will it take, in your view, to become a leader in augmented reality search?

Brian:  I’m more of a content guy, so I tend to focus on things like UI, quality of creative, etc..  From that perspective, I am looking forward to evolving beyond the “post-it” text overlay user-experience we see now in AR search. I was impressed with the TAT Augmented ID concept and hope we start seeing more smart design solutions like that emerging in the space.  There are some great new design approaches coming out of the location-aware space that should be applied to AR search. I’ve been studying the heads-up display designs being used in video games, and re-watching movies like Iron Man for ideas. This is another example where Hollywood has painted a polished picture of what AR can and should look like, and the masses have already accepted these design approaches.  So from that perspective, from my view the leaders in search will be delivering sexy, smart and simple solutions. It’s all about the S’s.

Today, I was very excited when Thomas Wrobel sent me a draft of, “Everything Everywhere: A proposal for an Augmented Reality Network system based on existing protocols and infrastructure.”

Thomas has kindly agreed to let me publish his draft, to open a discussion on this topic. The diagram opening this post (click image to enlarge) shows, “An example of how collaborative 3D-spaces could be shared over existing IRC networks.” It is from Thomas’ proposal. The full text of his paper is included later in this post.

“Can we try to avoid a browser war this time?”

Thomas notes in the closing remark to his paper:

“I am absolutely confident in my belief AR will become at least as important as the web has, and probably a lot more so. It will also face much the same hurdles and challenges getting established as that medium did. But, speaking as a web-developer, can we try to avoid a browser war this time?”

Thomas Wrobel has consistently posted insightful comments on how existing standards could be used for creating open augmented reality networks. But he expressed concern to me that his work and this paper not be overplayed:

“I’m hardly a leader, I’m just an amateur with a load of ideas on AR-related topics, some which might be useful, others might become unworkable. I don’t want anyone to get the impression this is how I think it has to, or should be done.”

I have brought/am bringing up this topic of using existing standards and infrastructure where possible for open augmented reality networks in all my interviews with members of the AR Consortium.

And I am finding agreement on a point that Robert Rice makes, “there is no perfect, ultimate solution *now*, but we have to do *something* to work from and refine/evolve.”

Thomas Wrobel makes what I consider some crucial opening suggestions. I take my hat off to him for thinking about this early, coming up with some clear, elegant, and practical ideas, and doing the work to articulate these ideas so others can participate in evolving them.  Massive props for that, many times over.

Good ideas on standards at an early stage of  a developing industry like augmented reality are like spring sunshine and April showers for new crops. No one knows what storms and pests the growing season will bring – but water and sunshine (open standards) are always a good start. And, personally, I can’t wait to see how this new industry unfolds (see Bruce Sterling’s Layar Conference awesome keynote : “At the Dawn of the Augmented Reality Industry.”)

Thomas Wrobel is:

“a web developer working for a small, brand-new company called Lost Again, which mostly works on ARGs (That is, the alternate reality games, not the augmented reality games, although there’s probably going to be big overlap there in the future). We developed two educational ARG games for the Netherlands with a company called res-nova.”

I have been following Alternate Reality Games  through the amazing work of Elan Lee and Fourth Wall Studios. Like Thomas, I think the intersection of ARGs and augmented realities is going to be very interesting. Thomas wanted me to point out that the website for his company with Bertine van Hövell, http://www.lostagain.nl/, is just a placeholder for now.

“Probably be up fully within a week or two. And, “despite the logo, we aren’t an AR company [yet], or a travel firm. The logos supposed to represent being lost in our minds.”

Thomas has been thinking about the topic of an open augmented reality network for a while now.  He is an artist also known as DarkFlame and his ARN network is included in this augmented reality concept for 2086 he did in 2006 (click on image below to enlarge).

Beyond IRC

Both Thomas and Mez Breeze made extensive and insightful comments on my last post, “Augmented Reality – Bigger Than the Web: Second Interview with Robert Rice.” And in particular they both picked up on something I am very interested in – the potential use of the Google Wave Web of protocols in creating open augmented reality networks.

Mez in her brilliant brainsplosion on social tesseracting takes on the very definition of information:

“Tish, when you ask Robert “…what is your approach to delivering a massively shared real time [augmented reality] experience that is like Wave not confined to a walled garden?” that’s an extremely relevant question + one that needs to be addressed while considering the entirety of the Reality-Virtual Continuum. I’ve recently finished a series of articles addressing this: the framework I’ve developed is termed “Social Tesseracting.”

I have recently begun exploring the Google Wave Web of Protocols which are nicely outlined in this post by J Aaron Farr which includes the very interesting diagram below (so more on Google Wave in another post).

But, as Thomas notes, while he demonstrates his ideas using IRC (Internet Relay Chat) they reach Beyond IRC:

“As mentioned before IRC has some drawbacks, which are due to its age or method of working. As such, future systems might yet prove better alternatives for a open AR network. One example of such a system is Google Wave. It shares many of the advantages of IRC (open, anyone can create a channel of data, different permission levels can be set and its free), while avoiding some critical restrictions. (The data can be persistent). I believe some of the ideas I’ve mentioned, and possibly even the proposed protocol string could be adapted for Google Wave or other future systems. I believe overall the principles are more important then any specific implementation to get to them”

Also Thomas pointed that while he uses markers to illustrate some of his examples, they are just a method for tracking.  What he is presenting is going to be transparent to the methodology of registration/tracking.

Tish Shute: You mostly use marker based examples but there is no reason why the principles you are suggesting will not be just as relevant as we move more into using more sophisticated image recognition tools is there?

Thomas Wrobel: No reason whatsoever. I mostly choose familiar markers as something that could be used now, with a lot of coding library’s already established for them. I think for most future AR use, markers will go completely…especially outside. Either things will be done purely by gps, object recognition, or the (in the case of advertising) markers will look like normal posters.

However, I do think traditional markers might “cling on” as being used for non geographical specific stuff at home. After all, if you need some reference points for moving mesh’s about in real time…(say, when playing a board game with a friend on the other side of the world)….then there’s probably nothing that’s going to be more practical then some simple bits of paper or card.

Everything Everywhere

-  A proposal for an Augmented Reality Network system based on existing protocols and infrastructure.

by Thomas Wrobel

The following paper is my vision of a open AR Network and potential methods to implement it with existing technologies. Specifically I’ll be focusing on a potential for a global outdoor AR network, although the ideas aren’t limited to that.

Of course I call it “my” vision, but I’m obviously not the first to have many of these ideas. I have been influenced and inspired by many things…

[Some of Thomas Wrobel’s influences – watched and played. Images  from Mitsuo Iso’s Denno Coil (Click to enlarge) top,  below from the game “Metroid Prime,” and Terminator, and the last from Buffy the Vampire Slayer!]

The AR Network.

When I speak of a future AR Network, I mean one as universal and as standard as the internet. One where people can connect from any number of devices, and without additional downloads, experience the majority of the content.
Where people can just point their phone, webcam, or pair of AR glasses anywhere were a virtual object should be, and they will see it. The user experience is seamless, AR comes to them without them needing to “prepare” their device for it.

From this point forward, I will refer to this future AR Network simple as the “Arn”.

The Arn should be an inclusive, and open platform where any number of devices can connect to, and anyone can make and host their own location-specific models or data.
It should allow people to communicate both publicly and privately, and not have their vision constantly cluttered with things they don’t want to see.

There’s two old, existing paradigms that I think can help reach this goal when they are combined.

The Internet Relay Photoshop.

IRC, or Internet Relay Chat was a chat system designed by Jarkko Oikarinen in the late 80′s.

Its a system where people meet on “channels”, they can talk in groups, or privately. Channels can be read-only, or open to all to contribute to. There is no restriction to the number of people that can participate in a given discussion, or the number of channels that can be formed. All servers are interconnected and pass messages from user to user over the network.

To me, this relatively old internet technology is a great template, or even foundation, for how the Arn could operate. Rather then text being exchanged, it would be mesh data (or links to mesh data), but other then that much of the same principles could apply.

People could join channels of information to view or contribute. Families could leave messages to each other scribbled in mid-air on private channels. Strangers can watch AR games being played between people in parks. People going into a restaurant could see the comments from recent guests hovering by the menu items.
None of this would have to be called up specially, if they are on the right channel when it was broadcast, they will see it.

The IRC paradigm becomes particularly powerful when combined with another one common to many computer users; that of a “Layer” in an art program, such as Photoshop or Paint Shop Pro.
As most of us know, layers allow us to separate out different components of a piece of art while editing, either to focus our attention on one piece, or to make future editing easier.

Now what if we simply have each “channel” of information represented as a layer?

Click to enlarge image above.

Having channels corresponding to layers is an easy and intuitive way for the Arn to operate. The user can login and contribute data to any channel, like IRC as well as adjusting the desired opacity and visual range of each layer, like they would a layer in Photoshop.

In this way they can get a custom view of the world, both with shared and personal AR elements visible at the same time.
They would not have to switch between various overlays to their world view, as they could see many at the same time.

Persistence of Data

With IRC or IRC-like system to communicate the data sent is mostly temporary data…broadcast on the fly from user to user and device to device. Retained in the users local logs, but not “hosted” anywhere.

I think for the majority of day to day purpose’s this is not so much a drawback, but actually desired for AR. Most casual communication doesn’t need to be recorded permanently in 3D space and, indeed, if it was, the cost of running such a service would increase exponentially with users and with time. Not to mention, our visual view of the world would get very cluttered very quickly. Imagine what your monitor would be like if it kept a history of every window you have ever opened and their positions!

So for most cases AR space should be treated like a 3D monitor letting us display many pieces of data from remote and local sources, and even to share them with others, but not being, by default, a permanent record for it all.

Most data will be analogous to pixels on a display, and if kept in records its only on the clients devices, not on the network itself.

However, occasionally we do want 3d data analogous to a web-page, such as (in the example above), the map layer. Data here should be persistent and visible to all that have that layered turned on.  I see no reason why hosting this data needs to use anything else but standard web-hosting with the (read only) #channel on the Arn merely providing a route to the data.

As the user logs onto the channel, the server, using a chat-bot, can send them a list of meshes with location data attached, and the Arn browser can simply pick the data to display that’s local to them. (Note 1: By doing it this way around, it allows some degree of anonymity to be possible, rather then the server knowing exactly where you are and feeding the specific correct string to you.)

We simply need to establish standards so this data can be pulled up and interpreted.

For instance, this standard could be as simple as a XML string pointing to a KML file on a server. This could then be then displayed in the users field of view at the co-ordinates specified.

In this way permanent data tied to locations, such as historical overlays or maps, could co-exist on the same protocol as temporary data such as mid-air chat’s or gaming related meshes.

There is also no reason why this shared-space/personal spaces based on channels of data has to be restricted to things given absolute co-ordinates.

(Different ways to access the same mesh)

It could work just as well with Markers and thus relative co-ordinates.

This would be mostly useful for indoor use, letting people logged onto a channel see the same meshes as everyone else on the markers. Thus allowing multi-player AR games, or AR games with observers very easily.

For example; games like Chess could be played between people with no additional code needed; You simply have a set of markers for only your own pieces, and as you move them the channel updates with the new positions, which are displayed in place in your opponents field of view.

This sort of game comes “free” with just having a generic system of shared space supporting markers.

It would also allow AR adverts down the street or in magazines to be viewed by simply logging onto the right AR channel

If markers are designed with URL data in them, this could even be a prompted or automatic process.
“There is visual data in this area on the following channel; #ABCD would you like to view this channel?”

Pros and Cons of using IRC or IRC-like systems

Pros;

• Anyone can write a IRC interface software.
• Anyone can create new IRC channels without cost
• Channels can have read and write permissions set.
• Users can easily have multiple channels open at once.
• Already established with thousands of severs worldwide.

Cons;

• 500-or-so character limit. 3D data must be linked too, not sent.
• Slow update rate. Lines of data can take a whole second or more to send.
• Non-persistent. Good for a 3d-view, not good for storage.

An example of how collaborative 3D-spaces could be shared over existing IRC networks;

Click on the image to enlarge

While in the long run I would hope for a dedicated AR network to be developed, with greater flexibility with persistence of data, there is a lot that can be done with the existing IRC system to implement the ideas mentioned above.

Below I will show an example of simple, crude, pseudo-protocol that could be fairly easily implemented to create shared AR spaces broadcast across IRC channels.

Its important to note, the goal here isn’t to exchange the mesh data itself on IRC, its to exchange links to the data.

Exchanging the mesh data directly within the 500 character IRC limit would be very hard, and liable to errors.

It’s also a waste of network bandwidth, as many people logged onto the channel might not have that object in their field of view, so their clients should not bother downloading it. (it should be up to the client browsers when to anticipate and cache mesh data).

Proposed Basic XML link exchange for AR;

Principle;
As user creates or changes an object, the clients software  posts a simple xml formatted string to
the IRC channel.
Anyone logged into that channel then sees that mesh displayed in the specified location.

This string could be formatted as follows;

<Mesh
ID=”DARKFLAME:1”
Obj=”http://www.darkflame.co.uk/mesh/church/chuch.kml”
Loc=”(49.5000123,-123.5000123)”
Permissions=”None”
LastUpdate=”12/12/0000,2012:12”
/>

This string allows other users client logged into the channel to automatically load the object from the URL and display it at the correct position in their field of view.
If the permissions are set to allow it, they could then move the object themselves, with the update being feeding back seamlessly to other users on the channel.

The objects posted are given an ID, which can be just the posters name, followed by a unique object number for that name. These unique ID’s would allow clients to track different instances of the same mesh, as well as making it easy to implement permissions. (if only the poster should be allowed to move this object, then the clients simply check if ID matches the user name posting the update. If its not, they can ignore it).

Next the objects need to be linked to a mesh.

The location of the objects mesh doesn’t have to be a fixed remotely-hosted url, it could be an IP address and port number of the user posting the mesh,hosted by the application posting the link to the channel.

Obj=”www.darkflame.co.uk/mesh/church/chuch.kml”
Obj=”123,223,14,23::3030”

The objects co-ordinates, likewise, need not be specified as absolute gps co-ordinates, but instead could refer to generic Marker.

Loc=”(49.5000123,-123.5000123)”
Loc=”Marker1”
Or relative to a marker;
Loc=”Marker4 (+0.0023,-0.0023)”
Or relative to a default plane;
Loc=”Default(+0.213,-0.123)”

The AR Browsers could then handle the association between the Markers pattern and its Name.
This way the markers are reusable, they do need unique markers to be printed for every new bit of AR they want to look at.
Users could just keep a set of generic markers handy, which they could simply assign to be Marker1,Marker2 etc for any AR use. (Note 2: As mentioned above specific makers could also contain a default ID name and channel built into their data, letting the Arn browser simply prompt the user if they want to see the model even if they aren’t in the right channel. This set up would be most useful for paper and even billboard advertising.)

The Default location could be a settable region, or marker, on the clients browser that defines a playable/user-able area in the field of view. Mostly useful for home use, this could typical be a square region on a users desk.

So, in the chess-game example, the client of the person making the moves simply updates the position relative to the Default every time they move their marker (which is tied to a chess piece mesh).
Then the (non-owners) clients software could automatically display it relative to their Default plane. This would make games like Chess, Checkers, Go or any other game involving merely moving objects about automatically very intuitive and easy to set up.

So by having meshes settable to absolute gps, marker-relative, or default-relative locations, reduces the bother necessary to experience AR content quite considerably, and makes “non-geo-specific” AR applications and games trivial to implement.

Next is permissions.

Mesh-permissions would be a simple string saying who else can update the data, if anyone.

eg;
Permissions=”None”
Permissions=”RandomPerson1, RandomPerson2”
Permissions=”All”

By default you could only update or move your own meshes. (identified by the ID of first posting). If you attempt to update anyone else’s,  their clients would just ignore it.

Thus in a game of chess, you can only move your own pieces. If you attempted to move your opponents (by reassigning  your own marker to their pieces Ids), the clients would just ignore that assignment. You’d only be fooling your own system.
Likewise, when pinning a message in mid-air for your friends to read, no one else can change that message without your permission, although copying it would be easy.  (Note 3: It’s important to note this sort of object-specific permission system is in addition to the global-permissions, or “user-modes” it’s possible to set for the IRC channels and users as a whole.)

Finally, as object data could change within all sorts of time-scales, the easiest way to keep everyone logged in up to date is to just have a time-stamp of when each model was last updated.

LastUpdate=”12/12/0000,2012:12”

This would not necessarily be the same as the XML string post date, because the models mesh might not be updated, but merely moved, and in such case the Arn browser shouldn’t redownload the mesh.

This sort of arrangement could be used as a standard today, and users wouldn’t have to constantly download special AR programs to view a single AR mesh.

In the long-term I would hope for more advanced methods to manipulate Arn-content online, analogous to Dom manipulation in web-pages. But for now, we should at least establish standard methods for devices to pull up meshes and overlay them in the correct position.

So, having a layered system could give the user a seamless blend of dynamic and static data with which to paint their world with.
I believe this is all relatively easy to achieve using modifications of existing web technology, combined with some basic graphics systems.

Local Data:

However, so far I have only talked about remote data.
What of programs originating on the device itself? This is, after all, how most AR software we have at the moment works.

I think, that just like the remote channels, local software should also be blended into the same list of layers.  People shouldn’t have to “Alt+Tab” out of one view of the world, to see another.
They should be able to see both at once, if they wish.

For instance, if your playing a AR game, why shouldn’t your chat window be viewable at the same time?

If you have skinned your environment with a custom view of the world, why shouldn’t you also see mapping or restaurant recommendations?

So local data and remote data should be blended in the same view.
How can AR software – of which I hope, there will be  thousands – seamlessly be expected to layer their graphics, not only with the real world, but with each other, and with online data too? Will games and software makers need to co-operate to allow their graphics to be integrated together with correct occlusion taken into account? A tall order, no?

I must confess though, my technology knowledge fails me here.

I can only guess special graphics drivers, or 3D APIs,  will have to be developed to let programs share their 3D world with that of a Arn browser.
Maybe programmes should simply treat themselves as a local-sever which the browser can connect too, and let the Arn handle all the rendering itself (although I imagine many games designers would find this quite limiting).
So I leave it as an exercise to the readers to discuss and propose the best methods by which this vision of a layered world could be realised..

Beyond IRC:

As mentioned before IRC has some drawbacks, which are due to its age or method of working.
As such, future systems might yet prove better alternatives for a open AR network.
One example of such a system is Google Wave.
It shares many of the advantages of IRC (open, anyone can create a channel of data, different permission levels can be set and its free), while avoiding some critical restrictions. (The data can be persistent).
I believe some of the ideas I’ve mentioned, and possibly even the proposed protocol string could be adapted for Google Wave or other future systems.
I believe overall the principles are more important then any specific implementation to get to them.

Summary;

⁃    In order for AR to flourish the user shouldn’t need to download a separate application for each mesh they want to see.
⁃     Having url’s embedded into QRCoded markers which point to standard mesh files like dxf or kml would be a way to do this right now.  The QR code would only have to be seen precisely  in shot once, then its borders could be used like a standard marker.

⁃    An augmented view of the world needs to support visual multitasking, and having  layers of information is the best way to do that.
⁃
⁃    Methods need to be devised to allow drastically different software to contribute to these layers, without restricting either the software’s rendering ability’s, or the users ability to pick and choose what layers of information he wants to see.

Last point;

I am absolutely confident in my belief AR will become at least as important as the web has, and probably a lot more so. It will also face much the same hurdles and challenges getting established as that medium did.
But, speaking as a web-developer, can we try to avoid a browser war this time?

Everything Everywhere , draft.
by Thomas Wrobel
Darkflame a t gmail

I first started talking to Robert Rice, CEO of Neogence Enterprises, Chairman of the AR Consortium, in 2008.  Robert was already actively working on creating the world’s first global augmented reality network.  But it took a few months before what Robert had said to me about impending explosion of  augmented reality into our lives really sunk in – “this is going to be much bigger than the Web!,” he extolled.

By January, 2009 I was convinced and I posted my first interview with Robert, “Is it OMG Finally for Augmented Reality?..” As I mentioned in the intro, I had recently tried out Wikitude and Nathan Freitas’s grafitti app on the streets of New York City and I was impressed.  Now, 7 months later, Augmented Reality has  not disappointed and there is an explosion of new applications, and the arrival of some of first commercial and practical toolsets, SDKs, and APIs for aspiring developers.

For more on this see my previous post, Augmented Reality’s Growth is Exponential: Ogmento – “Reality Reinvented,” talking with Ori Inbar, which is an introduction to my series of interviews with the key players in augmented reality and founding members of the ARConsortium – Int13, Metaio, Mobilizy, Neogence Enterprises, Ogmento, SPRXmobile, Tonchidot, and Total Immersion.

As I mentioned before, Maarten Lens-FitzGerald of SPRXmobile told me the other day that my first Interview with Robert Rice, in January of this year, was a key inspiration for SPRXmobile to get started on the development of Layar – a Mobile Augmented Reality Browser. Much more on Layar and Wikitude – world browser in my upcoming interviews with Maarten Lens-FitzGerald and Mark A. M. Kramer, respectively.

Recently, both Layar and Wikitude earned a mention in the white paper by Tim O’Reilly and John Battelle, Web Squared: Web 2.0 Five Years On. Web Squared is essential reading not only because it covers the underlying technological shifts of “Web Meets World,” which augmented reality is a vital part of;  but, crucially, Web Squared focuses on how there is a new opportunity for us all:

“The new direction for the Web, its collision course with the physical world, opens enormous new possibilities for business, and enormous new possibilities to make a difference on the world’s most pressing problems.”

I am currently working on a post on Green Tech AR which is one of the areas augmented reality can play an important role “in solving the world’s most pressing problems.” Augmented Reality has a lot to offer Green Tech development.  As Gavin Starks of AMEE said at Euro Foo in 2006, “climate change would be much easier to solve if you could see CO2.”

But really useful Green Tech AR requires still hard to do markerless object recognition (going beyond feature tracking and modified marker recognition), and a tight alignment of media/graphics with physical objects, in addition to a quite a high level of instrumentation of the physical world.  And for Green Tech AR to really shine, we are going to need innovators like Robert Rice who are working on, and solving, multiple really hard problems like:

“privacy, media persistence, spam, creating UI conventions, security, tagging and annotation standards, contextual search, intelligent agents, seamless integration and access of external sensors or data sources, telecom fragmentation, privilege and trust systems, and a variety of others.”

Recently Robert Rice presented at MoMo Amsterdam. Here is a drawing of him in action (picture below from wilgengebroed‘s Flickr Stream).

In his Twitter feed Robert Rice ( @RobertRice ) Robert reminds us: “By the way folks, what you see out there now as “augmented reality” is not what it is going to be in two years.”   Robert plans to show the first public demo of his “platform for platforms” at  ISMAR 2009.

Robert is writing up a series of White Papers currently.  I got a preview of the first, “The Future of Mobile – Ubiquitous Computing and Augmented Reality.”  Robert points out, “AR through the lens of the mobile industry and ubiquitous computing is almost overwhelming compared to AR as marker based marketing campaign.”

I asked Robert, “What are the key take-aways for investors interested in the augmented reality field at the moment:

“First, Mobile AR is going to be bigger than the web. Second, it is going to affect nearly every industry and aspect of life. Third, the emerging sector needs aggressive investment with long term returns. Get rich quick start ups in this space will blow through money and ultimately fail. We need smart VCs to jump in now and do it right. Fourth, AR has the potential to create a few hundred thousand jobs and entirely new professions. You want to kick start the economy or relive the golden days of 1990s innovation? Mobile AR is it.

Don’t be misguided by the gimmicky marketing applications now. Look ahead, and pay attention to what the visionaries are talking about right now. Find the right idea, help build the team, fund them, and then sit back and watch the world change. Also, AR has long term implications for smart cities, green tech, education, entertainment, and global industry. This is serious business, but it has to be done right. I’m more than happy to talk to any venture capitalist, angel investor, or company executive that wants to get a handle on what is out there, what is coming, and what the potential is. Understanding these is the first step to leveraging them for a competitive edge and building a new industry. Lastly, AR is not the same as last decade’s VR.”

Talking with Robert Rice

Picture of Robert Rice at MoMo from Guido van Nispen‘s Flickr Stream

Tish Shute: So perhaps we better start with an update on state of play with Neogence?

Robert Rice: Neogence is doing well actually. We don’t talk much about the fact that we are still a small startup and we face a lot of the usual obstacles related to that and being a small team. Fundraising has been extra difficult, mostly because people are just now beginning to see the potential in AR, but that is still colored by perceptions based on a lot of the gimmicky AR ad campaigns out there. Still, it is better than it was two years ago the idea of an AR startup was a bit of a joke to a lot of VCs we talked to. However, we do have an agreement from a new venture fund in Europe (which we can’t talk about yet) for our first round of funding, but we don’t expect to close that for several months.

If all goes well, we hope to debut our first public demo at ISMAR 2009 in Orlando to select individuals and a few press folks. We might release a few viral videos before then that are conceptual and about what we are building in the long run, but that depends on how things go over the next several weeks.

We are also very active in looking for and building strategic partnerships and relationships with other companies, and this is not restricted to the augmented reality or mobile sector. As I have said before, we are looking at this as a long term business venture and the industry as something that will be bigger than the web itself within ten years. We are doing typical contract work and custom AR solutions to keep the cash flow going and build up the corporate resume a bit. So, if you want something done, and better than the stuff you are seeing now with all of the generic “look at our brand in AR with markers and a webcam” you should definitely give us a call.

Tish Shute: Just to clarify because most of the recent press has been about browser type AR like Wikitude and Layar which are not in the purist sense AR ‘cos they do not have graphics tightly linked to physical world. Neogence, if I am correct, is focused on building a true AR platform in the sense I just described?

Robert Rice: Hrm, I have argued with a few others about the actual definition of AR. Some people prefer a narrow and limiting view (3D overlaid on video), but I think in terms of the market and the end-user, it is better to have a wider definition. In that sense, AR is purely the blend of real and virtual, with or without full 3D overlaid on video. If we go with that, then Wikitude, Layar, Sekai, NRU, and others all fit into the AR definition.

Anyway, you are correct. We are building a true platform for AR, and this is quite different from what others are marketing as AR browser “platforms.”

There are a few problems with the “AR Browsers” approach that no one seems to be noticing. One is that they are all trying to get people to build new applications for their browsers, when they should be trying to get people to create content that they can share and browse.

Second, someone using Layar is not going to see anything that is designed for Sekai or Wikitude.

Third the experiences are generally for one user. While I love all of these guys and think each of the teams has some real talent on it, the model is flawed until someone using Wikitude can see the same thing that someone using Layar or Sekai camera is seeing (provided they are in the same physical location).

While we are working on our own client side technologies that we hope will be useful and integrated with every mobile device and AR browser out there, our core focus is on connecting everything and everyone together, and facilitating the growth of the industry with the tools to create content, applications, and so forth. We want to solve the really difficult technical problems (some of which most people haven’t even considered yet, because of the perspective they are looking at the potential of AR with), and make it easy for everyone else to do the cool stuff. We want to be the facilitators.

If you really want an idea of where we are going or some of what has inspired us, you have GOT to read Dream Park, Rainbows End, and The Diamond Age. If you have heard me speak anywhere or read my blog, you know that I am continually suggesting these and others.

Anyway, short answer, yes, we are building a true platform for ubiquitous mobile augmented reality, and we are absolutely the first to be doing so. I hope to demo some of this in October at ISMAR, with a full commercial launch next year (10/10/10 at 1010am Hehe, seriously). We will probably launch a website soon for people to start signing up and building a community now (especially if you want in on the beta testing of the whole kibosh).

Tish: So just to clarify,  how will Neogence’s approach differ and fit into the  growing world of Augmented Reality tools that we have now, e.g.,  ARTookit, Imagination, Unifeye?

Robert: I guess you could say that we are trying to build the infrastructure for the global augmented reality network. This could be viewed as a service, or even a platform for platforms. If Neogence does its job right, anything you create using ARtoolkit, Unifeye, or Imagination would be applications you could ultimately link to, integrate with, or deploy on or through, what we are building, and not be tied to a specific set of hardware, browser, or walled garden.

Tish: You mention Neogence is going to provide a platform for platforms. Without knowing the details that sounds like a lot of centralization which prompts the inevitable question: “Who owns the data?” Do you think other AR applications or providers would resist a “Platform for Platforms?” I know the potential centralization power of Google Wave has already got people talking about these issues (one of the comments in my recent blog post was about how Google Wave protocol may be interesting for a least some parts of augmented reality communication).

Robert: It really depends on perception and how we end up building it. We aren’t talking about creating a closed system. As far as who owns the data, it depends on what data we are talking about. For the most part, I think that if the end-user creates something, they should own it and have control over it. They should also be able to do what they want with it, independent of everything else.

This is one thing that proponents of the smart cloud and the thin/dumb client don’t like to talk about. It sounds great on paper, but when you start thinking about it, all that does is strip away power from the end user. Case in point…Amazon recently wiped every copy of George Orwell’s 1984 from all Kindle devices. They claimed they didn’t have rights to distribute/publish it and it was available on accident. The scary thing though, is that they literally went into every kindle out there, found copies, and deleted them.

How would you like it if Microsoft suddenly decided to delete every copy of Microsoft Office? Or every file that had a .doc extension? That is a huge violation…we feel like we own what is on our computers. But with the whole cloud thing, your data is at the mercy of whoever is running the cloud servers. No privacy, no ownership, no control. And if the system breaks, all you will have is a pretty dumb device that can’t do much on its own. Now, that isn’t to say that the technical merits and benefits of a cloud model aren’t worth pursuing, they are.

But I think there needs to be some hybrid model. Don’t dumb down my computer or my smart phone, let’s keep pushing how much these devices can do. We should take full advantage of centralized and distributed systems, but in a hybrid mashup sense. That is what we are pursuing with our AR platform, while trying to protect ownership and intellectual property rights of the end user.

Tish: Earlier today I was telling you how impressed I was by Google Wave – it is quite mind blowing to experience massively multiplayer real time interaction on what will be an open internet wide platform – Wave is breaking new ground here and more than one person has mentioned its potential role in AR to me (see the comments to my recent post on Ogmento).

I know you are a strong advocate of this kind of real time shared experience being part of AR.  But we are only just beginning to see it emerge via Wave on the existing web – what will it take to have this kind of real time shared experience in AR!  We got briefly into the thick client, thin client, cloud versus P2P discussions – what is your approach to delivering a massively shared real time experience that is like Wave not confined to a walled garden?

Robert: I’m not a fan of any of those models as being stand alone or mutually exclusive. Again, the hybrid model with the best of both worlds is key. In the early stages of the emerging industry, you are likely to see some walled gardens (or perhaps a walled garden of walled gardens…).

No one knows how things are going to turn out in the next five to ten years and few people are thinking about it actively. For us though, I favor Alan Kay’s quote (pardon the paraphrasing): “To accurately predict the future, invent it”. That’s what we are doing. In the short term, there will be plenty of experimentation in the industry and a lot of model testing.

Tish: Do you think though Wave protocols might be useful as at least part of the picture for AR standards?  As you point out open standards and open protocols are going to be vital for shared experiences of AR.  Is it important to build off existing protocols to get the ball rolling and what do you see as being the important early protocols for AR?

Robert: I think for now, we will use a lot of existing protocols for communications and whatnot, as well as the usual standards for things like 3D models, animation, and so forth. This is only natural. However, as the industry and technology evolves, we will need entirely new ones. As far as I know there is no existing market standard for anything like the Holographic Doctor from Star Trek Voyager, and that type of thing is definitely in the pipeline for the future (sooner than you would think).

Tish: All the excitement at the arrival of the browser like mobile reality developments has been really great – I feel people are getting a taste for what it means to compute with anyone/anything, anywhere and and anytime.

Wikitude started the ball rolling. And with Wikitude.me it is the first to support user generated content. Now there is Layar, Sekai Camera also. But as you mentioned to me in an earlier chat, with Layar and Wikitude opening up “their are probably half dozen other apps coming out in short order with similar functionality (even the AR twitter thing has some similarities).”

What has been most exciting to you about these developments up to this point? What will these apps/platforms need to do to stand out in a crowd.  Up to now, these browser like AR experiences do nothing with close by objects. Do you see “world browsers” with near object recognition coming out in the near future. Could Wikitude do this with an integration of SRengine or Imagination?

Robert: Yes, Wikitude or Layar could do this (integrate with something else for “near” AR) and it would be a step in the right direction. Tagging things in the real world is the basic functionality that will grow from text tags to photos, videos, 3D objects, and all sorts of other types of data and meta data. This gets really fun when that data is generated by the object itself. First is just giving people the ability to tag something and share that tag with their friends, everything else grows from that. This sort of functionality is probably the most exciting in terms of near future advancement.

However, I think the idea of a stand-alone browser platform is a bit awkward…unless you also consider firefox a website browser platform. After all, you can create widgets (applications) for it. Anyway, the point is having access to the same data…if you put three people in a room, one for each browser, they should see and experience the same content, although the interface might be different (based on what browser and of course which hardware they are using). This means there needs to be some communication between whatever servers they are storing their data on (meaning, user tags) and some standard for how those tags are created.

Of course, if all they are doing is grabbing the GPS coordinates of the nearest subway station and telling you how far it is and in what direction, then they should all be able to see the same thing, regardless of the platform. But then, that isn’t really interesting is it? I could get the same info on a laptop with google maps.

This is part of the problem right now though…no one seems to be thinking about the bigger picture much. All of the effort is either on making the next cool ad campaign for a car or a movie, or creating a tool to tell you where the nearest thingamajig is, but in a really cool fashion on a mobile device.

No one is talking much about filtering data, privilege systems, standards, third party tools, interoperability, and so on. There is also little conversation about where hardware is going. Right now everyone is developing software based on what hardware is available. This needs to change where hardware is being developed to take advantage of new software coming out (this happened in the PC industry a while back and growth accelerated dramatically).

These are some of the reasons why I led the effort to start the AR Consortium. We brought CEOs from 8 different AR companies and startups together to start talking about these issues. We are still getting organized and have plans to expand the membership to other companies, but we want to do this right and we aren’t rushing things. The important thing is that we have started and there is at least a line of communication open now, where there wasn’t before.

I would expect to see the early movers expanding what they offer very soon, and they will probably lead the way in the short term. Definitely keep an eye on the companies involved in the AR Consortium. There are lots of very smart and motivated people there, and they are far ahead of all the experimental dabbling in AR we are beginning to see on youtube, twitter, and elsewhere.

Tish: When we had a discussion about what were the basics for an AR platform and an AR browser earlier, you talked about the difference between tools, a platform, and a AR browser – like Wikitude and Layar which should be about features/functionality e.g. to create treasure hunts AR geocaching, invisible AR yellow sticky notes you can leave at restaurants you don’t like, etc. Also you noted it should let you explore (browse) multiple formats, and open content content for AR – any data, information, or media that is linked to something in the real world and the visualization/interaction with the same.

Wikitude is a stepping stone to a true browser by your definition. But are we also seeing what you would define as an AR platform emerging – Unifeye, Wikitude (you can recap your definition if you like too)?

I think Wikitude hopes to provide the lego blocks for  augmented reality readers, browsers, applications, tools, and  platforms?

Robert: I expect some segmentation among the various AR companies that are out now, as they find their individual strengths and focus on them. Some will emphasize the client software (the browser), others will develop robust tools for creating content, SDKs/APIs will advance and facilitate rapid development of applications, etc. Neogence is ultimately working on the glue in the middle that ties everything together, makes it massively multiuser, persistent, and ubiquitous. Things like Unity3D have the potential to fill a need in the middleware space.

Tish: I know Blair McIntyre (see my interview with Blair here) and others are using Unity3D as an AR client, Could Unity3D become increasingly important?

Robert: It has the potential to become a favored middleware for providing the rendering layer. It already works nicely in regular browsers, and on several mobile platforms. Why code all the graphics rendering stuff from scratch when you can just license something and extend its features with AR functionality?

Tish: Now to ask your own question back to you! There seems to be a lot of reason to think that, eventually, there will be the kind of access to the iphone video API that augmented reality really requires and by that I mean more than we will get with OS 3.1 which is rumored to deliver only about half of what we really need for AR on the iphone – “not truly useful when you want to align video. with graphics.”  So:

“The iphone…future or failure? Seemingly anti-developer stance regarding augmented reality, and only a sliver of the global market share. Are we letting the short term glitz of Apple and the iPhone fad pull us in the wrong direction? Shouldnt we be focusing on symbian devices that have the lion’s share of the market? or should we be looking more at either other OSs (winmobile, android) or not at all and trying to create a new platform that is more MID and less smart phone with a hardware partner?”

Robert: Apple and the iphone are a bit problematic right now. There is no way I can go to a venture capitalist (at least in North America) and say hey we are building awesome AR applications for winmobile or symbian…they would either laugh or they simply wouldn’t get it. There is this false perception that the iphone is the ultimate mobile device, it is the sexiest, and the only thing that people want. Everyone wants a demo on the iphone, the media is mostly interested in iphone developments, and the apple fanatic market could give a fig about other devices. Other devices may have a larger market share or even better hardware, but we have to focus on the iphone right now at least in the demo stage to get any market attention and traction worth the time and effort.

In the future though, unless Apple changes its stance with their SDK and APIs, and starts adding hardware that is key for mobile AR (beyond what is there now), the market will move on without them. This is a really easy decision to make given Apple’s draconian policies and the fact that their percentage of the global market is miniscule. The smart companies are looking at the whole picture and not putting all of their eggs in the Apple basket.

Of course, once the wearable displays are commercially viable everything changes. Wearable computers with small screens or even no screens are going to be what everyone wants. The interface will go from handheld touch screens to virtual holographic interfaces that you interact with using your bare hands.

So for now, (the immediate short term), its all about the iphone. Taking mobile ubiquitous AR to the global market and building for the future will be based on something else. Hardware risks becoming a commodity or a closed platform. Do you really want to buy the Apple iGlasses and only see AR content that is compatible, where your best friend has a pair of WinGlasses and sees something entirely different? No. The hardware, and the client software (what people are calling the ar browser now) will become common and it won’t matter what brand you use, they will all be accessing the same content.

But at least for the forseeable future, we are building software for specific hardware, and the sexiest mobile on the block is the iphone. The second someone comes out with something much better and the paradigm shifts (software driving hardware instead of vice versa) everything changes.

Tish: How is the quest for sexy AR eyewear going.  I know we were checking out the Japanese eyewear with Adam Johnson from Genkii just now.  For the Neogence project – as you are going for a fully developed model of AR doesn’t this necessitate going beyond the iphone and getting the hardware companies moving on the eyewear?

Robert: The guys making wearable displays really need to get off the pot and stop paying lip service to mobile AR. If they don’t do something quick, I,  and others, are going to be scouring the planet looking for someone capable of building the lightweight stylish wearable displays with transparent lenses we are begging for. We aren’t going to be waiting around for hardware anymore. The AR Pandora’s box has been opened. I should note that many of us (AR Consortium members) have had less than pleasant experiences or communications with the half dozen companies or so that are making wearable displays. Either their visual design is terrible, the materials feel flimsy, the field of view is limited, or the companies are preoccupied with other business and government contracts. Any attention to the growing AR market is an afterthought and in a few cases condescending. AR is going to be a billion dollar industry in a very short time, and these guys are just leaving money on the table. If they were smart, they would be begging the CEOs from the AR Consortium to fly out to their offices and collaborate on building a pair of wicked sick glasses. The smart phone manufacturers should be doing the same thing, but I have to say that they at least seem to have some ambition and zeal to create better devices, so I can’t really complain too much there.

Anyway, to answer the rest of your question, we have to assume that the hardware guys, especially regarding the eyewear, is going to take a long time to develop and release the things we need for the ultimate AR experience. So, our goal is to start building things now for what is available. That means scaling things down and handicapping what AR can do, so it works on the “sexy” iphone. The important thing though is to start creating applications -now- so when the glasses are commercially available, there will be a wealth of content for people to access and use on day one.

As long as Apple isn’t playing nice, it is going to hurt everyone. Is it any surprise that they shut down Google Voice? There is a huge opportunity for someone to step up and leapfrog the rest of the industry. Give us the hardware and we will create amazing software for it. Don’t compete with the iphone, surpass it.

Tish: What is the state of play of current AR technology and toolkits?

Robert: The current crop of AR technology and toolkits is absolutely critical for this stage of the industry, and everyone should be leveraging it as much as possible. I talk down marker and image based tracking a lot, but I also like to point out that it is the necessary baseline that the industry is going to be built on. The problem is that there is only so much you can do with marker driven apps, and as creative people and marketing types start conceptualizing about all sorts of cool stuff for the future, they risk setting the expectations too high. It is one thing to show someone the future, it is another to say this is the future and its happening right now. This is why I cringe everytime I see a conceptual video presented as “our product DOES this” instead of “our product WILL DO this.” Something that simple can still cause the butterfly effect of raising expectations too high and contribute to overhyping.

Tish: One of the things that seems very exciting about the new Ogmento partnership is that experienced content producers  Brad Foxhoven and Brian Selzer from Blockade are now taking a leading role in AR.  What are the most exciting directions for content that you see emerging for AR in the next 12 months?

Robert: Virtual (well, augmented) pets, and multiuser mobile AR games (2-4 people) are probably going to lead in the next 12 months for content. Easy, accessible, engaging.

Tish: And are you at Neogence also involved in content partnerships?

Robert: Yes, we are in the process of finalizing some content partnerships with an eye for long term relationships. We are specifically looking for partners that want to find substantive ways to leverage AR technology, and not use it as a superficial gimmick or attraction that wears off after five minutes. I’m still cringing over the Proctor & Gamble Always campaign with AR.

Tish: So back to your observation about some of the tricky problems re creating a true global massively multiuser, ubiquitous, mobile AR platform – what are some of the main obstacles to this mission in our view? (aside from getting investment!)

Robert: Trying to explain it to people. The technical problems we can handle or have already solved. But trying to communicate what exactly we are doing is still tough. Not because it is overly complicated, but rather because it is so new and different. People are having a hard time grasping augmented reality beyond marker/webcam.

Tish: Which AR tools are most important right now?

Robert: Content is critical right now to show what the technology is capable of and to continue building the presence of augmented reality in the public mind the big benefit to integrated / unified platforms now is speed of development for content. I think that the flash artoolkit = papervision is rocking the planet right now. It is accessible, easy to learn, and lets people create something very quickly. More tools and middleware are coming out and this increases options for designers and developers.

Tish: What are your favorite papervision apps?

Robert: Hrm, I don’t have a favorite papervision app just yet, although I think the tech is solid. I expect to see a lot of stuff built on that platform in the near future. Especially as more ad agencies get on the bandwagon and start telling their IT guys to learn how to program flash so they can make something. Have you seen www.ronaldchevalier.com Not so much for the actual AR stuff, but because the whole thing is just brilliant. Its exactly like some cult figure spiritual guru would do with AR. I wish I had thought of it first actually. This is probably one of the best -seamless- implementations of AR in marketing where it fits…it isn’t just jammed in there for the sake of saying they used AR.

Tish: Do you think Apple is going open the iphone to the full potential of augmented reality anytime soon – a lot of expectations have been raised?

Robert: Apple is like that guy has a party at his house and owns this really awesome state of the art home theater in his basement, but makes everyone watch a movie in the living room on a regular TV with a VCR.

They need to get over themselves and quit being a wet blanket. Otherwise, we are taking the beer and pizza we brought, and going to someone else’s house. Sorry, the Apple thing is a bit of a sore point with me.

Tish: But will people leave all that candy and soda at the appstore?

Robert: I tell you what though, there is an opportunity for certain mobile phone manufacturers to give me a call and start talking to Neogence and the other members of the Consortium. We have some ideas and specs that could have a radical impact on the mobile market and stuff the IPhone in a box. Hint hint.

Tish: So what is your vision for the ARconsortium.  I know it kicked off with a letter to Apple about the video API.  What is the next step? There was a lot of hope that this year would be big for MIDs but this really hasn’t happened yet – do you think there is hope for a MID take off despite the lousy economy?)

Robert: MIDs? No, not yet. smart phones are too lucrative and too hot. It isn’t time yet for the MID to go mainstream. For that to happen, there needs to be a driving need (cough ubiquitous AR cough)

The AR consortium is mostly an informal affiliation. I expect that representatives from each member will probably meet at every significant conference to catch up over drinks. We are also going to be planning for our own members conference at least once a year. That will happen after we expand the membership though.

The main idea behind the consortium though was to open up a channel of communication between the CEOs so we could work together on standards, solving problems, collaborating, forming some partnerships, and using the collective to bang on the doors of companies like Apple and others. There is power in a group.

Tish: You mentioned there is a whole long conversation we can have about getting the eyewear.  As you point out true AR eyewear changes everything.  Can give a little road map of where this has to go?

Robert: There are essentially four or five main approaches, depending on whether or not you make the lenses special or if they are just plain. You would normally want them to be plain so people with prescription lenses wouldn’t have problems and would have the option to switch them out. Some types use a more prismatic approach for top down projection, or a corner piece mounts lasers and bounces them off the lens into the eye.  Another approach is embedding OLEDs or something else into the lenses themselves.

I really like the Lumus approach, but their product design isn’t quite there yet. If the wearables don’t look cool, people won’t use them. To be honest, if I had the money, I’d probably ask the Art Lebedev guys to design them based on someone else’s optical engineering. They designed the optimus maximus old keyboard…   brilliant industrial designers, loaded with engineers too. If these guys couldn’t build the glasses and make them look damn bad ass, I’d be shocked. Heck, I bet they could build the next gen MID while they were at it.

Tish: Getting the hardware innovation and software innovation feeding into each other would be really great.

Robert: Absolutely.

Tish: That would push the eyewear forward too wouldn’t it?

Robert: All it takes is one, and then the competitive landscape would fire right up.

Tish: What applications would the accurate gps enable?

Robert: Everything. for example, you know exactly where the phone is and where it is facing, that means you can put it on a table and hit a button, then move it somewhere else and do the same thing in a few minutes, you have a nearly accurate “mental” model of the whole place now you go back and start dropping virtual flower pots everywhere.

This is one area where I think the smart phone guys are missing the boat and taking the cheap route. It is possible to have very accurate GPS (down to a six inch area) with better chips and firmware, but it is cheaper to stick in old tech. Most apps today dont need that hyper accuracy, so they aren’t bothering. Mobile AR though, thats a different story.

With that level of accuracy, you would know exactly where the mobile device is, so all you would need to know is the direction it is facing (orientation), and you could solve one of the problems with registering exactly where 3D objects and augmented media is (it is more complicated than I am describing it, but we don’t need to get into that much detail here). You wouldn’t need markers anymore.

Tish: Isn’t Wikitude doing this with Wikitude.me their tagging app.?

Robert: Not really. That type of approach is on a very large scale using the accelerometers compass and GPS to determine where you are and what is in the distance. They (and others like Layar) don’t handle “near” AR. They effectively poll your GPS and then check a database to see what is nearby and what degree/distance it is and then they draw a representation on the screen. They don’t even need a mobile device’s camera at all.

Even if they did things up close, its still based on finding landmarks or on things that are broadcasting their location. For example, if they were standing near me, they might get “robert, 37 degrees, 15 meters away” but they wouldn’t be tracking me exactly as I walk around or have the ability to overlay graphics on ME.

Tish: I retweeted your #ar marketing using ARToolkit + flash (markers/webcams) = Photoshop pagecurl <six months. Bad design kills innovation. I know you like Dr Chevalier though!  What are some of the other AR marketing projects that you like. What would you like to see in terms of innovation in the next 6 months?

Robert: The marker/webcam approach is already becoming overused and cliche (tremendously fast). Older readers will remember the ubiquitous photoshop page curl that adorned nearly every website and graphic on the internet back in the day. It was horrible. Yes, the Dr. Chevalier stuff cracks me up.

I want to see some big companies or ad agencies really try to do something different with AR, preferably mobile. Take some risks, do something different. Don’t follow the crowd. Innovation? I want to see some wearable displays with transparent lenses, I want a mobile device specifically designed for ubiquitous AR, I want to see some experimenting with AR in the green tech sector, and I’d like to see someone get that GiFi wireless technology from that researcher in Australia and jam it into a smart mobile. I would also like my flying car and lunar vacation now, thank you. It is almost 2010 and no one has found that black obelisk yet.

Tish: So a few closing thoughts! What do you see as the next big thing? Hopes for the ar consortium?  Biggest bstacle for commercial AR?  And what is the coolest thing you have seen this year?!

Robert: The next big thing is what I’m working on hahaha. I hope the AR Consortium will grow and be the active catalyst in making AR mainstream, practical, and world changing.

The biggest obstacle is making sure that the right funding finds the right developers to develop the right technology and create kick ass applications.

The coolest thing I’ve seen this year would probably be the facade projection stuff (see below): Now, imagine that, but without the projector. Thats part of what I envision for AR in the future.

I am excited about going to Where Week 2009 – Where 2.0 and WhereCamp, this week (for more see Brady Forrest’s post).  Where Week will be total immersion for five days in a think tank with creators of the information landscapes of the future.

As you know, if you have read my previous post – here, I think the “Mobile Reality” panel is a must.  And I have been looking forward to hearing more about The Shape of Alpha from Aaron Straup Cope, Flickr, since Etech 2009 when I was introduced to Aaron by Mike Kuniavsky (see my interview with Mike Kuniavsky at Etech here and more on Mike’s concept “information shadows” in his Etech talk).

Shape of Alpha is revealing some fascinating possibilities for mining geotagged Flickr images.

As Tim O’Reilly noted in a tweet, Aaron Straup Cope’s recent post, The Absence and the Anchor, describes, “some of the surprising things Flickr is learning about people from geotagged photos.” Aaron’s post also announces that the “donut hole shapes” are available for developers to use with their developer magic via the Flickr API.

“If the shapefiles themselves are uncharted territory, the donut holes are the fuzzy horizon even further off in the distance. We’re not really sure where this will take us but we’re pretty sure there’s something to it all so we’re eager to share it with people and see what they can make of it too.”

For more on shape files see Aaron’s blog post about “some experimental work that I’d been doing with the shapefile data we derive from geotagged photos.”

Creating the Information Landscapes of the Future

I have been thinking and writing a lot about augmented reality lately.  And key thought leaders in this space like Blair MacIntyre, Robert Rice (see my interview here), and Ori Inbar (see my interview here), have clued me in to how vital it is, for an ubiquitous experience, for us to find ways to allow people to fill in the stories that can be used for augmented reality.

As Ori noted in conclusion to our recent conversation:

“in order to have a ubiquitous experience like Robert Rice and others are striving for, you’ll need to 3d map the world. Google earth like apps are going to help but it is not going to be sufficient. So let’s leverage people. Google became successful in part by making people work with them.  Each time you create a link from your blog to my blog their search engines learn from it.  So let’s find ways to make people create information that can be used for AR.”

Jim Purbrick, another key thinker in this area (interview upcoming), also notes:

“you can imagine a crowd sourced set of hints for any location so, AR knows roughly where it is and can do photosynth style matching  to find out exactly what it’s looking at and get the extra data it needs about that thing (humans are really good image recognition systems, and are also pretty good at interfacing with networks) instead of marking up real objects with ids you take pictures of real objects, tag them and then search them based on images from your ar system.”

Ori Inbar suggested to me an idea that I really liked – the notion of bread crumbs where, “You don’t have a constant view of what is happening when you walk but you get images and text and all sorts of things from people who walked there before – like breadcrumbs.“  And as Design Under Sky points out about Shape of Alpha:

“The truly amazing part of this process is how the “community” has the authority to provide areas previously unmapped.  By uploading personal photos of areas not covered by mapping software, members have the power of further shrinking our world through greater visual access and understanding of locations one might not be willing or unable to visit.”

Aaron Straup Cope, Flickr, Todd E. Kurt, ThingM and Mike Kuniavsky, ThingM

The Locative Media Manifesto

@stamen’s tweet brought André Lemos’ brilliant, thought provoking, ” Locative Media Manifesto,” to my attention.  I am also looking forward to hearing about how old maps “can shed light on modern geography when placed in counterpoint to the state of art in modern maps from Google or Microsoft” from Michal Migurski, Stamen Design, who will present Flea Market Mapping at Where 2.0.

André Lemos writes:

“After uploading to Matrix up there – Internet 1.0 – now is the time to “download cyberspace,” information about things down here – Internet 2.0. We are not dealing with what is virtual up there, but of what to do with all this information about things and places down here! How can we relate to things and places? And now that these things and places are provided with digital information and Internet connections? Do we invoke Heidegger and Lefevbre?”

I will leave it to people smarter than I to invoke Heidegger and Lefevbre as Andre Lemos does so eloquently in Locative Media Manifesto. But by reminding us artists and activists created the term “locative media” to “question the mass use of LBS (location based services) and LBT (location based technologies,”  the manifesto delivers 30 principles to inspire creators of Locative Media and explorers of the, “current dimension of cyberculture, comprising the era of “cyberspace leaking into the real world” (Russel, 1999); an era of the “internet of things.”

I feel well primed for Where Week by my visit to the ITP Spring Show, 2009 last Sunday. It was an interaction riot, jam packed with brilliance and off beat explorations of locative media which I experienced through the senses of my 9 year old.  His pick for best of show is below. But he had many favorites and I have put some pictures up on my FLickr stream with links to the creator’s sites.  One of my favorite projects Alexander Reeder’s S Ring – “seducing people by talking with your hands,” is up on my Posterous blog.  You can see a list of the extensive media coverage the show got here.

Loose Interaction Topologies

The picture above is of a game of mud pong  in Tom Gerhardt’s Mud Tub.  The mud interface – “a smart tub with some mud” knows the topology of the mud and where your hand is. Mud Tub takes advantage of  a complex material – to explore loose interaction topologies, including as seen above a game of Mud Pong.  Loose interaction topologies are a way we can explore meaning in “the internet of things.”

Tom explained his own exploration of the internet of things to me very succinctly:

“I am not trying to make mud better. I am trying to make computers better with mud.”

He elaborates on the value of Mud Tub in this regard on his site, dirtycomputing:

“The Mud Tub occupies a space similar to other experimental human-computer interfaces, like, multi-touch surfaces, body controllers, augmented reality systems, etc, which push the boundaries of codified interaction models, and drive the development of innovative software applications. Beyond its role as a research topic, the Mud Tub also exists as an open-sourced hardware/software platform on which interactive artists and designers explore new methods for creating and displaying their work.”

Recently, I talked to Ori Inbar (above), formerly senior vice- president at SAP.  Ori is on a mission to make augmented reality commercially successful not in 5, 10, or 15 years, but now. Ori is the founder of Pookatak Games – a video game company, “with a vision to upgrade the way people experience the world.” Ori will be participating May 20th, in O’Reilly’s Where 2.0 panel, “Mobile Reality” -  an event not to be missed IMO.

The taste for computing anywhere anytime has entered human culture via the iphone and is spreading like chocolate cake and pizza at a preschool party (see why the iPhone changed everything).  And while the full flowering of the next step is yet to come – computing anywhere, anytime by anyone and anything (“the internet of things”), our love for these first devices capable of being mediating artifacts for ubiquitous computing (Adam Greenfield) is a vital first step to free us from our tethers to computer screens, and fulfill the promise of augmented reality.

If you need more convincing on the pivotal role augmented reality will play as the web moves into the world, check out Tim O’Reilly’s recent comments in this video clip posted on Augmented Times and here early last year.

From another perspective, the gloomy specter of economic and environmental catastrophe  is driving a movement to “infuse intelligence into the way the world work’s.” But the challenge for a smart planet is not just about making environments smart, it is about using smart environments to enable people to act smarter (see my interview with Adam Greenfield).

We need a rapid upgrade in both the way the world works, and the way we experience the world.

((Note:  It is time to read (if you haven’t already) Bruce Sterling’s Caryatids (Cory Doctorow’s book of the year for 2009) “as a software design manual” (see Julian Bleeker) because Caryatids reveals the Gordian knots of human folly, greed, compassion and desire entwined in near future designs for technologies to save the world.))

Ori Inbar, worked with Shai Agassi (Shai is now leading the world changing Better Place ) driving Netweaver from a mere concept to a “major, major business for SAP.” So Ori has already been through the cycle of working in a very small startup and growing it into a billion dollar business.  He has both the experience and the passion to realize his vision for augmented reality.

At Pookatak, he explains :

“We design “reality experiences” that make users’ immediate environments more significant to them. We wish to free young and old from getting lost in front of the screen. By delivering the world’s information to people’s field of view, and by weaving real world objects into interactive narratives, we help people rediscover the real world.”

Pookatak will release their first game this summer. Currently it is under wraps. But Ori gives us some glimpses of what is to come in the interview below.

In addition to founding Pookatak, Ori is involved in a broader effort to move augmented reality forward. On his blog, Games Alfresco – he recently welcomed a new partner, Rouli Nir, Ori has focused his eye of wisdom on every significant recent advance in Augmented Reality (check out this essence of Ori’s thinking in a fast paced video presentation for WARM ‘09).

Also Ori is one of the organizers of the interactive media track at ISMAR 2009.  At ISMAR this year, Ori explained, “we are trying to bring in people that develop interactive experiences for consumers, beyond the traditional attendees coming from a research perspective.”

In the interview below, Ori explains much of his thinking on how augmented reality will become commercially successful.  Enjoy it, think about it, and share it. And most importantly, if you can, get involved with ISMAR 2009.

Ori  has inspired me to participate in ISMAR this year.  Ori pointed out:

The call for papers is on, and this year it targets well beyond the typical research papers audience and into interactive media and art folks.

There are plenty of opportunities such as:

Art Gallery

Demonstrations

Tutorial

Workshops

It’s a huge opportunity to shape the emergence of augmented reality.

Interview With Ori Inbar

Making Augmented Reality Commercially Successful

Tish Shute: You are considered a key trail blazer in AR and you have the go to blog for augmented reality!  What are the most important lessons you have learned researching, writing, and developing AR in the last couple of years?

Ori Inbar: You need to have a vision. You need to know where this is going to go in ten or fifteen or twenty years. But you’ve got to start with something really simple that makes use of the technology you have on hand. And do something that is practical, that people will like, and something they would actually want to buy. Its as simple as that. I’m currently looking at what we could do with existing technology. First of all, you have to put it in front of people. Right now most people have never heard about the term augmented reality. Go into the street, and ask 100 people about it, maybe 2 would know about it. So you need to put it in front of people because most people think it’s still science fiction or a special effect you see in movies, not something you can experience in real life.

Tish: It seems to me to that for augmented reality applications to become popular with existing technology the key breakthrough would be getting people to hold up their phones. What are the obstacles to getting people to use their mobile devices like this?

Ori: There’s a really nice cartoon by Tonchidot (below) – the Japanese company behind the Sekai Camera. It’s an illustration showing the evolution of man, from ape to man (holding a cell phone looking down), to the developed man holding a device like a camera – in front of its eyes.

Which is exactly what you’re talking about. People ask, “are people going to walk with this like that all day long?” Probably not. I mean you have to build it in a way that doesn’t require them to hold it like that all the time. People are used to this gesture with the ubiquitous digital cameras. I tested one of my prototypes on a two and a half year old girl. She had no problem holding it just like she holds a camera.

Tish: Blair MacIntyre mentioned, “The problem with the mobile phone as a AR device is a problem of awareness,” i.e., you have to have a way of letting people know when there’s something interesting wherever they are. One of the issues regarding this is if you get too many alerts, then you tune them out.

Ori: First of all Blair is one of the people in academia that get it. Because he looks at it from an experience perspective. Not just as an interesting technical problem to solve. Let’s start with getting people to enjoy this new experience. The AR demos so far were mostly eye candies, and mostly for advertising – the GE AR ad created a lot of buzz; but you look at it for 10 seconds and you forget about it.  You need to build something that people would want to experience over time and would be willing to pay for. I think that’s the big test, right?

Now in terms of having a ubiquitous experience where you’re continously connected, it doesn’t have to be an overwhelming experience. Just like some of the social media tools we’re using today, we decide when to connect, and we filter out the trash. You could get alerts only for things that really matter to you, not for everything that happens in your immediate environment.

There will be many layers of information, and it’ll be up to you to pick the ones you want to experience. The real benefit is that you get the information in your own field of view and in context of where you are or what you do.

Tish: So what are you working on these days?

Ori: We are working on a little app that targets a very different audience than what you’d expect: pre schoolers. We think we can encourage them to get away from a PC or TV screen and learn something while playing – in the real world. You’ll hear more about it as soon as this summer. Nuff said.

But, it is a small application that will run on the iPhone. People ask how many pre-schoolers own iPhones? Well, their parents do.

Tish: Yes there are certainly many New York kids with iPhones – my kid now has my old iphone.  He has pretty much switched from playing games on his DS to the iPhone. I noticed in your WARM video you place a big emphasis on AR as something that will get kids away from screens and engaged with reality.  This is something parents will approve of!

Ori: Yes I saw something really interesting at my kids’ party one day; they were all sitting around the room – looking down at their own DS screens.  You could play the DS anywhere, but kids would usually play it on the sofa, looking at the screen, isolated from the world. With an iPhone and a camera, and the application we’re producing, reality becomes part of the game. Yes that makes it all of a sudden much more interesting for parents. Because kids are spending so much time in front of the screen, all of a sudden they’re something that will encourage them to interact with real objects, real things. Every parent I’ve talked to loves that idea.

Tish: Yes that is what is cool about the work of Kati London – I think I saw someone say this on Twitter, “Kati puts the computer in the game not the game in the computer.”

Ori: Yes, kids are spending more time in front of games and the computer because it’s more interesting. It captivates them with “game pleasures ” that tap into their brain’s dopamine circuitry – constantly seeking reward and satisfaction. So you’re not going to be able to tell them to go back to playing in reality without these pleasures. We have to study these mechanics from games and bring them into reality. It’s about programming real life; and augmented reality helps you achieve that.

Here’s an example: cause and effect; in a game when you do something you always get an immediate effect. You’re good, you get a reward. You’re not good, you get a cue to improve. In real life you do things and you could wait 2 or 3 years until you actually get feedback (if you’re lucky). Augmented Reality allows you to bring these mechanics into the real world. I think that’s going to help kids rediscover reality, in a new sense, which is what every parent is dreaming about.

Tish: I don’t know how much you can say about your app. But in regard to doing augmented reality on the iPhone.. there’s no compass. Is this a limitation?

Ori: True, no compass yet. But the camera gives you a lot of information that you can interact with. When you run the application, you see the world in front of you, and if the app can recognize real life objects – it can put virtual elements on top of it.

Tish: But not with any accuracy unless you’re using markers. Are you using markers?

Ori: We’re using natural feature recognition. It doesn’t have to be an ugly looking marker. It can be any image.

Tish: So you’re using image recognition. Are you working with one of these image recognition startup companies (list here )?

Ori: We’re working with one of those. What’s unique about it is it runs very nicely on any cell phone, and on the iPhone it works the best. For this first app, it doesn’t really matter where you are physically; the geolocation is not part of the experience.

Tish: For a truly engaging AR experience we will need more of a backend than is currently available?

Ori: I call the backend the cloud, where you have all this information and ways to access it from anywhere. Actually I think it’s become pretty mature today. If you look at the different elements required to enable an augmented reality experience to work, you have – first – the user whose always in the center. Then you have the lens. The lens can be an iPhone, or glasses, even a projector. The lens allows you to watch, sense and track information in the real world: people, places, things. Then in the backend you have the cloud where you store and retrieve information.

So if you look at the maturity of these different elements, I think the cloud is in pretty good shape. Because there’s so much information we’re collecting and storing. Anything from Google, Wikipedia, Facebook, all that kind of stuff, it’s a lot of useful information you can access from anywhere using APIs. And a lot of it is also starting to include geolocation information. Take Loopt or Google’s friends service that allows you to see where your friends are and what they’re doing. There’s tons of information out there and it’s pretty easy to access it. Now what do you do with it is the question?

Wikitude is such a simple and brilliant application and nobody thought about doing it until this guy from Salzburg did. It doesn’t have any sophisticated visual tracking. It knows your position and it’s simply looking at the angle you’re pointing to. Based on these parameters it brings information from Wikipedia that pertains to your field of view. So most of it was already there. It’s just a matter of connecting the pieces in an experience that is valuable for people.

Tish: It is the uptake of even a very simple technology that puts the magic in it.

Ori:  Yes, take Twitter. If you go to its homepage it looks like a very simple boring app but it is something that is both enjoyable and very useful to people.

Why you should participate in ISMAR 2009

Tish: I know that you are involved in organizing  ISMAR (picture above from Ori’s post on “ISMAR 2009: The World’s Best Augmented Reality Event…,“) and there is a call out for papers and for volunteers, can you tell me more about it?

Ori: Yes, we hope to have the first ISMAR where we practice what we have just discussed: let’s build on all the research invested so far and instead of thinking only about 5-10 years from now, let’s see what we can do today. So we are bringing people in from other disciplines – artists, interactive media developers and people from the entertainment industry.  The goal is to use the technology to make something interesting for people – again, something that people would buy, and making it commercially successful.  Many people either don’t know about ISMAR because in the past it was a pure engineering-orientated event and people  from a commercial perspective of AR weren’t attracted to it.  The Chair of the Event this year is based in Florida and he is going to bring in a lot of people from the entertainment industry such as Disney. I think this will transform this event into something more like SIGGRAPH – more of an industry event.  As one of the organizers of the interactive media track we are trying to bring in people that want to build applications for consumers.

Tish: In terms of AR applications what are the flagships today?

Ori: There are very few because it’s just the beginning. There’s one tiny studio in France called Int 13 . They’ve created maybe the first commercial game running on a mobile device using AR technology. It’s called Kweekies. It was one of the contenders for the Nokia Mobile innovation awards. They were one of the ten finalists, but they didn’t win it. It’s looks really cool. It’s somethng that runs on your desk, with a marker. Many AR folks say markers are the past, markers are ugly. But it’s still a cool experience. I think people will go for it.

Tish: Yes I think we will have to look to small companies that are free to think creatively to lead the way.  It seems many games companies are tied up pulling off huge big budget projects and enterprise is still catching up on how to use social media!

Ori: Yes, last year I was in the game development conference (GDC); there was no mention of augmented reality – not on the exhibition floor, none of the sessions, nobody talked about it. I was stunned. Then this year, there was a little a change. There were like three demos on the exhibition floor, Metaio, Vuzix and a Dutch company called Beyond Reality.  And then there was Blair’s talk, which was very very cool. The room was packed with people. And after the talk there were dozens of people lining up to talk with him about the topic. There was definitely interest, but still on the very edge. The video game industry is still a hit driven business and publishers spend upward of 20-30 million dollar to create the best AAA game possible. They just can’t take the risk. So it’s going to come from smaller companies, from outsiders coming in with a vision and understanding on how to put the AR pieces together to create a totally new experience.

Tish: But the basic tool set is there isn’t it?

Ori: I talked to some folks at the games developer conference, many folks with MMO background, and they have great ideas about AR. It’s great to see different people with different views on what’s needed first. “Joe the Programmer” had this idea of creating a small piece of hardware that you can put in every house and provide accurate geospatial information in your home. That could  open up many opportunities for AR experiences in homes.

Tish: Don’t you think we have enormous resources in terms of image databases that provide a great basis for augmented reality.  I was talking to Aaron Cope at ETech about The Shape of Alpha – Flickr’s vernacular mapping project using all the geotagged photos in Flickr. That is such cool project. Aaron will be speaking at Where 2.0 also.

Ori: Think of Google Earth. Google Earth leveraged communities to basically map all the major cities around the world into 3D models. And that is an essential step to be able to do augmented reality outdoors. Because if you had to model everything from scratch, it wouldn’t be realistic.

Augmented Reality and Becoming Greener.

Tish: I am really interested in how AR interfaces might be useful to some of the emerging energy identity/metering projects like AMEE and WATTZON because I think it is very important that people have very intuitive, immediate, and enjoyable ways to relate to energy data so they can make greener choices.

Ori: Back in the day I had an idea to build an Augmented Reality application to become greener. You look at things around your home with the camera and it  recognizes its green gas footprint and makes recommendations to reduce it.  I guess it was a bit too early to do that based on visual recognition alone…you’d need  additional sensors that would provide related information about what you are looking at.

Tish: Well as there is more interest in Green technology do you think we may see VC interest in some green AR projects now?

Ori: I talked to some of the investment folks, Angels as well as VC’s about AR and they had no clue what it is. There’s a need for a whole lot of education. And there are no proof points (as in successful investments in this domain), and counter to popular belief – they don’t like risk so much…

Tish: And consumer adoption must lead the way, right?

Ori: Just like with every emerging technology in history, people never bought the technology, they bought the content, the apps, the benefits that came on top of the technology. Whether it was VHS winning over Beta Max, or BluRay winning over HD. It’s always because of more/better content. Look at the video game console war: Xbox, and Nintendo did better than Sony just because they had more and better games. Even Windows was a success thanks to its applications. People bought it for the applications not the OS. The content is the first to drive demand.

Tish: One of the challenges to giving people new ways to relate to their energy consumption is that you can just have them looking at graphs of how bad they have been in the past you – that may make them feel bad but that doesn’t necessarily give them ways or motivation to change. There perhaps needs to be more immediate relationship to the data to facilitate change. I think the mantra for optimization of anything from energy usage to supply chains is timely, actionable data?

Ori: There are a lot of ideas about measuring information and displaying it to people. For example, the Prius hybrid car, one of its interesting features – which is kind of game like – is a constant display of your current fuel consumption. That alone changes how people drive because they try to beat the “Score” and as a result conserve more fuel. That model can be applied to our homes…

Tish: Yes that is something I am very interested in. I have been following several projects in this area – one of my favorites is the Arduino, Current Cost/Tweetawatt, Pachube integrations I saw at Homecamp.

You joined a start up with Shai Agassi which was bought out by SAP right? He has a brilliant approach with Better Place.

Ori:  I think what’s really unique about Better Place’s approach is that he doesn’t require people to change their behavior. People are still going to have their own cars. They’ll be able to drive as far as they want, and for the same (or lower cost). Its not necessarily about a new technology, electric cars have been around for a long time but there was no way people were going to be limited by the 50 or 70 mile range and Better Place is solving that problem. With its infrastructure of charging spots and battery switching stations, drivers are going to be able to drive anywhere. And it’ll be similar to having to stop once in a while to refuel your car. The price maybe even lower than what you pay today for your transportation needs – and you’ll stop generating green gas. It’s a clever way of taking technology to a whole new level without changing the behavior of people.

Tish: Better Place is a classic example of things as a service isn’t it?  It is basically a utility company.

Ori: It is similar to a phone carrier model.  You pay for a membership that gives you access to the car (equivalent to the phone) and electricity (equivalent to the phone line) for the same price of fuel cost today. And as bonus you get to save the world.

How the iphone changed the game for AR – and the iphone versus Android

Picture from Ori’s post, “GDC 2009: Why the iphone changed everything”

Ori: And back to AR, you have to take the same approach, because nobody’s wants to don those huge head mounted displays or backpacks. You have to take advantage of people’s current behavior: they already carry their iPhones or similar devices.

Tish: As we discussed, you just have to get people raising up their phones and looking through them when that is a useful thing to do. Both Wikitude and Nathan Freitas’s graffiti app were enough to get me interested in the evolutionary step of raising my phone! Nathan’s graffiti app is nice. You leave a marker for your graffiti so other people can find view/add their own – a nice primal experience like pissing on the lamp post to let your pack know where you’ve been.  Also the graffiti app taps into a long history of  NYC street culture around tagging and graffiti art (see my interview, “Is it OMG finally for Augmented Reality?”).

Ori: The app store has fundamentally changed the mobile gaming industry. Last year they were in shambles. There was no growth. Everybody was complaining, “we can’t handle it, there’s a million phones, and you have to test it on each phone. And carriers suck, they don’t care about sharing and promoting your content. Everything was bad. This year mobile gaming is the hottest thing. And it’s all because of the iPhone. It changed the game.

Tish: How do you think Android is going to get traction against the iphone?

Ori: Well the number one thing is the form factor – the iPhone is just much cooler than the G1. Its OK but it doesn’t have the same feel. People thought it was going to be easy to clone the iPhone but none of the attempts succeeded so far.

Tish: How much does it matter for AR not being able to runs things persistently in the background on the iphone?

Ori: Actually they have add a such a capability in OS 3.  You can now make use of a background service.

Tish: OS 3 will open up new possibilities for AR?

Ori: The access to the video API is still not public.  But there is a new Microsoft application – Microsoft Tag that makes use of that API which means it is probably OK to use it.

Tish: (I ask Ori for his card and he shows me how to read it with my iphone.) Oh nice you have an AR card, of course!

In Search of Pong for Augmented Reality

Tish: So how will AR begin to, as Blair’s friend put’s it, “facilitate a killer existence,” particularly as we are probably looking at some new and perhaps pricey hardware?

Ori: You could take the Better Place approach. We’re going to give you a great experience and we’ll include the devices as part of that experience for the same price. Let’s say you subscribe to an AR experience  which offers access to multiuser, support, and all the information you need wherever you go – exactly according to the vision. You pay for a subscription on a monthly basis and included in that cost we give you a better device that offers a  better AR experience. It’s following the phone carrier approach, but in a good way.

But first of all we do need our Pong! I was sitting with a couple of AR game enthusiasts at the GDC and we were asking ourselves, “how do we create the first pong for AR?”

Was Pong a multiplayer game? Not necessarily! Did it connect to the network? No! We have to create the first dot in a long line of dots that will bring us to our destination.

Tish: You haven’t seen a Pong yet have you?

Ori: Not yet. I mean there’s maybe a handful of games and apps out there, but I don’t think any of them is a Pong yet. Still, it’s getting closer.

Tish: Kati London is doing some very interesting work on bringing games into reality, isn’t she?

Ori: Yes, she works with Frank Lanz at Area/Code. He teaches at NYU and has designed games for the “Come Out and Play” festival here in Manhattan. And a lot of these games are actually low tech.

Tish: Yes I have a big alternate reality game blog brewing that I haven’t had time to write yet!

Ori: The city is the gameboard is their slogan. It’s going to be a great playground for AR games. The city becomes a theme park. The city could become an even bigger touristic attraction. People will come to the city to be part of these games. So you’re having thousands of people running around the city playing all sorts of games from laser-tag style to history adventures, to treasure hunts.

Composing Reality

Tish: So why haven’t you focused on one of these kinds of games with your company?

Ori: We have a couple of scenarios along these lines that we’re planning for 2010-11. But first focus on what’s possible today.

Tish: And what’s stopping you from doing those kind of games today?

Ori: Many things. The devices are not there yet, location services are not accurate enough, ubiquitous sensors are not  there yet.

Tish: You think alternate reality gaming needs more “ubiquity” than is currently available?

Ori: Not necessarily. People are doing alternate reality games with no “ubiquity” at all. But my interest is to add the visual aspect. I believe humans are mostly driven visually.

Jane McGonigal said in a talk at GDC, that AR would allow us to program reality, which is exactly how I look at it. Once you can recognize things, some of it with WiFi and RFID and all sorts of sensors. But visual sensors is always going to be the ultimate way to recognize things. And once you recognize things and know what they are, and can pull information about those things (or people and places) from the internet, you can program it (visually). You could program it to be fictional, like in a video game, or it could be programmed as non-fictional, like a documentary. And that allows you to do things that before were unimaginable.

Tish: But you can’t forget the visual, it is primary the connection to peoples’ primary sensory relationships.

Ori: Yes, it’s like you go to a grocery store and you pick your vegetables, a lot of it is by sight and by touch. And what if you could also see just by looking at it that it’s from a local store, and that it’s organic?

Tish: It goes beyond overlays really?

Ori: By the way, I don’t like the term ‘overlay’. I know that’s how it looks: you either overlay or superimpose, but I’m still searching for a better term. A term I prefer to use is “composing reality”. Just like painters, they use brushstrokes and colors and compose a painting. We need to take the real element and the virtual element and compose them into something new. It’s not just about slapping one on top of the other.

Tish: yes I think the idea of dashboards is not so appealing.

Pookatak Games

Tish: Do you want to explain the evolution of your company? You have an interesting history of success with high end enterprise applications.

Ori: Since I was a kid I wanted to invent and create things. When I discovered software, that was a really cool way of actually creating things from nothing. From thin air; and you can do it very quickly. That’s what brought me into software. But I was always looking for the intersection between technology and art. Looking for ways to bring these things together. In the early nineties virtual reality was doing it. It had the appeal of cutting edge technology that can be combined with art. But then, as we all know, it crashed. So I joined Shai Agassi’s startup (who is now doing Better Place) back in the early nineties. I was one of the first employees in his startup which was developing multimedia products. I was leading the development of one of its flagship product. At some point we realized the technology could be great for an enterprise environment.

It was a really great experience. First going through this cycle from a very small startup and growing into this multi billion dollar business. I was responsible for defining and marketing SAP’s platform, which was called Netweaver. It was just an idea when we joined SAP and by the time I left it was a major, major business for SAP. I learned about the challenges of building a platform. No matter what purpose you’re building it for, it typically has similar rules. It’s definitely not just about the technology; the content that comes with it is really key to making a platform successful.

The third part of this platform trifecta is the community. If you don’t build a community, you won’t get the critical mass required for adoption. It may be your own platform but it’s not necessarily the people’s platform. That experience is very key to what we’re doing today. Now, a new industry is being born on the basis of a remarkable technology. But to drive adoption, first we’ll need good content. The content will be created using today’s technology with internal tools developed to simplify the process. Next step would be to make the tools used internally – available to other developers. Help scale the industry, enable innovation on a larger scale. That way we have a chance to create a platform. So it isn’t really just about my company. I’m so passionate about augmented reality, I want to it to become a healthy and successful industry for the next 5, 10, 15 years.

Tish: Yes I am so ready to be liberated from the sitting behind a computing screen! And I know that all this hardware is murdering the environment.

Ori: There’s ‘s the book by Rolf Hainich which is called “The End Of Hardware. ” It’s about hardware for augmented-reality. Once you use goggles or other AR interfaces you eliminate the need for screens, laptops, etc. It’s going to be great for the environment. You have read Rainbow’s End, right? According to the book in few years there will barely be any (visible) hardware. At least it’ll have a much smaller footprint for the environment. And it’ll touch every aspect of life, everything you do. It’ll change the way you interact with the world.

The Illusive Eyewear for Immersive AR.

Friend of Ori’s in San Francisco wearing retro AR goggles (from Games Alfresco, Ori’s roundup of GDC 2009)

Tish:OK lets talk about goggles.

Ori: Goggles are going to happen, we want to be hands free.

It’s going to happen because it’s just a more intuitive way to use this technology. But above all it has to look cool. Because if it’s not, if it’s a big headset, then maybe a small percent of the population might use it, but most people won’t. It has to look like an accessory, like new cool eyeglasses that you just must wear.

I recently talked to a friend, who runs an industrial design firm, and has experience in designing such glasses for companies like Microvision and Lumux. He says that when you try to bring the images so close to our eyes – there are some really hard problems to solve. Otherwise it can become really annoying and cause dizzyness.

But I’m optimistic. I believe it’s going to happen 3 to 5 years from now. It’s already starting now: Vuzix announced goggles that will be available this year. Some AR apps that are going to take advantage of next year. Initially only a fraction of the population will use it. And that’s going to help advance it and make it better and better. But it’s going to take time until it reaches the mass market.

Tish: In virtual worlds we have seen, I think, a lot of mistakes in terms of reinventing the wheel and producing too many proprietary versions of the same thing and not enough concerted effort on standards and open platforms that could create a vibrant ecosystem.  How can augmented reality not make the same mistakes?

Ori: There are some early AR open source efforts ARTookit, ARtag but it is not a movement yet.  One of the things we’re trying to do at ISMAR this year is to put together  discussions around key industry issues, such as standards. Some people say it’s too early, you have to have a defacto standard to start from. But pretty soon it’s going to be too late. Just like with virtual worlds, all of a sudden you have all these islands that don’t talk to each other. Why get to that point if we can plan to avoid it? Let’s start thinking about it right now. On the other front there are devices. There are pockets of people working on adapting devices for AR, second guessing the hardware companies. Why not get them together with the Intels and Nvidias of the world, and discuss what this device should be able to do. And then compete to make it happen.

Tish: How much luck are you having with this discussion part?

Ori: People are very interested in doing this. We proposed these panels for ISMAR. And I’ve got some key people already on board. They have tons of input, they want to get involved. We’ll see how much we can actually get out of it.

Tish: In virtual worlds it was a while before vibrant opensource communities developed.  OpenSim has I think been the breakthrough community in this regard.

Ori: You have to think about the elements up front. The dream job is to architect the industry. Say we agree on the required pieces. Then we could help the right companies succeed in delivering the pieces. Next, we have to collaborate so that these pieces talk to each other. And eventually these communication methods will become defacto standards and most developers will adopt it.

Tish: So I’m going to put you in the role. You’ve got your dream job. You’re going to architect this community. So what are the key pieces and where would you like to see the open source communities take hold first?

Ori: Open source will not be exclusive. It’s going to live side by side with proprietary technology.

The key pieces? You have the user at the center. And the user interacts with a lens. The lens includes both the hardware and the software. And then the lens senses and interacts with the world, which includes people, things and places. And these people-things-places emit information – about who they are, where they are, what they’re doing, etc  – which is then stored in the cloud.

And then you have the content providers, the people and companies, composers who weave AR experiences through the pieces we mentioned before. These composers need a platform that glues these pieces together. Pieces of the platform will be on the lens, and in the world, and in the cloud. If you manage to remove the frictions, and connect these pieces into an experience that people like – then you have a platform. What the platform does it reduces the overhead and accelerates innovation.

Tish: Another problem virtual worlds faced in their development was their isolation from the world wide web.  Will augmented reality avoid this plight?

Ori:  Yes, I believe the key, like you said before, is not to reinvent the wheel. The cloud is already there.  Take Wikitude for example, all Mobilizy had to do is build  a relatively simple client app, connected to wikipedia, and all of a sudden it offered a wealth of information in your field of view.

I think we can learn a lot from web 2.0. For example, in order to have a ubiquitous experience like Robert Rice and others are striving for, you’ll need to 3d map the world. Google earth like apps are going to help but it is not going to be sufficient. So let’s leverage people. Google became successful in part by making people work with them.  Each time you create a link from your blog to my blog their search engines learn from it.  So let’s find ways to make people create information that can be used for AR.

Ori Inbar directed Wiki Mouse – a WIKI Film co-created by a swarm of movie makers around the world.

Neogence is on stealth mode with an immersive mobile augmented reality platform – “tools, sdk, and infrastructure plus some applications.” They are probably six months away from YouTubing anything according to CEO, Robert Rice.  But Robert rustled up this pic for me – a Google street view of Neogence R&D labs: “the patio on the lower left is where I do a lot of pacing and smoking my pipe and the porch and office upstairs is where  a lot of  meetings have been held.”

Robert Rice (@RobertRice ), CEO of Neogence, recently tweeted:

I’m changing my name to Robert Mobile Ubiquitous Geospatial Augmented Rice. I’m betting on radical changes in next 18 months.

Although Robert’s new AR platform is still under wraps, I think you will get a good idea of what direction he is going in from this interview (full text at end of  this post). Robert is the author of “MMO Evolution” and is a key developer and thought leader in persistent immersive environments, simulations, virtual worlds and massively multiplayer games as well as large scale communities and social networking.

It is OMG finally, at least, for minimally immersive but truly useful AR.

Since the launch of Android a new generation of useful augmented reality applications like Wikitude are emerging.

After the last NYC Tech Meetup, my  friend Nathan Freitas,  (@NatDefreitas),  or rather Nathan Mobile Meets Social Freitas, demoed for me a cool graffiti app  he has developed on Android.   You leave a marker for your graffiti so other people can find view/add their own – a nice primal experience like pissing on the lamp post to let your pack know where you’ve been.  Also the graffiti app taps into a long history of  NYC street culture around tagging and graffiti art.  For more cool mobile projects Nathan is working on – Vote Report and data collection for mass events, a guide to pubs and nightlife in New York City, and more, see his blog, “Nathan’s OpenIdeals. With Camera, GPS, compass, and accelerometer, and APIs on Android for temperature, light meters, (no hardware yet), Nathan says Android:

“seems to be the platform most likely to socialize the idea that sensor data could be a piece of every application.”

As Nathan is fond of saying:

The compass is a killer app enabler!

Also see OpenIntents for some interesting Android Sensor projects.

Wikitude was one of Thomas Wrobel’s two top AR milestones for 2008 (see Gamesalfreso):

Wikitude I think. Seems the first released, useful, AR software.

AR Geisha doll is also a remarkable breakout for AR – but useful, nah.

I asked Robert if he also thought Wikitude and AR Geisha doll as  significant breakthroughs:

Yes,  these are among the first attempts to get away from the novelty of simply rendering a 3D object based on a marker and making it interesting.

Remember, one of the biggest risks that AR has, is being branded as “novelty”, which means “cool for five minutes but ultimately a waste of time.” I think we have a ways to go before something is truly useful, but as 2009 progresses we should start seeing some effort here. I’d guess 2010 before something really useful comes out…at least something practical.

Now, having said that, I should say that I expect entertainment and games to take the lead (as usual), although there are a few companies really trying to leverage AR and video/graphics compositing for marketing (brochures) and location based methods (kiosks, large screen projections, etc.)

So when is it “OMG finally!” for massively multiuser augmented reality?

The picture above is from benjapo’s portfolio on istockphoto – also see the istock video here.

Alex Soojung-Kim Pang (who weighed in recently on the twitter-baby debates – see my KickBee Posterous blog) challenges design assumptions for augmented reality that take as a given the user’s desire for numerous private enhancements to their reality.

Alex points out less will probably be more so that enhancements do not impinge on shared experience.  See his write up of a talk he gave at the Royal College of Art, “and the end of my own private Shibuya.” Photo below by Stéfan, “Karaoke in Shibuya“

Part of the pleasure of these streetscapes is precisely that they’re collectively experienced, rather than individual visions: for even a brief period, we share with other postmodern, globe-hopping flaneurs and expatriates and temporary natives the light of the ABC-Mart sign and storefront.

It is collective experience of enhanced, augmented, virtual or real experiences that interests me too. This is one of the reasons I find Pachube and the EEML project of Haque Design and Research so interesting.

Extended Environments Markup Language (EEML), a protocol for sharing sensor data between remote responsive environments, both physical and virtual. It can be used to facilitate direct connections between any two environments; it can also be used to facilitate many-to-many connections as implemented by the web service Pachube, which enables people to tag and share real time sensor data from objects, devices and spaces around the world.

“Distinctions between virtual and real are as quaint and outmoded as distinctions between mind and body” (Usman Haque)

Usman Haque (founder of Pachube and Haque Design and Research) points out this is an underlying premise of his work – and augmented reality (full interview coming up soon!).

The pictures above show the Haque Design project, Remote:

‘Remote’ connects together two spaces, one in Boston the other in Second Life, and treats them as a single contiguous environment, bound together by the internet so that things that occur in one space affect things that happen in the other and vice versa – remotely controlling each other.

There was a discussion in twitter recently about how the terms like Second Life, Exit Reality, Virtual Worlds are misleading and outmoded. As Robert pointed out we need:

one word please…that sums up virtual and/or augmented reality, interactive, immersive, virtual worlds, mmorpgs, simulations, etc… also, I really don’t like the term “augmented reality” or “mixed reality”. Neither is all that great. And NO “matrix” or “metaverse” please

Robert argues strongly that there is a stultification both in virtual world technology – much of what we call virtual world technology was already, basically, where it is now in the mid 90’s. And MMOGs have devolved into gameplay design “that emphasizes the single player experience and does nothing to take advantage of the potential of the massively connected internet.”

Robert suggested I take a cruise through a new Virtual Space -  CoolIris to find some good pictures for this post (note the partnership between CoolIris and Seesmic to Streamline Video Blogging. I added the Cooliris Plugin to Firefox and typed Augmented Reality into search and soon I was cruising a highway of images and links. The Road Map image grabbed my attention (see below). It shows the continua that the Metaverse RoadMap authors thought are likely to influence the ways in which the Metaverse unfolds. It is “a map of the spectrum of technologies and applications ranging from augmentation to simulation; and the spectrum ranging from intimate (identity-focused)external (world-focused)”

Quite to my surprise, when I clicked out of CoolIris to the source for the image, I found it had been drawn from a post I wrote in May 2007, Hybridized Digital/Physical Worlds: Where Pop and Corporate Cultures Mingle. My post talks about a number of hybridization experiments that were bringing together lifelogging, sensors everywhere, simulation, virtual worlds, and augmentation.

The striking difference from 2007 to now is that we have definitely moved on from mere experimentation. And the poles of the continua intimate/extimate, augmentation/simulation as expressed in the Metaverse Roadmap are now becoming entwined (note the picture above seems to be slightly different to the one used in the road map as published here – perhaps I had an early version?)

“Augmented Reality is not just about overlaying data…” (Robert Rice)

The screenshot above is from TotalImmersions video demoing Augmented Reality with 3D Cell Phones. Also see video of their immersive games, and FutureScope kiosks here and here.

Gamesalfreso noted that Will Wright, delivered the best augmented reality quote of the year. When describing AR as the way of the future for games, Will Wright said:

“Games could increase our awareness of our immediate environment, rather than distract us from it”.

Robert points out in this interview the term Augmented Reality itself has become associated with a very limited understanding of what “enhancing your specific reality,” is really about. Robert notes:

it is inherently about who YOU are, WHERE you are, WHAT you are doing, WHAT is around you, etc.

When I talk about AR, I try to expand the definition a little bit. Usually, when you talk to someone about augmented reality, the first thing that comes to mind is overlaying 3D graphics on a video stream. I think though, that it should more properly be any media that is specific to your location and the context of what you are doing (or want to do)…augmenting or enhancing your specific reality.

In this sense, anything that at least knows who you are (your ID, mobile phone #, etc.), where you are (GPS coord or a specific place like a cafe), and gives you relevant data, information, or media = augmented reality. Sure, you can make things more interactive or immersive, but that is the minimum.

So, in this case, yes, I think there will be networked applications in the next 18 months…mostly things that are enhanced by friends lists (you are here, your friend is over there). These will be *application specific*. My team at Neogence is already going beyond this, building a platform and infrastructure for other applications to be developed on…all networked through the same backbone. Now, in this context (the science fiction AR that we all dream about), no I do not see anyone else trying to leap a generation or two ahead of the industry to build a massively multiuser shared AR space. Expect to see things like multi-user AR games, virtual pets, kiosk marketing, magic book, “gee whiz” presentations (tradeshow booths, entertainment parks, etc.), and so forth.

Goggle’s Are Not The Secret Sauce…

AR Cat left and Robert Rice right

What has come to be associated with the term Augmented Reality, in the popular imagination – an idea of 3D graphics projected over markers that has been forever waiting for the advent of “wicked next generation transparent wearable displays” – nirvana for augmented reality. While such displays may be nirvana for AR (and they could be with us in less than twenty four months), Goggles are not the “secret sauce” of AR as Robert points out.

All the glasses are, is another display device. At the end of the day, it doesn’t matter if you are looking at an LCD monitor, an IPhone, a head mounted display, or a pair of wicked next generation transparent wearable displays that magically draw directly on your retinas.

The real tricky stuff is what happens on the backend…making it all persistent, massively multiuser, intelligent, interoperable, realistic, etc. etc.

There has been quite a buzz going around about the new Vuzix Eyewear, and recently Robert talked with Vuzix and checked The Wrap 920AV eyewear out:

Vuzix is not alone in pursuing the ultimate in hardware, at least as far as wearable displays. However, I think they are much farther than the rest of the pack in vision, roadmap, and execution. They have put together a team that has a sense of urgency and ambition that will blow the industry away. After talking to them, I got the feeling that they really know what they are doing and there is a lot of mind blowing stuff in their pipeline. I’m sure they are one of the few companies that really gets it and has a clear vision of the future. Definitely my first choice to work with.

Hybrid Augmented/Virtual Reality

Cory Ondrejka posted this picture of the anybots telepresence robot and “congrats to Trevor Blackwell and the rest of the Anybots team on the launch of QA at CES.”  Cory (one of the founders and former CTO of Second Life) also made some predictions for Virtual Worlds, some optimistic and some less so, including “the increasing need to be able to diversify the Second Life product offering to begin truly rebuilding the code base.”

Robert is unabashedly irritated with the state of play in Virtual Worlds and MMOGS:

Unless both industries (Virtual Worlds and MMOGs) have some serious upheaval or radical new approaches, they will quickly be eclipsed by AR, which will eventually evolve into something hybrid..AR/VR depending on your level of access and hardware.

I’d like to see someone grab an engine like Offset, Crytek, HERO, or Unreal 3, and smack on a fat MMO server infrastructure (Eve or Bigworld)…toss in the right tools, and you would see a revolution and renaissance occur at the same time in the virtual world space. All the puzzle pieces are there, just no one is putting them together the right way.

I did just find out that Nortel’s WebAlive is powered by the Unreal 3 engine. You can try WebAlive out here.

Robert points out how rare it has become to see people really push virtual worlds technology and MMOGs into entirely new directions.  Although, of course, there are exceptions.  I managed to engage some interest from Robert in the possibilities the opensource modular architecture of OpenSim opens up, and the augmented reality experiments from Georgia Tech with Second Life (screenshot below) got praise from Robert for trying to do something new. (Georgia tech have also put out a virtual pet app for the iphone ).

But while Robert clearly has zero patience for virtual world technology which he sees stuck in the mid nineties, he notes:

the innovative and wonderful stuff about SL isn’t SL, it is what people are doing and creating on their own with terrible tools *IN* SL [Second Life].

The immersive mobile augmented reality platform Robert is building, he hopes, will generate this kind of user creativity but with 21st century tools.

So is it “OMG” finally for the Augmented Reality we have dreamed about?

According to Robert:

It really boils down to a markerless solution and a good application.

In the interview below we cover a number of topics including business models for Augmented Reality, e.g., how business models based on micro-transactions and virtual goods will translate to Augmented Reality.

Many of the challenges to becoming mainstream faced by virtual worlds  are similar to the challenges AR must overcome. Robert discusses these including the interface/gui that is a critical element for AR, solving the riddle of one world or many, patent wars in Virtual Worlds and Augmented Reality, the role of Augmented Reality in the future of sustainable computing, and what interoperability is about.

The Back Story for AR/VR…

In case you want to get up to speed on the required background reading for  Augmented Reality. This is Robert’s required reading list and Denno Coil is an absolute must see (feel free to add to this list in the comments, please).

“If you want to see the things that have inspired our vision of what we want to build, check out:

* Dream Park by Larry Niven and Steven Barnes
* Rainbows End by Vernor Vinge
* Spook Country by William Gibson
* Halting State by Charles Stross
* The Diamond Age by Neal Stephenson
* Donnerjack by Roger Zelazny and Jane Lindskold
* Otherland by Tad Williams
* Neuromancer by William Gibson
* Idoru by Wiliam Gibson
* Cryptonomicon by Neal Stephenson

and watch the whole anime of Denno Coil (subbed NOT dubbed!)”

Screenshot from Denno Coil from Concrete Badger.

Interview With Robert Rice

Tish Shute: I am glad to hear that you are working on this [an immersive mobile augmented reality platform]!

Robert Rice: We switched gears from MMO stuff about a year ago and we are finally getting some traction. It is very hard doing anything in this economy right now, but we found an opportunity to take AR to a new level beyond what you see on youtube. AR is still too “cute” and novelty. We don’t want to play around.

Tish Shute: I like Wikitude ‘cos it even manages to do something useful!

Robert Rice: Yeah, useful = traction. Now that we are getting near a prototype we are starting to get a lot of interest even though we are still technically way under the radar.

Tish Shute: r u funded?

Robert Rice: privately funded, some revenues from an early license, and ongoing discussions with several institutional investors. So, we have some funding, but nothing spectacular just yet.

Tish Shute: are you just developing an AR platform?

Robert Rice: hrm, sort of, but not just that. By platform I mean tools, sdk, and infrastructure plus some applications. The idea is to build something that facilitates everyone else making cool things and useful applications for different industries/sectors

Tish Shute: Yes that is the cool thing to do but isn’t that hard to fund!

(Robert grins) Well, that depends on the business model. We’ve got that figured out. I’d be absolutely happy if everyone and their brother were making applications on our stuff that gives us an edge on market penetration/saturation. There are plenty examples that prove the model. If you give people free and easy to use tools, they will run with it. ARtoolkit for example, has tons of people making nifty things and posting videos on youtube that has pushed them to the forefront as THE AR middleware to use right now, or heck, look at youtube free service, and they dominate video sharing.  Sure there will be a lot of “noise”, but there will also be a lot of “signal” that will rise to the top, facilitating and enabling is creating value in its own right.

Tish Shute: But how do you expect to monetize?

Robert Rice: There are a good half a dozen ways to monetize AR or an AR platform.

Tish Shute: What are your top 3?

Robert Rice: hrm, microtransactions, localized mobile advertising, and enterprise solutions (visualization)

Tish Shute: Do you think the consumer market will give the lead?

Robert Rice: I’m not sure. We are getting people from academia, intelligence, defense, border security, and some corporate types knocking on our door already, and pretty aggressively. It may be that those sectors push AR before consumer entertainment really kicks off.

But going back to a discussion we had earlier – yes working with “no markers” is a big deal.

Tish Shute: Can you talk about what you are doing there or is it still under wraps?

Robert Rice: I can say that between some university tech transfer and some of our own proprietary stuff, we are using some fairly common visual tracking technology. if you are really plugged into the AR scene, you will know there are probably half a dozen visual tracking methods out there. We just looked for the best one, licensed it for commercial use, and then started working our magic. This is a very small piece of the overall effort, but worth noting.

The downside with working with university tech is that it is usually based on research, incomplete, and not wrapped up in a nice commercial package on the upside, it can be a good start to build on.

Tish Shute: As you know I am very interested in “technology that matters” in particular tech that can help us accomplish the urgent goal of sustainable living.

Robert Rice:: oh, I’m pretty keen on sustainable living as well…after I sell off a few companies and have money of my own, I’m going to get into arcologies
❦
Robert grins

The interesting thing with the visual stuff combined with our other tech, is that we can make things multiuser, persistent, dynamic, and mobile.
The markers (fiducials) are really really limiting outside of basic applications. You can’t really plaster everyone and everything with a marker.  And they are, by nature, static (even if they are animated or whatever).

Also… our stuff works indoors and outdoors even without a GPS connection.
❦
Robert grins

Tish Shute: Now that does sound interesting!

Robert Rice: Yeah, with visual, you don’t need a compass or accelerometers either. Less hardware : )

You start with wifi triangulation or gps coord to get a “brute” location, and then you use the visual stuff for down to the meter accuracy and that by nature, gives you your orientation and positioning.

Tish Shute: Wow this is beginning to sound very interesting!

Robert Rice: Once you have that, it doesn’t matter where you go, it continues to track and continually refines areas you have been before. We’ve spent the last year figuring all this out. There are so many problems and obstacles that are going to be developing in the future for anyone trying to do what we are, but we have already discovered solutions.

oh, visual tracking = gesture based interfaces too that’s going to take some work, but its doable.  The real pain in the ass there isn’t the actual tracking, it is in the interface design.

That’s something that almost every AR company, venture, and research program is missing out on entirely. They are so focused on making cute things with markers.  They are missing the larger problems of AR Spam, interface, iconography, GUI, metaphor, interoperability, privacy, identity.

Tish Shute: So how are you dealing with all that!!

Robert Rice: We took the backwards approach of trying to think where we want things to be in ten years (and we read all the cool books…Vinge, Stephenson, Gibson, etc.) and then we spent time trying to think of what the potential problems are….like AR spam. Its bad enough when a giant penis flies by in second life, we don’t want that to happen in a global wireless AR platform.

Tish Shute: Do you have a prototype yet?

Robert Rice: hrm, 6 months away from youtubing something. Problem has been slow funding, which equals slow development. We also don’t want to show our cards too soon…too many potential competitors out there.

❦
Robert grins

Tish Shute: when you say microtransactions what is the business potential there?

Robert Rice: hrm last year I think, $1.5B was spent on virtual items. That’s games and virtual worlds. That should hit $5B in a couple of years. That’s basically people buying and selling things like WoW gold or items in SL or whatever. microtransactions, is basically the same thing, but in AR space.

Why couldn’t a 3D artist make a wicked animated 3D dragon, and then sell it to someone else? With AR, you could sit it on your shoulder. With a good scripting engine, you could train it to do stuff. Thats what I want to enable.

tools + sdk + platform = enabling people to make and create. Add in a commerce level (microtransactions) and wala.

Tish Shute: At the moment all of these virtual goods are very platform specific, is that a problem for you?

Robert Rice: Not at all. This is at a higher level. You have to switch mental models when you talk about what AR could or should be. For example, lets contrast the web and virtual worlds. For every virtual world you go to, you have to download a whole new client. Imagine if that model was applied to the web… you would need a brand new browser for every website you went to. That is just so…wrong.

It’s the same thing for AR…people are thinking about it with the same mental and business models and development philosophies as virtual worlds or web.  There are some things and aspects that work fine, but not everything.

Virtual worlds, are, by nature, necessarily different and walled gardens. The idea of 100% open and interoperable virtual worlds is a red herring…it sounds good but in practice it is a really dumb idea.

Tish Shute: I was wondering if you had a way to leverage all the 3D content already created ‘cos that would jump start things in AR wouldn’t it?

Robert Rice: Oh yeah, that’s easy. They all use the same polygons. Any virtual item in any game or virtual world is likely created with 3D studio or maya or something similar would be easy to convert and use.

Tish Shute: So people could bring their WoW weapons into your system?

Robert Rice: Not legally, but sure. Its just a 3D model with a texture.  It doesn’t matter if you use corel draw or photoshop or paintshop pro….or one screwdriver or another. Part of my team’s advantage, is that we are all experienced in MMORPG and virtual world design and development. We know the tools, the tech, and what works and what doesn’t.

Tish Shute: But some of the 3D content created in the social worlds is what has most value to people.

Robert Rice: Right, and that can be exported out easily.

Tish Shute: But back to “real” life applications. Is you platform really markerless?

Robert Rice: Yes.  marker = printed icon or glyph, also known as a fiducial

Tish Shute: But u must have some marker?

Robert Rice: hrm, more accurately, you need a point of reference.

Visual tracking has been around for more than a decade.  Lots of work for robots and other sectors.

Robert Rice: But isn’t the specificity of reference n terms of RL applications a vital key, for example, for a database of things?

Robert grin That is a different problem…tracking, registration, mapping, positioning, etc. That question has to do with mapping which is related to visual tracking, but not the same thing. We have a rather unique approach to some of this that I can’t discuss (patent pending).

Tish Shute: But for example, to create an augmented natural history of food – say I want to point at the slab of meat on my plate and know where that cow came from, what feed lot how it was treated etc

Robert Rice: That is not possible without ubiquitous nanotechnology. Shall I explain?

Tish Shute: Yes please!

Robert Rice: Ok, lets step back a minute and turn that burger back into a cow… the first problem (of this particular situation) is differentiating from one cow to another since most cows look alike, you can either attempt to discriminate visually (cow patterns) or use a much simpler option, like giving each cow a rfid chip in their bell, or hoof

Now, most people would try to figure out how to jam all sorts of info in the rfid chip, which sounds like a good idea, but isn’t, the trick would be to simply use the rfid to store a unique identifier with is then linked to a database elsewhere, or hoof.

That database should continually be updated with whatever relevant information you need so as you get close with your AR laptop, wearable displays, or embedded brain chip, you get the identifier broadcast, then you get the info downloaded to you, and it “sticks” to the cow with the generic visual tracking (object following, even simple bounding box is sufficient for a slow moving cow)

So, up to that point, you can get tons of information about that specific cow, that cow population (remember, AR is not just about overlaying data…it is inherently about who YOU are, WHERE you are, WHAT you are doing, WHAT is around you, etc.) Tie in data visualisation and some farmer tools and all sorts of other things happen. Now, lets move the timeline ahead a bit.

The butcher gets the cow and does his handiwork…because we know all the info about the cow, all of the meat can be properly labeled and marked. Ideally, with a UPC code or a unique glyph (somewhat problematic depending on how many unique glyphs you can create) so, while you are in the grocery store, you can access the relevant shopping data…age of cow, state of origin, type of feed, how many spots, how much body fat, which butcher, whatever not because of what is inside the package, but the package itself.

Getting back to your hamburger, the problem is that it is a burger…there is nothing to distinguish that burger from another one at the table…unless you stuck a rfid chip in it or splattered it with ink and a unique glyph, or maybe a special one of a kind plate.

However, a properly designed AR system could say “hey! that/s a hamburger! and I know I am at Fat Daddy’s Burger Joint in Raleigh North Carolina on Glenwood Avenue, and I know that they cook their burgers this particular way, and their meat supplier is those guys over there, and they usually get their cow meat from a farm out in Utah”

With ubiquitous nanomites or whatever, then its not that far out to consider edible nanos that are in the meat and that broad cast info so a slab of meat can tell you about itself and broadcast that to the general public.

Tish Shute: What useful scenarios can we create without the nanomites?

Robert Rice: If it wasn’t a burger or a consumable organic, the scenario changes.

Tish Shute: What is the time scale on nanomites?

Robert Rice: ehhhhhhh 20 years minimum if we are lucky. They sound good on paper, but there is a whole book worth of problems and why they are so far off…as consumer grade, all over the place, type of stuff.

Tish Shute: Did you see the Nokia Home Control center?

Robert Rice: Yes, I saw the Nokia stuff.

AR for sensors, like security systems, temperature control, etc. all become “sources of data” that a AR system can visualize. So yes, thats easily doable. You could do that in a short period of time with some half decent engineers.

The trick of what Nokia is doing is aggregating sensor data from a building/home/facility, mashing it together, and sending the mobile device alerts and data visualization conceptually rather simple, but no one has done it right or well yet.

It wouldn’t surprise me if Nokia pulled it off.

Tish Shute: yes and if they do and someone does an AR interface to it that would be an inflection point for AR?

Robert Rice: In a roundabout way, yes. You could get data directly from your house, or get it through your mobile device and in either case, use the AR for visualization and control.

The interface/gui is a critical element for AR. That is one of the areas where it, as an industry, risks doing a bad job and turning into just a fad or another novelty like VR.  Virtual worlds have been struggling with that for a while, but MMORPGs have had the effect of extending their life cycle

Tish Shute: Yes VWs have not solved the interface problem.

Robert Rice: The interface is one of their problems yes. Most virtual worlds are stuck in 1996/98

Tish Shute: If AR  is inherently about who YOU are, WHERE you are, WHAT you are doing, WHAT is around you, etc. seems that it is the ideal interface for home control?

Robert Rice: Well for home control, you must know:

1) Who am I? Am I authorized to know this information? Am I a guest?

2) Where am I? Is this my house? or someone elses?

3) What am I doing? Do I want to make all the doors lock? Turn on or off lights? Open the garage door? Trigger the security alarm?

So the same questions apply

I’d say that all virtual worlds are stuck in the mid 90s. They are at least a decade behind the game worlds…in technology, design, implementation, architecture, etc. etc. In my opinion, things like Second Life are shameful in how they are presented as state of the art, innovative, ground breaking, new, wonderful, and world changing.

But thats another topic of conversation : )

Tish Shute: Well for me the contribution of VWs is the presence enabled real time interaction with application (as 3D info machine) and context with other people.

Robert Rice: Oh,there is no doubt that they are greatly useful and have a phenomenal amount of potential.

They *could* be all those things I just said that SL isn’t…the problem is that they are either just existing, or they are meandering around without any real focus or direction. They aren’t evolving.

Even MMORPGs are losing their way and beginning to stagnate terribly

Tish Shute: yes I agree

Robert Rice: But, AR has the potential to change a lot of things.

Im sure you have seen the yellowbook commercials? The technologies you are seeing here are doable in hrm, a year or less maybe. The tricky part is the interactivity and AI…that is, the content. Everything else isn’t a problem. The avatar there could be photorealistic or stylized like a WoW character.

You could do that to some degree with markers for registration but dynamically changing the content linked to those markers is a little weird

(by the way, for the record, I like markers just fine, I just don’t think they are useful for real-world mobile applications)

I also think that the guys that want to dust the planet with miniature rfid chips are on crack and are going about it the wrong way

Tish Shute: A high level of interactivity is hard though. Isn’t it? Even in VWs it is very limited.

Robert Rice: it depends if you can track what the user is doing, and interpret that properly. Interactive is also a very lose term.

Clicking a button and making a light blink could be considered interactive.

Tish Shute: In VWs a high level of interactivity would  be to wield a virtual hammer and have a real nail go in! is physics part of the problem?

Robert Rice: physics aren’t difficult, plenty of middleware out there for it. The problem with that isnt so much the physics as much as it is the scale and purpose

Tish Shute: well for robotics?

Robert Rice: that gets into a conversation about meshes, textures, and volumetric collision detection and stuff

Tish Shute: virtual robotics?

Robert Rice: You mean teleremote/telepresence of real robots?

Tish Shute: yes!

Robert Rice: ah, for that, you need some tactile feedback and some other stuff – doable, but insanely difficult. That’s why you don’t see a whole lot of remote controlled surgery robots all over the place.

They do exist…

Tish Shute: Will AR contribute to sustainable living by freeing us from some of our energy hogging devices?

Robert Rice: AR will ultimately encourage energy saving and recycling. where did I leave a light on at? where is the nearest trash can? what is the UV index outside today?

Yes, computers are energy hogs, but as we start seeing larger SSD drives, more efficient CPUs (even if the number of cores increases in multiples), and so on, the power will go down.

Also, think about this…wearable displays potentially use less energy than LCD monitors on your desk.

Tish Shute: Yes I should pick the brains of my intel chums on energy saving!

Robert Rice: Getting rid of the monitor and switching to solid state drives will save an assload of power. Yes, I said assload.

Tell your intel chums to quit screwing around with single core mobile CPUs. We need multiple cores, that are smaller, faster, and use less power.

Tish Shute: Is AR is the sustainable future of VW and MMOGs?

Robert Rice: The fun stuff will happen when they are both integrated in some fashion.

Tish Shute: So perhaps this is why the Georgia guys are thinking in trying to combine AR and SL (see video here).

Robert Rice: That first video was pretty damn cool. It just pains me that they are using SL for it. And omg, all those markers on the table.

Although, I could care less about seeing my SL avatar on my coffee table. I would rather see an avatar representing ME in the real world, moving around in a virtual world that is a “to scale” replica of the real world. That is MUCH more interesting and innovative.

But even if I don’t like where they are going, or that they are using SL, the important thing is that they are doing something and forging ahead. I have a massive amount of respect for anyone, private, government, or academic, that is doing that.

And yes, the door (or window, or looking glass) has to work both ways for maximum potential, at least, that’s what Id like to see. They don’t *have* to, but it would be rather cool.

And going back to sustainability, AR has the potential to make monitors generally obsolete, laptops too. That’s a lot of power hungry devices with all sorts of metals and batteries inside.

But, even if the tech was absolutely crazy awesome right this minute, it would take a little while for consumer adoption.

Tish Shute: But AR unleashes the mobile device?

Robert Rice: Yes, AR is going to be built on powerful mobile devices for the near future, eventually embedded comps in clothing and whatnot. But that is a ways off

Entertainment is going to be the first huge driver.

Tish Shute: So people will get used to having a pet virtual dragon on their shoulder first?

Robert Rice: Yes, virtual dragon is way cool, easy tech for games, and can eventually be leveraged into a smart agent which becomes a practical application…agent based contextual search, etc. Yes, entertainment will also drive people to get used to the tech

Tish Shute: Oh thanks for turning me on to gamesalfresco!

Robert Rice: Ive noticed that the good stuff usually gets linked to there. They don’t list my blog, but that’s what I get for staying under the radar and not posting often. But anyway, gamesalfresco is the first place I send people that need a crash course in AR. Great site, great owner.

Tish Shute: So are you in agreement with Thomas Wrobel’s positioning of  Wikitude and AR Geisha doll as being significant milestones for AR?

Robert Rice: Yes,  these are among the first attempts to get away from the novelty of simply rendering a 3D object based on a marker and making it interesting.

Remember, one of the biggest risks that AR has, is being branded as “novelty”, which means “cool for five minutes but ultimately a waste of time.” I think we have a ways to go before something is truly useful, but as 2009 progresses we should start seeing some effort here. I’d guess 2010 before something really useful comes out…at least something practical.

Now, having said that, I should say that I expect entertainment and games to take the lead (as usual), although there are a few companies really trying to leverage AR and video/graphics compositing for marketing (brochures) and location based methods (kiosks, large screen projections, etc.)

Tish Shute: Many people would say SnowCrash (metaverse) is now and Halting State (AR) is ten years from now. But you are seeing a development timeline for some popular AR apps in the next 18 months?

Robert Rice: Anyone that says SnowCrash is -now- is living in a box. Virtual Worlds, Virtual Reality, and immersive tech in general stopped innovating in the mid 90s. I’m continually flabbergasted at the number of people that think that things like Second Life are state-of-the-art or innovative. You might as well try to market a walkman as cutting edge, even though we have IPods out there.

Id like to see someone grab an engine like offset, crytek, hero, or unreal 3, and smack on a fat mmo server infrastructure (eve or big world)…toss in the right tools, and you would see a revolution and renaissance occur at the same time in the virtual world space. All the puzzle pieces are there, just no one is putting them together the right way.

Tish Shute: Why doesn’t anyone do that?

Robert Rice: Its not cheap, people will only fund a copy of something that exists already, people fear change and innovation, etc, The list goes on. The right money goes to the wrong people all the time.

Alternatively stated, there is a lot of “right idea, wrong implementation”

MMORPGs carried the torch and have made huge strides on the technology front, but have devolved in design. More often than not the gameplay emphasizes the single player experience and does nothing to take advantage of the potential of the massively connected internet.

Unless both industries have some serious upheaval or radical new approaches, they will quickly be eclipsed by AR, which will eventually evolve into something hybrid..AR/VR depending on your level of access and hardware.

But yes, I’d say that the next 18 months are going to be very interesting with a lot of money being thrown around, new ventures, and plenty of content/applications. I expect most of this will be centered on single user AR experienced through a mobile device with a screen (iphone, android, etc.). I expect that there will be a significant boost after Vuzix releases some of their wearable *transparent* displays, putting Microvision back into the “has potential but is too quiet” position.

Tish Shute: AR conjurs an image in many people’s minds of dreadful head gear!

Robert Rice: Yes, it is either transparent wearable displays (in eyeglass formfactor) or nothing. HMDs with miniature LCD or OLED displays are good for streaming video, but for the mobile ubiquitous AR we all dream about, it has to be something that looks and feels like a pair of Oakleys.

I should also mention that several different types and modes of AR are going to find themselves being defined and refined over the next two years as we continue to blaze new trails, establish a lexicon (we keep borrowing terms from games, VR, virtual worlds, mmorpgs), and really work out the how as well as the why.

Even though the idea of AR has been around for a long time, the technology is just beginning to emerge, and very few people are even looking far enough ahead to figure out the problems and solutions that the tech creates. Really, who is thinking about how to deal with AR spam right now?

Tish Shute: Do you see any successful networked AR applications emerging in the next 18 months?

Robert Rice: Yes and no.

When I talk about AR, I try to expand the definition a little bit. Usually, when you talk to someone about augmented reality, the first thing that comes to mind is overlaying 3D graphics on a video stream. I think though, that it should more properly be any media that is specific to your location and the context of what you are doing (or want to do)…augmenting or enhancing your specific reality.

In this sense, anything that at least knows who you are (your ID, mobile phone #, etc.), where you are (GPS coord or a specific place like a cafe), and gives you relevant data, information, or media = augmented reality. Sure, you can make things more interactive or immersive, but that is the minimum.

So, in this case, yes, I think there will be networked applications in the next 18 months…mostly things that are enhanced by friends lists (you are here, your friend is over there). These will be *application specific*. My team at Neogence is already going beyond this, building a platform and infrastructure for other applications to be developed on…all networked through the same backbone. Now, in this context (the science fiction AR that we all dream about), no I do not see anyone else trying to leap a generation or two ahead of the industry to build a massively multiuser shared AR space. Expect to see things like multi-user AR games, virtual pets, kiosk marketing, magic book, “gee whiz” presentations (tradeshow booths, entertainment parks, etc.), and so forth.

The big thing I’m worried about is AR becoming the next silicon valley trend…once they realize the potential, an enormous amount of capital will flow to a bunch of startups with half baked ideas, weak business models, ten year old tech, and a lot of overhyped marketing. That is the very thing that will kill this technology as something that has true power and potential to literally change the way we interact with each other, our surroundings, information, and media.

Tish Shute: Do you think AR has value for a project like Pachube that helps us connect dtat from lots of different environments and sensor actuator data?

Robert Rice: I think that AR has value as an interface to this data (essentially data visualization based on information streaming from a sensor or source that is interpreted in some dynamic graphical manner that has meaning). This is one of the “big areas” where ubiquitous augmented reality and wearable computing will really shine. I’ll definitely be keeping an eye on Pachube .

Tish Shute: I can’t help it! I am really interested to hear more about the Vuzix glasses?

Robert Rice: Yeah, everyone is getting hung up on the glasses as the end-all be all and having markers everywhere too.

All the glasses are, is another display device. At the end of the day, it doesnt matter if you are looking at a lcd monitor, a iphone, a head mounted display, or a pair of wicked next generation transparent wearable displays that magically draw directly on your retinas.

The real tricky stuff is what happens on the backend…making it all persistent, massively multiuser, intelligent, interoperable, realistic, etc. etc.

I think that we are within 24 months of the magic wearables (these new ones by vuzix are probably the real first generation attempt at doing it right). They wont be perfect, but I expect they will be functional…and once we have functional, we can start doing the good stuff.

Tish Shute: You mentioned you disappointement with VWs and MMORPGs earlier.  Could you tell me more about that?

Robert Rice: Yeah, there was an evolutionary divergence between virtual worlds and mmorpgs a while back. One stagnated almost completely, and the other leapt ahead in one sense and devolved horribly in the other sense. Neither is where the state of the art should be.  That is a whole other conversation, and probably a second book.

Tish Shute: So making AR persistent, massively multiuser, intelligent, interoperable, realistic, etc. etc. that is where your efforts are going?

Robert Rice: Yes. I fully expect that the hardware is almost ready for it. You can cobble together some amazing things in the lab right now, and I think commercial viability is imminent. The real value (as far as I’m concerned) is in making it mobile, wireless, persistent, and massively multiuser. You could argue that augmented reality will take over where virtual reality failed and become internet 3, internet one being the internet, internet two being the web…

mmorpgs are nothing more than single player games in a multiuser environment these days. I’m more than a bit bitter about it. All the right money went to the wrong people, and the best games we have are barely shadows of what we could have had by now.

Tish Shute: Are there any open source AR platform dev projects?

Robert Rice: open source? hrm, Im sure there are multiple ones out there

if not entirely open source, there are plenty of things to experiment with that are generally free if you aren’t trying to sell something, DART and ARTOOLKIT come to mind as very accessible applications.

Marker based AR is very important right now…it is easy, low tech, understandable, highly customizable, and most importantly, accessible to the average joe. Ultimately though, we need a method of pure tracking…no markers glued to everything on the planet, no “billions of RFIDs” embedded in every square inch of every object on the planet, etc.

Tish Shute: What do you mean by interoperability in AR? And what do you think about the development of standards?

Robert Rice: Ooh, good question.

Ok, so the internet is basically computers communicating with computers, and the web is mostly pages linking to other pages (I’m greatly oversimplifying here). Hold this thought for a minute.

Switch over to MMORPGs. If you want to play in one (or a virtual world), you need to download a client that is specific to that world. One client does not work with another world. There are plenty of efforts to change this, but they are all barking up the wrong tree. The specific uniqueness of each world defeats the need and purpose of true interoperability, unless you completely reinvent the whole thing with a common backbone, features, functionality, etc. The very nature of virtual worlds and mmorpgs rebels against this.You absolutely do not want an avatar from second life running around in world of warcraft (for reasons that should be obvious).

On the other hand, with the web, you can use just about any client (browser) to access nearly any website (some requiring plugins or whatever).

The thing with augmented reality, is how do we go about making this? I’ve seen a few people thinking about this from the wrong perspective. There was a question at the last techcrunch to the Sekai Camera guys (a conceptual AR application for the iphone) where someone on the panel wanted to know how website owners would convert their content for augmented reality. BZZZZZT! That is a fundamental misunderstanding of what AR is, or could be, and it falls into the same trap I see a lot of people doing…and that is looking at AR through the web 2.0 lens or the virtual world lens. It is absolutely fundamentally different at the core…sure there are similarities: it has social networking/media applications and properties, and it has 3D graphics, but it stops there.

Ubiquitous augmented reality will be dramatically different depending on which standards, approaches, and philosophies get the most traction first. Will you walk down the street with your AR glasses and have a pop up every 30 feet asking you if you want to access the AR content on another server? Will you then have to register, subscribe, or whatever?

Or will all AR content be mediated by one sole master control server deep in the bowels of google? What about some other option? Will you need different sets of glasses to access different features and content from multiple sources?

At the end of the day, it should not matter what brand of glasses you are wearing, you should never have to deal with AR server popups to join/subscribe, and so forth.

Interoperability, in the context of what I was saying earlier, is the sense of how to build the infrastructure so all of this is seamless to the end user, but still maintaining the features/functionality necessary for all of what augmented reality promises us…I dont want to see everything in AR space, I want to be able to tune in or filter out some things, and I want to customize the snot out of what I see (perhaps changing metaphors or “holoscapes”), and so on. It all has to work together and simplify the end-user experience or it won’t get anywhere

Tish Shute: So what caused the stagnation of new development and devolution of MMOGs in you opinion?

Robert Rice: yes, look at all the hope and hype for the mmorpgs released in the last 12 months really, what is different or better? Now, what is worse?

I bet any decent mmorpg gamer could give you a list of 2 or 3 things for the first question and 20-30 things for the second.

And, VWs seem to be stuck in a feedback loop

Tish Shute: feedback loop?

Robert Rice: Imagine nailing one of your feet to the ground and then trying to run ’round and ’round and ’round.

Tish Shute: Why do you think this happened to VWs?

Robert Rice: Men in suits and flashy watches.

actually, hang on…..

I saw a video clip the other day from a conference about using various virtual and game technologies for simulations and other real world applications several people were talking about “avatar technology” and how theirs was better than their competitions and what not.

Now, can you tell me what “avatar technology” is? Avatar technology is a red herring. Avatar technology is the same thing as calling a toaster a new “fire technology.”

Robert Rice: The problem is that a lot of people that don’t have a clue about what they are doing are selling the tech to other people that have no clue what they are buying, but they feel like the should for some unknown reason.

That is happening all over the government, academic, and industrial sectors now with a few companies selling virtual worlds (again, mid 90-s tech) as the ultimate solution to all problems.

Anyway, getting back to your question

Once virtual reality started getting some buzz, some people got greedy and jumped into the avatar/virtual world thing and tried making it commercial too soon half of the 3D chat worlds were being jammed into platforms for virtual shopping malls.

Most of the money funding tech R&D started funneling towards VRML, and doing 3D in web pages, etc.

Tish Shute: yes horrible idea trying make web pages 3D IMHO

Robert Rice: The money people got involved too soon, and then the greedy people jumped in and tried patenting everything possible. Take a look at the worlds.com patent for 3D worlds.

They filed it back in 2000 or so and it was awarded in 07 (it shouldnt have been in my opinion) now they are suing everyone they can.

Tish Shute: Will there be patent wars in AR?

Robert Rice: Yes, the AR patent wars will be legendary once people start waking up to the real potential here.

The only solution is for everyone to band together and pre-emptively patent or make public domain every possible patentable concept, technology, or implementation for AR otherwise, you haven’t seen anything yet.

Tish Shute: Is the AR community organized enough to do that yet?

Robert Rice: That depends on how my company fares in the next six months.

Tish Shute: Will you patent or make your tech public domain?

Robert Rice: I plan on patenting the snot out of everything we can possibly think of, and then giving away our content creation tools and SDK stuff for free. The whole goal of what we are trying to build is to empower the end user and facilitate the creation of a wonderful world of augmented reality.

There are some things we will make public domain for sure, on top of that

Tish Shute: So back to my question on networked real time experience. Will we have networked Real time AR experiences in the next 18 months

Robert Rice: It is possible, yes. Other than what we are doing, I am not aware of anyone else taking the same approach we are, but the potential for an “under the radar venture” (much like my own company) is definitely there.

Tish Shute: Will you use cloud computing?

Robert Rice: I think that’s overrated and probably another attempt at the whole “thin client” model that some companies have been pushing for the last 20 years.

It sounds good on paper, but ultimately takes power and control away from the end user.

Tish Shute: cloud computing?

Robert Rice: Yes. You know, we aren’t playing around, We are totally building “THE AR” that everyone keeps dreaming about. None of this cute stuff you see on youtube. Actually, if you want to see the things that have inspired our vision of what we want to build, check out:

* Dream Park by Larry Niven and Steven Barnes
* Rainbows End by Vernor Vinge
* Spook Country by William Gibson
* Halting State by Charles Stross
* The Diamond Age by Neal Stephenson
* Donnerjack by Roger Zelazny and Jane Lindskold
* Otherland by Tad Williams
* Neuromancer by William Gibson
* Idoru by Wiliam Gibson
* Cryptonomicon by Neal Stephenson

and watch the whole anime of Denno Coil (subbed NOT dubbed!).

Tish Shute: So scaling the real time experience won’t be a problem in your project hehe

Cos no sharding allowed in AR right

And if you have lots of API calls?

Robert Rice:: haha, sharding is one of the dumbest things to happen to the VW/MMO industry

It is a solution to a technical problem that was relevant 15 years ago.

Tish Shute: so why did it stick (i know men in suits)

Robert Rice: it stuck because “thats what the other guys did” and the mmo designers are too lazy to reconcile gameplay for PvP and RP gamers

However, there is a curious problem between dealing with “one world” and “anyone can start their own custom AR server”

Tish Shute: Now that is a very interesting problem the one world and own AR server

Robert Rice: It took me a few weeks of not sleeping to figure that one out. It gets back to the interoperability issue

Tish Shute: What did you come up with?

Robert Rice: a solution. Thats all I can say for now on that.

Tish Shute: eeextra seeekrit!

Well I will definitely have to bug you on that.

The problem has produced some creativity in OpenSim with people coming up with hybrids of p2p and oneworld

Robert Rice: As far as virtual worlds are concerned, they need to look at the problem from a different perspective. They are trying to make all virtual worlds interoperable intead of creating a new model for interoperable worlds that new ones will be created to adhere to.

Tish Shute: well some people are. I would say most OpenSim developers see their modular approach doing this.  And you choose to interoperate based on what modules you have activated and then social agreements…

Robert Rice: hrm, thats a start, but that only works on a functional and social level – doesn’t account for content (story, mythos, game rules), unique data (my +3 sword), or the concepts of commerce, inherent value, and intellectual property

Enabling my WoW avatar to run around in SL and vice versa creates more problems than it solves.

Its like two alien races working hard to make sure that their two spaceships can dock but no one is paying any attention to the fact that race A breathes nitrogen and race B breathes sulpher.

Its technically possible, but they are missing the boat on the content side of the problem.

Tish Shute: Yes but don’t you think when a modular open source tech for virtual worlds  becomes pervasive, what will happen is that those interested in a similar genre will increasingly use the module in ways that allows their content to interoperate if they want it too

Robert Rice: everyone has to use the same backend tech, and the front end clients need to adhere to the same standards. Bu I have to admit, I haven’t been paying much attention to the vw space in the last 9 months or so.

Oh I have to run now.  But download and install cooliris. I promise you will be blown away and will start using it to search for images and videos

Its frigging awesome.

Tish Shute: Will do!  Thanks so much great talking to you. I can’t wait for your launch.