Above is an image above  from Total Immersion’s augmented reality experience developed for the “Networked City” exhibition in South Korea, – “a fun scenario created for a u-City’s infrastructure and city management service”

“To the naked eye, the exhibit looks like a bare bones model of a city. But when visitors put on the special AR goggles a whole new world unfolds – as graphics overlaid on the city model.” (Games Alfresco)

“The Networked City,” is a large scale augmented virtuality of a scenario for a networked city. But my guess, reading the     Korea IT Times, is the plan is to move from an augmented virtuality to an augmented reality as Incheon Free Economic Zone  (IFEZ) realizes its vision to become a leading u-City – where reality is turned “inside out” (see Inside Out: Interaction Design for Augmented Reality ).  If you are not familiar with South Korea’s u-Cities, check out this post for a short primer (and note Google Trends search on Augmented Reality shows  South Korea leaving everyone else in the dust).

Ubiquitous computing and augmented reality are like adenine and thymine – a DNA base pair.

A sky view of Incheon Free Economic Zone (from Korean IT Times). For more on the IFEZ vision to become a leading u-City see here.

Korea IT Times writes about the u-city concept:

“Korea began using the term u-City after accepting the concept of ubiquitous computing, a post-desktop model of human-computer interaction created by Mark Weiser, the chief technologist of the Xerox Palo Alto Research Center in California, in 1998. There have been a lot of research in this field since 2002. As a result, many local governments in Korea have applied this concept to various development projects since 2005 based on a practical approach to it.”

The back story to many of my recent posts, including this one, is an understanding of a relationship between ubiquitous computing and augmented reality that emerged, for me, in a February conversation with Adam Greenfield, Towards a Newer Urbanism: Talking Cities, Networks, and Publics with Adam Greenfield.  In cased you missed it, here is the link again because I think it holds up very well considering the rapid developments of recent months.  Also, importantly for this post, it includes a discussion of  moving on from Weiserian visions.

Adam Greenfield’s Speedbird is one of my key sources for understanding “networked urbanism,” and the list he makes of the elements of networked urbanism here (also see the comments) – is my mantra for thinking about the DNA base pair relationship of augmented reality and ubiquitous computing.

Adam Greenfield’s, “summary of what those of us who are thinking, writing and speaking about networked urbanism seem to be seeing” is:

1. From latent to explicit; 2. From browse to search; 3. From held to shared; 4. From expiring to persistent; 5. From deferred to real-time; 6. From passive to interactive; 7. From component to resource; 8. From constant to variable; 9. From wayfinding to wayshowing; 10. From object to service; 11. From vehicle to mobility; 12. From community to social network; 13. From ownership to use; 14. From consumer to constituent.

Augmented Reality – Making Visible the Invisible

The screenshot above is one of  the coolest “making visible the invisible” AR applications. It was developed at Columbia University Graphics and User Interface Lab where Steven Feiner is Director (see the deep list of projects from the lab here).  This app “shows carbon monoxide levels projected over New York City. The height of each ball reflects concentrations of the pollutant.” Credit: Sean White and Steven Feiner  (via Technology Review).

The recent emergence of “magic lens” augmented reality apps for our smart phones – Wikitude, Layar, Acrossair, Sekai Camera, and many others now, have given us a new window into our cities. But we are yet to realize the full potential of the AR/ubicomp base pair that can “make visible the invisible” and give us new opportunities to relate to the invisible data ecosystems of our cities, not merely as a spectator experience,  but as an interactive, in context, real time opportunity to reimagine social relations.

Mark Shepard says in his curatorial statement for, “Toward the Sentient City:” (Much more soon on this very significant exhibit which runs from Sept. 17th to Nov. 7th, 2009.)

“In place of natural weather systems, however, today we find the dataclouds of 21st century urban space increasingly shaping our experience of this city and the choices we make there.”

Augmented Reality, as Joe Lamantia points out, is becoming the great “ambassador of ubiqitous computing.” AR is. “…mak[ing] it possible to experience the new world of ubiquitous computing by reifying the digital layer that permeates our inside-out world,” and we are only just glimpsing the razor thin end of the wedge in this regard.

I am still working on my Gov 2.0 Summit write up  and, amongst other things, I will talk about how an emerging new social contract around open data, here in the US,  will put augmented reality  apps center stage  – “doing stuff that matters.” At Gov 2.0 Expo Showcase Tim O’Reilly tweeted:

Tim O’Reilly Really enjoyed @capttaco (Digital Arch Design) @ #gov20e: “Augmented Reality could be a new public infrastructure” http://bit.ly/18iCx

Also see Tim O’Reilly and Jennifer Pahlka on Forbes.com discuss the The “Web Squared” Era -  “the Web Squared era is an era of augmented reality arriving (like the sensor revolution) stealthily, in more pedestrian clothes than we expected.… …our world will have “information shadows.” Augmented reality amounts to information shadows made visible.”

Again there is back story to how I came to think about Information Shadows in relation to augmented reality.  So in case your missed it the first time, here is the link to a conversation that began in a hallway meeting between Tim O’Reilly, Mike Kuniavsky, ThingM, Usman Haque, Pachube, and Gavin Starks, AMEE, at ETech earlier this year,  “Dematerializing the World, Shadows, Subscriptions and Things as Services: Talking With Mike Kuniavsky at ETech 2009.”

Slide from Augmented City Lab @ Picnic ’09

So What’s Next for Mobile Augmented Reality?

These videos from Daniel Wagner’s team from Graz University of Technology showing Realtime Panorama Mapping and Tracking on Mobile Phones and Creating an Indoor Panorama in Realtime, as Rouli from Games Alfresco points out,  indicate that there is a lot in store for us at ISMAR09.

We may not be so impressed by directory style/”post it” AR anymore, as these applications have become common place so quickly!  But while these early mobile AR apps may be disappointing in relation to some futurist visions of AR – merely AR/ubicomp appetizers,  there are still good implementations of this model coming out (see new comers to the app store Bionic Eye and RobotVision). And Layar, always on the ball, has upped the ante for the new cohort of AR Browsers with Layar 3D.

But as Bruce Sterling notes here:

*In AR, everybody wants to be the platform and the browser, and nobody wants to be the boring old geolocative database. Look how Tim [creator of RobotVision] here, who is like one guy working on his weekends, can boldly fold-in the multi-billion dollar, multi-million user empires of Apple iPhone, Microsoft Bing, Flickr, and Twitter, all under his right thumb

(watch video here)

But if  you looking for something more from AR, you probably won’t have to wait too long.  The two pioneering companies in AR, Total Immersion – founded in 1999, and Metaio – founded in 2003 are both coming out with “mobile augmented reality platforms” in a matter of weeks (see press releases here and here).  And both companies, it seems, will deploy much more sophisticated AR rendering and tracking than we have seen to date.

I approached Bruno Uzzan, founder and CEO of Total Immersion, for an interview as part of my look at the new industry of augmented reality through the eyes of the founding members of the AR Consortium. These consortium members are some of the first commercial augmented reality companies.

The interview below with Bruno began early this summer and then we both went on vacation and it picks up after the announcement of the partnership between Total Immersion and Int13.

The significance of this announcement is that Total Immersion is now positioned to take the augmented reality experiences they have developed for a number of top brands onto multiple mobile platforms with, “Int13′s very clever embedded solution that allows our [Total Immersion's] solutions to work across many [mobile] platforms,” while Int13 gets to extend their reach.

Total Immersion has a 50 person R&D team and their two main focuses have been, firstly getting:

“Augmented Reality to work with as many platforms as possible – PC, Mac, Mobile, Game Consoles, all those are the platforms that we are targeting. We are currently doing lot of work in the R & D team in cross platform compatibility….”

and, secondly:

“Our R&D guys are working on the real world interacting more with the virtual world.  And I have started seeing some results which are pretty much crazy and this will be ready for next year.”

Pandora’s Box – Shared Augmented Realities

Spes or “Hope”; engraving by Sebald Beham, German c1540 (see Wikipedia article on Pandora’s Box)

There are many weaknesses to the mobile smart phone AR experiences we have now, and the lack of near field object recognition (to date), and difficulties with accurate positioning aren’t the only ones.  Note re solving positioning problems in mobile AR, we are yet to see AR  leverage public libraries for analyzing scenes like Flickr’s geo tagged photos, see Aaron Straup Copes’s work on “The Shape of Alpha.” And for more on this my post here.

But, as Joe Lamantia points out:

“One of the weakest aspects of the existing interaction patterns for augmented reality is their reliance on single-person, socially disconnected user experiences.”

In my view, The Pandora’s Box of Augmented Realities is an open, distributed, multiuser augmented reality framework, fully integrated with the internet and world wide web.

As Yochai Benkler has pointed out many times, and argues again in, Capital, Power, and the Next Step in Decentralization, it is “open, collaborative, distributed practices that have been at the core of what made the Internet.”  We have to try to make sure that open, collaborative, distributed practices are at the core of mobile augmented reality.

Can Google Wave be the basis for an Open, Distributed, Multiuser Augmented Reality Framework?

I have been exploring the idea of using Google Wave protocol as the basis for a distributed, multiuser open augmented reality framework with a small group of AR enthusiasts and developers. And I am happy to say the proposal is beginning to get fleshed out a little.  New collaborators are welcome both for “gear heady” input and use case suggestions (but re the latter, you can’t just say everything you see in Denno Coil..!).

This effort started with Thomas Wrobel’s  proposal for an Open AR Framework prototyped on IRC – see here, and click to enlarge the image above of, “Sky Writer: Basic Concept for an Open Multi-source AR Framework.”

But recently we began looking at the Wave Federation Protocol.  And, if you check out this post, and this post, you may get a glimpse of why Google Wave protocol might be a good basis for an open, distributed, AR Framework.  You will notice, if you study what Google Wave has done with the XMPP protocol, that many of  the elements of networked urbanism that Adam Greenfield describes resonate strongly with what is being attempted in Wave.

But enough said for now!  Regardless of the details of implementation,  Google Wave or an AR protocol built from scratch (phew! the latter does seem like a lot of work) -  an open, distributed, multiuser AR framework integrated with the internet and web would explode the potential of AR, creating new possibilities for data flows, mashups ,and shared augmented realities.

And we are excited by Google Wave because, as Thomas puts it:

“The really great thing wave does ….(aside from being an open standard backed by a major player…hopefully leading to thousands of worldwide servers )….is that it allows anyone to create any number of waves, set precisely who can view or edit them, and for them to be able to be updated quickly and continuously (and even simultaneously!) Better yet, changes will (if necessary) propagate to all the other servers sharing that wave. It does all this right now. From my eyes this does a lot of the work of an AR infrastructure already.

I cant see any other protocol actually doing anything like this at the moment, although correct me if I’m wrong, as alternatives are always welcome :)”

Also, Thomas notes, “even the playback system (that is, the ability to playback the changes made to a wave since its creation) …this could give us automatically some of the ideas Jeremy Hight has mentioned in his visionary work here,  and here on “the geo spatial web, interlinked locations and data, immersive augmentation and open source geo augmentation.”

One of the many reasons why an Open, distributed AR Framework would be so cool is it would open up all kinds of possibilities for GeoAR by providing the over-arching standard protocol for communication of updates necessary for the substandards that will facilitate GeoAR.

Also important to note is the  Wave Federation Protocol allows anyone:

“to run wave servers and become wave providers, for themselves, or as services for their users, and to “federate” waves, that is, to share waves with each other and with Google Wave. – “the federation gateway and a federation proxy and is based on open extension to XMPP core [RFC3920] protocol to allow near real-time communication between two wave servers.” See Reuven Cohen’s blog for more here and here, “HTTP is Dead, Long Live the Real Time Cloud.”

Still some people have expressed concern that an AR Framework using Google Wave protocol would give Google disproportionate influence.   Will Google-specific functionality be an issue?  How much stuff is Google specific just because no one else is using it (yet)? And how much is Google specific because it holds no value to anyone else but Google? These are some of the questions that have come up.

You are going to see a variety of suggestions for standards and specs for open AR coming out out in the next few months which as, Robert Rice of the AR Consortium points out is: “a good thing, we need that competition early on to settle down on best case.” Recently,  Mobilizy have offered up an ARML (“an augmented reality mark-up language specification based on the OpenGIS® KML Encoding Standard (OGC KML) with extensions”) for consideration see here.

So it is, perhaps, also important to note, that an Open AR Framework should be neutral/transparent to techniques of  “reality recognition,”  and methodologies of registration/tracking, allowing various ones to work on the system as new techniques evolve, and to support as many evolving standards as possible.

Augmented Reality developers, like Total Immersion and others with powerful rendering/tracking AR software, should be able use an Open AR Framework to exchange the data which their tracking will use. And the tracking/rendering problems they and other researchers have solved are much harder than figuring out data exchange on on a standard infrastructure or protocol!

So I pricked up my ears when I heard Bruno Uzzan, CEO of Total Immersion -  the first and currently the largest augmented reality company, with a 50 person R&D team in France and offices in LA, where Bruno himself is now based, say: “Total Immersion is  only months away from launching shared mobile augmented reality experiences using near field object recognition/tracking across multiple platforms” (for more details read my conversation with Bruno Uzzan below).

I was happy when I asked Bruno about the possibilities for developing an open, distributed, multiuser augmented reality framework fully integrated with the internet and world wide web (possibly using Google Wave protocols), and he replied:

“I think this is feasible. I think that’s doable, that’s just  in my opinion. I mean some people might have another kind of opinion but I think that that’s definitely doable.”

Total Immersion – working with the “symbiosis between augmented reality and brands”

Total Immersion has created many of the best known and most ambitious augmented reality experiences for major brands to date, including Mattel’s new AR toys to be released in conjunction with the James Cameron film Avatar, and AR baseball cards for Topps, video here (or click screenshot above), and the UK’s first augmented reality books.

Bruno founded Total Immersion 10 years ago when he was just 27. And the kind of conviction it took to survive as an augmented reality business in the decade before augmented reality captured the world’s attention is remarkable.

AR’s first steps out into the world after 17 years as predominantly a lab science maybe “wobbly” (what new technology isn’t), and sometimes gloriously kitsch – check out this riotus video of the 3D Interactive Live Show Total Immersion produced in Korea (also see the Total Immersion Augmented Reality Blog for more on the TI’s turn key  Interactive 3D Live Show Solution).

As Lamantia points out here, ” projecting mixed realities into public, common, or social spaces makes them social by default.”

However, the potential for shared location based augmented reality experiences is as yet untapped.  So I see the entry of the most experienced commercial augmented reality company into mobile as pretty interesting.   While  smart phone AR still has significant limitations, and it certainly does differ from some of the futurist dreams of AR (see Mok Oh’s post here on his disappointment in this regard), it is significant that Total Immersion is committing to becoming a leader in mobile AR.

Our smart phones, the powerful networked sensor devices that so many people carry in their pockets, have proved themselves a “good enough for now”  mediating device for early manifestations of the ubiquitous computing and augmented reality base pair.  And now AR and ubicomp is mixed in the  rich, messy soup of everyday life, commerce, business, marketing, art, entertainment, and government, we should get ready to see these technologies grow up fast, and unfold in some surprising ways that lab science didn’t necessarily predict.

And, perhaps, the new dialogue between scientists and entrepreneurs may spur both communities to outdo themselves.

Particularly, as Joe Ludwig notes: “It seems to me that the biggest disconnect between the academics and the entrepreneurs is that they disagree on how far we are from the finish line.”

See the comments’s on Ori Inbar’s post, Augmented Reality Entrepreneurship: Natural Evolution or Intelligent Design?, for  a courteous but spirited discussion on the potential benefits and frictions of the newly expanded AR community of  researchers and  entrepreneurs.

As Blair MacIntyre (see my long conversation with Blair here) notes:

“not all academics and researchers are only interested in the traditional models of impact. Case in point: I wouldn’t be building unpublishable games, nor investing so much time talking to the press, entrepreneurs and VCs if I did not believe strongly in the value of the impact I am having by doing that — and I know others with the same attitude.”

In this vein, check out the Marble Game (video here) developed by Steve Feiner and his team at Columbia U. It’s enabled by Goblin XNA, an open source AR framework built on top of Microsoft’s XNA, which powers XBox live games, Zune games, and some Windows games. For more about Goblin XNA and AR from Columbia U see here.  (Hat tip to Brian Jepson for this link)

While we are still waiting for the kind of sexy AR specs – nothing totally game changing in Gigantico’s AR eyewear rounup (maybe note this Apple patent), that might get wide adoption. But at least researchers are not afraid to explore the possibilities of AR Goggles.

But how far are we now, with or without sexy goggles,  from a fuller expression of the base pair DNA of ubiquitous computing and augmented reality?

We may have a LAN of things before we have an Internet of Things

The picture above is a workshop I attended at Conflux last weekend – Fish ‘n microChips, with Natalie Jeremijenko. We are at the site of the Amphibious Architecture project (a commissioned work for Toward the Sentient City) and “a collaborative project with xClinic, The Living and other intelligent creatures.”

We are probably as far off some grand futurist visions of ubiquitious computing as we are some of the futurist visions of augmented reality. But as it turns out that may not be a bad thing! Recently, @mikekuniavsky noted in a tweet:

“Another argument for the LAN of Things before the Internet of Things: http://tinyurl.com/lgp9uq”

Bert Moore, in the article Mike linked to points out, the grand vision of an “internet of things” with everything connected to everything  can “distract people from thinking about the benefits of RFID in smaller, more easily implemented and cost-justified applications.”  The same argument I think applies to sensor networks and augmented reality.

In New York City, a series of commissioned works for the Architectural League of New York’s exhibit, “Toward the Sentient City” are giving us the opportunity to dip our toes into the ocean of a “networked urbanism.”   For only a small budget, two of the five commissioned works, Amphibeous Architecture and Natural Fuse demonstrate how sensor networks can allow us to explore new kinds of communities – connecting people to environments in interesting ways to create new forms of social agency.

“Amphibeous Architecture” -  from The Living Architecture Lab at Columbia University Graduate School of Architecture, Planning and Preservation (Directors David Benjamin and Soo-in Yang) and Natalie Jeremijenko, Environmental Health Clinic at New York University, uses a skillfully built (electronics and water are notoriously hard to mix) array of partially submerged sensors to pierce the blinding, reflective surfaces of the rivers  surrounding Manhattan and to create a new two way relationship with the ecosystem below – the water, our neighbors the fish and even a beaver that lives in the water surrounding Manhattan.

Image from Toward the Sentient City

In a similar spirit, “Natural Fuse” – Usman Haque, creative director, Nitipak ‘Dot’ Samsen, designer, Ai Hasegawa, designer, Cesar Harada, designer, Barbara Jasinowicz, producer, creates a network of people and electronically assisted plants to explore what it takes to work together on energy consumption and to experience the consequences of “selfish” and “unselfish” behavior interactively before it is too late to modify our actions.

The “Greedy Switch“ from Natural Fuse on the left. On the right “The System” – click to enlarge.

Much more to come in another post on these works, and “Toward the Sentient City.”  Also an update on how Pachube – an important part of both these projects and a very important contribution to ubiquitous computing because it creates the opportunity to connect environments and create mashups from diverse sensor data feeds – has matured since my interview with Pachube founder, Usman Haque, “Pachube, Patching the Planet,” in January this year.

In the picture above Natalie Jeremijenko, and Jonathan Laventhol give the Amphibious Architecture sensor array a last look over, as it will soon be lowered into the East River. Jonathan is on a busman’s holiday to help out at the pre launch of Amphibious Architecture, nr Manhattan Bridge, NYC.

I was very happy to get  a chance to talk to Jonathan Laventhol - more on our conversation in another post. Jonathan Laventhol is CTO of Imagination – one of the world’s leading design, events, and branding agencies. We talked about the importance of  Pachube, which Jonathan called the “The Facebook of Data,”  and  how the symbiosis between brands and augmented reality, and healthcare applications, would  be key to augmented reality emerging into the mainstream.

Natalie Jeremijenko’s workshop at Conflux on the social negotiation of technology and how “everyware” can give us the chance to experience new forms of agency and connection was a totally inspiring.  And I will cover this too in another post.  I have so much awesome stuff  to write about at the moment!

None of the projects in, “Toward the Sentient City,” included a mobile augmented reality, or “magic lens” component, but they all pointed to why “enchanted windows into our newly inside-out reality” are going to be so important. And why the DNA base pair of ubicomp and augmented reality can really do stuff that matters.

Shangri- La – “Transfigured City”

Screenshot from Shangri-La episode Transfigured City

In my AR Consortium founder member interview series, I have found that, understandably, the visionary founders of these first augmented reality companies are a little reticent about sharing their full vision.  They are basically on stealth mode in this regard.  So as you will not, from my interview with Total Immersion founder and CEO, Bruno Uzzan, get a fully drawn scenario of his vision for a next generation of shared augmented reality experiences, here’s a really interesting anime episode from the anime Shangri La called, Transfigured City, to mull over instead.

As you can tell from this rather long and circuitous intro to my my conversation with Bruno Uzzan, I  have been investigating shared augmented realities pretty intensively recently. And Mike Kuniavsky pointed me to Shangri-La, and Transfigured City, in a conversation with Mark Shepard, after Mark’s presentation at Conflux, Sentient City Survival Kit.

Mike Kuniavsky with Tod E. Kurt is founder of ThingM, a ubiquitous computing device studio. Also Mike Kuniavsky researches, designs and writes about people’s experiences at the intersection of technology and everyday life – see Mikes blog Orange Cone.  And I interviewed Mike at Etech- see here.

In Transfigured City, the “Metal Age” group has to figure out how to share and communicate in a city transfigured by augmented realities/virtualities, where no-one sees the same place in the same way.  Only one character can figure out from her previous experience of the city the relationship between the transfigured city and how it used to be.

The conversation I had with Mike Kuniavsky on The Transfigured City continued at a picnic in Washington Square Park the next day with Elizabeth Goodman, who I met at Etech when she gave a brilliant presentation, Designing for Urban Green Space.  We covered so many areas at the picnic related to ubiquitous computing and augmented realities that this conversation probably deserves a post of its own (my writing to do list is growing longer!).

The Plot Synopsis for Shangri La:

“In the mid-21st century, the international committee decided to forcefully reduce CO2 emission levels to mitigate the global warming crisis. As a result, the economic market was transferred mainly into the trade of carbon. A great earthquake destroys much of Japan, yet the carbon tax placed on the country is not lifted, so Tokyo is turned into the world’s largest “jungle-polis” that absorbs carbon dioxide. Project Atlas is commenced to plan the rebuilding of Tokyo and oversee the government organization, which the Metal Age group opposes due to its oppressive nature. However, Atlas is only built with enough room for 3,500,000 people and most people are not allowed to migrate into the city. The disparity between the elite within Atlas and the refugees living in the jungles outside of its walls set up the background of the story.”

Talking With Bruno Uzzan

Tish Shute: We won’t have fully opened the Pandora’s Box of Augmented Realities until we have ubiquitous, shared augmented realities, will we?

Bruno Uzzan: Yes. The most important for augmented reality is the experience we want to share. Now we are working on the cell phone, we can potentially do some marketing components that we already have developed now on cell phone. Done. It’s working.

But the most interesting part of it is how these new components [cell phone AR] will be used for marketing campaigns by brands. And we are also pretty much well positioned to transform some of the AR that we currently have working on Mac and PC and to transform these to applications working on mobile devices.

Tish Shute: We haven’t really experienced yet what it means to actually share mobile AR experiences?

Bruno Uzzan: It’s hard — we did a Facebook app. It’s a first try, it has a way to go.  But to go more and more into social, is the way forward for us – to share and expand AR experiences. But yes, I mean what you’re seeing is how two people on two different applications can share that same expanse.  For sure we are going in that direction. We are currently working on those kind of solutions. How people can share and experience together at the same time. That’s how we start creating excitement in augmented reality, and it’s coming up.

It’s a new market and there’s so much more in store for augmented reality. You know, some people are telling me, don’t you believe that augmented reality is a gimmick? It will be a trend for a few weeks or a few months and then gone? I say, you’re kidding me. This is only the beginning. I mean I can assure you that the applications that are on the market today are one percent of what we will have five years from now.

Tish Shute: I agree.

Bruno Uzzan: And I’m sure that augmented reality will be a part of a lot of components that we are currently using today – GPS, web browser, glasses, I mean there are so many applications that will come up shortly. This is only the beginning. I’m completely convinced that augmented reality will be in three years from now what virtual reality is today, which is a billion dollar market.  I know that it’s not just a gimmick of a few weeks or a few months, because so many brands are jumping into it, spending money, exploring solutions.  I know that it’s not just short term -what they are willing to do and we are willing to do, but also middle and long term. And that’s what makes this adventure pretty much unique and what makes creating a cutting edge technology, very, very much exciting for us.

Tish Shute: First could you explain more to me about your partnership with Int13. I am not sure I understand what is in the arrangement from Total Immersion’s POV. I mean what happens re your own mobile software development? Haven’t you only been licensed the Int13 SDK for a limited period of time and have limited access to all it’s power? Stephane from Int13 said to Ori on Games Alfresco, here, “we have licensed the SDK4 for two years,” and then Ori asks, “but you have basically kept the power to yourselves, right?” So if they are the only ones that can enhance it and develop the software, where will  TI be in two years in mobile if you haven’t really had the chance to develop your own software .

Bruno Uzzan: Actually it’s a real win-win situation. Int13 is a very small company and they have so many requests they can’t possibly fulfill them all. So  this is a way for both of us to be, as quickly as possible, the first mobile provider for all the requests we have. Also they give us exclusivity so nobody else can use INT13 SDK for such applications.  I think that it is a good partnership,

And concerning our own mobile application… First of all we have currently some mobile applications working. But with Int13 we have a mobile solution that can work on many different devices. That’s a fact and that’s working. And, believe me you will hear from us a lot more about this soon. We are fully independent on our mobile development. The reason we closed the partnership with Int 13 is  to be able to deploy mobile in a broad way.

I mean you know that the difficulty with AR mobile is that each separate device needs some customization. Working on the iPhone is different from working on the Nokia, different from working on the Palm; it’s different from working on the Samsung. Each of them have their own operating system inside and so we were interested in Int13′s very clever embedded solution that allows our solutions to work across many platforms.

The reason we are working with Int13 is that we are able to work on so many mobile devices, thanks to Int13. And in the mobile AR race that we are currently in, the next two years will be extremely important to us…

Tish Shute: OK, that definitely clarifies it a lot. So Int13 has done an embedded solution to allow TI developed AR solutions to work easily across many devices?

Bruno Uzzan: Yes  they have kind of an embedded solution, a way to address extremely quickly new cell phone… But, currently on our side, we are in discussions with a mobile company… and that only refers to some very specific mobile devices.  And what they have is also a way to embed deeper our technology into mobile, so that we can have quicker… applications that work on a large number of cell phones.

Tish Shute: So, basically it means you don’t have to go through some complicated negotiations with each of the cell phone companies, is what you are saying?

Bruno Uzzan: Not only negotiations, but also hard development. You know? Working on the Windows mobile is completely different from working on the Palm OS. You know, that’s different! Its a big work, to have a mobile application working on many other devices. So, INt13,  provides us a way for us to save some time and some development cost too.

Tish Shute: And Int13 doesn’t have powerful AR development tools like D’fusion right?

Bruno Uzzan: Right! That’s right. That’s why we say it’s a true win-win solution. They can benefit from our work too. And we can benefit from their work, in order to deploy quicker and faster mobile solutions.

Tish Shute: Now, the second thing is… there is a lot of debate and disagreement about how far mobile augmented reality is from delivering something more that the “post it” approach that has been much publicized in recent months, via all the AR browser apps.

But from my understanding from the conversation we had earlier this summer (see below), Total Immersion is targeting a much higher level of mobile augmented reality than we’ve seen to date?

Bruno: Yes the browser apps we have seen are a kind of augmented reality, but not exactly the way we see it. Let me explain you why. With this kind of application it’s true that you can overlay 3D-information and video. That’s a fact. So, in a sense, that’s augmented reality. But the way that they are working on the position of the 3D on that video is that they are using compass and GPS-information.. so it means that this AR solution will work only on some building and some physical objects that are FIXED. In a fixed and known position.

So you want to go to a theater?

The theater is here, for sure it will not move, so you know the position of the theater, and that’s a fact that you can superimpose an object on the theater. That’s what can be done currently. What we are achieving and what we are doing on mobile is more than that. We want to be able to port our solution with trading cards, with brands, into a smart phone.

I’m assuming that you want a can, a drink can, to be able to trigger an experience. The only way you can do it is to be able to understand what the can, it is. And the current solutions that are out there can’t do that, it’s impossible.

Tish Shute: Right, yes. There’s no near-field object at all in these early browser apps.

Bruno Uzzan: And the solution we have is that we can recognize a can and then — in a very, very precise way and that activates geo-location, so we can superimpose 3D. I mean in that case, it opens up all the applications that we currently have, so they could work on mobile.

Tish Shute: So for example, if you’re working with a soft drink company, people can trigger that experience wherever they see that can?

Bruno Uzzan: Correct.

Tish Shute: Yes. Yes, I assumed that was what you’re doing

Bruno Uzzan: We believe — and maybe that’s not the case, but we believe that our marker-less tracking technology is pretty much unique on the mobile devices.

I haven’t seen yet, from anyone, a full augmented reality mobile solution working.

I really see AR being part of the Web 3.0 next generation. I mean the vision I have is that, you know — today, when you want to have information, you go on a website and then you find your information. AR — and the future is that I think it will be part of the opposite. You want to have information about a product, you just show it to your computer and the information will automatically pop up. I see here a new way to market some key messages, a new way to get information is that some physical product by themselves could be a way to get information, and you don’t have to search anymore for them, it’s coming out to you.

AR is definitely for me, one of these components. Another thing that AR is a solution, another thing that AR itself will create these kind of results in how information is being displayed. But I’m seeing  here a way that could be part of a new way to have access to information. And that’s part of the vision I have. Whatever, if it is through mobile phone or web or PC, Mac, whatever, I really believe that now this kind of new generation of receiving information will come shortly and could be a kind of a new — could be part of the new 3.0 generation of the web.

Tish Shute: My friend Gene Becker did an interesting post recently on some of the current limitations of mobile AR where he pointed out the problem of:

“Simplistic, non-standard data formats – POIs, the geo-annotated data that many of these apps display, are mostly very simple one-dimensional points of lat/long coordinates, plus a few bytes of metadata. Despite their simplicity there has been no real standardization of POI formats; so far, data providers and AR app developers are only giving lip service to open interoperability. Furthermore, they are not looking ahead to future capabilities that will require more sophisticated data representations. At the same time, there is a large community of GIS, mapping and Geoweb experts who have defined open formats such as GeoRSS, GeoJSON and KML that may be suitable for mobile AR use and standardization.”

Bruno Uzzan: That’s interesting. I mean — I know exactly what his is referring to. He is mainly referring to a localization and how you can have a quick, accurate localization.  If you look at current solutions, and you look at this 3-D superimposing on the video, the 3-D is shaking a lot. I don’t know if you see that in some of these early efforts.

It’s hard to use because the 3-D, you know, is  part of the magic of augmented reality, that is when the 3-D is being inserted in a very easy way and smooth way in your solution. Here, when you see this overlay, 2-D or 3-D overlaid on the video, it’s shaking a lot. One reason for this is that the GPS compass is not accurate enough to coordinate the perfect location of the user. And here, what Gene says is interesting. I think we are addressing this localization issue in a pretty smart way.

But to be frank with you, I don’t believe mobile augmented reality in the extremely short term — I’m talking about three weeks, one, two months is mature enough for good AR applications.  It will be shortly.  But for now it is more proof of concept than a true and easy application to use.

But we are starting to see a lot of new application coming out, but I really believe that marketing and entertainment are the two key markets for AR right now.

I’ve been working ten years in augmented reality. And, eight years ago, when I was talking about augmented reality, I was E.T., you know? Nobody understood what I said, and I thought it was crazy. And now, today, yes it’s completely different.

Tish Shute: The Pandora’s Box of Augmented Realities, in my view, is an open, universal and standard, distributed, multiuser, augmented reality framework fully integrated with the internet and world wide web. I have been looking into Google Wave protocols as a basis for this would you be interested in this? Do you think it is feasable?

Bruno Uzzan: I think this is feasible. I think that’s doable, that’s just  in my opinion. I mean some people might have another kind of opinion but I think that that’s definitely doable.

Tish Shute: Yes I suppose an open AR Framework involves cooperation and collaboration, it is more about business and politics than technological problems.

Bruno Uzzan: Yes!  Actually the Web is politics. Business is politics.

Tish Shute: I would be interested if anyone in your R&D team would be interested in looking at some of the ideas that are emerging in our little discussion of Google Wave and an Open AR Framework  to offer feedback. it is an interesting time now to input on the Wave Federation Protocol docs because nothing is set it stone right now.

Bruno Uzzan: Just shoot me an email, I’ll try to put you in touch with the right person and, and a team member that can input on this.

Tish Shute: For mobile augmented reality the best thing we’ve got now is the phone, right?

Bruno Uzzan: Right.

Tish Shute: And the only way we can use the phone is by holding it up, right?  Isn’t this a bit of an an obstacle as you introduce better object recognition and tracking?  People are going to have to stop moving to use their phone. What do you feel about that experience? Isn’t AR eyewear and essential part of a tightly registered AR experience?

Bruno Uzzan: We don’t do hardware and we don’t have the current solution for eyewear that would do all we need for a good mobile AR experience, so I guess we don’t have the current answer for that.  But we are beginning to see the next generation of this — of these glasses.

Tish Shute: But you’re happy enough with the mobile experience of augmented reality on smart phones that you’re investing in this next generation of software for this.

Bruno Uzzan: Yes, I know. We know that some application will not work on the iPhone. And yes, whatever you do, you still need to hold the iPhone, so it means that you can’t play with your hands anymore. So we know that partially, some AR solutions  we have on other platforms will lose the magical effectivities on just the iPhone.

But I’m starting to see on the market some glasses that could perhaps be not too expensive — that’s a challenge!  And easy to use — that’s another big challenge. And, that could fit on anybody’s faces and head — there’s another big challenge. So yes, I’m starting to see that, but so far AR glasses are only applicable for some very, very specific application, like design or theme park or, you know, some specific location where it makes sense to move forward with glasses.

I don’t believe that kids will use glasses for — in our toys and for games in the next months or maybe othe next one or two years. But maybe something will come out shortly and that could be a big breakthrough, and enable us to think another way. But  from what we have seen so far and from what we know in this hardware market, I don’t believe that currently there is a workable solution.

Note: The following section of the interview took place earlier in the Summer.

Tish Shute: You are the first commercial AR company  – you started in 1999 right?

Bruno Uzzan: Yes you are right. We started the extremely early in this augmented reality market. We were the first company worldwide to start doing augmented reality and to start promoting augmented reality. So it’s true, we are pretty old players although the market has been getting bigger and bigger for the last year and a half. So for a long time we were only in the market, and the market was not really there.

But for the past 8 months, the company has been growing really fast.

Tish Shute: Yes I’m sure. Congratulations for hanging in there long enough to get the pay off!

Bruno Uzzan: You know, my background is Financial. So I have been driving the company for many years in a very cash efficient way. So we have been waiting for the markets to reach maturity before starting make some investments. So that’s the reason we are still here, and that’s the reason I think we managed pretty smartly the cash that we raised for the company.

Tish Shute: Yes there is a saying that when a market takes off you can tell a pioneers because they are the ones with the arrows in their backs. But I am glad you are dodging the arrows!

Bruno Uzzan: You know, I’ve always driven the company with revenue. And because revenue was not there at the beginning I was extremely cautious about the cash. So now that the company is getting some revenue, for sure we are making more and more investments, and taking advantage of our situation as a worldwide leader of augmented reality.

This situation is not easy as it appears today but it’s now getting better, as you can see, AR, Augmented Reality, has very good momentum and we are benefiting a lot from all this momentum for augmented reality right now.

Tish Shute: You’ve been very involved in researching developing augmented reality tools. Are you still as active in the research area, or are you too busy keeping up with work for hire now, to be working on research and building new technology for Augmented Reality?

Bruno Uzzan: Both. First of all, we are part of lot of projects either directly with clients like Mattel or with some partners that are using our technology to promote and develop other AR projects. From what we he have seen, many, many, many, projects augmented projects have been done currently with our solutions.

To continue with your previous question. So we are being perceived as this leader in that space, and we  have some pretty heavy demand for our services. But we are coming up with new technology, of course, still connected to Augmented Reality.  But, our R & D is working in two different directions, which of course also bind together.

The first one is platform developments. So we want Augmented Reality to work with as many platforms as possible – PC, Mac, Mobile, Game Consoles, all those are the platforms that we are targeting. We are currently doing lot of work in the R & D team in cross platform compatibility.

Tish Shute: Robert Rice said recently, “markers and webcams equal Photoshop page curls…”

Bruno Uzzan: Yes. There are so many concerns with markers. The quality is extremely bad. As soon as you hide a part of the marker, a slight part of the marker, you’re dead. You can’t track any more of the object. So compared to our solution where I want to say play with cards or where you are going to play with a Mattel toy, even if you hide a part of the toy, it’s still working.

Tish Shute: But you haven’t offered the public an SDK to your engine right? Basically the way people get access to your tools is working in a partnership with Total Immersion right?

Bruno Uzzan: Correct.

Tish Shute: Do you think in the future you might open your SDK? Are you considering that?

Bruno Uzzan: Yes, it would be interesting.

Tish Shute: So that is something we can see coming soon?

Bruno Uzzan: Maybe, because it’s true that Total Immersion is starting to be mature enough for these kind of tools. The only thing is that we have to respect good timing for that.  It’s a big decision. You know what I mean?  It is a big, big decision. We would then compete with others using our technology.

Tish Shute: Oh I know, it is a big decision when you have so much skin in the game! But it would be nice to have your SDK being THE platform for AR, wouldn’t it?

Bruno Uzzan: It is a really big decision that we can’t just take like that, you know.  There are a lot of friends who told me you have to be extremely careful about timing. This timing is pretty much connected to the maturity of the market. For sure, we see the market being more and more mature. But, there are a lot of low hanging fruits we still want to address

To get the best value possible for all the publicity we have and all the clients we have now.

Tish Shute: Yes, I know. You’ve been in this game so long. Now, there is an interesting question here though about tools and platforms because you know, A.R., augmented reality has already expanded  beyond its kind of original purist definition. And when I talk to people  about augmented reality, there are actually lot of different ideas and priorities of where the tools should go right now. You know, obviously we have these kind of browser-like applications, but these browser like applications are not dealing with recognizing near field objects yet.  What are your priorities for tool development and what are your priorities for AR development in the future? What areas are you going to focus on? Oh dear that is a rambling question!

Bruno Uzzan: [laughter]  So, one of our first priorities is we need to create our software with one development, one installer, one software that can be spread on different platforms. The same application, the same software can be used either on a PC, Mac, phone or console. So that’s a lot of work, because that means that our platform has to address many many different devices and that’s a big priority for us because we received this request from our clients. We want to be able to use one application on many different platforms and devices. So, that’s the first one.

And the second one is to add more and more interactivity between the real and the virtual world. So, we are working on some improvements to add some real components that will interact with virtual, and that also part of our big strategy and direction and these two worlds can more and more be bridged together, linked together so they can interact  one with the other.

Our R&D guys are working on the real world interacting more with the virtual world.  And I have started seeing some results which are pretty much crazy and this will be ready for next year.

There are so many different directions for interaction between the real world and virtual world to develop.  I’m sure ten years from now you’re going to have AR applications everywhere.  Its not just temporary fashion stuff or a gimmick for few months. I mean we are getting there, its getting stronger and stronger and we are getting a good adoption rate from our consumers. They like it, they test it, they play with it and brands wants more, people want more and its getting bigger and bigger.

Tish Shute: Yea and I totally agree, its not a gimmick because the interaction between “virtual” and “real” enhances the magic of both. Another question about you RandD operation. Is your R&D still in France or have you moved totally out to LA.

Bruno Uzzan: We are 50 people in France and I started this LA office two years ago and I moved permanently two years to LA. So I’m now permanently located in the US to take care of the US office, knowing that revenues are really getting bigger and bigger in the US. So it means that we are getting a lot of traction, working with large company and now I’m currently located in the US.

Tish Shute: My sister lives in Paris. Could I visit your R&D lab at some point? I’d love to visit!

Bruno Uzzan: Yeah sure sure sure. I mean if you want to go. You won’t have access to all the research. But if you want to go out and meet all the team please do.

Tish Shute: I’d love to.

Bruno Uzzan: No problem. Shoot me an Email you and I will introduce you to Eric Gehl, COO, he is the COO of the French team. And he can definitely take care of that.

Tish Shute: That would be fun. Thank you!

Recently, AR browser applications have really caught the imagination of the web community, eg., Layar and Wikitude?  Where do you think the most important market for AR is at the moment, entertainment,  green tech, business, education?

Bruno Uzzan: I think that all that you mention will be important. The first one that did grab my attention is entertainment particularly dual marketing, because they always searching for new ways to interact with players or the consumers.  But it’s just the tip of the iceberg, you know, I mean medical applications could be huge using augmented reality. Education, and edutainment is definitely using more and more augmented reality components.  And I will just be submitting with big companies – that are considering using augmentation for education. Museums are very important too. Also augmentation as a kind of free sales tool, you know there are so many applications, design, architecture – so many directions that it’s hard to say today which one will take the lead.

But I do believe that on the short term the ones that are really really moving fast are the entertainment business and the digital marketing business.

Tish Shute: What do you think are the biggest shortcomings with current augmented reality and what are the obstacles that no one has solved yet?

Bruno Uzzan: I think the cell phone is not fully ready for augmented reality – a lot of people are working on that but there are still a lot of constraints to get the augmented reality working on a cell phone and I think that from what I heard a lot of manufacturers and a lot of companies are working from direction that are going to help us a lot to develop some great cell phone applications.

And I think that’s one of the biggest part of the game. All the applications that you see on cell phones so far are just gimmicks – the next big key is how to transform some gimmick cell phone application to a real, industrial, robust application that’s going to work on a cell phone. So I think that’s a big challenge for this year.

Most of what we see now is just matching and overlaying some 2d components in a video. This is not what I call AR.  You’re far away – with this kind of application, you are far away from doing the registration that we need to do – you can’t do it. So here’s the challenge: “how can you get a Topps is an application working on cell phone. That’s the big challenge  How we can make that work!” You can’t today get a real AR Topps application working on cell phone because there’s no cell phone  that’s actually ready. But we are working on it and the first one that can make that work, it’s going to be huge.

When you are working with good AR components you need a lot of CPU and GPU programs. So today new cell phone have started to be more and more ready for augmented reality but you need a really good cell phone to make it work. You can’t choose an old cell phone to make it work because you have some recognition, you have some tracking, you have some rendering, so you can’t choose a Nokia cell phone two years old to make that work. For sure the newest iPhone is the one that can make it work, but that’s it for now. There is a lot of research – from large cell phone companies – to get more CPU and GPU into their cell phone.  But so far we are also waiting for these devices to be released to consumers.

Tish Shute: And the current economic climate has put a damper on MIDs hasn’t it. But who can tell? It depends what price points some new MID came out at right?

Bruno Uzzan: Correct.

Tish Shute: Yes,I agree. But basically what’s interesting, the interesting thing is, the iPhone can deliver so much of what is necessary and even if Apple hasn’t given access to the full power of the iphone to AR developers yet, there is really no going back now – the mobile augmented reality cat is out of the bag!

Bruno Uzzan: You’re right, you’re fully right.

Sekai Camera has a slick new demo video out that is already causing a stir in the Japanese press (see Beyond the Beyond).  This video shows a ton of stuff going on! (A friend who lives in Tokyo pointed out to me that, in Japan, people are used to working with “busier” mobile UIs.)

Takahito Iguchi, founder of Tonchidot the company that has created Sekai Camera, is ultra cool.  Coming to augmented reality from the worlds of anime and manga culture, he is  already a successful entrepreneur with excellent sartorial taste (as Bruce Sterling notes).  Before turning his attention to AR, Iguchi-san was founder of Digitao, where he pioneered a blogging + social networking service “chibikki (Little Diary).” Also Iguchi-san spent time at JUST Systems and Scitron & Art, where he developed innovative multimedia platforms and web services.

But Takahito Iguchi doesn’t give interviews in English.  So recently, as part of my series of interviews with members of the AR Consortium, I found myself talking to the brilliant,  CFO of Tonchidot, Ken Inoue. Inoue-san’s specialties include the Japanese mobile market, start-up finance, alliances, new business development, and international expansion.

And while, perhaps, I would have liked to learn more about how cool Japanese sub-cultures are informing the future of AR, with every business analyst under the sun opining on the future of this young industry, it is good to hear directly from an augmented reality CFO who is actually shaping business development on the ground.  And Tonchidot is one of  AR’s most interesting start ups.

With Tonchidot, I think we are beginning to taste a magic brew as augmented reality, long nurtured only in lab scientist cultures, meets business savvy and fearless creativity.

Bruce Sterling posted the video below, noting:

“Tonchidot tearin’ it up at the department store. Check out that exceedingly weird and/or clever AR-iPhone pistol grip device that kicks in around 2:20.”

The AR Commons

In the interview below, Ken Inoue also describes an important organization that Tonchidot has helped create – the AR Commons.

Ken Inoue: We feel that public data, such as landmarks, government facilities, and public transport should be shared. We see an AR world where people can readily and easily access information by just seeing – quick, easy, and efficient.  And because of this ease and intuitiveness, children, the elderly and handicapped will surely benefit.  AR could help create a safer society.  Warnings, alerts, and safety information could save lives and avoid disasters.  These are what we, and AR Commons would like to tackle in the not so distant future.

An AR Commons is something we should all be thinking about. “Augmented reality could be a new public infrastructure,” as Tim O’Reilly noted in Twitter. I will discuss this more in my upcoming post on the recent Gov 2.0 Summit which was was an extraordinary event – an historic manifestation of the current wave of transformation in the nature of Government that Carl Malamud described in his address, “By The People,” available as video, audio and text here.  Carl Malamud received a standing ovation at the Summit.

Malamud pointed out:

“We are now witnessing a third wave of change – an Internet wave – where the underpinnings and machinery of government are used not only by bureaucrats and civil servants, but by the people.”

Talking with Ken Inoue, CFO, Tonchidot

Tish Shute: There has been some skepticism lately that augmented reality experiences will live up to the recent hype (see this post for example). But Tonchidot has a reputation for creativity, as you pointed out, “we are not “AR lab scientists” – we are from the worlds of multimedia, visual arts, publishing, lovers of manga and anime and Japanese sub-culture…. ” What is Tonchidot’s approach to designing AR experiences that can deliver wonder, curiosity and discovery – the emotions of AR, despite the limitations of GPS+compass implementations of mobile AR?

Ken Inoue: We have been facing skepticism ever since we started!  It doesn’t really bother us, it never has.  As for the opposite, the recent hype, well, we will have to live with that too.  We are aware of the hype cycle, the obstacles that lie ahead.  We are not rejoicing, and we will be prepared.  By the way, I didn’t think the said article was skeptical at all – in fact, I took it as great advice.

Tish Shute: Most augmented reality experiences are at the moment about one person experiencing multiple streams of content, we haven’t seen any multiuser realtime interaction in augmented reality yet, for example, people teaming up to accomplish some goal?  What do you think will be the most exciting aspects of shared augmented reality experiences? And, we are yet to see a really mobile augmented reality game get a mass audience.  Pong was a landmark game for the PC.  It really excited people because there was a “Wow! my physical action is changing what is seen?”  What would be an equivalent Wow! experience for augmented reality?

Ken Inoue: We wish we had an answer to that! We are talking to many game developers, and everyone has different ideas…  we want to test them all! We are striving to be a social application, and we are thinking hard.  But often times, users find new uses and come up with really unexpected, but ingenious ideas… that’s the nature of social experiences, I guess.

Tish Shute: It is a year since you demoed at TC50.  What have been the most exciting developments in augmented reality this year and what have been the biggest disappointments?

Ken Inoue: We’re definitely excited about what other start-ups in the field are doing across the ocean.  We get a lot of stimulation, and we see it as something close to a great sporting rivalry, but only, we aren’t that great yet….  Our disappointment was that we weren’t able to release our app this summer….

Tish Shute: I know you can’t give too many details about your upcoming iphone launch because you are in “stealth mode” and because of Apple’s NDA. But I will start with a general question: “Do you think that Apple is going down the right path with what they are or aren’t making available to developers?”

Ken Inoue: Looking back at Apple’s short history in iPhone and AppStore, they’ve slowly but steadily headed in the path of more openness. And what with the FCC making an inquiry to Apple about the rejected Google Voice application, they’re forced to be more friendly and open to developers, whether they want to or not…

Tish Shute: How is Tonchidot going to differentiate itself in an exploding field of new augmented reality companies?

Ken Inoue: Well, I feel that the market for augmented reality is still in such a nascent stage, that the priority for many of us is cooperation, rather than cut-throat competition.  That’s the rationale for the AR Consortium that was founded by Robert Rice and others very recently, and something that we completely subscribe to.  In Japan, Tonchidot is the central proponent in AR Commons, an organization which has already started building a social database for AR.

Tish Shute: What do you think of the augmented reality applications released recently?

Ken Inoue: There are now so many cool AR apps out there – we’d like to think that our presentation at TC50 back in September 2008 stimulated fellow developers just a little bit.  Many AR applications and services seem to capture the benefits of AR in some way or another very well.  I think maybe the difference between our service and what some others are doing, is that we are initially focused on UGC (user generated content) – not on business applications and tools.  However, it’s just a matter of prioritization, I think – it seems we all share the same dream!

Tish Shute: Yes I like the way you have taken these the concepts “world camera”  and “air tagging” and focused on the social aspects – social tagging.  Wikitude now has a way for users to create tags – wikitude.me which is a big step forward too I think.

Ken Inoue: Yes, indeed!  It seems they have done a great job.  Their success, and the success of everyone else helps us too, since it generates media attention, and also ideas for how it can be applied to the real world and real businesses.

Tish Shute: There is a a growing development of AR browser like experiences, Wikitude, Layar, and Sekai Camera but they are not true browser experiences (in the sense that we experience web browsers) as they don’t share AR data across browsers. How can we move towards a situation of sharing augmented reality data? What are the obstacles to sharing AR data across browsers now?  I guess these obstacles are business obstacles mainly, not technical obstacles. But what do you think?

Ken Inoue: Because AR is in many ways location dependent, geographic coverage always will be a challenge for anyone.  This means that collaboration makes sense. I don’t think there are many technical obstacles, and some things can already be shared though open APIs. The issue of sharing AR data can not be solved by any one company – We believe we must make collaborative efforts. 

As I mentioned, we helped create an organization called AR Commons which has already started building a social database for AR in Japan.   However, sharing ALL data on this platform will be a challenge, since so many interests will need to be aligned.  Not all info is shared on the internet, and some prefer closed and secure environments.

Tish Shute: What is your vision for AR Commons in the next 12 months?

Ken Inoue: We feel that public data, such as landmarks, government facilities, and public transport should be shared. We see an AR world where people can readily and easily access information by just seeing – quick, easy, and efficient.  And because of this ease and intuitiveness, children, the elderly and handicapped will surely benefit.  AR could help create a safer society.  Warnings, alerts, and safety information could save lives and avoid disasters.  These are what we, and AR Commons would like to tackle in the not so distant future.

Tish Shute: What is  the business model for Sekai Camera?  Do you have to subscribe to create? Otherwise just view?

Ken Inoue: All users can create AirTags – we want to allow all users to start AirTagging and add value to our service.  We wanted everybody to make tags, and we didn’t want to put a hurdle on it.

So, users can create text, voice, image/photo tags and can add comments on the tags – much like blogging and twitter. We will also mash up with many other social services which will strengthen the “social” aspect of our app.

Tish Shute: are you aiming for something close to the real time experience of Twitter?  And what will attract users over other social location based apps like Bright Kite using 2 dimensional maps?

Ken Inoue: Our service is very close to real-time already – only, because of the location specific aspect, it will be different.  It will definitely be something new.  Maps will also be integrated.

Tish Shute: And Sekai camera will work anywhere in the world?

Ken Inoue: We have named and designed it to be global!

However, it’s definitely easier for any company to focus on your home market first.  Being a Japanese company, we are initially concentrating on the Japanese market.  It’s still the second largest economy in the world, one of the leaders in the mobile internet market, full of geeks and early adopters of new technologies.  And what’s more, we already have a great buzz here, and it’s easier to talk and collaborate with local partners.  For any company building AR apps, geography and platform may be the difficult decisions to make, since first-mover advantage may become quite significant…  We are lucky to have such a large and hungry home market.

Tish Shute: Yes you have Denno Coil too. One of my big inspirations!

Ken Inoue: Oh, you know about it!  How surprising!

Tish Shute: You mentioned  CEO did a talk session with the creator of denno coil recently.

Ken Inoue: Yes, “Denno Coil” shows us what the future could be, and is very inspiring.  We actually didn’t know about Denno Coil until afterwards, although it was broadcast on national TV.

There is a picture on the web article of our talk session – on the right, you see our CEO, Takahito Iguchi, on the left, Mitsuo Iso, creator of Denno Coil.  Iso-san knew about the Sekai camera, and in fact, gave us lots of hints and advice on how to make it better.  He is a real technology lover – a mac lover, and iphone lover.

Tish Shute: Iguchi-san is a very inspiring and charismatic thinker and I would love to know about some of his imaginings for augmented realities. What are his AR imaginings for the next step after air tagging! What does Iguchi-san see Tonchidot doing in 2010? And then beyond that? And, what are some augmented realities he would like to see even beyond the limitations of current technologies?

Ken Inoue: We believe the possibilities are infinite! There are so many things we can and would like to do, but so limited resources.. So here again, what we and other fellow AR pioneers will be doing will depend on how we prioritize.  We would like to keep our plans secret for now.

Tish Shute: Does Iguchi-san see Tonchidot doing more with image recognition and the tight alignment of graphics with physical objects in the near future?

Ken Inoue: Yes, definitely!  We are already in talks with potential partners. There are some great technologies here in Japan, which were just waiting for us!

Tish Shute: And when will we get the kind of eyeware that would really change everything? (I noticed one Japanese company that is producing eyewear - what is their potential? Are their other eyewear initiatives in Japan?  What does Tonchidot think will be key to pushing this kind of hardware development for AR forward?

Ken Inoue: Yes indeed, the world of Denno Coil is not too far away….  There are actually many projects going on in Japan, and we are definitely interested in hardware development.  We are not short of world-class hardware developers here in Japan, and we have been approached by quite a few.

Tish Shute: I know you got some criticism for showing a concept video at last year’s TechCrunch50 which people felt didn’t show the technology you had actually developed. Do you have all the functionality shown in your video working now?

Ken Inoue: Hmm.. We did get criticism, and so it seems did TechCrunch – but we got far more praise and support!  I guess we really felt we needed to get the idea out there – As Robert said in your interview -  it’s hard to make people understand the full potential of AR.  And unless you show something like that in video form, it’s difficult to make people understand.  We showed TC50 our working prototype on the iPhone, and made it clear that the video was a vision of the future.  Because of the language barrier, we used simple phrases like, “Look up, not down” and “AirTag”.  TC50 let us make the presentation, for which we are very very thankful.

Tish Shute: Oh I love the term Air Tagging. It is a brilliant term!  Robert Rice noted it has the ring of terms like xerox and kleenex – i.e. a brand that becomes the “thing” and no longer a brand, congrats!  Sekai (World) Camera is really nice concept too!

Ken Inoue: Thanks!

Tish Shute: Recently @rhymo of SPRXMobile tweeted that Samsung NL was calling #augmentedreality the Optical Internet.  The resulting Twitter discussion gave a pretty resounding the thumbs down to the term Optical Internet with no’s from @bruces and my friend Gene Becker.

RT @genebecker: No @Rhymo, Optical Internet misses the point that #AR will be multimodal, multisensory, social, contextual

I tweeted that I thought Tonchidot my be able to improve on the term augmented reality considering your great track record with word smithing.  Has the Tonchidot team got any ideas for a better term?

Ken Inoue: *** Good question – the term “AR” is too techy/difficult…..  we agree.
But we haven’t thought of a good alternative term yet…

Tish Shute: Who came up with the term “air tags?”

Ken Inoue: Our CEO, Takahito Iguchi did.  He has a talent for creating names, phrases…  and the future, we hope.
Our members are not “AR lab scientists” – we are from the worlds of multimedia, visual arts, publishing, lovers of manga and anime and Japanese sub-culture….

Tish Shute: You mentioned Tonchidot has been very involved in Android development community in Japan. can you tell me more about this and what have been areas Tonchidot has been most interested in? What do the Tonchidot developers think have been the most exciting new developments with Android?

Ken Inoue: Yes,  core members of our tech team are key members of the Android movement in Japan, and we are influenced greatly by what’s happening there.  Their openness is very very attractive indeed!  It was a tough decision whether to choose Android or iPhone as our first application platform. There are pros and cons.

The android dev community is unofficial, of course, but we have been invited to speak and do demos very often -  one of our demos is in the media – shooting games on Android.  It was quite a while back, and our app is now far ahead.

Tish Shute: But Sekai Camera will be released on the iphone?

Ken Inoue: YES, if  all goes well – as many have pointed out, iPhone is not PERFECT – no device is, at least currently.

Tish Shute: Yes and how is the iphone uptake in Japan – the big plus in the US is the big user base?

Ken Inoue: Yes that’s the big difference.   In Japan, Softbank, the #3 carrier is marketing it – for now. They don’t release numbers, but I think there are 1M handsets already sold.  Still very small compared to other markets.  BTW, In Japan, roughly 35M handsets were sold last year, dropping from 50M in previous years.

Tish Shute: Yes it seems at the moment application developers are forced to choose between the US market and the rest of the world! So what is the status of Android in the Japanese mobile market – the iphone is pretty tiny

Ken Inoue: We just had a release of the first Android phone by NTT DoCoMo a couple of months ago, so still very very early.

Tish Shute: So Android phones market is even smaller than the iphone

Ken Inoue: Yes, and so our decision to release on the iPhone -
We haven’t provided our app for android yet – just demos. It’s too small of a market, at least for now.

Tish Shute: Robert put out an interesting question: “Are we letting the short term glitz of Apple and the iPhone fad pull us in the wrong direction? Shouldnt we be focusing on symbian devices that have the lion’s share of the market? or should we be looking more at either other OSs (winmobile, android) or not at all and trying to create a new platform that is more MID and less smart phone with a hardware partner?”

Ken Inoue: Good point. We certainly don’t wish to be Apple dependent, or dependent on anyone.  As much as we like Apple and iPhone, we will surely create apps for other platforms. We always get question/requests to create symbian apps, and we would like to do that – but in order of prioritization- we’re a small start-up.

Tish Shute: There are obstacles to creating AR apps on symbian devices aren’t there?

Ken Inoue: The AR experience we can provide on iPhone and android, can not be replicated on conventional phones.  However, we haven’t examined possibilities on Symbian in detail yet, so we can’t say much.

Tish Shute: iphone adoption in the US has really put augmented reality on the map.

Ken Inoue: It certainly has!

In Japan, it is rumored that iPhone will soon be marketed by multiple carriers, in addition to Softbank. That will be a boost for us.  Apple is moving gradually to a multi-carrier strategy, I believe.  With content getting richer, Apple will be required to partner with carriers with strong infrastructure.

Tish Shute: Recently I have  been exploring the strengths of Google Wave protocol for some aspects of mobile augmented reality.

And this is, perhaps, a question for the Tech team perhaps?  Do the Tonchidot devlopers think Google Wave would be an interesting jumping off point for some augmented reality standards?

Ken Inoue: Our tech members haven’t been able to examine this in detail yet – but we are definitely excited!

Today, I was very excited when Thomas Wrobel sent me a draft of, “Everything Everywhere: A proposal for an Augmented Reality Network system based on existing protocols and infrastructure.”

Thomas has kindly agreed to let me publish his draft, to open a discussion on this topic. The diagram opening this post (click image to enlarge) shows, “An example of how collaborative 3D-spaces could be shared over existing IRC networks.” It is from Thomas’ proposal. The full text of his paper is included later in this post.

“Can we try to avoid a browser war this time?”

Thomas notes in the closing remark to his paper:

“I am absolutely confident in my belief AR will become at least as important as the web has, and probably a lot more so. It will also face much the same hurdles and challenges getting established as that medium did. But, speaking as a web-developer, can we try to avoid a browser war this time?”

Thomas Wrobel has consistently posted insightful comments on how existing standards could be used for creating open augmented reality networks. But he expressed concern to me that his work and this paper not be overplayed:

“I’m hardly a leader, I’m just an amateur with a load of ideas on AR-related topics, some which might be useful, others might become unworkable. I don’t want anyone to get the impression this is how I think it has to, or should be done.”

I have brought/am bringing up this topic of using existing standards and infrastructure where possible for open augmented reality networks in all my interviews with members of the AR Consortium.

And I am finding agreement on a point that Robert Rice makes, “there is no perfect, ultimate solution *now*, but we have to do *something* to work from and refine/evolve.”

Thomas Wrobel makes what I consider some crucial opening suggestions. I take my hat off to him for thinking about this early, coming up with some clear, elegant, and practical ideas, and doing the work to articulate these ideas so others can participate in evolving them.  Massive props for that, many times over.

Good ideas on standards at an early stage of  a developing industry like augmented reality are like spring sunshine and April showers for new crops. No one knows what storms and pests the growing season will bring – but water and sunshine (open standards) are always a good start. And, personally, I can’t wait to see how this new industry unfolds (see Bruce Sterling’s Layar Conference awesome keynote : “At the Dawn of the Augmented Reality Industry.”)

Thomas Wrobel is:

“a web developer working for a small, brand-new company called Lost Again, which mostly works on ARGs (That is, the alternate reality games, not the augmented reality games, although there’s probably going to be big overlap there in the future). We developed two educational ARG games for the Netherlands with a company called res-nova.”

I have been following Alternate Reality Games  through the amazing work of Elan Lee and Fourth Wall Studios. Like Thomas, I think the intersection of ARGs and augmented realities is going to be very interesting. Thomas wanted me to point out that the website for his company with Bertine van Hövell, http://www.lostagain.nl/, is just a placeholder for now.

“Probably be up fully within a week or two. And, “despite the logo, we aren’t an AR company [yet], or a travel firm. The logos supposed to represent being lost in our minds.”

Thomas has been thinking about the topic of an open augmented reality network for a while now.  He is an artist also known as DarkFlame and his ARN network is included in this augmented reality concept for 2086 he did in 2006 (click on image below to enlarge).

Beyond IRC

Both Thomas and Mez Breeze made extensive and insightful comments on my last post, “Augmented Reality – Bigger Than the Web: Second Interview with Robert Rice.” And in particular they both picked up on something I am very interested in – the potential use of the Google Wave Web of protocols in creating open augmented reality networks.

Mez in her brilliant brainsplosion on social tesseracting takes on the very definition of information:

“Tish, when you ask Robert “…what is your approach to delivering a massively shared real time [augmented reality] experience that is like Wave not confined to a walled garden?” that’s an extremely relevant question + one that needs to be addressed while considering the entirety of the Reality-Virtual Continuum. I’ve recently finished a series of articles addressing this: the framework I’ve developed is termed “Social Tesseracting.”

I have recently begun exploring the Google Wave Web of Protocols which are nicely outlined in this post by J Aaron Farr which includes the very interesting diagram below (so more on Google Wave in another post).

But, as Thomas notes, while he demonstrates his ideas using IRC (Internet Relay Chat) they reach Beyond IRC:

“As mentioned before IRC has some drawbacks, which are due to its age or method of working. As such, future systems might yet prove better alternatives for a open AR network. One example of such a system is Google Wave. It shares many of the advantages of IRC (open, anyone can create a channel of data, different permission levels can be set and its free), while avoiding some critical restrictions. (The data can be persistent). I believe some of the ideas I’ve mentioned, and possibly even the proposed protocol string could be adapted for Google Wave or other future systems. I believe overall the principles are more important then any specific implementation to get to them”

Also Thomas pointed that while he uses markers to illustrate some of his examples, they are just a method for tracking.  What he is presenting is going to be transparent to the methodology of registration/tracking.

Tish Shute: You mostly use marker based examples but there is no reason why the principles you are suggesting will not be just as relevant as we move more into using more sophisticated image recognition tools is there?

Thomas Wrobel: No reason whatsoever. I mostly choose familiar markers as something that could be used now, with a lot of coding library’s already established for them. I think for most future AR use, markers will go completely…especially outside. Either things will be done purely by gps, object recognition, or the (in the case of advertising) markers will look like normal posters.

However, I do think traditional markers might “cling on” as being used for non geographical specific stuff at home. After all, if you need some reference points for moving mesh’s about in real time…(say, when playing a board game with a friend on the other side of the world)….then there’s probably nothing that’s going to be more practical then some simple bits of paper or card.

Everything Everywhere

-  A proposal for an Augmented Reality Network system based on existing protocols and infrastructure.

by Thomas Wrobel

The following paper is my vision of a open AR Network and potential methods to implement it with existing technologies. Specifically I’ll be focusing on a potential for a global outdoor AR network, although the ideas aren’t limited to that.

Of course I call it “my” vision, but I’m obviously not the first to have many of these ideas. I have been influenced and inspired by many things…

[Some of Thomas Wrobel’s influences – watched and played. Images  from Mitsuo Iso’s Denno Coil (Click to enlarge) top,  below from the game “Metroid Prime,” and Terminator, and the last from Buffy the Vampire Slayer!]

The AR Network.

When I speak of a future AR Network, I mean one as universal and as standard as the internet. One where people can connect from any number of devices, and without additional downloads, experience the majority of the content.
Where people can just point their phone, webcam, or pair of AR glasses anywhere were a virtual object should be, and they will see it. The user experience is seamless, AR comes to them without them needing to “prepare” their device for it.

From this point forward, I will refer to this future AR Network simple as the “Arn”.

The Arn should be an inclusive, and open platform where any number of devices can connect to, and anyone can make and host their own location-specific models or data.
It should allow people to communicate both publicly and privately, and not have their vision constantly cluttered with things they don’t want to see.

There’s two old, existing paradigms that I think can help reach this goal when they are combined.

The Internet Relay Photoshop.

IRC, or Internet Relay Chat was a chat system designed by Jarkko Oikarinen in the late 80′s.

Its a system where people meet on “channels”, they can talk in groups, or privately. Channels can be read-only, or open to all to contribute to. There is no restriction to the number of people that can participate in a given discussion, or the number of channels that can be formed. All servers are interconnected and pass messages from user to user over the network.

To me, this relatively old internet technology is a great template, or even foundation, for how the Arn could operate. Rather then text being exchanged, it would be mesh data (or links to mesh data), but other then that much of the same principles could apply.

People could join channels of information to view or contribute. Families could leave messages to each other scribbled in mid-air on private channels. Strangers can watch AR games being played between people in parks. People going into a restaurant could see the comments from recent guests hovering by the menu items.
None of this would have to be called up specially, if they are on the right channel when it was broadcast, they will see it.

The IRC paradigm becomes particularly powerful when combined with another one common to many computer users; that of a “Layer” in an art program, such as Photoshop or Paint Shop Pro.
As most of us know, layers allow us to separate out different components of a piece of art while editing, either to focus our attention on one piece, or to make future editing easier.

Now what if we simply have each “channel” of information represented as a layer?

Click to enlarge image above.

Having channels corresponding to layers is an easy and intuitive way for the Arn to operate. The user can login and contribute data to any channel, like IRC as well as adjusting the desired opacity and visual range of each layer, like they would a layer in Photoshop.

In this way they can get a custom view of the world, both with shared and personal AR elements visible at the same time.
They would not have to switch between various overlays to their world view, as they could see many at the same time.

Persistence of Data

With IRC or IRC-like system to communicate the data sent is mostly temporary data…broadcast on the fly from user to user and device to device. Retained in the users local logs, but not “hosted” anywhere.

I think for the majority of day to day purpose’s this is not so much a drawback, but actually desired for AR. Most casual communication doesn’t need to be recorded permanently in 3D space and, indeed, if it was, the cost of running such a service would increase exponentially with users and with time. Not to mention, our visual view of the world would get very cluttered very quickly. Imagine what your monitor would be like if it kept a history of every window you have ever opened and their positions!

So for most cases AR space should be treated like a 3D monitor letting us display many pieces of data from remote and local sources, and even to share them with others, but not being, by default, a permanent record for it all.

Most data will be analogous to pixels on a display, and if kept in records its only on the clients devices, not on the network itself.

However, occasionally we do want 3d data analogous to a web-page, such as (in the example above), the map layer. Data here should be persistent and visible to all that have that layered turned on.  I see no reason why hosting this data needs to use anything else but standard web-hosting with the (read only) #channel on the Arn merely providing a route to the data.

As the user logs onto the channel, the server, using a chat-bot, can send them a list of meshes with location data attached, and the Arn browser can simply pick the data to display that’s local to them. (Note 1: By doing it this way around, it allows some degree of anonymity to be possible, rather then the server knowing exactly where you are and feeding the specific correct string to you.)

We simply need to establish standards so this data can be pulled up and interpreted.

For instance, this standard could be as simple as a XML string pointing to a KML file on a server. This could then be then displayed in the users field of view at the co-ordinates specified.

In this way permanent data tied to locations, such as historical overlays or maps, could co-exist on the same protocol as temporary data such as mid-air chat’s or gaming related meshes.

There is also no reason why this shared-space/personal spaces based on channels of data has to be restricted to things given absolute co-ordinates.

(Different ways to access the same mesh)

It could work just as well with Markers and thus relative co-ordinates.

This would be mostly useful for indoor use, letting people logged onto a channel see the same meshes as everyone else on the markers. Thus allowing multi-player AR games, or AR games with observers very easily.

For example; games like Chess could be played between people with no additional code needed; You simply have a set of markers for only your own pieces, and as you move them the channel updates with the new positions, which are displayed in place in your opponents field of view.

This sort of game comes “free” with just having a generic system of shared space supporting markers.

It would also allow AR adverts down the street or in magazines to be viewed by simply logging onto the right AR channel

If markers are designed with URL data in them, this could even be a prompted or automatic process.
“There is visual data in this area on the following channel; #ABCD would you like to view this channel?”

Pros and Cons of using IRC or IRC-like systems

Pros;

• Anyone can write a IRC interface software.
• Anyone can create new IRC channels without cost
• Channels can have read and write permissions set.
• Users can easily have multiple channels open at once.
• Already established with thousands of severs worldwide.

Cons;

• 500-or-so character limit. 3D data must be linked too, not sent.
• Slow update rate. Lines of data can take a whole second or more to send.
• Non-persistent. Good for a 3d-view, not good for storage.

An example of how collaborative 3D-spaces could be shared over existing IRC networks;

Click on the image to enlarge

While in the long run I would hope for a dedicated AR network to be developed, with greater flexibility with persistence of data, there is a lot that can be done with the existing IRC system to implement the ideas mentioned above.

Below I will show an example of simple, crude, pseudo-protocol that could be fairly easily implemented to create shared AR spaces broadcast across IRC channels.

Its important to note, the goal here isn’t to exchange the mesh data itself on IRC, its to exchange links to the data.

Exchanging the mesh data directly within the 500 character IRC limit would be very hard, and liable to errors.

It’s also a waste of network bandwidth, as many people logged onto the channel might not have that object in their field of view, so their clients should not bother downloading it. (it should be up to the client browsers when to anticipate and cache mesh data).

Proposed Basic XML link exchange for AR;

Principle;
As user creates or changes an object, the clients software  posts a simple xml formatted string to
the IRC channel.
Anyone logged into that channel then sees that mesh displayed in the specified location.

This string could be formatted as follows;

<Mesh
ID=”DARKFLAME:1”
Obj=”http://www.darkflame.co.uk/mesh/church/chuch.kml”
Loc=”(49.5000123,-123.5000123)”
Permissions=”None”
LastUpdate=”12/12/0000,2012:12”
/>

This string allows other users client logged into the channel to automatically load the object from the URL and display it at the correct position in their field of view.
If the permissions are set to allow it, they could then move the object themselves, with the update being feeding back seamlessly to other users on the channel.

The objects posted are given an ID, which can be just the posters name, followed by a unique object number for that name. These unique ID’s would allow clients to track different instances of the same mesh, as well as making it easy to implement permissions. (if only the poster should be allowed to move this object, then the clients simply check if ID matches the user name posting the update. If its not, they can ignore it).

Next the objects need to be linked to a mesh.

The location of the objects mesh doesn’t have to be a fixed remotely-hosted url, it could be an IP address and port number of the user posting the mesh,hosted by the application posting the link to the channel.

Obj=”www.darkflame.co.uk/mesh/church/chuch.kml”
Obj=”123,223,14,23::3030”

The objects co-ordinates, likewise, need not be specified as absolute gps co-ordinates, but instead could refer to generic Marker.

Loc=”(49.5000123,-123.5000123)”
Loc=”Marker1”
Or relative to a marker;
Loc=”Marker4 (+0.0023,-0.0023)”
Or relative to a default plane;
Loc=”Default(+0.213,-0.123)”

The AR Browsers could then handle the association between the Markers pattern and its Name.
This way the markers are reusable, they do need unique markers to be printed for every new bit of AR they want to look at.
Users could just keep a set of generic markers handy, which they could simply assign to be Marker1,Marker2 etc for any AR use. (Note 2: As mentioned above specific makers could also contain a default ID name and channel built into their data, letting the Arn browser simply prompt the user if they want to see the model even if they aren’t in the right channel. This set up would be most useful for paper and even billboard advertising.)

The Default location could be a settable region, or marker, on the clients browser that defines a playable/user-able area in the field of view. Mostly useful for home use, this could typical be a square region on a users desk.

So, in the chess-game example, the client of the person making the moves simply updates the position relative to the Default every time they move their marker (which is tied to a chess piece mesh).
Then the (non-owners) clients software could automatically display it relative to their Default plane. This would make games like Chess, Checkers, Go or any other game involving merely moving objects about automatically very intuitive and easy to set up.

So by having meshes settable to absolute gps, marker-relative, or default-relative locations, reduces the bother necessary to experience AR content quite considerably, and makes “non-geo-specific” AR applications and games trivial to implement.

Next is permissions.

Mesh-permissions would be a simple string saying who else can update the data, if anyone.

eg;
Permissions=”None”
Permissions=”RandomPerson1, RandomPerson2”
Permissions=”All”

By default you could only update or move your own meshes. (identified by the ID of first posting). If you attempt to update anyone else’s,  their clients would just ignore it.

Thus in a game of chess, you can only move your own pieces. If you attempted to move your opponents (by reassigning  your own marker to their pieces Ids), the clients would just ignore that assignment. You’d only be fooling your own system.
Likewise, when pinning a message in mid-air for your friends to read, no one else can change that message without your permission, although copying it would be easy.  (Note 3: It’s important to note this sort of object-specific permission system is in addition to the global-permissions, or “user-modes” it’s possible to set for the IRC channels and users as a whole.)

Finally, as object data could change within all sorts of time-scales, the easiest way to keep everyone logged in up to date is to just have a time-stamp of when each model was last updated.

LastUpdate=”12/12/0000,2012:12”

This would not necessarily be the same as the XML string post date, because the models mesh might not be updated, but merely moved, and in such case the Arn browser shouldn’t redownload the mesh.

This sort of arrangement could be used as a standard today, and users wouldn’t have to constantly download special AR programs to view a single AR mesh.

In the long-term I would hope for more advanced methods to manipulate Arn-content online, analogous to Dom manipulation in web-pages. But for now, we should at least establish standard methods for devices to pull up meshes and overlay them in the correct position.

So, having a layered system could give the user a seamless blend of dynamic and static data with which to paint their world with.
I believe this is all relatively easy to achieve using modifications of existing web technology, combined with some basic graphics systems.

Local Data:

However, so far I have only talked about remote data.
What of programs originating on the device itself? This is, after all, how most AR software we have at the moment works.

I think, that just like the remote channels, local software should also be blended into the same list of layers.  People shouldn’t have to “Alt+Tab” out of one view of the world, to see another.
They should be able to see both at once, if they wish.

For instance, if your playing a AR game, why shouldn’t your chat window be viewable at the same time?

If you have skinned your environment with a custom view of the world, why shouldn’t you also see mapping or restaurant recommendations?

So local data and remote data should be blended in the same view.
How can AR software – of which I hope, there will be  thousands – seamlessly be expected to layer their graphics, not only with the real world, but with each other, and with online data too? Will games and software makers need to co-operate to allow their graphics to be integrated together with correct occlusion taken into account? A tall order, no?

I must confess though, my technology knowledge fails me here.

I can only guess special graphics drivers, or 3D APIs,  will have to be developed to let programs share their 3D world with that of a Arn browser.
Maybe programmes should simply treat themselves as a local-sever which the browser can connect too, and let the Arn handle all the rendering itself (although I imagine many games designers would find this quite limiting).
So I leave it as an exercise to the readers to discuss and propose the best methods by which this vision of a layered world could be realised..

Beyond IRC:

As mentioned before IRC has some drawbacks, which are due to its age or method of working.
As such, future systems might yet prove better alternatives for a open AR network.
One example of such a system is Google Wave.
It shares many of the advantages of IRC (open, anyone can create a channel of data, different permission levels can be set and its free), while avoiding some critical restrictions. (The data can be persistent).
I believe some of the ideas I’ve mentioned, and possibly even the proposed protocol string could be adapted for Google Wave or other future systems.
I believe overall the principles are more important then any specific implementation to get to them.

Summary;

⁃    In order for AR to flourish the user shouldn’t need to download a separate application for each mesh they want to see.
⁃     Having url’s embedded into QRCoded markers which point to standard mesh files like dxf or kml would be a way to do this right now.  The QR code would only have to be seen precisely  in shot once, then its borders could be used like a standard marker.

⁃    An augmented view of the world needs to support visual multitasking, and having  layers of information is the best way to do that.
⁃
⁃    Methods need to be devised to allow drastically different software to contribute to these layers, without restricting either the software’s rendering ability’s, or the users ability to pick and choose what layers of information he wants to see.

Last point;

I am absolutely confident in my belief AR will become at least as important as the web has, and probably a lot more so. It will also face much the same hurdles and challenges getting established as that medium did.
But, speaking as a web-developer, can we try to avoid a browser war this time?

Everything Everywhere , draft.
by Thomas Wrobel
Darkflame a t gmail

Neogence is on stealth mode with an immersive mobile augmented reality platform – “tools, sdk, and infrastructure plus some applications.” They are probably six months away from YouTubing anything according to CEO, Robert Rice.  But Robert rustled up this pic for me – a Google street view of Neogence R&D labs: “the patio on the lower left is where I do a lot of pacing and smoking my pipe and the porch and office upstairs is where  a lot of  meetings have been held.”

Robert Rice (@RobertRice ), CEO of Neogence, recently tweeted:

I’m changing my name to Robert Mobile Ubiquitous Geospatial Augmented Rice. I’m betting on radical changes in next 18 months.

Although Robert’s new AR platform is still under wraps, I think you will get a good idea of what direction he is going in from this interview (full text at end of  this post). Robert is the author of “MMO Evolution” and is a key developer and thought leader in persistent immersive environments, simulations, virtual worlds and massively multiplayer games as well as large scale communities and social networking.

It is OMG finally, at least, for minimally immersive but truly useful AR.

Since the launch of Android a new generation of useful augmented reality applications like Wikitude are emerging.

After the last NYC Tech Meetup, my  friend Nathan Freitas,  (@NatDefreitas),  or rather Nathan Mobile Meets Social Freitas, demoed for me a cool graffiti app  he has developed on Android.   You leave a marker for your graffiti so other people can find view/add their own – a nice primal experience like pissing on the lamp post to let your pack know where you’ve been.  Also the graffiti app taps into a long history of  NYC street culture around tagging and graffiti art.  For more cool mobile projects Nathan is working on – Vote Report and data collection for mass events, a guide to pubs and nightlife in New York City, and more, see his blog, “Nathan’s OpenIdeals. With Camera, GPS, compass, and accelerometer, and APIs on Android for temperature, light meters, (no hardware yet), Nathan says Android:

“seems to be the platform most likely to socialize the idea that sensor data could be a piece of every application.”

As Nathan is fond of saying:

The compass is a killer app enabler!

Also see OpenIntents for some interesting Android Sensor projects.

Wikitude was one of Thomas Wrobel’s two top AR milestones for 2008 (see Gamesalfreso):

Wikitude I think. Seems the first released, useful, AR software.

AR Geisha doll is also a remarkable breakout for AR – but useful, nah.

I asked Robert if he also thought Wikitude and AR Geisha doll as  significant breakthroughs:

Yes,  these are among the first attempts to get away from the novelty of simply rendering a 3D object based on a marker and making it interesting.

Remember, one of the biggest risks that AR has, is being branded as “novelty”, which means “cool for five minutes but ultimately a waste of time.” I think we have a ways to go before something is truly useful, but as 2009 progresses we should start seeing some effort here. I’d guess 2010 before something really useful comes out…at least something practical.

Now, having said that, I should say that I expect entertainment and games to take the lead (as usual), although there are a few companies really trying to leverage AR and video/graphics compositing for marketing (brochures) and location based methods (kiosks, large screen projections, etc.)

So when is it “OMG finally!” for massively multiuser augmented reality?

The picture above is from benjapo’s portfolio on istockphoto – also see the istock video here.

Alex Soojung-Kim Pang (who weighed in recently on the twitter-baby debates – see my KickBee Posterous blog) challenges design assumptions for augmented reality that take as a given the user’s desire for numerous private enhancements to their reality.

Alex points out less will probably be more so that enhancements do not impinge on shared experience.  See his write up of a talk he gave at the Royal College of Art, “and the end of my own private Shibuya.” Photo below by Stéfan, “Karaoke in Shibuya“

Part of the pleasure of these streetscapes is precisely that they’re collectively experienced, rather than individual visions: for even a brief period, we share with other postmodern, globe-hopping flaneurs and expatriates and temporary natives the light of the ABC-Mart sign and storefront.

It is collective experience of enhanced, augmented, virtual or real experiences that interests me too. This is one of the reasons I find Pachube and the EEML project of Haque Design and Research so interesting.

Extended Environments Markup Language (EEML), a protocol for sharing sensor data between remote responsive environments, both physical and virtual. It can be used to facilitate direct connections between any two environments; it can also be used to facilitate many-to-many connections as implemented by the web service Pachube, which enables people to tag and share real time sensor data from objects, devices and spaces around the world.

“Distinctions between virtual and real are as quaint and outmoded as distinctions between mind and body” (Usman Haque)

Usman Haque (founder of Pachube and Haque Design and Research) points out this is an underlying premise of his work – and augmented reality (full interview coming up soon!).

The pictures above show the Haque Design project, Remote:

‘Remote’ connects together two spaces, one in Boston the other in Second Life, and treats them as a single contiguous environment, bound together by the internet so that things that occur in one space affect things that happen in the other and vice versa – remotely controlling each other.

There was a discussion in twitter recently about how the terms like Second Life, Exit Reality, Virtual Worlds are misleading and outmoded. As Robert pointed out we need:

one word please…that sums up virtual and/or augmented reality, interactive, immersive, virtual worlds, mmorpgs, simulations, etc… also, I really don’t like the term “augmented reality” or “mixed reality”. Neither is all that great. And NO “matrix” or “metaverse” please

Robert argues strongly that there is a stultification both in virtual world technology – much of what we call virtual world technology was already, basically, where it is now in the mid 90’s. And MMOGs have devolved into gameplay design “that emphasizes the single player experience and does nothing to take advantage of the potential of the massively connected internet.”

Robert suggested I take a cruise through a new Virtual Space -  CoolIris to find some good pictures for this post (note the partnership between CoolIris and Seesmic to Streamline Video Blogging. I added the Cooliris Plugin to Firefox and typed Augmented Reality into search and soon I was cruising a highway of images and links. The Road Map image grabbed my attention (see below). It shows the continua that the Metaverse RoadMap authors thought are likely to influence the ways in which the Metaverse unfolds. It is “a map of the spectrum of technologies and applications ranging from augmentation to simulation; and the spectrum ranging from intimate (identity-focused)external (world-focused)”

Quite to my surprise, when I clicked out of CoolIris to the source for the image, I found it had been drawn from a post I wrote in May 2007, Hybridized Digital/Physical Worlds: Where Pop and Corporate Cultures Mingle. My post talks about a number of hybridization experiments that were bringing together lifelogging, sensors everywhere, simulation, virtual worlds, and augmentation.

The striking difference from 2007 to now is that we have definitely moved on from mere experimentation. And the poles of the continua intimate/extimate, augmentation/simulation as expressed in the Metaverse Roadmap are now becoming entwined (note the picture above seems to be slightly different to the one used in the road map as published here – perhaps I had an early version?)

“Augmented Reality is not just about overlaying data…” (Robert Rice)

The screenshot above is from TotalImmersions video demoing Augmented Reality with 3D Cell Phones. Also see video of their immersive games, and FutureScope kiosks here and here.

Gamesalfreso noted that Will Wright, delivered the best augmented reality quote of the year. When describing AR as the way of the future for games, Will Wright said:

“Games could increase our awareness of our immediate environment, rather than distract us from it”.

Robert points out in this interview the term Augmented Reality itself has become associated with a very limited understanding of what “enhancing your specific reality,” is really about. Robert notes:

it is inherently about who YOU are, WHERE you are, WHAT you are doing, WHAT is around you, etc.

When I talk about AR, I try to expand the definition a little bit. Usually, when you talk to someone about augmented reality, the first thing that comes to mind is overlaying 3D graphics on a video stream. I think though, that it should more properly be any media that is specific to your location and the context of what you are doing (or want to do)…augmenting or enhancing your specific reality.

In this sense, anything that at least knows who you are (your ID, mobile phone #, etc.), where you are (GPS coord or a specific place like a cafe), and gives you relevant data, information, or media = augmented reality. Sure, you can make things more interactive or immersive, but that is the minimum.

So, in this case, yes, I think there will be networked applications in the next 18 months…mostly things that are enhanced by friends lists (you are here, your friend is over there). These will be *application specific*. My team at Neogence is already going beyond this, building a platform and infrastructure for other applications to be developed on…all networked through the same backbone. Now, in this context (the science fiction AR that we all dream about), no I do not see anyone else trying to leap a generation or two ahead of the industry to build a massively multiuser shared AR space. Expect to see things like multi-user AR games, virtual pets, kiosk marketing, magic book, “gee whiz” presentations (tradeshow booths, entertainment parks, etc.), and so forth.

Goggle’s Are Not The Secret Sauce…

AR Cat left and Robert Rice right

What has come to be associated with the term Augmented Reality, in the popular imagination – an idea of 3D graphics projected over markers that has been forever waiting for the advent of “wicked next generation transparent wearable displays” – nirvana for augmented reality. While such displays may be nirvana for AR (and they could be with us in less than twenty four months), Goggles are not the “secret sauce” of AR as Robert points out.

All the glasses are, is another display device. At the end of the day, it doesn’t matter if you are looking at an LCD monitor, an IPhone, a head mounted display, or a pair of wicked next generation transparent wearable displays that magically draw directly on your retinas.

The real tricky stuff is what happens on the backend…making it all persistent, massively multiuser, intelligent, interoperable, realistic, etc. etc.

There has been quite a buzz going around about the new Vuzix Eyewear, and recently Robert talked with Vuzix and checked The Wrap 920AV eyewear out:

Vuzix is not alone in pursuing the ultimate in hardware, at least as far as wearable displays. However, I think they are much farther than the rest of the pack in vision, roadmap, and execution. They have put together a team that has a sense of urgency and ambition that will blow the industry away. After talking to them, I got the feeling that they really know what they are doing and there is a lot of mind blowing stuff in their pipeline. I’m sure they are one of the few companies that really gets it and has a clear vision of the future. Definitely my first choice to work with.

Hybrid Augmented/Virtual Reality

Cory Ondrejka posted this picture of the anybots telepresence robot and “congrats to Trevor Blackwell and the rest of the Anybots team on the launch of QA at CES.”  Cory (one of the founders and former CTO of Second Life) also made some predictions for Virtual Worlds, some optimistic and some less so, including “the increasing need to be able to diversify the Second Life product offering to begin truly rebuilding the code base.”

Robert is unabashedly irritated with the state of play in Virtual Worlds and MMOGS:

Unless both industries (Virtual Worlds and MMOGs) have some serious upheaval or radical new approaches, they will quickly be eclipsed by AR, which will eventually evolve into something hybrid..AR/VR depending on your level of access and hardware.

I’d like to see someone grab an engine like Offset, Crytek, HERO, or Unreal 3, and smack on a fat MMO server infrastructure (Eve or Bigworld)…toss in the right tools, and you would see a revolution and renaissance occur at the same time in the virtual world space. All the puzzle pieces are there, just no one is putting them together the right way.

I did just find out that Nortel’s WebAlive is powered by the Unreal 3 engine. You can try WebAlive out here.

Robert points out how rare it has become to see people really push virtual worlds technology and MMOGs into entirely new directions.  Although, of course, there are exceptions.  I managed to engage some interest from Robert in the possibilities the opensource modular architecture of OpenSim opens up, and the augmented reality experiments from Georgia Tech with Second Life (screenshot below) got praise from Robert for trying to do something new. (Georgia tech have also put out a virtual pet app for the iphone ).

But while Robert clearly has zero patience for virtual world technology which he sees stuck in the mid nineties, he notes:

the innovative and wonderful stuff about SL isn’t SL, it is what people are doing and creating on their own with terrible tools *IN* SL [Second Life].

The immersive mobile augmented reality platform Robert is building, he hopes, will generate this kind of user creativity but with 21st century tools.

So is it “OMG” finally for the Augmented Reality we have dreamed about?

According to Robert:

It really boils down to a markerless solution and a good application.

In the interview below we cover a number of topics including business models for Augmented Reality, e.g., how business models based on micro-transactions and virtual goods will translate to Augmented Reality.

Many of the challenges to becoming mainstream faced by virtual worlds  are similar to the challenges AR must overcome. Robert discusses these including the interface/gui that is a critical element for AR, solving the riddle of one world or many, patent wars in Virtual Worlds and Augmented Reality, the role of Augmented Reality in the future of sustainable computing, and what interoperability is about.

The Back Story for AR/VR…

In case you want to get up to speed on the required background reading for  Augmented Reality. This is Robert’s required reading list and Denno Coil is an absolute must see (feel free to add to this list in the comments, please).

“If you want to see the things that have inspired our vision of what we want to build, check out:

* Dream Park by Larry Niven and Steven Barnes
* Rainbows End by Vernor Vinge
* Spook Country by William Gibson
* Halting State by Charles Stross
* The Diamond Age by Neal Stephenson
* Donnerjack by Roger Zelazny and Jane Lindskold
* Otherland by Tad Williams
* Neuromancer by William Gibson
* Idoru by Wiliam Gibson
* Cryptonomicon by Neal Stephenson

and watch the whole anime of Denno Coil (subbed NOT dubbed!)”

Screenshot from Denno Coil from Concrete Badger.

Interview With Robert Rice

Tish Shute: I am glad to hear that you are working on this [an immersive mobile augmented reality platform]!

Robert Rice: We switched gears from MMO stuff about a year ago and we are finally getting some traction. It is very hard doing anything in this economy right now, but we found an opportunity to take AR to a new level beyond what you see on youtube. AR is still too “cute” and novelty. We don’t want to play around.

Tish Shute: I like Wikitude ‘cos it even manages to do something useful!

Robert Rice: Yeah, useful = traction. Now that we are getting near a prototype we are starting to get a lot of interest even though we are still technically way under the radar.

Tish Shute: r u funded?

Robert Rice: privately funded, some revenues from an early license, and ongoing discussions with several institutional investors. So, we have some funding, but nothing spectacular just yet.

Tish Shute: are you just developing an AR platform?

Robert Rice: hrm, sort of, but not just that. By platform I mean tools, sdk, and infrastructure plus some applications. The idea is to build something that facilitates everyone else making cool things and useful applications for different industries/sectors

Tish Shute: Yes that is the cool thing to do but isn’t that hard to fund!

(Robert grins) Well, that depends on the business model. We’ve got that figured out. I’d be absolutely happy if everyone and their brother were making applications on our stuff that gives us an edge on market penetration/saturation. There are plenty examples that prove the model. If you give people free and easy to use tools, they will run with it. ARtoolkit for example, has tons of people making nifty things and posting videos on youtube that has pushed them to the forefront as THE AR middleware to use right now, or heck, look at youtube free service, and they dominate video sharing.  Sure there will be a lot of “noise”, but there will also be a lot of “signal” that will rise to the top, facilitating and enabling is creating value in its own right.

Tish Shute: But how do you expect to monetize?

Robert Rice: There are a good half a dozen ways to monetize AR or an AR platform.

Tish Shute: What are your top 3?

Robert Rice: hrm, microtransactions, localized mobile advertising, and enterprise solutions (visualization)

Tish Shute: Do you think the consumer market will give the lead?

Robert Rice: I’m not sure. We are getting people from academia, intelligence, defense, border security, and some corporate types knocking on our door already, and pretty aggressively. It may be that those sectors push AR before consumer entertainment really kicks off.

But going back to a discussion we had earlier – yes working with “no markers” is a big deal.

Tish Shute: Can you talk about what you are doing there or is it still under wraps?

Robert Rice: I can say that between some university tech transfer and some of our own proprietary stuff, we are using some fairly common visual tracking technology. if you are really plugged into the AR scene, you will know there are probably half a dozen visual tracking methods out there. We just looked for the best one, licensed it for commercial use, and then started working our magic. This is a very small piece of the overall effort, but worth noting.

The downside with working with university tech is that it is usually based on research, incomplete, and not wrapped up in a nice commercial package on the upside, it can be a good start to build on.

Tish Shute: As you know I am very interested in “technology that matters” in particular tech that can help us accomplish the urgent goal of sustainable living.

Robert Rice:: oh, I’m pretty keen on sustainable living as well…after I sell off a few companies and have money of my own, I’m going to get into arcologies
❦
Robert grins

The interesting thing with the visual stuff combined with our other tech, is that we can make things multiuser, persistent, dynamic, and mobile.
The markers (fiducials) are really really limiting outside of basic applications. You can’t really plaster everyone and everything with a marker.  And they are, by nature, static (even if they are animated or whatever).

Also… our stuff works indoors and outdoors even without a GPS connection.
❦
Robert grins

Tish Shute: Now that does sound interesting!

Robert Rice: Yeah, with visual, you don’t need a compass or accelerometers either. Less hardware : )

You start with wifi triangulation or gps coord to get a “brute” location, and then you use the visual stuff for down to the meter accuracy and that by nature, gives you your orientation and positioning.

Tish Shute: Wow this is beginning to sound very interesting!

Robert Rice: Once you have that, it doesn’t matter where you go, it continues to track and continually refines areas you have been before. We’ve spent the last year figuring all this out. There are so many problems and obstacles that are going to be developing in the future for anyone trying to do what we are, but we have already discovered solutions.

oh, visual tracking = gesture based interfaces too that’s going to take some work, but its doable.  The real pain in the ass there isn’t the actual tracking, it is in the interface design.

That’s something that almost every AR company, venture, and research program is missing out on entirely. They are so focused on making cute things with markers.  They are missing the larger problems of AR Spam, interface, iconography, GUI, metaphor, interoperability, privacy, identity.

Tish Shute: So how are you dealing with all that!!

Robert Rice: We took the backwards approach of trying to think where we want things to be in ten years (and we read all the cool books…Vinge, Stephenson, Gibson, etc.) and then we spent time trying to think of what the potential problems are….like AR spam. Its bad enough when a giant penis flies by in second life, we don’t want that to happen in a global wireless AR platform.

Tish Shute: Do you have a prototype yet?

Robert Rice: hrm, 6 months away from youtubing something. Problem has been slow funding, which equals slow development. We also don’t want to show our cards too soon…too many potential competitors out there.

❦
Robert grins

Tish Shute: when you say microtransactions what is the business potential there?

Robert Rice: hrm last year I think, $1.5B was spent on virtual items. That’s games and virtual worlds. That should hit $5B in a couple of years. That’s basically people buying and selling things like WoW gold or items in SL or whatever. microtransactions, is basically the same thing, but in AR space.

Why couldn’t a 3D artist make a wicked animated 3D dragon, and then sell it to someone else? With AR, you could sit it on your shoulder. With a good scripting engine, you could train it to do stuff. Thats what I want to enable.

tools + sdk + platform = enabling people to make and create. Add in a commerce level (microtransactions) and wala.

Tish Shute: At the moment all of these virtual goods are very platform specific, is that a problem for you?

Robert Rice: Not at all. This is at a higher level. You have to switch mental models when you talk about what AR could or should be. For example, lets contrast the web and virtual worlds. For every virtual world you go to, you have to download a whole new client. Imagine if that model was applied to the web… you would need a brand new browser for every website you went to. That is just so…wrong.

It’s the same thing for AR…people are thinking about it with the same mental and business models and development philosophies as virtual worlds or web.  There are some things and aspects that work fine, but not everything.

Virtual worlds, are, by nature, necessarily different and walled gardens. The idea of 100% open and interoperable virtual worlds is a red herring…it sounds good but in practice it is a really dumb idea.

Tish Shute: I was wondering if you had a way to leverage all the 3D content already created ‘cos that would jump start things in AR wouldn’t it?

Robert Rice: Oh yeah, that’s easy. They all use the same polygons. Any virtual item in any game or virtual world is likely created with 3D studio or maya or something similar would be easy to convert and use.

Tish Shute: So people could bring their WoW weapons into your system?

Robert Rice: Not legally, but sure. Its just a 3D model with a texture.  It doesn’t matter if you use corel draw or photoshop or paintshop pro….or one screwdriver or another. Part of my team’s advantage, is that we are all experienced in MMORPG and virtual world design and development. We know the tools, the tech, and what works and what doesn’t.

Tish Shute: But some of the 3D content created in the social worlds is what has most value to people.

Robert Rice: Right, and that can be exported out easily.

Tish Shute: But back to “real” life applications. Is you platform really markerless?

Robert Rice: Yes.  marker = printed icon or glyph, also known as a fiducial

Tish Shute: But u must have some marker?

Robert Rice: hrm, more accurately, you need a point of reference.

Visual tracking has been around for more than a decade.  Lots of work for robots and other sectors.

Robert Rice: But isn’t the specificity of reference n terms of RL applications a vital key, for example, for a database of things?

Robert grin That is a different problem…tracking, registration, mapping, positioning, etc. That question has to do with mapping which is related to visual tracking, but not the same thing. We have a rather unique approach to some of this that I can’t discuss (patent pending).

Tish Shute: But for example, to create an augmented natural history of food – say I want to point at the slab of meat on my plate and know where that cow came from, what feed lot how it was treated etc

Robert Rice: That is not possible without ubiquitous nanotechnology. Shall I explain?

Tish Shute: Yes please!

Robert Rice: Ok, lets step back a minute and turn that burger back into a cow… the first problem (of this particular situation) is differentiating from one cow to another since most cows look alike, you can either attempt to discriminate visually (cow patterns) or use a much simpler option, like giving each cow a rfid chip in their bell, or hoof

Now, most people would try to figure out how to jam all sorts of info in the rfid chip, which sounds like a good idea, but isn’t, the trick would be to simply use the rfid to store a unique identifier with is then linked to a database elsewhere, or hoof.

That database should continually be updated with whatever relevant information you need so as you get close with your AR laptop, wearable displays, or embedded brain chip, you get the identifier broadcast, then you get the info downloaded to you, and it “sticks” to the cow with the generic visual tracking (object following, even simple bounding box is sufficient for a slow moving cow)

So, up to that point, you can get tons of information about that specific cow, that cow population (remember, AR is not just about overlaying data…it is inherently about who YOU are, WHERE you are, WHAT you are doing, WHAT is around you, etc.) Tie in data visualisation and some farmer tools and all sorts of other things happen. Now, lets move the timeline ahead a bit.

The butcher gets the cow and does his handiwork…because we know all the info about the cow, all of the meat can be properly labeled and marked. Ideally, with a UPC code or a unique glyph (somewhat problematic depending on how many unique glyphs you can create) so, while you are in the grocery store, you can access the relevant shopping data…age of cow, state of origin, type of feed, how many spots, how much body fat, which butcher, whatever not because of what is inside the package, but the package itself.

Getting back to your hamburger, the problem is that it is a burger…there is nothing to distinguish that burger from another one at the table…unless you stuck a rfid chip in it or splattered it with ink and a unique glyph, or maybe a special one of a kind plate.

However, a properly designed AR system could say “hey! that/s a hamburger! and I know I am at Fat Daddy’s Burger Joint in Raleigh North Carolina on Glenwood Avenue, and I know that they cook their burgers this particular way, and their meat supplier is those guys over there, and they usually get their cow meat from a farm out in Utah”

With ubiquitous nanomites or whatever, then its not that far out to consider edible nanos that are in the meat and that broad cast info so a slab of meat can tell you about itself and broadcast that to the general public.

Tish Shute: What useful scenarios can we create without the nanomites?

Robert Rice: If it wasn’t a burger or a consumable organic, the scenario changes.

Tish Shute: What is the time scale on nanomites?

Robert Rice: ehhhhhhh 20 years minimum if we are lucky. They sound good on paper, but there is a whole book worth of problems and why they are so far off…as consumer grade, all over the place, type of stuff.

Tish Shute: Did you see the Nokia Home Control center?

Robert Rice: Yes, I saw the Nokia stuff.

AR for sensors, like security systems, temperature control, etc. all become “sources of data” that a AR system can visualize. So yes, thats easily doable. You could do that in a short period of time with some half decent engineers.

The trick of what Nokia is doing is aggregating sensor data from a building/home/facility, mashing it together, and sending the mobile device alerts and data visualization conceptually rather simple, but no one has done it right or well yet.

It wouldn’t surprise me if Nokia pulled it off.

Tish Shute: yes and if they do and someone does an AR interface to it that would be an inflection point for AR?

Robert Rice: In a roundabout way, yes. You could get data directly from your house, or get it through your mobile device and in either case, use the AR for visualization and control.

The interface/gui is a critical element for AR. That is one of the areas where it, as an industry, risks doing a bad job and turning into just a fad or another novelty like VR.  Virtual worlds have been struggling with that for a while, but MMORPGs have had the effect of extending their life cycle

Tish Shute: Yes VWs have not solved the interface problem.

Robert Rice: The interface is one of their problems yes. Most virtual worlds are stuck in 1996/98

Tish Shute: If AR  is inherently about who YOU are, WHERE you are, WHAT you are doing, WHAT is around you, etc. seems that it is the ideal interface for home control?

Robert Rice: Well for home control, you must know:

1) Who am I? Am I authorized to know this information? Am I a guest?

2) Where am I? Is this my house? or someone elses?

3) What am I doing? Do I want to make all the doors lock? Turn on or off lights? Open the garage door? Trigger the security alarm?

So the same questions apply

I’d say that all virtual worlds are stuck in the mid 90s. They are at least a decade behind the game worlds…in technology, design, implementation, architecture, etc. etc. In my opinion, things like Second Life are shameful in how they are presented as state of the art, innovative, ground breaking, new, wonderful, and world changing.

But thats another topic of conversation : )

Tish Shute: Well for me the contribution of VWs is the presence enabled real time interaction with application (as 3D info machine) and context with other people.

Robert Rice: Oh,there is no doubt that they are greatly useful and have a phenomenal amount of potential.

They *could* be all those things I just said that SL isn’t…the problem is that they are either just existing, or they are meandering around without any real focus or direction. They aren’t evolving.

Even MMORPGs are losing their way and beginning to stagnate terribly

Tish Shute: yes I agree

Robert Rice: But, AR has the potential to change a lot of things.

Im sure you have seen the yellowbook commercials? The technologies you are seeing here are doable in hrm, a year or less maybe. The tricky part is the interactivity and AI…that is, the content. Everything else isn’t a problem. The avatar there could be photorealistic or stylized like a WoW character.

You could do that to some degree with markers for registration but dynamically changing the content linked to those markers is a little weird

(by the way, for the record, I like markers just fine, I just don’t think they are useful for real-world mobile applications)

I also think that the guys that want to dust the planet with miniature rfid chips are on crack and are going about it the wrong way

Tish Shute: A high level of interactivity is hard though. Isn’t it? Even in VWs it is very limited.

Robert Rice: it depends if you can track what the user is doing, and interpret that properly. Interactive is also a very lose term.

Clicking a button and making a light blink could be considered interactive.

Tish Shute: In VWs a high level of interactivity would  be to wield a virtual hammer and have a real nail go in! is physics part of the problem?

Robert Rice: physics aren’t difficult, plenty of middleware out there for it. The problem with that isnt so much the physics as much as it is the scale and purpose

Tish Shute: well for robotics?

Robert Rice: that gets into a conversation about meshes, textures, and volumetric collision detection and stuff

Tish Shute: virtual robotics?

Robert Rice: You mean teleremote/telepresence of real robots?

Tish Shute: yes!

Robert Rice: ah, for that, you need some tactile feedback and some other stuff – doable, but insanely difficult. That’s why you don’t see a whole lot of remote controlled surgery robots all over the place.

They do exist…

Tish Shute: Will AR contribute to sustainable living by freeing us from some of our energy hogging devices?

Robert Rice: AR will ultimately encourage energy saving and recycling. where did I leave a light on at? where is the nearest trash can? what is the UV index outside today?

Yes, computers are energy hogs, but as we start seeing larger SSD drives, more efficient CPUs (even if the number of cores increases in multiples), and so on, the power will go down.

Also, think about this…wearable displays potentially use less energy than LCD monitors on your desk.

Tish Shute: Yes I should pick the brains of my intel chums on energy saving!

Robert Rice: Getting rid of the monitor and switching to solid state drives will save an assload of power. Yes, I said assload.

Tell your intel chums to quit screwing around with single core mobile CPUs. We need multiple cores, that are smaller, faster, and use less power.

Tish Shute: Is AR is the sustainable future of VW and MMOGs?

Robert Rice: The fun stuff will happen when they are both integrated in some fashion.

Tish Shute: So perhaps this is why the Georgia guys are thinking in trying to combine AR and SL (see video here).

Robert Rice: That first video was pretty damn cool. It just pains me that they are using SL for it. And omg, all those markers on the table.

Although, I could care less about seeing my SL avatar on my coffee table. I would rather see an avatar representing ME in the real world, moving around in a virtual world that is a “to scale” replica of the real world. That is MUCH more interesting and innovative.

But even if I don’t like where they are going, or that they are using SL, the important thing is that they are doing something and forging ahead. I have a massive amount of respect for anyone, private, government, or academic, that is doing that.

And yes, the door (or window, or looking glass) has to work both ways for maximum potential, at least, that’s what Id like to see. They don’t *have* to, but it would be rather cool.

And going back to sustainability, AR has the potential to make monitors generally obsolete, laptops too. That’s a lot of power hungry devices with all sorts of metals and batteries inside.

But, even if the tech was absolutely crazy awesome right this minute, it would take a little while for consumer adoption.

Tish Shute: But AR unleashes the mobile device?

Robert Rice: Yes, AR is going to be built on powerful mobile devices for the near future, eventually embedded comps in clothing and whatnot. But that is a ways off

Entertainment is going to be the first huge driver.

Tish Shute: So people will get used to having a pet virtual dragon on their shoulder first?

Robert Rice: Yes, virtual dragon is way cool, easy tech for games, and can eventually be leveraged into a smart agent which becomes a practical application…agent based contextual search, etc. Yes, entertainment will also drive people to get used to the tech

Tish Shute: Oh thanks for turning me on to gamesalfresco!

Robert Rice: Ive noticed that the good stuff usually gets linked to there. They don’t list my blog, but that’s what I get for staying under the radar and not posting often. But anyway, gamesalfresco is the first place I send people that need a crash course in AR. Great site, great owner.

Tish Shute: So are you in agreement with Thomas Wrobel’s positioning of  Wikitude and AR Geisha doll as being significant milestones for AR?

Robert Rice: Yes,  these are among the first attempts to get away from the novelty of simply rendering a 3D object based on a marker and making it interesting.

Remember, one of the biggest risks that AR has, is being branded as “novelty”, which means “cool for five minutes but ultimately a waste of time.” I think we have a ways to go before something is truly useful, but as 2009 progresses we should start seeing some effort here. I’d guess 2010 before something really useful comes out…at least something practical.

Now, having said that, I should say that I expect entertainment and games to take the lead (as usual), although there are a few companies really trying to leverage AR and video/graphics compositing for marketing (brochures) and location based methods (kiosks, large screen projections, etc.)

Tish Shute: Many people would say SnowCrash (metaverse) is now and Halting State (AR) is ten years from now. But you are seeing a development timeline for some popular AR apps in the next 18 months?

Robert Rice: Anyone that says SnowCrash is -now- is living in a box. Virtual Worlds, Virtual Reality, and immersive tech in general stopped innovating in the mid 90s. I’m continually flabbergasted at the number of people that think that things like Second Life are state-of-the-art or innovative. You might as well try to market a walkman as cutting edge, even though we have IPods out there.

Id like to see someone grab an engine like offset, crytek, hero, or unreal 3, and smack on a fat mmo server infrastructure (eve or big world)…toss in the right tools, and you would see a revolution and renaissance occur at the same time in the virtual world space. All the puzzle pieces are there, just no one is putting them together the right way.

Tish Shute: Why doesn’t anyone do that?

Robert Rice: Its not cheap, people will only fund a copy of something that exists already, people fear change and innovation, etc, The list goes on. The right money goes to the wrong people all the time.

Alternatively stated, there is a lot of “right idea, wrong implementation”

MMORPGs carried the torch and have made huge strides on the technology front, but have devolved in design. More often than not the gameplay emphasizes the single player experience and does nothing to take advantage of the potential of the massively connected internet.

Unless both industries have some serious upheaval or radical new approaches, they will quickly be eclipsed by AR, which will eventually evolve into something hybrid..AR/VR depending on your level of access and hardware.

But yes, I’d say that the next 18 months are going to be very interesting with a lot of money being thrown around, new ventures, and plenty of content/applications. I expect most of this will be centered on single user AR experienced through a mobile device with a screen (iphone, android, etc.). I expect that there will be a significant boost after Vuzix releases some of their wearable *transparent* displays, putting Microvision back into the “has potential but is too quiet” position.

Tish Shute: AR conjurs an image in many people’s minds of dreadful head gear!

Robert Rice: Yes, it is either transparent wearable displays (in eyeglass formfactor) or nothing. HMDs with miniature LCD or OLED displays are good for streaming video, but for the mobile ubiquitous AR we all dream about, it has to be something that looks and feels like a pair of Oakleys.

I should also mention that several different types and modes of AR are going to find themselves being defined and refined over the next two years as we continue to blaze new trails, establish a lexicon (we keep borrowing terms from games, VR, virtual worlds, mmorpgs), and really work out the how as well as the why.

Even though the idea of AR has been around for a long time, the technology is just beginning to emerge, and very few people are even looking far enough ahead to figure out the problems and solutions that the tech creates. Really, who is thinking about how to deal with AR spam right now?

Tish Shute: Do you see any successful networked AR applications emerging in the next 18 months?

Robert Rice: Yes and no.

When I talk about AR, I try to expand the definition a little bit. Usually, when you talk to someone about augmented reality, the first thing that comes to mind is overlaying 3D graphics on a video stream. I think though, that it should more properly be any media that is specific to your location and the context of what you are doing (or want to do)…augmenting or enhancing your specific reality.

In this sense, anything that at least knows who you are (your ID, mobile phone #, etc.), where you are (GPS coord or a specific place like a cafe), and gives you relevant data, information, or media = augmented reality. Sure, you can make things more interactive or immersive, but that is the minimum.

So, in this case, yes, I think there will be networked applications in the next 18 months…mostly things that are enhanced by friends lists (you are here, your friend is over there). These will be *application specific*. My team at Neogence is already going beyond this, building a platform and infrastructure for other applications to be developed on…all networked through the same backbone. Now, in this context (the science fiction AR that we all dream about), no I do not see anyone else trying to leap a generation or two ahead of the industry to build a massively multiuser shared AR space. Expect to see things like multi-user AR games, virtual pets, kiosk marketing, magic book, “gee whiz” presentations (tradeshow booths, entertainment parks, etc.), and so forth.

The big thing I’m worried about is AR becoming the next silicon valley trend…once they realize the potential, an enormous amount of capital will flow to a bunch of startups with half baked ideas, weak business models, ten year old tech, and a lot of overhyped marketing. That is the very thing that will kill this technology as something that has true power and potential to literally change the way we interact with each other, our surroundings, information, and media.

Tish Shute: Do you think AR has value for a project like Pachube that helps us connect dtat from lots of different environments and sensor actuator data?

Robert Rice: I think that AR has value as an interface to this data (essentially data visualization based on information streaming from a sensor or source that is interpreted in some dynamic graphical manner that has meaning). This is one of the “big areas” where ubiquitous augmented reality and wearable computing will really shine. I’ll definitely be keeping an eye on Pachube .

Tish Shute: I can’t help it! I am really interested to hear more about the Vuzix glasses?

Robert Rice: Yeah, everyone is getting hung up on the glasses as the end-all be all and having markers everywhere too.

All the glasses are, is another display device. At the end of the day, it doesnt matter if you are looking at a lcd monitor, a iphone, a head mounted display, or a pair of wicked next generation transparent wearable displays that magically draw directly on your retinas.

The real tricky stuff is what happens on the backend…making it all persistent, massively multiuser, intelligent, interoperable, realistic, etc. etc.

I think that we are within 24 months of the magic wearables (these new ones by vuzix are probably the real first generation attempt at doing it right). They wont be perfect, but I expect they will be functional…and once we have functional, we can start doing the good stuff.

Tish Shute: You mentioned you disappointement with VWs and MMORPGs earlier.  Could you tell me more about that?

Robert Rice: Yeah, there was an evolutionary divergence between virtual worlds and mmorpgs a while back. One stagnated almost completely, and the other leapt ahead in one sense and devolved horribly in the other sense. Neither is where the state of the art should be.  That is a whole other conversation, and probably a second book.

Tish Shute: So making AR persistent, massively multiuser, intelligent, interoperable, realistic, etc. etc. that is where your efforts are going?

Robert Rice: Yes. I fully expect that the hardware is almost ready for it. You can cobble together some amazing things in the lab right now, and I think commercial viability is imminent. The real value (as far as I’m concerned) is in making it mobile, wireless, persistent, and massively multiuser. You could argue that augmented reality will take over where virtual reality failed and become internet 3, internet one being the internet, internet two being the web…

mmorpgs are nothing more than single player games in a multiuser environment these days. I’m more than a bit bitter about it. All the right money went to the wrong people, and the best games we have are barely shadows of what we could have had by now.

Tish Shute: Are there any open source AR platform dev projects?

Robert Rice: open source? hrm, Im sure there are multiple ones out there

if not entirely open source, there are plenty of things to experiment with that are generally free if you aren’t trying to sell something, DART and ARTOOLKIT come to mind as very accessible applications.

Marker based AR is very important right now…it is easy, low tech, understandable, highly customizable, and most importantly, accessible to the average joe. Ultimately though, we need a method of pure tracking…no markers glued to everything on the planet, no “billions of RFIDs” embedded in every square inch of every object on the planet, etc.

Tish Shute: What do you mean by interoperability in AR? And what do you think about the development of standards?

Robert Rice: Ooh, good question.

Ok, so the internet is basically computers communicating with computers, and the web is mostly pages linking to other pages (I’m greatly oversimplifying here). Hold this thought for a minute.

Switch over to MMORPGs. If you want to play in one (or a virtual world), you need to download a client that is specific to that world. One client does not work with another world. There are plenty of efforts to change this, but they are all barking up the wrong tree. The specific uniqueness of each world defeats the need and purpose of true interoperability, unless you completely reinvent the whole thing with a common backbone, features, functionality, etc. The very nature of virtual worlds and mmorpgs rebels against this.You absolutely do not want an avatar from second life running around in world of warcraft (for reasons that should be obvious).

On the other hand, with the web, you can use just about any client (browser) to access nearly any website (some requiring plugins or whatever).

The thing with augmented reality, is how do we go about making this? I’ve seen a few people thinking about this from the wrong perspective. There was a question at the last techcrunch to the Sekai Camera guys (a conceptual AR application for the iphone) where someone on the panel wanted to know how website owners would convert their content for augmented reality. BZZZZZT! That is a fundamental misunderstanding of what AR is, or could be, and it falls into the same trap I see a lot of people doing…and that is looking at AR through the web 2.0 lens or the virtual world lens. It is absolutely fundamentally different at the core…sure there are similarities: it has social networking/media applications and properties, and it has 3D graphics, but it stops there.

Ubiquitous augmented reality will be dramatically different depending on which standards, approaches, and philosophies get the most traction first. Will you walk down the street with your AR glasses and have a pop up every 30 feet asking you if you want to access the AR content on another server? Will you then have to register, subscribe, or whatever?

Or will all AR content be mediated by one sole master control server deep in the bowels of google? What about some other option? Will you need different sets of glasses to access different features and content from multiple sources?

At the end of the day, it should not matter what brand of glasses you are wearing, you should never have to deal with AR server popups to join/subscribe, and so forth.

Interoperability, in the context of what I was saying earlier, is the sense of how to build the infrastructure so all of this is seamless to the end user, but still maintaining the features/functionality necessary for all of what augmented reality promises us…I dont want to see everything in AR space, I want to be able to tune in or filter out some things, and I want to customize the snot out of what I see (perhaps changing metaphors or “holoscapes”), and so on. It all has to work together and simplify the end-user experience or it won’t get anywhere

Tish Shute: So what caused the stagnation of new development and devolution of MMOGs in you opinion?

Robert Rice: yes, look at all the hope and hype for the mmorpgs released in the last 12 months really, what is different or better? Now, what is worse?

I bet any decent mmorpg gamer could give you a list of 2 or 3 things for the first question and 20-30 things for the second.

And, VWs seem to be stuck in a feedback loop

Tish Shute: feedback loop?

Robert Rice: Imagine nailing one of your feet to the ground and then trying to run ’round and ’round and ’round.

Tish Shute: Why do you think this happened to VWs?

Robert Rice: Men in suits and flashy watches.

actually, hang on…..

I saw a video clip the other day from a conference about using various virtual and game technologies for simulations and other real world applications several people were talking about “avatar technology” and how theirs was better than their competitions and what not.

Now, can you tell me what “avatar technology” is? Avatar technology is a red herring. Avatar technology is the same thing as calling a toaster a new “fire technology.”

Robert Rice: The problem is that a lot of people that don’t have a clue about what they are doing are selling the tech to other people that have no clue what they are buying, but they feel like the should for some unknown reason.

That is happening all over the government, academic, and industrial sectors now with a few companies selling virtual worlds (again, mid 90-s tech) as the ultimate solution to all problems.

Anyway, getting back to your question

Once virtual reality started getting some buzz, some people got greedy and jumped into the avatar/virtual world thing and tried making it commercial too soon half of the 3D chat worlds were being jammed into platforms for virtual shopping malls.

Most of the money funding tech R&D started funneling towards VRML, and doing 3D in web pages, etc.

Tish Shute: yes horrible idea trying make web pages 3D IMHO

Robert Rice: The money people got involved too soon, and then the greedy people jumped in and tried patenting everything possible. Take a look at the worlds.com patent for 3D worlds.

They filed it back in 2000 or so and it was awarded in 07 (it shouldnt have been in my opinion) now they are suing everyone they can.

Tish Shute: Will there be patent wars in AR?

Robert Rice: Yes, the AR patent wars will be legendary once people start waking up to the real potential here.

The only solution is for everyone to band together and pre-emptively patent or make public domain every possible patentable concept, technology, or implementation for AR otherwise, you haven’t seen anything yet.

Tish Shute: Is the AR community organized enough to do that yet?

Robert Rice: That depends on how my company fares in the next six months.

Tish Shute: Will you patent or make your tech public domain?

Robert Rice: I plan on patenting the snot out of everything we can possibly think of, and then giving away our content creation tools and SDK stuff for free. The whole goal of what we are trying to build is to empower the end user and facilitate the creation of a wonderful world of augmented reality.

There are some things we will make public domain for sure, on top of that

Tish Shute: So back to my question on networked real time experience. Will we have networked Real time AR experiences in the next 18 months

Robert Rice: It is possible, yes. Other than what we are doing, I am not aware of anyone else taking the same approach we are, but the potential for an “under the radar venture” (much like my own company) is definitely there.

Tish Shute: Will you use cloud computing?

Robert Rice: I think that’s overrated and probably another attempt at the whole “thin client” model that some companies have been pushing for the last 20 years.

It sounds good on paper, but ultimately takes power and control away from the end user.

Tish Shute: cloud computing?

Robert Rice: Yes. You know, we aren’t playing around, We are totally building “THE AR” that everyone keeps dreaming about. None of this cute stuff you see on youtube. Actually, if you want to see the things that have inspired our vision of what we want to build, check out:

* Dream Park by Larry Niven and Steven Barnes
* Rainbows End by Vernor Vinge
* Spook Country by William Gibson
* Halting State by Charles Stross
* The Diamond Age by Neal Stephenson
* Donnerjack by Roger Zelazny and Jane Lindskold
* Otherland by Tad Williams
* Neuromancer by William Gibson
* Idoru by Wiliam Gibson
* Cryptonomicon by Neal Stephenson

and watch the whole anime of Denno Coil (subbed NOT dubbed!).

Tish Shute: So scaling the real time experience won’t be a problem in your project hehe

Cos no sharding allowed in AR right

And if you have lots of API calls?

Robert Rice:: haha, sharding is one of the dumbest things to happen to the VW/MMO industry

It is a solution to a technical problem that was relevant 15 years ago.

Tish Shute: so why did it stick (i know men in suits)

Robert Rice: it stuck because “thats what the other guys did” and the mmo designers are too lazy to reconcile gameplay for PvP and RP gamers

However, there is a curious problem between dealing with “one world” and “anyone can start their own custom AR server”

Tish Shute: Now that is a very interesting problem the one world and own AR server

Robert Rice: It took me a few weeks of not sleeping to figure that one out. It gets back to the interoperability issue

Tish Shute: What did you come up with?

Robert Rice: a solution. Thats all I can say for now on that.

Tish Shute: eeextra seeekrit!

Well I will definitely have to bug you on that.

The problem has produced some creativity in OpenSim with people coming up with hybrids of p2p and oneworld

Robert Rice: As far as virtual worlds are concerned, they need to look at the problem from a different perspective. They are trying to make all virtual worlds interoperable intead of creating a new model for interoperable worlds that new ones will be created to adhere to.

Tish Shute: well some people are. I would say most OpenSim developers see their modular approach doing this.  And you choose to interoperate based on what modules you have activated and then social agreements…

Robert Rice: hrm, thats a start, but that only works on a functional and social level – doesn’t account for content (story, mythos, game rules), unique data (my +3 sword), or the concepts of commerce, inherent value, and intellectual property

Enabling my WoW avatar to run around in SL and vice versa creates more problems than it solves.

Its like two alien races working hard to make sure that their two spaceships can dock but no one is paying any attention to the fact that race A breathes nitrogen and race B breathes sulpher.

Its technically possible, but they are missing the boat on the content side of the problem.

Tish Shute: Yes but don’t you think when a modular open source tech for virtual worlds  becomes pervasive, what will happen is that those interested in a similar genre will increasingly use the module in ways that allows their content to interoperate if they want it too

Robert Rice: everyone has to use the same backend tech, and the front end clients need to adhere to the same standards. Bu I have to admit, I haven’t been paying much attention to the vw space in the last 9 months or so.

Oh I have to run now.  But download and install cooliris. I promise you will be blown away and will start using it to search for images and videos

Its frigging awesome.

Tish Shute: Will do!  Thanks so much great talking to you. I can’t wait for your launch.