The Missing Manual for The Future (or The Future: The Missing Manual)

O’Reilly Media, is famous for is producing  “missing manuals” for new technologies, but thinking of O’Reilly as just a publisher of books would be like saying Facebook is just a website (this came up in the discussion at Media Round Table at Web 2.0 Expo, NY, 2010).   In recent weeks, I managed to catch Tim O’Reilly at several events, Maker Faire, Web 2.0 Expo, Hadoop World, and the free webcast Tim did with John Battelle on The Battle for the Internet Economy (although Tim spoke several other times during this period!).

It occurred to me, as I immersed myself in the depth and breadth of innovation showcased and discussed at these events that Tim O’Reilly, and the O’Reilly team, are creating, The Missing Manual for the Future.

As Tim puts it, we are “changing the world by spreading the knowledge of innovators.” Tim uses a quote from William Gibson to illuminate what is at the heart of the O’Reilly project:

“The Future is here, it is just not evenly distributed yet.” (William Gibson).

But Tim O’Reilly makes another point about the future when he speaks.  The future unfolds unexpectedly – so we must invent for an unknown future not a known future, or as Alex Steffen put it so well in his post, Why Our Bright Green Futures Will Be Weirder Than We Think, – “The world we need is one we’ve never yet seen.” The magic of attending an O”Reilly event is that it gives you a chance to work on this koan in interesting ways, and to take more responsibility for how things turn out.

Tim O’Reilly also urges that we think more deeply about what we are doing.  His keynote for Hadoop World , NYC, billed as, “The Business of Data” turned towards “The Consequences of Living in a World of Data.” The 900 strong crowd at Hadoop World was probably one of the most savvy crowds in the world about the business of data, so this was a nice turn.

Web 2.0 Expo with the theme, Platforms for Growth, was a deep dive into the business of innovation.  Tim O’Reilly’s keynote at Web 2.0 Expo,  “Thinking Hard About The Future” (or rather “thinking a little bit creatively or differently about the future)  – see video here, developed the call he made at Web 2.0 Expo 2008, to “work on stuff that matters,” into a Four Cylinder Engine for Innovation.   The first of the four cylinders in the firing order is, “Having Fun!” But,  at Maker Faire, Web 2.0 Expo, and Hadoop World I got an inside look at the workings of all four cylinders, and there is more to come, I am sure, as the other O’Reilly events unfold over the coming months including,  Web 2.0 Summit, Strata (a new O’Reilly conference on The Business of Data), and Where 2.0, 2011.

In a free webcast, last week (recording here), previewing Web 2.0 Summit, John Battelle and Tim O’Reilly discussed the Points of Control Map which is developing into a fun and useful tool to examine a very serious topic, “The Battle for the Internet Economy,” and how the “increasingly direct conflicts between its major players” could effect “people, government and the future of technology innovation.”   In my previous post, Platforms for Growth and Points of Control for Augmented Reality, I had a great conversation with Chris Arkenberg using this map as a springboard.  More on Points of Control later in this post.

The Four Cylinders of Innovation

click to enlarge

From Jet Ponies to Jet Packs: The First Cylinder of Innovation – “Have Fun”

The “maker” energy and its spirit of play, and the courage to create, hack, reinvent and re-purpose everything and anything, is a quintessential example of the first cylinder of innovation firing big.  Many “maker” projects also go on to fire on all four cylinders.   But the Maker forte definitely is in the first cylinder zone (and safety third as some of the rides, including Jet Ponies, warned).  The photo opening this post by Marc de Vinck – for more pics see here, is of Tim riding The Jet Ponies at Maker Faire which took  the New York Hall of Science by storm in late September – see The New York Times coverage here.  The ride was “built by the dastardly danger-hackers at the Madagascar Institute.“ See this wonderful interview with Hackett on his work to design “our specific jets from a patent that was filed in 1960s by a Mr. Lockwood, for Valveless Pulse Jets.” Hackett points out:

“Louder than god, glowing white-hot and looking like the trombone of the Apocalypse, pulse jets are also really shitty, inefficient engines,”

But, he adds:

“I have always wanted a jetpack, and one of the reasons I learned to build these things was to further that goal.”

This grand vision behind the Jet Ponies is a key to firing, The Second Cylinder of Innovation,  “Hey, we can change the world!”

But Jet Ponies, as a stepping stone to jet packs, also really struck a chord for me as I have been devoting a lot of time lately to the emerging Augmented Reality industry, a technology which was lumped in the same category of sci fi chimera as jet packs until very recently.

Data is the Gasoline

“The faces are coming from the sky.  The locations are coming from the sky.  All these apps depend on something, somewhere up.  And that, to me, was always the heart of Web 2.0.  And I am so delighted that people are finally getting it.  Because for a long time, people thought, ‘Oh, Web 2.0, it’s about lightweight advertising supported in a web start up.’  So I went, ‘No, no, no. It’s about the fact that we’re building these giant database subsystems in the sky that are going to drive applications.’  And now, of course, the same application is on your PC, it’s on your phone, it’s on you iPad.  And clearly, the applications are just sort of an interface to something that is being driven from the cloud, and that is fabulous.  That’s the difference.  People get it now.” (Tim O’Reilly, said this as part of a response to the first questioner at the Media Round table Web 2.0 Expo)

Answering questions about the importance of “Having Fun” to innovation doesn’t look quite as fun as riding Jet Ponies! Photo above from luca.sartoni’s Flickr stream

“ the data that is generated by the sensors and the applications that use that data is going to be where people are going to be innovative.” (Tim O’Reilly)

During the Media Round Table, I had a chance to ask Tim more about the role of bottom up innovation in a world where big data is the gasoline for increasingly sophisticated engines – platforms integrating machine to machine intelligence and real time analytics.

Tish Shute: You brought up Maker Faire in your keynote, and again now.  I was there, which not many people in the audience were  [not too many hands went up when Tim asked during his keynote].  But I think one of the things that struck me was the jet ponies – they were just earthshaking to stand near.  They made the ground tremble; they made the world shake.  Yet, most of your keynote, and most of what’s on our minds here, at Web 2.0 Expo, is extracting intelligence from the big data [in the sky], and algorithmic intelligences are the jet engines of the internet.  And of course, not to be forgotten, as we are here in New York City, where the trading markets are creating the air we breathe  [although we probably don't realize it until we lose our mortgage or something] and these algorithmic economies or “robot casinos” as Kevin Slavin put it, are all about speed – it’s not just real-time, issues of latency are  so critical that co-location is key to winning the game of the markets.  [Kevin Slavin brilliantly unpacks this in his talk, "Loitering on the Motherboard." For more in this see my conversation with Kevin Slavin below].

So my question is, who’s making the jet ponies for the algorithmic economies in the sky that you just described?   How can we make a play from the bottom up?  I always feel Ushahidi is one of the jet ponies of the data algorithmic space [because of their great work to bring human and machine intelligence together to solve problems in crisis situations].  But who do you think is doing exciting work and how can we ensure that this powerful world of data and algorithmic intelligences does not become hidden in a closed black box [only really accessible to elite players like the NYC trading markets]?

Tim O’Reilly: “Well, I think there’s certainly a lot of interesting things happening in, say, the financial services that a lot of, kind of, the Internet folks are kind of blind to.  I think that there are companies like Next Jump which are really good with data and good with algorithms. But kind of speaking specifically to the maker side of this, that whole sensor enabled world which is going to produce data is in its infancy.  What we have that I think is so powerful right now is we have the first portable sensor platform.  I said in my talk the other day, you know, your phone has ears, it has eyes, it has a sense of where it is.  And these are all available to application developers. You know, you can compare, say, Dodgeball to Foursquare, you can see how different… Dodgeball is Foursquare in the tele-type era.  Foursquare is now possible because there are so many more capabilities on the phone.

And I think that we are going to see a lot of other areas that are revolutionized by the sensors in the device.  It could well be that some of them will come explicitly out of the maker kind of projects, or it could just be that make is sort of a proxy for them.  So yeah, Arduino is this great maker sensor platform, but hey, here’s a consumer sensor platform [holding up phone]. Maybe we vaulted past the maker stage already and we just didn’t know it.

And that’s not entirely true, because Arduino is building a whole economy of special purpose devices.  But it feels a little bit like the days when people rolling their own PCs coexisted with the rise of Dell, who was a kid in his college dorm room who made his own PCs and sold them on the net, but figured out how to scale it pretty quickly and get good at it. But there were still a lot of garage shops, you know, ‘I’ll make a PC and sell it to you’ people for probably a decade before there was really a clue that that was a commodity industry.  In fact, I do think the sensor platforms are going to become a commodity industry.  And the data that is generated by the sensors and the applications that use that data is going to be where people are going to be innovative.”

The internet operating system is a data operating system and it is happening in real time (Tim O’Reilly)

click to enlarge the image above  – a slide from Mike Olsen’s  (CEO of Cloudera) keynote at Hadoop World

Not only do we have a portable sensor platform in our pockets  but developers also have powerful platforms and tools to make sense of data that fuel our apps.   Opensource Hadoop makes available, to anyone with some data munching chops, the power to work with giant unstructured databases and do the kind of real time analytics previously only available to giants like Google.  Big players like Yahoo, Facebook, and Twitter use Hadoop (Jonathon Gray from Facebook noted they add 10TB a day).   But, as this great roundup of Hadoop World points out, while Hadoop gets the press for handling petabytes of data , Mike Olsen (CEO of Cloudera) noted, the fastest growing area of users are working with clusters smaller than 10TB and over half of the Hadoop clusters were under 10TB in size.

Four Square: A Platform for Growth with an ecosytem built on top of data that exists in the real world

As an augmented reality enthusiast it is not hard to guess that one of my favorite platforms for growth is Four Square.  See Dennis Crowley’s keynote at Web 2.0 Expo here.  The Four Square API has been available to developers since November 2009,  and there are already a number of  interesting applications, and there will be many more to come.  The screen shot above is of geopollster – see the gallery of Four Square apps here.

@dens tweeted recently  “Politics + @Foursquare = @GeoPollster” http://geopollster.com <- I love love love that people are using 4SQ to think about election tools

As Kati London pointed out in her keynote, Four Square is the “kind of augmented reality that is aimed at shifting or changing a person’s social reality, e.g. the mayor badges in Four Square that change my relationship to the people and the place I am in, and augment engagement and reputation through socially driven consumer tie ins.” We are already see augmented reality developers beginning to work with the Four Square API – see here, Foursquare + Augmented Reality + Virtual Graffiti = ARstreets.

As augmented reality development tools mature, Four Square will, increasingly, become an important platform for creative AR developers interested in integrating the power of this platform for augmented engagement and reputation with “device aided augmented reality that can shift visual experiences of situated geolocal experiences.” With the Qualcomm vision based augmented reality SDK now available for download, and Wave in a Box soon? to be released, and an ARWave client working on Android (almost!), I have been exploring the Four Square API in my non existent spare time!!

The Four Square API also offers some interesting possibilities for exploring games that take the complex economy of Four Square – not personal data but aggregates of behavior, as their subject matter (for more on this see my conversation with Kevin Slavin later in this post and in an upcoming post).

I took this picture of Dennis at Where 2.0, 2009 at the beginning of Four Square’s phenomenal growth (they are at 4 million plus users now).

Pachube (Patch-Bay): a web service for storing and sharing sensor, energy and environmental data

Eighteen months ago, I interviewed Usman Haque (architect and director, Haque Design + Research) and founder of Pachube – see Pachube, Patching the Planet.   Usman pointed me to this wonderful evocative image from T.R. Oke’s book, “Boundary Layer Climates” (original photo source Prof. L. E. Mount’s The Climatic Physiology of the Pig).  “It’s the same piglets, in the same box, but on the right hand side the temperature has been increased. This small change in how the space is “programmed” has dramatically changed the way the ‘inhabitants’ relate to each other and how they relate to their space.”

The Challenge of Connecting people and environments.

At Web 2.0 Expo, I got the opportunity to talk with Usman Haque again.   Pachube, is becoming an established platform now, Usman explained.  They have a development team of eleven and robust back end.  And, they will now be spending some more time on the front end, including a redesign of the website, making “it a lot easier to widgetize the entire website so that you will be able to take almost any element and embed that into your own website.” And, as Usman mentioned in his presentation, they are working on an augmented reality interface, Porthole, for facilities management and, “as a consumer-oriented application that extends the universe of Pachube data into the context of AR – a ‘porthole’ into Pachube’s data environments..  Usman is also contributing to the AR standards discussion and on the program committee now for the W3C group on augmented reality.

Click to enlarge the image above from Chris Burman’s paper for the W3C, Portholes and Plumbing: how AR erases boundaries between “physical” and “virtual”

Pachube, is sometimes described as the Facebook for Data or an analogy Usman prefers, a Twitter for Sensors.  At Web 2.0 Expo, I had an amazing opportunity to hear from Twitter and Facebook about their strategies as platforms for growth.  This gave me lots of fuel for questions about Pachube’s approach to developing their platform.  Simplicity was a theme that Facebook  and Twitter both affirmed as a key.  One of Pachube’s challenges will be to deliver ease of use, and the equivalent of Facebook’s “like” and  Twitter’s “follow” to gain mass appeal.

Here is a brief excerpt from my upcoming conversation with Usman:

Tish Shute: So as a platform you see Pachube as having more in common with Twitter – a Twitter for Sensors. In what ways is Pachube similar to Twitter?

Usman Haque: Well we are the Twitter of sensors, devices & machines in the sense that, really, the API that enables all this communication is important, much more so than the website itself. It is where, basically, most of the millions of our hits actually go, is to the backend. And we’ve now got dozens of applications built on top of the system, a little bit like Twitter’s applications; you know, all the apps are the important part.

But we are actually going to be doing some quite exciting things with API keys that we haven’t really spoken that much about in public. But we have come up with a pretty innovative solution to make almost every resource have granular privacy options on it, now discussed here.

At Hadoop World, Tim O’Reilly also raised some interesting broader questions that are very relevant to Pachube’s vision to “patch the planet”, e.g, the problem of digital identity in the age of sensors? (Smart phones already know their users by the way they walk!) And, “How should we think about privacy in a world where data can be triangulated?”

Usman talked about Pachube’s approach to both the technical aspects of how to build a massively scalable system, and the conceptual aspects of how people connect to each other, and what they might do with these new opportunities to connect environments and sensor data  (see my earlier talk with Usman, Pachube, Patching the Planet, for a detailed explanation of some of the concepts behind Pachube).

I look forward to posting this conversation. Pachube is growing, and Usman always goes beyond the familiar tropes of connecting human and machine intelligence.

2nd Cylinder of Innovation: “Hey Can We Change the World!”

The possibilities for reimagining of the role of data in healthcare produced some of the most powerful “Hey Can We Change the World” moments for me at both Web 2.0 Expo and Hadoop World.  The slide above is from Esther Dyson’s brilliant Ignite presentation, “What you can and can’t learn from your genes?” are here,   Tim O’Reilly also brought up the powerful role real time data analytics can play in improving healthcare in his Hadoop World Keynote.  Also see Alex Howard’s post, 10 Lessons for Gov 2.0 from Web 2.0 for some more great, “hey we can change the world moments” at Web 2.0 Expo.  The keynote from Lukas Biewald of CrowdFlower and Leila Chirayath Janah of Samasource (screen shot below) in particular, is a provocative exploration of the future of work in the new ecologies of human and machine intelligence.

Changing the World When Our Lives Are Increasingly Shaped by Forces Invisible To Us?

Click to enlarge

Mike Olsen, CEO of Cloudera, noted that “the largest area of data growth does not come from humans interacting with machines; rather, it’s from machines interacting with each other” (see here in Minor Technical Difficulties).   One of the most interesting presentations at Web 2.0 Expo was Kevin Slavin’s, “Loitering on the Motherboard,” which, as Tim O’Reilly pointed out in his keynote at Hadoop World, is a talk that raises all kinds of questions about a system where big players are gaming the data for their own ends.

Kevin Slavin, a founder of Area/Code, notes the operating system of our mortgage, life insurance, the operating system of currencies and gold is now governed by machine to machine intelligence and algorithimic economies outside of human cognitive processes.  The markets are now legible only to bots in an algorithmic arms race with bots surveilling bots, and throwing off false information in a bid for counter-surveillance.  He showed some slides of the eery but beautiful visualizations of traces of the trading bots created from the Nanex API.

The screenshot above is from the Nanex: Crop Circle of the Day – Quote Stuffing and Strange Sequences.  “The common theme with the charts shown on this page is they are all generated in code and are algorithmic. Some demonstrate bizarre price or size cycling, some demonstrate large burst of quotes in extremely short time frames and some will demonstrate both…” This one is a zoom of the NSDQ “Wild Thing.”  Wild price/size repeater from NSDQ running at 1,000 quotes per second, effecting the BBO along the way (I love the great names Nanex gives the different patterns and traces produced by the trading bots).

Nanex supplies a real-time data feed comprising trade and quote data for all US equity, option, and futures exchanges. They have archived this data since 2004 and have created and used numerous tools to “sift through the enormous dataset: approximately 2.5 trillion quotes and trades as of June 2010.” May 6th 2010 (day of the flash crash), had approximately 7.6 billion trade, quote, level 2, and depth records.

Kevin points out that our lives are being shaped by criteria invisible to us and the old hackneyed tropes of machine to machine intelligence such a robots reading HUDs in English are long worn out.  The latter point is, perhaps, something for us augmented reality geeks absorbed in ideas of “making the invisible visible” to chew on.

Changing a world shaped by forces that are, increasingly, invisible to us presents a huge challenge.

But I had the glimmer of a, “Hey Can We Change the World” moment, when I attended Kevin Slavin founder of Area/Code’s presentation and had a conversation with him after his talk.  Could games take these complex economies as their subject matter?  The economies of  Farmville and games like WoW are not opaque at all, and these are environments with complex economic behavior, “where you can actually have enough data to understand what it is” – “it’s not so much about personal data.  It’s more about, like, aggregate behaviors.” “Games that can really model those, and play with those, and take those as the subject the way that Monopoly takes Monopoly as a subject could be really interesting.” Kevin made many fascinating points – more to come on this topic.

Photo by James Duncan Davidson, of Kevin Slavin speaking at Web 2.0 Expo NY, 2010, from the O’Reilly Conferences Flickr stream

Here is the beginning of our conversation:

Talking With Kevin Slavin

Tish Shute: You began your talk today about visibility and where some of the algorithmic masters of disguise went to work, after they had solved the math behind stealth bombers.  I thought perhaps you were leading into ideas about a reverse surveillance society.

But you surprised me, as I felt you made visibility itself kind of a non-issue by the end of your presentation and that counter surveillance became basically a time and speed issue.  Now I am not sure quite how to imagine a counter-surveillance society, something I try to think about…

Kevin Slavin: Well, let’s see.  There’s a couple ways to think about it.  I think one point is just that when we talk about counter-surveillance, we usually locate that as something that comes from  the bottom up, something that comes from the population. Think about the way the plane spotters discovered the CIA black rendition flights.

I think in general, when people talk about counter surveillance, or sousveillance, they imagine it as an inversion of the traditional relationship between the people and the state.

But that’s what’s interesting. What’s happening now, is that there are forms of surveillance and counter-surveillance that are in play beyond any human perceptual horizons. These forms are at their most sophisticated in financial services, in the markets.

If you were a bot, and could read the market legibly (which humans cannot), what you would see, effectively, are bots that are surveilling bots. Then you have bots that are throwing off false information in a bid for counter-surveillance. Many of the bots are, themselves, surveilling other bots; each one of them is trying to figure out what all the other ones are going to do. In essence, it’s an algorithmic arms race, and game theory has become concrete, since the theories are code, the code is action, and the action affects, let’s say: your mortgage.

And so, basically what you have is you have this series of algorithms that are all looking to discern each other, while also trying to prevent themselves from being discerned. I think of the tunnels under the trenches in WWI, tunnels to surveil the trenches, and then, later, tunnels to surveil the tunnels. There’s a few examples of this kind of thing.  But It’s especially strange when it’s computer code, and at the magnitude we’re seeing today.

All of it, as noted in the talk, accounting for 70% of all the trades in the market. 70% of the market trades are never touched by human hands or even seen by human eyes; they don’t move through a conventional cognitive process.  And that’s why you get things like the Credit Suisse algorithm, it was buying, selling 200,000 shares of stocks to itself over and over and over again. It was a bug and it slowed the market to a crawl.

Credit Suisse was fined, in essence, for failing to control an algorithm. Maybe that’s the first time an algorithm was treated like a human, in a way. As if the algorithm broke the law, and Credit Suisse was responsible for letting it do so. For me, that feels like a threshold event.

It’s not that humans never made mistakes when trading on the market. But when algorithms err, they err with magnitude.

The idea that we now have bugs in the United States market economy is really worth looking at.  If Apple can’t keep code bugs from the most simple iPhone apps in a closed and regulated ecosystem, I’m pretty certain we’ll have a lot more Credit Suisse type bugs in the future.

And that will be pretty interesting. There will be viruses, and the operating system they will operate on will be the operating system of the United States. The operating system of your pension, your house, your life insurance. The operating system of currencies and gold.

Tish Shute: I was hard-pressed by the end of your talk to think of like, “Well, what would be the equivalent of, sort of a people’s uprising to create a better fairer society in this kind of world where, really, the things that affect the key aspects of lives most are going on beyond human perception at an algorithmic level?”  But you made a pretty radical suggestion at the end…

Kevin Slavin: Well I think increasingly the markets have become delaminated from anything meaningful. First from goods, then from fundamentals, and now finally from homo sapiens. So that’s hard to fight.

It’s the race towards abstraction that makes it impossible to simply “resist.” The latest version in the long series of fiscal catastrophes was based on Wall Street finding goods that could be rolled up and sold with false valuations, but goods that would take a long time to fail. Mortgages are handy like that. It’s the tradition of extending the abstraction as long as possible, until finally the bill arrives and the banks fail. I don’t know if that’s something to rise up against or not. It’s like a rally against evil.

But really, I think the point is that it won’t be the people that rise up. It will be the financial services themselves that rise up. They’ll just detach completely.

That was harder to do with cotton or with wheat, with simple futures; they keep financial services tied to the ground.  So what we’re doing is creating increasingly complex financial instruments that are further and further removed from anything you can touch.  Like the way a mortgage is abstract. But, of course, the bottom line is that at the end of that mortgage lies someone’s home.

It’s said that Wall Street is now moving onto life insurance, because that’s going to take even longer to fail.  They’re doing the exact same thing. The word is that they are rolling up CDOs made out of crap life insurance policies, same way they rolled them up with crap mortgages a few years ago.

And those will probably take, I don’t know, 15 or 20 years to unwrap and unravel.  But what you see in the meantime, is that they are looking for things that are increasingly abstract, intangible, removed as far as possible from the experience of everyday life.

So maybe this is good. Maybe that’s financial services rising up. Lifting off. I think best case scenario now is that they actually leave humans alone altogether.  That, someday, they are just trading, effectively, completely arbitrary goods, the stocks could be anything at all, maybe for crops that no longer exist, and I’m just saying that then these bots would no longer affect what we do and what we are, it would just be a robot casino, an invisible paradise in the air.

People are the platform: How Games Can Be Engines of Innovation in Our Lives

See the video of Games that Know Where We Live here (screen shot above)

Kati London, Senior Producer, Area/Code, in her keynote showed how games that know where we live can shift players perspectives – from device aided augmented reality that can shift visual experiences of situated geolocal experiences to a kind of augmented reality that is aimed at shifting or changing a person’s social reality, e.g. the mayor badges in Four Square that change my relationship to the people and the place I am in, and augment engagement and reputation through socially driven consumer tie ins.

Area/Code has recently developed two games for the Knight Foundation that take people as the platform.  Macon Money, uses very simple games dynamics (for more see the video of Kati’s keynote) in a game designed to help “Knight’s continuing efforts to support revitalizing Macon and creating a vibrant college town.”

The other game that Area/Code has designed with the support of the Knight Foundation  is for the Biloxi and Gulf Coast community, a game called Battlestorm.  “The game’s purpose is to increase awareness about natural disasters and change the way people prepare for them.”

3rd Cylinder of Innovation: Build products, business models and entire industries.

Glympse – real-time, private location tracking

Julianne Pepitone, Yahoo! Finance, nailed the essence of Web 2.0 Expo, NYC, this year in her post, Web 2.0 Expo startups are big on neighborhoods, storytelling.  She writes:

“At the Web 2.0 Expo in New York City this week, executives from big sites like Facebook, Twitter and Pandora all spoke about industry trends. But the showcase of 27 startup tech companies stole the show.”

Listen carefully to Tim O’Reilly and Fred Wilson, Union Square Ventures, question their picks from the startup showcase at Web 2.0 Expo.  Also see this video of Fred and Tim discussing their conversations with all the start ups.  This  is one of the clearest public windows onto both how to present your company to VC, and how to figure out what are the most important questions for you as an entrepreneur  building a business in a world of data.

Glympse successfully pitches their “jet pony” strategy for a location based business, and is Fred’s pick.  They hold up well under pressure and answer Tim and Fred’s hard questions about how their start up will not get overtaken by an encumbent player with resources and market share before they can gain traction.   food52 responds to Tim’s probing about their strategy for business data analytics that he points out are vital if they want to survive with the small margins of ecommerce.  There is a list of all the participants in the start up showcase in Brady’s post here. hour.ly was the audience pick.

Shazam for Faces!

My favorite start up was a biometric service doing face, iris, and finger print matching, Tactical Information Systems.

Tim and Fred also liked them, and they have an interesting discussion about the merits or not of approaching your platform through a narrow first application as Tactical Information Systems are with WanderID -  an application to help identifying lost Alzheimer patients.  As Fred pointed out, they are potentially the Shazam for faces, so why start so small?

I  had asked TIS the same question when I met them in the “speed dating” session.  This is just their first toe in the water as they are a two person company at the moment. Their vision for their platform is big.  Mary Haskett and Dr Alex Kilpatrick, the founders of this quintessential jet pony for the algorithmic economies in the sky, are not only a partnership with the credentials to do a  Shazam for faces – see their bios here, they are the people I would want to be running a Shazam for faces!  They really get the consequences of living in a world of data – check out Dr Kilpatrick’s absolute killer Ignite talk, “Defeating Big Brother.” (screenshot below)

How Can Augmented Reality Add Value to the Real Time Internet/Data Operating System?

planefinder.net – an augmented reality app that lets you find information about planes by pointing your phone at the sky, “including flight number, aircraft registration, speed, altitude and how far away it is” (via MacLife).

The new opportunities in the algorithmic economies in the sky were center stage at Web 2.0 Expo and there are some interesting AR apps for the real time internet/data operating system emerging, like planefinder.net.  But Augmented Reality was still pretty low profile at Web 2.0 Expo (except that NVidia augmented reality demo attracted a lot of attention at the sponsors expo).  However, everyone working in the emerging industry of AR should recognize that apps big on “neighborhoods and story telling” are heading right up the AR street, and that platforms like Four Square and Pachube present enormous opportunity to explore the possibilities of AR.  And if augmented reality enthusiasts are not already paying attention to real time data analytics, and Hadoop, they should be (see this post for an excellent round up on Hadoop World).

At Hadoop World, Tim O’Reilly referenced the great tagline from the  IBM commercial:

“Would you be willing to cross the street — blindfolded — on data that was five minutes old? Five hours? Five days?”

As I have noted in several earlier posts – see here and here for starters,  we are just seeing the tools  for developing near field, vision based, mobile, social AR become widely available to developers, so there should be a new level of AR apps emerging through 2011.  There is a wonderful discussion in the comments of this post by Mac Slocum, “How Augmented Reality Apps Can Catch On,” between Mac, Raimo one of the founders of Layar, and Chris Arkenberg on what constitutes a platform for growth for augmented reality.

Mac’s post, the comments and Chris Arkenberg’s post on the latest edition of the Gartner Hype Cycle, that rather curiously placed Augmented reality almost at the peak of inflated expectations. really got me excited about exploring an idea I have been thinking about for a while, which is to get the AR community to discuss the Points of Control map.    See my discussion with Chris Arkenberg here, Platforms for Growth and Points of Control for Augmented Reality. The recording of  John Battelle’s and Tim O’Reilly’s webcast on Points of Control is posted here.

The interactive Points of Control map is an amazing tool to think with! Check it out in movements, territory and movements, acquisition mode.  There is a competition for the most interesting comment and most interesting acquisition suggestion.  The prize is a ticket to Web 2.0 Summit!

What is the Future of Social?

The recent “defection” from Google to Facebook – see Lars Rasmussen, Father Of Google Maps And Google Wave, Heads To Facebook,  is as MG Siegler of TechCrunch points out, “the biggest one since Chrome OS lead Matthew Papakipos made the same jump in June” (TechCrunch also notes “current Facebook CTO Bret Taylor was heavily involved in the launch of Google Maps”).

These moves have drawn my particular attention as did Bret Taylor’s response in his conversation with Brady Forrest at Web 2.0 Expo to Brady’s question, “How soon until we get the Facebook firehose?”

If you have been reading Ugotrade you already know how important I think an open, distributed, standard for real-time communications such as the very innovative Wave Federation Protocol could be for AR development  -  see ARWave and my presentation at MoMo13, Amsterdam last year, The Next Wave of AR: Mobile Social Interaction Right Here, Right Now!

The anticipated release of  Wave in a Box, has raised hopes in the developer community that  WFP will soon become easier to work with, and hopefully more widely adopted.  Like many others, I wonder what will happen to Wave in a Box now?

But the innovation of Wave is deep and broad (and as many have pointed out hugely ambitious).  Perhaps the boldest attempt yet to innovate both at the low level of architecture (where Google is so powerful) and at the high level of the Mark Zuckerberg, “big idea,” which as Tim O’Reilly notes is, “What is the future of social?” MG Siegler noted Facebook Groups Is Sort Of Like Google Wave For Human Beings.

But I deeply hope that the open, distributed standard part of the Wave big idea is not lost in the mix here.

Fourth Cylinder of Innovation: Keep the Ecosystem Going, Create More Value than You Capture

The Points of Control map is interactive, so please click here or on the image above for the full experience.

Tim O’Reilly points out that there is a worrisome dark side to the Points of Control Map – see Tim’s keynote here.  To paraphrase some or his points:

There are companies on the map that are forgetting to think about creating a sustainable ecosystem.  Rather than growing the pie, they are trying to divide up the pie and that threatens to cause the fourth cylinder of innovation to misfire.  This fourth cylinder is essential to the ecosystem.

Tim O’Reilly looks back to the lessons of the personal computing industry which was incredibly vital and creative, and lots of people made money until a couple of big players “sucked all the air out of the ecosystem” and innovation had to go elsewhere.

The Power of Platforms is to create value not just for your company but for other people.   Create value for yourself by creating value for other people.  Tim O’Reilly used the wonderful example of  Henry Ford inventing the weekend so that there would be enough people with time and money to buy his mass produced cars.  Think about building the ecosystem that will support the future your are going to build.  Grow the pie rather than cut up the pie.  This will be the vital fourth cylinder of innovation in a Web Squared world.

Tim O’Reilly has long proposed that  Web 2.0 is all about harnessing collective intelligence,  But as Gartner predicts, “By year end 2012, physical sensors will create 20 percent of non-video internet traffic.” Yet another previously unevenly distributed future is going mainstream, and if you haven’t read it already, now is the time to read this paper by Tim O’Reilly and John Batelle, Web Squared: Web 2.0 Five Years On.

The Consequences of Living in a World of Data

To bring this very long post to a close!  Here are just a few of the key questions re The Consequences of Living in a World of Data that Tim O’Reilly raised during his keynote for Hadoop World:

“How would we solve the problem of digital identity in the age of sensors? (Our smart phones are able to know their users by the way they walk – their gait!)

“How will we input data when our devices are smart enough to listen on their own?”

“How should we think about privacy in a world where data can be triangulated?”

“We are moving to a world in which every device generates useful data, in which every action creates information shadows on the net.”

“Shouldn’t we regulate the misuse of data rather than the possession of it?”

“How do we avoid a data arms race?”

“Create more value than you capture.”

I have been so busy working on Augmented Reality Event, Jun 2nd, 3rd, Santa Clara, CA, in recent weeks that I have barely had time to post!  But it’s getting hard to contain my excitement about ARE2010.  If you haven’t already seen the sneak preview of the schedule  – see here.

Augmented Reality Event will be an unique opportunity to see how the complete vision of AR is emerging, one that will include visual recognition of real life objects, sensors to enable interaction with physical objects, and the long anticipated, comfortable, sexy, AR eyewear.  Six companies with be presenting and exhibiting AR glasses and HMDs at Augmented Reality Event.

Keynote luminaries will include: Bruce Sterling, “AR Prophet,” Will Wright (Spore, The Sims & now AR and The Stupid Fun Club), Jesse Schell (who will give the AR follow up to his now viral Dice talk), and Blaise Aguera y Arcas (Microsoft, Bing), whose talk on augmented reality maps was one of the most popular talks at TED this year.

Over 40 AR companies from all over the world are represented, and there will be speakers from Qualcomm, Google, Microsoft, Nokia, Intel, and Nvidia.  Over 80 augmented reality experts and entrepreneurs are presenting, including a AR titans, Total Immersion and Metaio.   Mobilizy, Layar, Ogmento, TagWhat, Tonchidot, and Zenitum (just to name a few of the stellar augmented reality start ups coming – for more, and a list of speakers see here) will demonstrate AR is not just a vision, but actually drives significant businesses today,

Five Augmented Reality industry start ups will be on the hot seat in a Launchpad competition with $10,000 prize for the winner, thanks to the generosity of  Qualcomm, a leading developer and innovator of advanced wireless technologies, and the featured sponsor of Augmented Reality Event.  Zenitum, a leading augmented reality company, with a technology portfolio that includes both location and advanced vision-based sensory input is the Gold sponsor for ARE2010, and Adobe are sponsoring the ARt Gala & Reception.  Many thanks to the sponsors for making ARE2010 possible.

Readers of this post can use my discount code TISH245 to register here for a $245 price for the whole 2 day event.

But just to add some balance to my total enthusiasm for the rocket like trajectory of augmented reality in recent months, this post will also look at an area that has been, to date, disappointing for AR developers.   While Nokia will have a couple of speakers at ARE2010, and certainly they have a very impressive history in augmented reality research, they have not delivered the magic brew to attract the augmented reality developer community.   Will this change with the release of the N8 (see video opening this post)?  I wanted to find out if AR developers see a light at the end of the tunnel with N8 as a platform for AR.  So I asked some AR developers and entrepreneurs, “How attractive is the N8 for the AR community?”

Nokia’s apparent nonchalance about augmented reality is in sharp contrast to the bear hug shown in this Verizon Droid Augmented Reality Ad featuring Layar!  As Bruce Sterling notes,

*Hollywood-up for Layar by cooking up some o’ those big-time sci-fi production values.
All magic, all the time, Mr. Roboto.

And it is clear that everyone has high hopes that iPhone SDK 4 delivers fully for AR developers this time! For more on this listen to @chrisgrayson’s 3rd video on Mobile AR here.

But before I get back to my question about the AR community’s thoughts on N8 as a platform for augmented reality, let me crow just a little more about the upcoming awesomeness of Augmented Reality Event.  As the chair of the technology track, ARE2010 is on my mind night and day at the moment!

The tools for the kind of augmented reality we have always dreamed of  – the zero click interface to a heads up, hyperlocal view, bringing you computing anywhere, anytime, with anything, are evolving fast.  Powerful algorithms for search, and machine learning, combined with cooperating cloud data services, will soon be bringing apps that learn by context accumulation to your AR view (see Siri).  Google Goggles, who will have a speaker at ARE2010, have already announced their intent to open APIs, putting a vital tool of visual search in developer’s hands.

A bunch of technologies are maturing these days and helping bring AR to consumers.  Mobile devices that pack all the ingredients for AR, location-based services, see thru video goggles, visual search, innovations in 3D mapping, Simple Geo’s innovative a approach to aggregating geo data, and the next generation of “Street View” such as Earthmine, and, very importantly, open standards and software for AR that will bring many new opportunities for creativity, and monetization.   These are just some of the aspects of this energetic technology convergence that will be represented at  Augmented Reality Event.

In the area of open standards, I am especially excited about the debut of Polaris at Augmented Reality Event.  Polaris is an AR browser, from Blair MacIntyre and the Georgia Tech team, based as much as possible on existing standards and tech.  More on this and ARWave in another post!  Also, I am excited to be demoing ARWave at the Com.Geo 2010 conference, June 21 – 23, in Washington.  Again more on this later.

But for now back to my question opening this post:

“Are Nokia back in the AR?”  and “How attractive is the N8 to the AR community?”

First I asked, my co-chair on Augmented Reality Event, Ori Inbar of Ogmento – one of the augmented reality start ups who’s star is on the rise, and Ori (also known for his writing on Games Alfresco) is one of the key thought leaders in the emerging augmented reality industry.

Ori Inbar:

“Well, the hardware spec is impressive – but that’s not what makes it a good AR device. What’s really important is reach, distribution, and ease of development – areas in which the N8 still has a lot to prove.

- Will the N8 become as prevailing as the iPhone or lead Android phones or will the Nokia landscape remain fragemented with too many variants of phones to support?

- Will the Ovi store pick up steam and become a lucrative chanel for selling apps?

While we wait for these questions to be answered – we’ll keep developing games for Android and the iPhone “

So there you go.

Ori is right that augmented reality developers like the hardware specs.  Here are some comments from Patrick O’Shaughnessey of Patched Reality, Peter Meier, Metaio, and whurley of Chaotic Moon and Robert Rice, Mirascape.

Patrick O’Shaughnessey:

“Looks like it has everything one would want to make both looking glass and magic mirror AR, and built-in face recognition software to boot!  I wonder if they allow you to use both cameras at once making AR video conferencing possilbe?”

“Dedicated graphics processor with OpenGL 2.0 enables 3D graphics” nice!

“Looks like it also does Flash (Flash Lite 4.0, anyway). Not sure if Flash Light provides access to the camera. Could be a nice way to do quick and dirty POC’s on the device”

Peter Meier:

“Metaio thinks the N8 is a very promising device for AR. Especially in Europe we expect a lot of reach.  Symbian is hard to develop for, but on the other hand is much more efficient in terms of real time performance than Android and less restrictive on APIs than iPhone.”

whurley:

“It looks really great and perfect for ar in many regards.”

“clarity of a 12 MP camera with Carl Zeiss optics will give the Nokia N8 some clear advantages in the AR world.  The clarity of this camera will allow for more advanced applications of augmented reality such as image/object recognition”.

“At first glance this is definitely very impressive from a hardware specification. I am surprised though, at the lack of buzz about the device. I have heard virtually nothing about it in the usual sources I monitor, and I wonder about the marketing.

Given the technical superiority and the difficulty Nokia has had penetrating at least the North American market, maybe something catchier like the GR-8 (Nokia Great) or M8 (Nokia Mate) might be a bit more user friendly and easier to market. I particularly like “Nokia M8″…take it with you wherever you go, can’t live without it, etc.

Anyway, as awesome as the specifications are, I would like to echo some of the other comments here…tools, community, ease of development, etc. are critical parts of the ecosystem that would make this ideal for AR development, above and beyond the pure hardware specs. I think Nokia has to really re-think some of its strategy to engage consumers, attract the developers, and establish a new beachhead in North America.”

Talking with Michael Halbherr at Where 2.0

I talked to Michael Halbherr briefly at Where 2.0 before the release of details on the N8, so, of course, he was unable to be as explicit as he might have been about the N8.  Below is a lightly edited transcript.  At Where 2.0, Michael Halbherr  gave a clear explanation of  Nokia’s emphasis on a global strategy.   And, as this blow by blow account on the Google Earth Blog noted, “They’re doing some cool stuff with their handheld maps. In particular, their Ovi Maps (turn-by-turn navigation) are now completely free and becoming very popular.”

But the question remains:  Where is AR is on the Nokia roadmap?

Michael Halbherr (Nokia) on stage at the O’Reilly Where 2.0 Conference 2010 in San Jose, California. Photo by James Duncan Davidson. See Where 2.0 Flickr set here.

Tish Shute: Can I ask you a few questions about Augmented Reality?

Michael Halbherr: So for me augmented reality and mixed reality is essentially a way you search.  So when you look at where we come from, as the company that acquired Gate 5, we believe the phone is essentially the gate that basically combines or connects your virtual and your real world in both ways.  So we can have virtual overlays on top of reality, and you can also record and push it back.

But when you look at the way we do it, I think it is actually an extension of mapping, right?  So you will have the map, which is an abstraction of reality, and then basically with a flick you can go to real reality.  And I think to really understand what you see, you need to actually really know what you see, and an image doesn’t do that.  But if you have compass, and GPS, and 3D models, which is what Navteq is capturing, we can actual do line of sight calculations and you can really say, “OK.  He is looking at that building.” And you can click on the building because you have that understanding.  And you can combine that.  And that leads to a completely new way for how people interact with their environment.

So the here and now, and me here and now, and what next is a key innovation in our focus hereafter.

Tish Shute: I suppose for AR enthusiasts we were all disappointed when we saw the N900 with Maemo, which was very exciting as an operating system, didn’t have a compass.

Michael Halbherr: Yeah, but you know, that is just a matter of timing.  At the end of the day, Meego is our, I think, high-end push.  You will see this Mego device have what you need it to have.  So I wouldn’t worry too much about that.  We are building all the  phones, and everything has everything.

I think it is not just Meego.  It is the whole thing around it.  It is the QT, it is the web run time, it is the whole Meego system.

Tish Shute: Yes we have used QT for some development on the ARWave project, it is very good.

Michael Halbherr: Yeah, and the web run times, it’s open…So Nokia is about open and emotional.  Others are maybe open and more utilitarian or closed, and we are more open and more emotional.  And I think that is what we want to basically do with this platform.

And we are working very hard.  And so whatever I say has been built almost a year ago.  So there is a lot that we are building that we don’t talk about before we announce.  And we try to do this as a global offer, so we can’t launch a little service in the US, which maybe is a problem for us.  But we are basically building complete solutions for global markets.

Tish Shute: Yes.  And when I heard you talk just now on Nokia’s global strategy just now, it all fits in.  But from the point of view from someone who lives here in the US, it seems Nokia is disappointing us.

So is Nokia enabling augmented reality in any ways that noone else is?   I mean if you are interested in augmented reality, the backend and data is vital, e.g. the Navteq part.  And Apple has excited us with the amazing performance of their hardware, and they have taken steps like controlling the manufacture of their chips, powerful cpu and gpu is very important for augmented reality experiences too? Is there a need for a chip set geared to augmented reality specifically?

Michael Halbherr: I’m not sure you have to go that far.  I think at the end of the day it is the data, right?  The map data and the 3D data that is actually extremely important.

Tish Shute: Yeah, I agree with you that the data is vitally important.

Michael Halbherr: Because at the end of the day, when you have a real abstract view, it is really important, because the camera or the view is only a viewer, right?  And people don’t really know what they see.  It is just they plaster on it believing that the person sees what he sees.  So I think real augmented reality or mixed reality is really basically having a real picture, but having, actually, the understanding of the 3D logic of the picture.  And for that, you need to capture the data which Navteq is doing.

Tish Shute: The original dream of augmented reality is 3D media and graphics tightly registered to the physical world.  But actually, it is the data…being able to integrate the streams of data and sensor fusion techniques that will make this interesting and possible right?

Michael Halbherr: Again, it is overlaying virtual elements on real elements.  It is like you are taking your glasses and now you can see information attached.  And of course, you need to make sure it is not overloaded, and that everybody works with it, and it is precise.  So that is basically it.  So I think that is why we are investing a lot in the precision of the maps and the 3D elements of the maps.

At this point we had to leave our seats has the set was being broken down for the next event at Where 2.0.  Check out all the videos for Where 2.0 here,  it was a watershed event for all things hyperlocal, geo, and augmented!

I am excited about going to Where Week 2009 – Where 2.0 and WhereCamp, this week (for more see Brady Forrest’s post).  Where Week will be total immersion for five days in a think tank with creators of the information landscapes of the future.

As you know, if you have read my previous post – here, I think the “Mobile Reality” panel is a must.  And I have been looking forward to hearing more about The Shape of Alpha from Aaron Straup Cope, Flickr, since Etech 2009 when I was introduced to Aaron by Mike Kuniavsky (see my interview with Mike Kuniavsky at Etech here and more on Mike’s concept “information shadows” in his Etech talk).

Shape of Alpha is revealing some fascinating possibilities for mining geotagged Flickr images.

As Tim O’Reilly noted in a tweet, Aaron Straup Cope’s recent post, The Absence and the Anchor, describes, “some of the surprising things Flickr is learning about people from geotagged photos.” Aaron’s post also announces that the “donut hole shapes” are available for developers to use with their developer magic via the Flickr API.

“If the shapefiles themselves are uncharted territory, the donut holes are the fuzzy horizon even further off in the distance. We’re not really sure where this will take us but we’re pretty sure there’s something to it all so we’re eager to share it with people and see what they can make of it too.”

For more on shape files see Aaron’s blog post about “some experimental work that I’d been doing with the shapefile data we derive from geotagged photos.”

Creating the Information Landscapes of the Future

I have been thinking and writing a lot about augmented reality lately.  And key thought leaders in this space like Blair MacIntyre, Robert Rice (see my interview here), and Ori Inbar (see my interview here), have clued me in to how vital it is, for an ubiquitous experience, for us to find ways to allow people to fill in the stories that can be used for augmented reality.

As Ori noted in conclusion to our recent conversation:

“in order to have a ubiquitous experience like Robert Rice and others are striving for, you’ll need to 3d map the world. Google earth like apps are going to help but it is not going to be sufficient. So let’s leverage people. Google became successful in part by making people work with them.  Each time you create a link from your blog to my blog their search engines learn from it.  So let’s find ways to make people create information that can be used for AR.”

Jim Purbrick, another key thinker in this area (interview upcoming), also notes:

“you can imagine a crowd sourced set of hints for any location so, AR knows roughly where it is and can do photosynth style matching  to find out exactly what it’s looking at and get the extra data it needs about that thing (humans are really good image recognition systems, and are also pretty good at interfacing with networks) instead of marking up real objects with ids you take pictures of real objects, tag them and then search them based on images from your ar system.”

Ori Inbar suggested to me an idea that I really liked – the notion of bread crumbs where, “You don’t have a constant view of what is happening when you walk but you get images and text and all sorts of things from people who walked there before – like breadcrumbs.“  And as Design Under Sky points out about Shape of Alpha:

“The truly amazing part of this process is how the “community” has the authority to provide areas previously unmapped.  By uploading personal photos of areas not covered by mapping software, members have the power of further shrinking our world through greater visual access and understanding of locations one might not be willing or unable to visit.”

Aaron Straup Cope, Flickr, Todd E. Kurt, ThingM and Mike Kuniavsky, ThingM

The Locative Media Manifesto

@stamen’s tweet brought André Lemos’ brilliant, thought provoking, ” Locative Media Manifesto,” to my attention.  I am also looking forward to hearing about how old maps “can shed light on modern geography when placed in counterpoint to the state of art in modern maps from Google or Microsoft” from Michal Migurski, Stamen Design, who will present Flea Market Mapping at Where 2.0.

André Lemos writes:

“After uploading to Matrix up there – Internet 1.0 – now is the time to “download cyberspace,” information about things down here – Internet 2.0. We are not dealing with what is virtual up there, but of what to do with all this information about things and places down here! How can we relate to things and places? And now that these things and places are provided with digital information and Internet connections? Do we invoke Heidegger and Lefevbre?”

I will leave it to people smarter than I to invoke Heidegger and Lefevbre as Andre Lemos does so eloquently in Locative Media Manifesto. But by reminding us artists and activists created the term “locative media” to “question the mass use of LBS (location based services) and LBT (location based technologies,”  the manifesto delivers 30 principles to inspire creators of Locative Media and explorers of the, “current dimension of cyberculture, comprising the era of “cyberspace leaking into the real world” (Russel, 1999); an era of the “internet of things.”

I feel well primed for Where Week by my visit to the ITP Spring Show, 2009 last Sunday. It was an interaction riot, jam packed with brilliance and off beat explorations of locative media which I experienced through the senses of my 9 year old.  His pick for best of show is below. But he had many favorites and I have put some pictures up on my FLickr stream with links to the creator’s sites.  One of my favorite projects Alexander Reeder’s S Ring – “seducing people by talking with your hands,” is up on my Posterous blog.  You can see a list of the extensive media coverage the show got here.

Loose Interaction Topologies

The picture above is of a game of mud pong  in Tom Gerhardt’s Mud Tub.  The mud interface – “a smart tub with some mud” knows the topology of the mud and where your hand is. Mud Tub takes advantage of  a complex material – to explore loose interaction topologies, including as seen above a game of Mud Pong.  Loose interaction topologies are a way we can explore meaning in “the internet of things.”

Tom explained his own exploration of the internet of things to me very succinctly:

“I am not trying to make mud better. I am trying to make computers better with mud.”

He elaborates on the value of Mud Tub in this regard on his site, dirtycomputing:

“The Mud Tub occupies a space similar to other experimental human-computer interfaces, like, multi-touch surfaces, body controllers, augmented reality systems, etc, which push the boundaries of codified interaction models, and drive the development of innovative software applications. Beyond its role as a research topic, the Mud Tub also exists as an open-sourced hardware/software platform on which interactive artists and designers explore new methods for creating and displaying their work.”

Recently, I talked to Ori Inbar (above), formerly senior vice- president at SAP.  Ori is on a mission to make augmented reality commercially successful not in 5, 10, or 15 years, but now. Ori is the founder of Pookatak Games – a video game company, “with a vision to upgrade the way people experience the world.” Ori will be participating May 20th, in O’Reilly’s Where 2.0 panel, “Mobile Reality” -  an event not to be missed IMO.

The taste for computing anywhere anytime has entered human culture via the iphone and is spreading like chocolate cake and pizza at a preschool party (see why the iPhone changed everything).  And while the full flowering of the next step is yet to come – computing anywhere, anytime by anyone and anything (“the internet of things”), our love for these first devices capable of being mediating artifacts for ubiquitous computing (Adam Greenfield) is a vital first step to free us from our tethers to computer screens, and fulfill the promise of augmented reality.

If you need more convincing on the pivotal role augmented reality will play as the web moves into the world, check out Tim O’Reilly’s recent comments in this video clip posted on Augmented Times and here early last year.

From another perspective, the gloomy specter of economic and environmental catastrophe  is driving a movement to “infuse intelligence into the way the world work’s.” But the challenge for a smart planet is not just about making environments smart, it is about using smart environments to enable people to act smarter (see my interview with Adam Greenfield).

We need a rapid upgrade in both the way the world works, and the way we experience the world.

((Note:  It is time to read (if you haven’t already) Bruce Sterling’s Caryatids (Cory Doctorow’s book of the year for 2009) “as a software design manual” (see Julian Bleeker) because Caryatids reveals the Gordian knots of human folly, greed, compassion and desire entwined in near future designs for technologies to save the world.))

Ori Inbar, worked with Shai Agassi (Shai is now leading the world changing Better Place ) driving Netweaver from a mere concept to a “major, major business for SAP.” So Ori has already been through the cycle of working in a very small startup and growing it into a billion dollar business.  He has both the experience and the passion to realize his vision for augmented reality.

At Pookatak, he explains :

“We design “reality experiences” that make users’ immediate environments more significant to them. We wish to free young and old from getting lost in front of the screen. By delivering the world’s information to people’s field of view, and by weaving real world objects into interactive narratives, we help people rediscover the real world.”

Pookatak will release their first game this summer. Currently it is under wraps. But Ori gives us some glimpses of what is to come in the interview below.

In addition to founding Pookatak, Ori is involved in a broader effort to move augmented reality forward. On his blog, Games Alfresco – he recently welcomed a new partner, Rouli Nir, Ori has focused his eye of wisdom on every significant recent advance in Augmented Reality (check out this essence of Ori’s thinking in a fast paced video presentation for WARM ‘09).

Also Ori is one of the organizers of the interactive media track at ISMAR 2009.  At ISMAR this year, Ori explained, “we are trying to bring in people that develop interactive experiences for consumers, beyond the traditional attendees coming from a research perspective.”

In the interview below, Ori explains much of his thinking on how augmented reality will become commercially successful.  Enjoy it, think about it, and share it. And most importantly, if you can, get involved with ISMAR 2009.

Ori  has inspired me to participate in ISMAR this year.  Ori pointed out:

The call for papers is on, and this year it targets well beyond the typical research papers audience and into interactive media and art folks.

There are plenty of opportunities such as:

Art Gallery

Demonstrations

Tutorial

Workshops

It’s a huge opportunity to shape the emergence of augmented reality.

Interview With Ori Inbar

Making Augmented Reality Commercially Successful

Tish Shute: You are considered a key trail blazer in AR and you have the go to blog for augmented reality!  What are the most important lessons you have learned researching, writing, and developing AR in the last couple of years?

Ori Inbar: You need to have a vision. You need to know where this is going to go in ten or fifteen or twenty years. But you’ve got to start with something really simple that makes use of the technology you have on hand. And do something that is practical, that people will like, and something they would actually want to buy. Its as simple as that. I’m currently looking at what we could do with existing technology. First of all, you have to put it in front of people. Right now most people have never heard about the term augmented reality. Go into the street, and ask 100 people about it, maybe 2 would know about it. So you need to put it in front of people because most people think it’s still science fiction or a special effect you see in movies, not something you can experience in real life.

Tish: It seems to me to that for augmented reality applications to become popular with existing technology the key breakthrough would be getting people to hold up their phones. What are the obstacles to getting people to use their mobile devices like this?

Ori: There’s a really nice cartoon by Tonchidot (below) – the Japanese company behind the Sekai Camera. It’s an illustration showing the evolution of man, from ape to man (holding a cell phone looking down), to the developed man holding a device like a camera – in front of its eyes.

Which is exactly what you’re talking about. People ask, “are people going to walk with this like that all day long?” Probably not. I mean you have to build it in a way that doesn’t require them to hold it like that all the time. People are used to this gesture with the ubiquitous digital cameras. I tested one of my prototypes on a two and a half year old girl. She had no problem holding it just like she holds a camera.

Tish: Blair MacIntyre mentioned, “The problem with the mobile phone as a AR device is a problem of awareness,” i.e., you have to have a way of letting people know when there’s something interesting wherever they are. One of the issues regarding this is if you get too many alerts, then you tune them out.

Ori: First of all Blair is one of the people in academia that get it. Because he looks at it from an experience perspective. Not just as an interesting technical problem to solve. Let’s start with getting people to enjoy this new experience. The AR demos so far were mostly eye candies, and mostly for advertising – the GE AR ad created a lot of buzz; but you look at it for 10 seconds and you forget about it.  You need to build something that people would want to experience over time and would be willing to pay for. I think that’s the big test, right?

Now in terms of having a ubiquitous experience where you’re continously connected, it doesn’t have to be an overwhelming experience. Just like some of the social media tools we’re using today, we decide when to connect, and we filter out the trash. You could get alerts only for things that really matter to you, not for everything that happens in your immediate environment.

There will be many layers of information, and it’ll be up to you to pick the ones you want to experience. The real benefit is that you get the information in your own field of view and in context of where you are or what you do.

Tish: So what are you working on these days?

Ori: We are working on a little app that targets a very different audience than what you’d expect: pre schoolers. We think we can encourage them to get away from a PC or TV screen and learn something while playing – in the real world. You’ll hear more about it as soon as this summer. Nuff said.

But, it is a small application that will run on the iPhone. People ask how many pre-schoolers own iPhones? Well, their parents do.

Tish: Yes there are certainly many New York kids with iPhones – my kid now has my old iphone.  He has pretty much switched from playing games on his DS to the iPhone. I noticed in your WARM video you place a big emphasis on AR as something that will get kids away from screens and engaged with reality.  This is something parents will approve of!

Ori: Yes I saw something really interesting at my kids’ party one day; they were all sitting around the room – looking down at their own DS screens.  You could play the DS anywhere, but kids would usually play it on the sofa, looking at the screen, isolated from the world. With an iPhone and a camera, and the application we’re producing, reality becomes part of the game. Yes that makes it all of a sudden much more interesting for parents. Because kids are spending so much time in front of the screen, all of a sudden they’re something that will encourage them to interact with real objects, real things. Every parent I’ve talked to loves that idea.

Tish: Yes that is what is cool about the work of Kati London – I think I saw someone say this on Twitter, “Kati puts the computer in the game not the game in the computer.”

Ori: Yes, kids are spending more time in front of games and the computer because it’s more interesting. It captivates them with “game pleasures ” that tap into their brain’s dopamine circuitry – constantly seeking reward and satisfaction. So you’re not going to be able to tell them to go back to playing in reality without these pleasures. We have to study these mechanics from games and bring them into reality. It’s about programming real life; and augmented reality helps you achieve that.

Here’s an example: cause and effect; in a game when you do something you always get an immediate effect. You’re good, you get a reward. You’re not good, you get a cue to improve. In real life you do things and you could wait 2 or 3 years until you actually get feedback (if you’re lucky). Augmented Reality allows you to bring these mechanics into the real world. I think that’s going to help kids rediscover reality, in a new sense, which is what every parent is dreaming about.

Tish: I don’t know how much you can say about your app. But in regard to doing augmented reality on the iPhone.. there’s no compass. Is this a limitation?

Ori: True, no compass yet. But the camera gives you a lot of information that you can interact with. When you run the application, you see the world in front of you, and if the app can recognize real life objects – it can put virtual elements on top of it.

Tish: But not with any accuracy unless you’re using markers. Are you using markers?

Ori: We’re using natural feature recognition. It doesn’t have to be an ugly looking marker. It can be any image.

Tish: So you’re using image recognition. Are you working with one of these image recognition startup companies (list here )?

Ori: We’re working with one of those. What’s unique about it is it runs very nicely on any cell phone, and on the iPhone it works the best. For this first app, it doesn’t really matter where you are physically; the geolocation is not part of the experience.

Tish: For a truly engaging AR experience we will need more of a backend than is currently available?

Ori: I call the backend the cloud, where you have all this information and ways to access it from anywhere. Actually I think it’s become pretty mature today. If you look at the different elements required to enable an augmented reality experience to work, you have – first – the user whose always in the center. Then you have the lens. The lens can be an iPhone, or glasses, even a projector. The lens allows you to watch, sense and track information in the real world: people, places, things. Then in the backend you have the cloud where you store and retrieve information.

So if you look at the maturity of these different elements, I think the cloud is in pretty good shape. Because there’s so much information we’re collecting and storing. Anything from Google, Wikipedia, Facebook, all that kind of stuff, it’s a lot of useful information you can access from anywhere using APIs. And a lot of it is also starting to include geolocation information. Take Loopt or Google’s friends service that allows you to see where your friends are and what they’re doing. There’s tons of information out there and it’s pretty easy to access it. Now what do you do with it is the question?

Wikitude is such a simple and brilliant application and nobody thought about doing it until this guy from Salzburg did. It doesn’t have any sophisticated visual tracking. It knows your position and it’s simply looking at the angle you’re pointing to. Based on these parameters it brings information from Wikipedia that pertains to your field of view. So most of it was already there. It’s just a matter of connecting the pieces in an experience that is valuable for people.

Tish: It is the uptake of even a very simple technology that puts the magic in it.

Ori:  Yes, take Twitter. If you go to its homepage it looks like a very simple boring app but it is something that is both enjoyable and very useful to people.

Why you should participate in ISMAR 2009

Tish: I know that you are involved in organizing  ISMAR (picture above from Ori’s post on “ISMAR 2009: The World’s Best Augmented Reality Event…,“) and there is a call out for papers and for volunteers, can you tell me more about it?

Ori: Yes, we hope to have the first ISMAR where we practice what we have just discussed: let’s build on all the research invested so far and instead of thinking only about 5-10 years from now, let’s see what we can do today. So we are bringing people in from other disciplines – artists, interactive media developers and people from the entertainment industry.  The goal is to use the technology to make something interesting for people – again, something that people would buy, and making it commercially successful.  Many people either don’t know about ISMAR because in the past it was a pure engineering-orientated event and people  from a commercial perspective of AR weren’t attracted to it.  The Chair of the Event this year is based in Florida and he is going to bring in a lot of people from the entertainment industry such as Disney. I think this will transform this event into something more like SIGGRAPH – more of an industry event.  As one of the organizers of the interactive media track we are trying to bring in people that want to build applications for consumers.

Tish: In terms of AR applications what are the flagships today?

Ori: There are very few because it’s just the beginning. There’s one tiny studio in France called Int 13 . They’ve created maybe the first commercial game running on a mobile device using AR technology. It’s called Kweekies. It was one of the contenders for the Nokia Mobile innovation awards. They were one of the ten finalists, but they didn’t win it. It’s looks really cool. It’s somethng that runs on your desk, with a marker. Many AR folks say markers are the past, markers are ugly. But it’s still a cool experience. I think people will go for it.

Tish: Yes I think we will have to look to small companies that are free to think creatively to lead the way.  It seems many games companies are tied up pulling off huge big budget projects and enterprise is still catching up on how to use social media!

Ori: Yes, last year I was in the game development conference (GDC); there was no mention of augmented reality – not on the exhibition floor, none of the sessions, nobody talked about it. I was stunned. Then this year, there was a little a change. There were like three demos on the exhibition floor, Metaio, Vuzix and a Dutch company called Beyond Reality.  And then there was Blair’s talk, which was very very cool. The room was packed with people. And after the talk there were dozens of people lining up to talk with him about the topic. There was definitely interest, but still on the very edge. The video game industry is still a hit driven business and publishers spend upward of 20-30 million dollar to create the best AAA game possible. They just can’t take the risk. So it’s going to come from smaller companies, from outsiders coming in with a vision and understanding on how to put the AR pieces together to create a totally new experience.

Tish: But the basic tool set is there isn’t it?

Ori: I talked to some folks at the games developer conference, many folks with MMO background, and they have great ideas about AR. It’s great to see different people with different views on what’s needed first. “Joe the Programmer” had this idea of creating a small piece of hardware that you can put in every house and provide accurate geospatial information in your home. That could  open up many opportunities for AR experiences in homes.

Tish: Don’t you think we have enormous resources in terms of image databases that provide a great basis for augmented reality.  I was talking to Aaron Cope at ETech about The Shape of Alpha – Flickr’s vernacular mapping project using all the geotagged photos in Flickr. That is such cool project. Aaron will be speaking at Where 2.0 also.

Ori: Think of Google Earth. Google Earth leveraged communities to basically map all the major cities around the world into 3D models. And that is an essential step to be able to do augmented reality outdoors. Because if you had to model everything from scratch, it wouldn’t be realistic.

Augmented Reality and Becoming Greener.

Tish: I am really interested in how AR interfaces might be useful to some of the emerging energy identity/metering projects like AMEE and WATTZON because I think it is very important that people have very intuitive, immediate, and enjoyable ways to relate to energy data so they can make greener choices.

Ori: Back in the day I had an idea to build an Augmented Reality application to become greener. You look at things around your home with the camera and it  recognizes its green gas footprint and makes recommendations to reduce it.  I guess it was a bit too early to do that based on visual recognition alone…you’d need  additional sensors that would provide related information about what you are looking at.

Tish: Well as there is more interest in Green technology do you think we may see VC interest in some green AR projects now?

Ori: I talked to some of the investment folks, Angels as well as VC’s about AR and they had no clue what it is. There’s a need for a whole lot of education. And there are no proof points (as in successful investments in this domain), and counter to popular belief – they don’t like risk so much…

Tish: And consumer adoption must lead the way, right?

Ori: Just like with every emerging technology in history, people never bought the technology, they bought the content, the apps, the benefits that came on top of the technology. Whether it was VHS winning over Beta Max, or BluRay winning over HD. It’s always because of more/better content. Look at the video game console war: Xbox, and Nintendo did better than Sony just because they had more and better games. Even Windows was a success thanks to its applications. People bought it for the applications not the OS. The content is the first to drive demand.

Tish: One of the challenges to giving people new ways to relate to their energy consumption is that you can just have them looking at graphs of how bad they have been in the past you – that may make them feel bad but that doesn’t necessarily give them ways or motivation to change. There perhaps needs to be more immediate relationship to the data to facilitate change. I think the mantra for optimization of anything from energy usage to supply chains is timely, actionable data?

Ori: There are a lot of ideas about measuring information and displaying it to people. For example, the Prius hybrid car, one of its interesting features – which is kind of game like – is a constant display of your current fuel consumption. That alone changes how people drive because they try to beat the “Score” and as a result conserve more fuel. That model can be applied to our homes…

Tish: Yes that is something I am very interested in. I have been following several projects in this area – one of my favorites is the Arduino, Current Cost/Tweetawatt, Pachube integrations I saw at Homecamp.

You joined a start up with Shai Agassi which was bought out by SAP right? He has a brilliant approach with Better Place.

Ori:  I think what’s really unique about Better Place’s approach is that he doesn’t require people to change their behavior. People are still going to have their own cars. They’ll be able to drive as far as they want, and for the same (or lower cost). Its not necessarily about a new technology, electric cars have been around for a long time but there was no way people were going to be limited by the 50 or 70 mile range and Better Place is solving that problem. With its infrastructure of charging spots and battery switching stations, drivers are going to be able to drive anywhere. And it’ll be similar to having to stop once in a while to refuel your car. The price maybe even lower than what you pay today for your transportation needs – and you’ll stop generating green gas. It’s a clever way of taking technology to a whole new level without changing the behavior of people.

Tish: Better Place is a classic example of things as a service isn’t it?  It is basically a utility company.

Ori: It is similar to a phone carrier model.  You pay for a membership that gives you access to the car (equivalent to the phone) and electricity (equivalent to the phone line) for the same price of fuel cost today. And as bonus you get to save the world.

How the iphone changed the game for AR – and the iphone versus Android

Picture from Ori’s post, “GDC 2009: Why the iphone changed everything”

Ori: And back to AR, you have to take the same approach, because nobody’s wants to don those huge head mounted displays or backpacks. You have to take advantage of people’s current behavior: they already carry their iPhones or similar devices.

Tish: As we discussed, you just have to get people raising up their phones and looking through them when that is a useful thing to do. Both Wikitude and Nathan Freitas’s graffiti app were enough to get me interested in the evolutionary step of raising my phone! Nathan’s graffiti app is nice. You leave a marker for your graffiti so other people can find view/add their own – a nice primal experience like pissing on the lamp post to let your pack know where you’ve been.  Also the graffiti app taps into a long history of  NYC street culture around tagging and graffiti art (see my interview, “Is it OMG finally for Augmented Reality?”).

Ori: The app store has fundamentally changed the mobile gaming industry. Last year they were in shambles. There was no growth. Everybody was complaining, “we can’t handle it, there’s a million phones, and you have to test it on each phone. And carriers suck, they don’t care about sharing and promoting your content. Everything was bad. This year mobile gaming is the hottest thing. And it’s all because of the iPhone. It changed the game.

Tish: How do you think Android is going to get traction against the iphone?

Ori: Well the number one thing is the form factor – the iPhone is just much cooler than the G1. Its OK but it doesn’t have the same feel. People thought it was going to be easy to clone the iPhone but none of the attempts succeeded so far.

Tish: How much does it matter for AR not being able to runs things persistently in the background on the iphone?

Ori: Actually they have add a such a capability in OS 3.  You can now make use of a background service.

Tish: OS 3 will open up new possibilities for AR?

Ori: The access to the video API is still not public.  But there is a new Microsoft application – Microsoft Tag that makes use of that API which means it is probably OK to use it.

Tish: (I ask Ori for his card and he shows me how to read it with my iphone.) Oh nice you have an AR card, of course!

In Search of Pong for Augmented Reality

Tish: So how will AR begin to, as Blair’s friend put’s it, “facilitate a killer existence,” particularly as we are probably looking at some new and perhaps pricey hardware?

Ori: You could take the Better Place approach. We’re going to give you a great experience and we’ll include the devices as part of that experience for the same price. Let’s say you subscribe to an AR experience  which offers access to multiuser, support, and all the information you need wherever you go – exactly according to the vision. You pay for a subscription on a monthly basis and included in that cost we give you a better device that offers a  better AR experience. It’s following the phone carrier approach, but in a good way.

But first of all we do need our Pong! I was sitting with a couple of AR game enthusiasts at the GDC and we were asking ourselves, “how do we create the first pong for AR?”

Was Pong a multiplayer game? Not necessarily! Did it connect to the network? No! We have to create the first dot in a long line of dots that will bring us to our destination.

Tish: You haven’t seen a Pong yet have you?

Ori: Not yet. I mean there’s maybe a handful of games and apps out there, but I don’t think any of them is a Pong yet. Still, it’s getting closer.

Tish: Kati London is doing some very interesting work on bringing games into reality, isn’t she?

Ori: Yes, she works with Frank Lanz at Area/Code. He teaches at NYU and has designed games for the “Come Out and Play” festival here in Manhattan. And a lot of these games are actually low tech.

Tish: Yes I have a big alternate reality game blog brewing that I haven’t had time to write yet!

Ori: The city is the gameboard is their slogan. It’s going to be a great playground for AR games. The city becomes a theme park. The city could become an even bigger touristic attraction. People will come to the city to be part of these games. So you’re having thousands of people running around the city playing all sorts of games from laser-tag style to history adventures, to treasure hunts.

Composing Reality

Tish: So why haven’t you focused on one of these kinds of games with your company?

Ori: We have a couple of scenarios along these lines that we’re planning for 2010-11. But first focus on what’s possible today.

Tish: And what’s stopping you from doing those kind of games today?

Ori: Many things. The devices are not there yet, location services are not accurate enough, ubiquitous sensors are not  there yet.

Tish: You think alternate reality gaming needs more “ubiquity” than is currently available?

Ori: Not necessarily. People are doing alternate reality games with no “ubiquity” at all. But my interest is to add the visual aspect. I believe humans are mostly driven visually.

Jane McGonigal said in a talk at GDC, that AR would allow us to program reality, which is exactly how I look at it. Once you can recognize things, some of it with WiFi and RFID and all sorts of sensors. But visual sensors is always going to be the ultimate way to recognize things. And once you recognize things and know what they are, and can pull information about those things (or people and places) from the internet, you can program it (visually). You could program it to be fictional, like in a video game, or it could be programmed as non-fictional, like a documentary. And that allows you to do things that before were unimaginable.

Tish: But you can’t forget the visual, it is primary the connection to peoples’ primary sensory relationships.

Ori: Yes, it’s like you go to a grocery store and you pick your vegetables, a lot of it is by sight and by touch. And what if you could also see just by looking at it that it’s from a local store, and that it’s organic?

Tish: It goes beyond overlays really?

Ori: By the way, I don’t like the term ‘overlay’. I know that’s how it looks: you either overlay or superimpose, but I’m still searching for a better term. A term I prefer to use is “composing reality”. Just like painters, they use brushstrokes and colors and compose a painting. We need to take the real element and the virtual element and compose them into something new. It’s not just about slapping one on top of the other.

Tish: yes I think the idea of dashboards is not so appealing.

Pookatak Games

Tish: Do you want to explain the evolution of your company? You have an interesting history of success with high end enterprise applications.

Ori: Since I was a kid I wanted to invent and create things. When I discovered software, that was a really cool way of actually creating things from nothing. From thin air; and you can do it very quickly. That’s what brought me into software. But I was always looking for the intersection between technology and art. Looking for ways to bring these things together. In the early nineties virtual reality was doing it. It had the appeal of cutting edge technology that can be combined with art. But then, as we all know, it crashed. So I joined Shai Agassi’s startup (who is now doing Better Place) back in the early nineties. I was one of the first employees in his startup which was developing multimedia products. I was leading the development of one of its flagship product. At some point we realized the technology could be great for an enterprise environment.

It was a really great experience. First going through this cycle from a very small startup and growing into this multi billion dollar business. I was responsible for defining and marketing SAP’s platform, which was called Netweaver. It was just an idea when we joined SAP and by the time I left it was a major, major business for SAP. I learned about the challenges of building a platform. No matter what purpose you’re building it for, it typically has similar rules. It’s definitely not just about the technology; the content that comes with it is really key to making a platform successful.

The third part of this platform trifecta is the community. If you don’t build a community, you won’t get the critical mass required for adoption. It may be your own platform but it’s not necessarily the people’s platform. That experience is very key to what we’re doing today. Now, a new industry is being born on the basis of a remarkable technology. But to drive adoption, first we’ll need good content. The content will be created using today’s technology with internal tools developed to simplify the process. Next step would be to make the tools used internally – available to other developers. Help scale the industry, enable innovation on a larger scale. That way we have a chance to create a platform. So it isn’t really just about my company. I’m so passionate about augmented reality, I want to it to become a healthy and successful industry for the next 5, 10, 15 years.

Tish: Yes I am so ready to be liberated from the sitting behind a computing screen! And I know that all this hardware is murdering the environment.

Ori: There’s ‘s the book by Rolf Hainich which is called “The End Of Hardware. ” It’s about hardware for augmented-reality. Once you use goggles or other AR interfaces you eliminate the need for screens, laptops, etc. It’s going to be great for the environment. You have read Rainbow’s End, right? According to the book in few years there will barely be any (visible) hardware. At least it’ll have a much smaller footprint for the environment. And it’ll touch every aspect of life, everything you do. It’ll change the way you interact with the world.

The Illusive Eyewear for Immersive AR.

Friend of Ori’s in San Francisco wearing retro AR goggles (from Games Alfresco, Ori’s roundup of GDC 2009)

Tish:OK lets talk about goggles.

Ori: Goggles are going to happen, we want to be hands free.

It’s going to happen because it’s just a more intuitive way to use this technology. But above all it has to look cool. Because if it’s not, if it’s a big headset, then maybe a small percent of the population might use it, but most people won’t. It has to look like an accessory, like new cool eyeglasses that you just must wear.

I recently talked to a friend, who runs an industrial design firm, and has experience in designing such glasses for companies like Microvision and Lumux. He says that when you try to bring the images so close to our eyes – there are some really hard problems to solve. Otherwise it can become really annoying and cause dizzyness.

But I’m optimistic. I believe it’s going to happen 3 to 5 years from now. It’s already starting now: Vuzix announced goggles that will be available this year. Some AR apps that are going to take advantage of next year. Initially only a fraction of the population will use it. And that’s going to help advance it and make it better and better. But it’s going to take time until it reaches the mass market.

Tish: In virtual worlds we have seen, I think, a lot of mistakes in terms of reinventing the wheel and producing too many proprietary versions of the same thing and not enough concerted effort on standards and open platforms that could create a vibrant ecosystem.  How can augmented reality not make the same mistakes?

Ori: There are some early AR open source efforts ARTookit, ARtag but it is not a movement yet.  One of the things we’re trying to do at ISMAR this year is to put together  discussions around key industry issues, such as standards. Some people say it’s too early, you have to have a defacto standard to start from. But pretty soon it’s going to be too late. Just like with virtual worlds, all of a sudden you have all these islands that don’t talk to each other. Why get to that point if we can plan to avoid it? Let’s start thinking about it right now. On the other front there are devices. There are pockets of people working on adapting devices for AR, second guessing the hardware companies. Why not get them together with the Intels and Nvidias of the world, and discuss what this device should be able to do. And then compete to make it happen.

Tish: How much luck are you having with this discussion part?

Ori: People are very interested in doing this. We proposed these panels for ISMAR. And I’ve got some key people already on board. They have tons of input, they want to get involved. We’ll see how much we can actually get out of it.

Tish: In virtual worlds it was a while before vibrant opensource communities developed.  OpenSim has I think been the breakthrough community in this regard.

Ori: You have to think about the elements up front. The dream job is to architect the industry. Say we agree on the required pieces. Then we could help the right companies succeed in delivering the pieces. Next, we have to collaborate so that these pieces talk to each other. And eventually these communication methods will become defacto standards and most developers will adopt it.

Tish: So I’m going to put you in the role. You’ve got your dream job. You’re going to architect this community. So what are the key pieces and where would you like to see the open source communities take hold first?

Ori: Open source will not be exclusive. It’s going to live side by side with proprietary technology.

The key pieces? You have the user at the center. And the user interacts with a lens. The lens includes both the hardware and the software. And then the lens senses and interacts with the world, which includes people, things and places. And these people-things-places emit information – about who they are, where they are, what they’re doing, etc  – which is then stored in the cloud.

And then you have the content providers, the people and companies, composers who weave AR experiences through the pieces we mentioned before. These composers need a platform that glues these pieces together. Pieces of the platform will be on the lens, and in the world, and in the cloud. If you manage to remove the frictions, and connect these pieces into an experience that people like – then you have a platform. What the platform does it reduces the overhead and accelerates innovation.

Tish: Another problem virtual worlds faced in their development was their isolation from the world wide web.  Will augmented reality avoid this plight?

Ori:  Yes, I believe the key, like you said before, is not to reinvent the wheel. The cloud is already there.  Take Wikitude for example, all Mobilizy had to do is build  a relatively simple client app, connected to wikipedia, and all of a sudden it offered a wealth of information in your field of view.

I think we can learn a lot from web 2.0. For example, in order to have a ubiquitous experience like Robert Rice and others are striving for, you’ll need to 3d map the world. Google earth like apps are going to help but it is not going to be sufficient. So let’s leverage people. Google became successful in part by making people work with them.  Each time you create a link from your blog to my blog their search engines learn from it.  So let’s find ways to make people create information that can be used for AR.

Ori Inbar directed Wiki Mouse – a WIKI Film co-created by a swarm of movie makers around the world.