The Missing Manual for The Future (or The Future: The Missing Manual)

O’Reilly Media, is famous for is producing  “missing manuals” for new technologies, but thinking of O’Reilly as just a publisher of books would be like saying Facebook is just a website (this came up in the discussion at Media Round Table at Web 2.0 Expo, NY, 2010).   In recent weeks, I managed to catch Tim O’Reilly at several events, Maker Faire, Web 2.0 Expo, Hadoop World, and the free webcast Tim did with John Battelle on The Battle for the Internet Economy (although Tim spoke several other times during this period!).

It occurred to me, as I immersed myself in the depth and breadth of innovation showcased and discussed at these events that Tim O’Reilly, and the O’Reilly team, are creating, The Missing Manual for the Future.

As Tim puts it, we are “changing the world by spreading the knowledge of innovators.” Tim uses a quote from William Gibson to illuminate what is at the heart of the O’Reilly project:

“The Future is here, it is just not evenly distributed yet.” (William Gibson).

But Tim O’Reilly makes another point about the future when he speaks.  The future unfolds unexpectedly – so we must invent for an unknown future not a known future, or as Alex Steffen put it so well in his post, Why Our Bright Green Futures Will Be Weirder Than We Think, – “The world we need is one we’ve never yet seen.” The magic of attending an O”Reilly event is that it gives you a chance to work on this koan in interesting ways, and to take more responsibility for how things turn out.

Tim O’Reilly also urges that we think more deeply about what we are doing.  His keynote for Hadoop World , NYC, billed as, “The Business of Data” turned towards “The Consequences of Living in a World of Data.” The 900 strong crowd at Hadoop World was probably one of the most savvy crowds in the world about the business of data, so this was a nice turn.

Web 2.0 Expo with the theme, Platforms for Growth, was a deep dive into the business of innovation.  Tim O’Reilly’s keynote at Web 2.0 Expo,  “Thinking Hard About The Future” (or rather “thinking a little bit creatively or differently about the future)  – see video here, developed the call he made at Web 2.0 Expo 2008, to “work on stuff that matters,” into a Four Cylinder Engine for Innovation.   The first of the four cylinders in the firing order is, “Having Fun!” But,  at Maker Faire, Web 2.0 Expo, and Hadoop World I got an inside look at the workings of all four cylinders, and there is more to come, I am sure, as the other O’Reilly events unfold over the coming months including,  Web 2.0 Summit, Strata (a new O’Reilly conference on The Business of Data), and Where 2.0, 2011.

In a free webcast, last week (recording here), previewing Web 2.0 Summit, John Battelle and Tim O’Reilly discussed the Points of Control Map which is developing into a fun and useful tool to examine a very serious topic, “The Battle for the Internet Economy,” and how the “increasingly direct conflicts between its major players” could effect “people, government and the future of technology innovation.”   In my previous post, Platforms for Growth and Points of Control for Augmented Reality, I had a great conversation with Chris Arkenberg using this map as a springboard.  More on Points of Control later in this post.

The Four Cylinders of Innovation

click to enlarge

From Jet Ponies to Jet Packs: The First Cylinder of Innovation – “Have Fun”

The “maker” energy and its spirit of play, and the courage to create, hack, reinvent and re-purpose everything and anything, is a quintessential example of the first cylinder of innovation firing big.  Many “maker” projects also go on to fire on all four cylinders.   But the Maker forte definitely is in the first cylinder zone (and safety third as some of the rides, including Jet Ponies, warned).  The photo opening this post by Marc de Vinck – for more pics see here, is of Tim riding The Jet Ponies at Maker Faire which took  the New York Hall of Science by storm in late September – see The New York Times coverage here.  The ride was “built by the dastardly danger-hackers at the Madagascar Institute.“ See this wonderful interview with Hackett on his work to design “our specific jets from a patent that was filed in 1960s by a Mr. Lockwood, for Valveless Pulse Jets.” Hackett points out:

“Louder than god, glowing white-hot and looking like the trombone of the Apocalypse, pulse jets are also really shitty, inefficient engines,”

But, he adds:

“I have always wanted a jetpack, and one of the reasons I learned to build these things was to further that goal.”

This grand vision behind the Jet Ponies is a key to firing, The Second Cylinder of Innovation,  “Hey, we can change the world!”

But Jet Ponies, as a stepping stone to jet packs, also really struck a chord for me as I have been devoting a lot of time lately to the emerging Augmented Reality industry, a technology which was lumped in the same category of sci fi chimera as jet packs until very recently.

Data is the Gasoline

“The faces are coming from the sky.  The locations are coming from the sky.  All these apps depend on something, somewhere up.  And that, to me, was always the heart of Web 2.0.  And I am so delighted that people are finally getting it.  Because for a long time, people thought, ‘Oh, Web 2.0, it’s about lightweight advertising supported in a web start up.’  So I went, ‘No, no, no. It’s about the fact that we’re building these giant database subsystems in the sky that are going to drive applications.’  And now, of course, the same application is on your PC, it’s on your phone, it’s on you iPad.  And clearly, the applications are just sort of an interface to something that is being driven from the cloud, and that is fabulous.  That’s the difference.  People get it now.” (Tim O’Reilly, said this as part of a response to the first questioner at the Media Round table Web 2.0 Expo)

Answering questions about the importance of “Having Fun” to innovation doesn’t look quite as fun as riding Jet Ponies! Photo above from luca.sartoni’s Flickr stream

“ the data that is generated by the sensors and the applications that use that data is going to be where people are going to be innovative.” (Tim O’Reilly)

During the Media Round Table, I had a chance to ask Tim more about the role of bottom up innovation in a world where big data is the gasoline for increasingly sophisticated engines – platforms integrating machine to machine intelligence and real time analytics.

Tish Shute: You brought up Maker Faire in your keynote, and again now.  I was there, which not many people in the audience were  [not too many hands went up when Tim asked during his keynote].  But I think one of the things that struck me was the jet ponies – they were just earthshaking to stand near.  They made the ground tremble; they made the world shake.  Yet, most of your keynote, and most of what’s on our minds here, at Web 2.0 Expo, is extracting intelligence from the big data [in the sky], and algorithmic intelligences are the jet engines of the internet.  And of course, not to be forgotten, as we are here in New York City, where the trading markets are creating the air we breathe  [although we probably don't realize it until we lose our mortgage or something] and these algorithmic economies or “robot casinos” as Kevin Slavin put it, are all about speed – it’s not just real-time, issues of latency are  so critical that co-location is key to winning the game of the markets.  [Kevin Slavin brilliantly unpacks this in his talk, "Loitering on the Motherboard." For more in this see my conversation with Kevin Slavin below].

So my question is, who’s making the jet ponies for the algorithmic economies in the sky that you just described?   How can we make a play from the bottom up?  I always feel Ushahidi is one of the jet ponies of the data algorithmic space [because of their great work to bring human and machine intelligence together to solve problems in crisis situations].  But who do you think is doing exciting work and how can we ensure that this powerful world of data and algorithmic intelligences does not become hidden in a closed black box [only really accessible to elite players like the NYC trading markets]?

Tim O’Reilly: “Well, I think there’s certainly a lot of interesting things happening in, say, the financial services that a lot of, kind of, the Internet folks are kind of blind to.  I think that there are companies like Next Jump which are really good with data and good with algorithms. But kind of speaking specifically to the maker side of this, that whole sensor enabled world which is going to produce data is in its infancy.  What we have that I think is so powerful right now is we have the first portable sensor platform.  I said in my talk the other day, you know, your phone has ears, it has eyes, it has a sense of where it is.  And these are all available to application developers. You know, you can compare, say, Dodgeball to Foursquare, you can see how different… Dodgeball is Foursquare in the tele-type era.  Foursquare is now possible because there are so many more capabilities on the phone.

And I think that we are going to see a lot of other areas that are revolutionized by the sensors in the device.  It could well be that some of them will come explicitly out of the maker kind of projects, or it could just be that make is sort of a proxy for them.  So yeah, Arduino is this great maker sensor platform, but hey, here’s a consumer sensor platform [holding up phone]. Maybe we vaulted past the maker stage already and we just didn’t know it.

And that’s not entirely true, because Arduino is building a whole economy of special purpose devices.  But it feels a little bit like the days when people rolling their own PCs coexisted with the rise of Dell, who was a kid in his college dorm room who made his own PCs and sold them on the net, but figured out how to scale it pretty quickly and get good at it. But there were still a lot of garage shops, you know, ‘I’ll make a PC and sell it to you’ people for probably a decade before there was really a clue that that was a commodity industry.  In fact, I do think the sensor platforms are going to become a commodity industry.  And the data that is generated by the sensors and the applications that use that data is going to be where people are going to be innovative.”

The internet operating system is a data operating system and it is happening in real time (Tim O’Reilly)

click to enlarge the image above  – a slide from Mike Olsen’s  (CEO of Cloudera) keynote at Hadoop World

Not only do we have a portable sensor platform in our pockets  but developers also have powerful platforms and tools to make sense of data that fuel our apps.   Opensource Hadoop makes available, to anyone with some data munching chops, the power to work with giant unstructured databases and do the kind of real time analytics previously only available to giants like Google.  Big players like Yahoo, Facebook, and Twitter use Hadoop (Jonathon Gray from Facebook noted they add 10TB a day).   But, as this great roundup of Hadoop World points out, while Hadoop gets the press for handling petabytes of data , Mike Olsen (CEO of Cloudera) noted, the fastest growing area of users are working with clusters smaller than 10TB and over half of the Hadoop clusters were under 10TB in size.

Four Square: A Platform for Growth with an ecosytem built on top of data that exists in the real world

As an augmented reality enthusiast it is not hard to guess that one of my favorite platforms for growth is Four Square.  See Dennis Crowley’s keynote at Web 2.0 Expo here.  The Four Square API has been available to developers since November 2009,  and there are already a number of  interesting applications, and there will be many more to come.  The screen shot above is of geopollster – see the gallery of Four Square apps here.

@dens tweeted recently  “Politics + @Foursquare = @GeoPollster” http://geopollster.com <- I love love love that people are using 4SQ to think about election tools

As Kati London pointed out in her keynote, Four Square is the “kind of augmented reality that is aimed at shifting or changing a person’s social reality, e.g. the mayor badges in Four Square that change my relationship to the people and the place I am in, and augment engagement and reputation through socially driven consumer tie ins.” We are already see augmented reality developers beginning to work with the Four Square API – see here, Foursquare + Augmented Reality + Virtual Graffiti = ARstreets.

As augmented reality development tools mature, Four Square will, increasingly, become an important platform for creative AR developers interested in integrating the power of this platform for augmented engagement and reputation with “device aided augmented reality that can shift visual experiences of situated geolocal experiences.” With the Qualcomm vision based augmented reality SDK now available for download, and Wave in a Box soon? to be released, and an ARWave client working on Android (almost!), I have been exploring the Four Square API in my non existent spare time!!

The Four Square API also offers some interesting possibilities for exploring games that take the complex economy of Four Square – not personal data but aggregates of behavior, as their subject matter (for more on this see my conversation with Kevin Slavin later in this post and in an upcoming post).

I took this picture of Dennis at Where 2.0, 2009 at the beginning of Four Square’s phenomenal growth (they are at 4 million plus users now).

Pachube (Patch-Bay): a web service for storing and sharing sensor, energy and environmental data

Eighteen months ago, I interviewed Usman Haque (architect and director, Haque Design + Research) and founder of Pachube – see Pachube, Patching the Planet.   Usman pointed me to this wonderful evocative image from T.R. Oke’s book, “Boundary Layer Climates” (original photo source Prof. L. E. Mount’s The Climatic Physiology of the Pig).  “It’s the same piglets, in the same box, but on the right hand side the temperature has been increased. This small change in how the space is “programmed” has dramatically changed the way the ‘inhabitants’ relate to each other and how they relate to their space.”

The Challenge of Connecting people and environments.

At Web 2.0 Expo, I got the opportunity to talk with Usman Haque again.   Pachube, is becoming an established platform now, Usman explained.  They have a development team of eleven and robust back end.  And, they will now be spending some more time on the front end, including a redesign of the website, making “it a lot easier to widgetize the entire website so that you will be able to take almost any element and embed that into your own website.” And, as Usman mentioned in his presentation, they are working on an augmented reality interface, Porthole, for facilities management and, “as a consumer-oriented application that extends the universe of Pachube data into the context of AR – a ‘porthole’ into Pachube’s data environments..  Usman is also contributing to the AR standards discussion and on the program committee now for the W3C group on augmented reality.

Click to enlarge the image above from Chris Burman’s paper for the W3C, Portholes and Plumbing: how AR erases boundaries between “physical” and “virtual”

Pachube, is sometimes described as the Facebook for Data or an analogy Usman prefers, a Twitter for Sensors.  At Web 2.0 Expo, I had an amazing opportunity to hear from Twitter and Facebook about their strategies as platforms for growth.  This gave me lots of fuel for questions about Pachube’s approach to developing their platform.  Simplicity was a theme that Facebook  and Twitter both affirmed as a key.  One of Pachube’s challenges will be to deliver ease of use, and the equivalent of Facebook’s “like” and  Twitter’s “follow” to gain mass appeal.

Here is a brief excerpt from my upcoming conversation with Usman:

Tish Shute: So as a platform you see Pachube as having more in common with Twitter – a Twitter for Sensors. In what ways is Pachube similar to Twitter?

Usman Haque: Well we are the Twitter of sensors, devices & machines in the sense that, really, the API that enables all this communication is important, much more so than the website itself. It is where, basically, most of the millions of our hits actually go, is to the backend. And we’ve now got dozens of applications built on top of the system, a little bit like Twitter’s applications; you know, all the apps are the important part.

But we are actually going to be doing some quite exciting things with API keys that we haven’t really spoken that much about in public. But we have come up with a pretty innovative solution to make almost every resource have granular privacy options on it, now discussed here.

At Hadoop World, Tim O’Reilly also raised some interesting broader questions that are very relevant to Pachube’s vision to “patch the planet”, e.g, the problem of digital identity in the age of sensors? (Smart phones already know their users by the way they walk!) And, “How should we think about privacy in a world where data can be triangulated?”

Usman talked about Pachube’s approach to both the technical aspects of how to build a massively scalable system, and the conceptual aspects of how people connect to each other, and what they might do with these new opportunities to connect environments and sensor data  (see my earlier talk with Usman, Pachube, Patching the Planet, for a detailed explanation of some of the concepts behind Pachube).

I look forward to posting this conversation. Pachube is growing, and Usman always goes beyond the familiar tropes of connecting human and machine intelligence.

2nd Cylinder of Innovation: “Hey Can We Change the World!”

The possibilities for reimagining of the role of data in healthcare produced some of the most powerful “Hey Can We Change the World” moments for me at both Web 2.0 Expo and Hadoop World.  The slide above is from Esther Dyson’s brilliant Ignite presentation, “What you can and can’t learn from your genes?” are here,   Tim O’Reilly also brought up the powerful role real time data analytics can play in improving healthcare in his Hadoop World Keynote.  Also see Alex Howard’s post, 10 Lessons for Gov 2.0 from Web 2.0 for some more great, “hey we can change the world moments” at Web 2.0 Expo.  The keynote from Lukas Biewald of CrowdFlower and Leila Chirayath Janah of Samasource (screen shot below) in particular, is a provocative exploration of the future of work in the new ecologies of human and machine intelligence.

Changing the World When Our Lives Are Increasingly Shaped by Forces Invisible To Us?

Click to enlarge

Mike Olsen, CEO of Cloudera, noted that “the largest area of data growth does not come from humans interacting with machines; rather, it’s from machines interacting with each other” (see here in Minor Technical Difficulties).   One of the most interesting presentations at Web 2.0 Expo was Kevin Slavin’s, “Loitering on the Motherboard,” which, as Tim O’Reilly pointed out in his keynote at Hadoop World, is a talk that raises all kinds of questions about a system where big players are gaming the data for their own ends.

Kevin Slavin, a founder of Area/Code, notes the operating system of our mortgage, life insurance, the operating system of currencies and gold is now governed by machine to machine intelligence and algorithimic economies outside of human cognitive processes.  The markets are now legible only to bots in an algorithmic arms race with bots surveilling bots, and throwing off false information in a bid for counter-surveillance.  He showed some slides of the eery but beautiful visualizations of traces of the trading bots created from the Nanex API.

The screenshot above is from the Nanex: Crop Circle of the Day – Quote Stuffing and Strange Sequences.  “The common theme with the charts shown on this page is they are all generated in code and are algorithmic. Some demonstrate bizarre price or size cycling, some demonstrate large burst of quotes in extremely short time frames and some will demonstrate both…” This one is a zoom of the NSDQ “Wild Thing.”  Wild price/size repeater from NSDQ running at 1,000 quotes per second, effecting the BBO along the way (I love the great names Nanex gives the different patterns and traces produced by the trading bots).

Nanex supplies a real-time data feed comprising trade and quote data for all US equity, option, and futures exchanges. They have archived this data since 2004 and have created and used numerous tools to “sift through the enormous dataset: approximately 2.5 trillion quotes and trades as of June 2010.” May 6th 2010 (day of the flash crash), had approximately 7.6 billion trade, quote, level 2, and depth records.

Kevin points out that our lives are being shaped by criteria invisible to us and the old hackneyed tropes of machine to machine intelligence such a robots reading HUDs in English are long worn out.  The latter point is, perhaps, something for us augmented reality geeks absorbed in ideas of “making the invisible visible” to chew on.

Changing a world shaped by forces that are, increasingly, invisible to us presents a huge challenge.

But I had the glimmer of a, “Hey Can We Change the World” moment, when I attended Kevin Slavin founder of Area/Code’s presentation and had a conversation with him after his talk.  Could games take these complex economies as their subject matter?  The economies of  Farmville and games like WoW are not opaque at all, and these are environments with complex economic behavior, “where you can actually have enough data to understand what it is” – “it’s not so much about personal data.  It’s more about, like, aggregate behaviors.” “Games that can really model those, and play with those, and take those as the subject the way that Monopoly takes Monopoly as a subject could be really interesting.” Kevin made many fascinating points – more to come on this topic.

Photo by James Duncan Davidson, of Kevin Slavin speaking at Web 2.0 Expo NY, 2010, from the O’Reilly Conferences Flickr stream

Here is the beginning of our conversation:

Talking With Kevin Slavin

Tish Shute: You began your talk today about visibility and where some of the algorithmic masters of disguise went to work, after they had solved the math behind stealth bombers.  I thought perhaps you were leading into ideas about a reverse surveillance society.

But you surprised me, as I felt you made visibility itself kind of a non-issue by the end of your presentation and that counter surveillance became basically a time and speed issue.  Now I am not sure quite how to imagine a counter-surveillance society, something I try to think about…

Kevin Slavin: Well, let’s see.  There’s a couple ways to think about it.  I think one point is just that when we talk about counter-surveillance, we usually locate that as something that comes from  the bottom up, something that comes from the population. Think about the way the plane spotters discovered the CIA black rendition flights.

I think in general, when people talk about counter surveillance, or sousveillance, they imagine it as an inversion of the traditional relationship between the people and the state.

But that’s what’s interesting. What’s happening now, is that there are forms of surveillance and counter-surveillance that are in play beyond any human perceptual horizons. These forms are at their most sophisticated in financial services, in the markets.

If you were a bot, and could read the market legibly (which humans cannot), what you would see, effectively, are bots that are surveilling bots. Then you have bots that are throwing off false information in a bid for counter-surveillance. Many of the bots are, themselves, surveilling other bots; each one of them is trying to figure out what all the other ones are going to do. In essence, it’s an algorithmic arms race, and game theory has become concrete, since the theories are code, the code is action, and the action affects, let’s say: your mortgage.

And so, basically what you have is you have this series of algorithms that are all looking to discern each other, while also trying to prevent themselves from being discerned. I think of the tunnels under the trenches in WWI, tunnels to surveil the trenches, and then, later, tunnels to surveil the tunnels. There’s a few examples of this kind of thing.  But It’s especially strange when it’s computer code, and at the magnitude we’re seeing today.

All of it, as noted in the talk, accounting for 70% of all the trades in the market. 70% of the market trades are never touched by human hands or even seen by human eyes; they don’t move through a conventional cognitive process.  And that’s why you get things like the Credit Suisse algorithm, it was buying, selling 200,000 shares of stocks to itself over and over and over again. It was a bug and it slowed the market to a crawl.

Credit Suisse was fined, in essence, for failing to control an algorithm. Maybe that’s the first time an algorithm was treated like a human, in a way. As if the algorithm broke the law, and Credit Suisse was responsible for letting it do so. For me, that feels like a threshold event.

It’s not that humans never made mistakes when trading on the market. But when algorithms err, they err with magnitude.

The idea that we now have bugs in the United States market economy is really worth looking at.  If Apple can’t keep code bugs from the most simple iPhone apps in a closed and regulated ecosystem, I’m pretty certain we’ll have a lot more Credit Suisse type bugs in the future.

And that will be pretty interesting. There will be viruses, and the operating system they will operate on will be the operating system of the United States. The operating system of your pension, your house, your life insurance. The operating system of currencies and gold.

Tish Shute: I was hard-pressed by the end of your talk to think of like, “Well, what would be the equivalent of, sort of a people’s uprising to create a better fairer society in this kind of world where, really, the things that affect the key aspects of lives most are going on beyond human perception at an algorithmic level?”  But you made a pretty radical suggestion at the end…

Kevin Slavin: Well I think increasingly the markets have become delaminated from anything meaningful. First from goods, then from fundamentals, and now finally from homo sapiens. So that’s hard to fight.

It’s the race towards abstraction that makes it impossible to simply “resist.” The latest version in the long series of fiscal catastrophes was based on Wall Street finding goods that could be rolled up and sold with false valuations, but goods that would take a long time to fail. Mortgages are handy like that. It’s the tradition of extending the abstraction as long as possible, until finally the bill arrives and the banks fail. I don’t know if that’s something to rise up against or not. It’s like a rally against evil.

But really, I think the point is that it won’t be the people that rise up. It will be the financial services themselves that rise up. They’ll just detach completely.

That was harder to do with cotton or with wheat, with simple futures; they keep financial services tied to the ground.  So what we’re doing is creating increasingly complex financial instruments that are further and further removed from anything you can touch.  Like the way a mortgage is abstract. But, of course, the bottom line is that at the end of that mortgage lies someone’s home.

It’s said that Wall Street is now moving onto life insurance, because that’s going to take even longer to fail.  They’re doing the exact same thing. The word is that they are rolling up CDOs made out of crap life insurance policies, same way they rolled them up with crap mortgages a few years ago.

And those will probably take, I don’t know, 15 or 20 years to unwrap and unravel.  But what you see in the meantime, is that they are looking for things that are increasingly abstract, intangible, removed as far as possible from the experience of everyday life.

So maybe this is good. Maybe that’s financial services rising up. Lifting off. I think best case scenario now is that they actually leave humans alone altogether.  That, someday, they are just trading, effectively, completely arbitrary goods, the stocks could be anything at all, maybe for crops that no longer exist, and I’m just saying that then these bots would no longer affect what we do and what we are, it would just be a robot casino, an invisible paradise in the air.

People are the platform: How Games Can Be Engines of Innovation in Our Lives

See the video of Games that Know Where We Live here (screen shot above)

Kati London, Senior Producer, Area/Code, in her keynote showed how games that know where we live can shift players perspectives – from device aided augmented reality that can shift visual experiences of situated geolocal experiences to a kind of augmented reality that is aimed at shifting or changing a person’s social reality, e.g. the mayor badges in Four Square that change my relationship to the people and the place I am in, and augment engagement and reputation through socially driven consumer tie ins.

Area/Code has recently developed two games for the Knight Foundation that take people as the platform.  Macon Money, uses very simple games dynamics (for more see the video of Kati’s keynote) in a game designed to help “Knight’s continuing efforts to support revitalizing Macon and creating a vibrant college town.”

The other game that Area/Code has designed with the support of the Knight Foundation  is for the Biloxi and Gulf Coast community, a game called Battlestorm.  “The game’s purpose is to increase awareness about natural disasters and change the way people prepare for them.”

3rd Cylinder of Innovation: Build products, business models and entire industries.

Glympse – real-time, private location tracking

Julianne Pepitone, Yahoo! Finance, nailed the essence of Web 2.0 Expo, NYC, this year in her post, Web 2.0 Expo startups are big on neighborhoods, storytelling.  She writes:

“At the Web 2.0 Expo in New York City this week, executives from big sites like Facebook, Twitter and Pandora all spoke about industry trends. But the showcase of 27 startup tech companies stole the show.”

Listen carefully to Tim O’Reilly and Fred Wilson, Union Square Ventures, question their picks from the startup showcase at Web 2.0 Expo.  Also see this video of Fred and Tim discussing their conversations with all the start ups.  This  is one of the clearest public windows onto both how to present your company to VC, and how to figure out what are the most important questions for you as an entrepreneur  building a business in a world of data.

Glympse successfully pitches their “jet pony” strategy for a location based business, and is Fred’s pick.  They hold up well under pressure and answer Tim and Fred’s hard questions about how their start up will not get overtaken by an encumbent player with resources and market share before they can gain traction.   food52 responds to Tim’s probing about their strategy for business data analytics that he points out are vital if they want to survive with the small margins of ecommerce.  There is a list of all the participants in the start up showcase in Brady’s post here. hour.ly was the audience pick.

Shazam for Faces!

My favorite start up was a biometric service doing face, iris, and finger print matching, Tactical Information Systems.

Tim and Fred also liked them, and they have an interesting discussion about the merits or not of approaching your platform through a narrow first application as Tactical Information Systems are with WanderID -  an application to help identifying lost Alzheimer patients.  As Fred pointed out, they are potentially the Shazam for faces, so why start so small?

I  had asked TIS the same question when I met them in the “speed dating” session.  This is just their first toe in the water as they are a two person company at the moment. Their vision for their platform is big.  Mary Haskett and Dr Alex Kilpatrick, the founders of this quintessential jet pony for the algorithmic economies in the sky, are not only a partnership with the credentials to do a  Shazam for faces – see their bios here, they are the people I would want to be running a Shazam for faces!  They really get the consequences of living in a world of data – check out Dr Kilpatrick’s absolute killer Ignite talk, “Defeating Big Brother.” (screenshot below)

How Can Augmented Reality Add Value to the Real Time Internet/Data Operating System?

planefinder.net – an augmented reality app that lets you find information about planes by pointing your phone at the sky, “including flight number, aircraft registration, speed, altitude and how far away it is” (via MacLife).

The new opportunities in the algorithmic economies in the sky were center stage at Web 2.0 Expo and there are some interesting AR apps for the real time internet/data operating system emerging, like planefinder.net.  But Augmented Reality was still pretty low profile at Web 2.0 Expo (except that NVidia augmented reality demo attracted a lot of attention at the sponsors expo).  However, everyone working in the emerging industry of AR should recognize that apps big on “neighborhoods and story telling” are heading right up the AR street, and that platforms like Four Square and Pachube present enormous opportunity to explore the possibilities of AR.  And if augmented reality enthusiasts are not already paying attention to real time data analytics, and Hadoop, they should be (see this post for an excellent round up on Hadoop World).

At Hadoop World, Tim O’Reilly referenced the great tagline from the  IBM commercial:

“Would you be willing to cross the street — blindfolded — on data that was five minutes old? Five hours? Five days?”

As I have noted in several earlier posts – see here and here for starters,  we are just seeing the tools  for developing near field, vision based, mobile, social AR become widely available to developers, so there should be a new level of AR apps emerging through 2011.  There is a wonderful discussion in the comments of this post by Mac Slocum, “How Augmented Reality Apps Can Catch On,” between Mac, Raimo one of the founders of Layar, and Chris Arkenberg on what constitutes a platform for growth for augmented reality.

Mac’s post, the comments and Chris Arkenberg’s post on the latest edition of the Gartner Hype Cycle, that rather curiously placed Augmented reality almost at the peak of inflated expectations. really got me excited about exploring an idea I have been thinking about for a while, which is to get the AR community to discuss the Points of Control map.    See my discussion with Chris Arkenberg here, Platforms for Growth and Points of Control for Augmented Reality. The recording of  John Battelle’s and Tim O’Reilly’s webcast on Points of Control is posted here.

The interactive Points of Control map is an amazing tool to think with! Check it out in movements, territory and movements, acquisition mode.  There is a competition for the most interesting comment and most interesting acquisition suggestion.  The prize is a ticket to Web 2.0 Summit!

What is the Future of Social?

The recent “defection” from Google to Facebook – see Lars Rasmussen, Father Of Google Maps And Google Wave, Heads To Facebook,  is as MG Siegler of TechCrunch points out, “the biggest one since Chrome OS lead Matthew Papakipos made the same jump in June” (TechCrunch also notes “current Facebook CTO Bret Taylor was heavily involved in the launch of Google Maps”).

These moves have drawn my particular attention as did Bret Taylor’s response in his conversation with Brady Forrest at Web 2.0 Expo to Brady’s question, “How soon until we get the Facebook firehose?”

If you have been reading Ugotrade you already know how important I think an open, distributed, standard for real-time communications such as the very innovative Wave Federation Protocol could be for AR development  -  see ARWave and my presentation at MoMo13, Amsterdam last year, The Next Wave of AR: Mobile Social Interaction Right Here, Right Now!

The anticipated release of  Wave in a Box, has raised hopes in the developer community that  WFP will soon become easier to work with, and hopefully more widely adopted.  Like many others, I wonder what will happen to Wave in a Box now?

But the innovation of Wave is deep and broad (and as many have pointed out hugely ambitious).  Perhaps the boldest attempt yet to innovate both at the low level of architecture (where Google is so powerful) and at the high level of the Mark Zuckerberg, “big idea,” which as Tim O’Reilly notes is, “What is the future of social?” MG Siegler noted Facebook Groups Is Sort Of Like Google Wave For Human Beings.

But I deeply hope that the open, distributed standard part of the Wave big idea is not lost in the mix here.

Fourth Cylinder of Innovation: Keep the Ecosystem Going, Create More Value than You Capture

The Points of Control map is interactive, so please click here or on the image above for the full experience.

Tim O’Reilly points out that there is a worrisome dark side to the Points of Control Map – see Tim’s keynote here.  To paraphrase some or his points:

There are companies on the map that are forgetting to think about creating a sustainable ecosystem.  Rather than growing the pie, they are trying to divide up the pie and that threatens to cause the fourth cylinder of innovation to misfire.  This fourth cylinder is essential to the ecosystem.

Tim O’Reilly looks back to the lessons of the personal computing industry which was incredibly vital and creative, and lots of people made money until a couple of big players “sucked all the air out of the ecosystem” and innovation had to go elsewhere.

The Power of Platforms is to create value not just for your company but for other people.   Create value for yourself by creating value for other people.  Tim O’Reilly used the wonderful example of  Henry Ford inventing the weekend so that there would be enough people with time and money to buy his mass produced cars.  Think about building the ecosystem that will support the future your are going to build.  Grow the pie rather than cut up the pie.  This will be the vital fourth cylinder of innovation in a Web Squared world.

Tim O’Reilly has long proposed that  Web 2.0 is all about harnessing collective intelligence,  But as Gartner predicts, “By year end 2012, physical sensors will create 20 percent of non-video internet traffic.” Yet another previously unevenly distributed future is going mainstream, and if you haven’t read it already, now is the time to read this paper by Tim O’Reilly and John Batelle, Web Squared: Web 2.0 Five Years On.

The Consequences of Living in a World of Data

To bring this very long post to a close!  Here are just a few of the key questions re The Consequences of Living in a World of Data that Tim O’Reilly raised during his keynote for Hadoop World:

“How would we solve the problem of digital identity in the age of sensors? (Our smart phones are able to know their users by the way they walk – their gait!)

“How will we input data when our devices are smart enough to listen on their own?”

“How should we think about privacy in a world where data can be triangulated?”

“We are moving to a world in which every device generates useful data, in which every action creates information shadows on the net.”

“Shouldn’t we regulate the misuse of data rather than the possession of it?”

“How do we avoid a data arms race?”

“Create more value than you capture.”

Click on the image below or here to watch this presentation and others from Momo13

Amphibious Architecture – “submerges ubiquitous computing into the water—that 90% of the Earth’s inhabitable volume that envelops New York City but remains under-explored and under-engaged.”

Toward the Sentient City, brought “architects and urban designers into a conversation that until now has been limited largely to technologists,” and created an extraordinary opportunity to investigate distributed architectures of participation of what we might call the “outernet.”  This is a timely conversation as “web squared,”  “smart things,” the “internet of things,” or the “outernet,” and their popular “ambassador” augmented reality are rapidly becoming everyone’s “business.” From “evil” marketers, to global corporations, environmentalists, artists and community activists -  everyone, it seems, is interested in the possibilities of this new frontier.

It is a challenging task to respond to, Toward the Sentient City, an exhibition whose backdrop includes a series of conversations on Situated Technologies – published by the Architectural League, from a circle of people who have been thinking, writing, and speaking on networked urbanism for many years now, including: Adam Greenfield, Mark Shepard, Matthew Fuller, Usman Haque, Benjamin H. Bratton, Natalie JeremiJenko, Laura Forlano, Dharma Dailey,  Philip Beesley, Omar Khan, Julian Bleeker, Nicolas Nova.  And the exhibition itself has a very thoughtful group of respondents, see posts from: Dan Hill, Martijn de Waal, Enrique Ramirez, and Mimi Zeiger.

But one of  Toward the Sentient City’s key accomplishments was to go beyond the rhetorical, and to put practical examples out into the world to organize a discussion on some of the ideas and possibilities of ubiquitous computing that have barely begun to emerge from academic research, and entrepreneurial blue skying.  As curator, Mark Shepard, explained:

“The aim is to provide concrete examples in the present around which to organize a discussion about just what kind of future we might want. Whether they’re prototypes or not, these commissions are concrete examples. They’re not abstract ideas. And we can go stand next to each other and look at and interact with something which is out there in the world behaving in the way it behaves, performing as it does, and we can then begin to have a discussion about it that is less dependent upon powers of rhetoric. So it’s not about me persuading you about an idea but it’s about us evaluating something that’s living and existing in this world. And that was really the intention of the show.”

The commissioned works - Amphibious Architecture, Breakout!, Natural Fuse, Too Smart City, and TrashTrack, that were the hub of Toward the Sentient City’s events, themes and texts, provided a unique glimpse at some of the possible dystopian and utopian futures of a “smart” city.  But, most importantly,  all the works questioned what might be new architectures of participation for a sentient city.

New Architectures of Participation: Hybrid Social Networks with Human and Non-human Participants .

Of the five works, Amphibious Architecture and Natural Fuse were particularly fascinating to me because they explored the possibilities of sensor networks to create new forms of distributed participation in networked ecosystems that connected the experience/trajectories of human and non human actors – fish, plants,  and people.

Both Amphibious Architecture, and  “Natural Fuse” – from Usman Haque and Haque Design + Research, gave exhibition attendees the chance to experience at a personal level our relationships with our non-human neighbors.

Amphibious Architecture from the The Living Architecture Lab at Columbia University Graduate School of Architecture, Planning and Preservation (Directors David Benjamin and Soo-in Yang) and Natalie Jeremijenko, Environmental Health Clinic at New York University, used a sensor array to “pierce the reflective surface of the water” that separates us from the underwater ecosystem below.  The sensor arrays just below the surface of the East River and  floating light array (see picture on left opening this post) create a new interface between people and fish whose movements and water quality are transmitted in light.

One could also SMS the fish and the single beaver that lives in the rivers surrounding NYC to find the conditions they were experiencing. But turning the city’s “back stories,” like the movements of “Yo beaver,” and the oxygen levels and water quality of the rivers into “fore stories,” is only one of the many ways Natalie JeremiJenko explores how we can engender the empathy necessary for humans and non humans to live in harmony and mutual benefit.

Toward the Sentient City also held workshops/presentations in conjunction with Conflux 2009. After her Conflux presentation, Natalie Jeremijenko of Amphibious Architecture (which is also a collaborative project between xClinic, The Living, “and other intelligent creatures on the East River”)  invited participants to enjoy a lunch of cross-species foods at the East River site. 

The cross-species lunch takes an existing interaction pattern through which people and fish are already communicating, i.e., people going to the river – the waterfront,  and feeding the fish Wonder Bread (which is bad for humans and fish); and transforms this desire to feed the fish into something which actually can remove the mercury content from the fish and our bodies by removing it from the food chain, so a previously inharmonious connection between people and fish, is redirected into a productive interaction benefitting both species.  As it turns out, food that is good for Fish (see pictures above), and removes mercury from their bodies can also be nutritious and tasty for humans.

Natural Fuse, from team members, Usman Haque, creative director, Nitipak ‘Dot’ Samsen, designer, Ai Hasegawa, designer, Cesar Harada, designer, Barbara Jasinowicz, producer, used sensors to  link humans and plants in network where we are accountable for how our behavior effects others in your ecosystem.

If you brought an ordinary plant to the exhibition, you could take home an electronically assisted plant and become part of a social network of humans and plants. This network of humans and electronically assisted plants is also a carbon sink and if  more energy is consumed than the total number of plants in the social network can offset, plants begin to die giving immediate feedback and consequences to being greedy about energy consumption. For more about joining the Natural Fuse network see here.

We are in the pre-dawn of  sensor networks like those Natural Fuse and Amphibious Architecture created – social networks  that link human and non human participants in entirely new ways are largely an uncharted territory. (Note: The upcoming Situated Technologies Pamphlet 6 – “Micro Public Places,” Marc Bohlen and Hans Frei, indicates it will continue the journey with an investigation of  “transparent and distributed participation.”)

Where Does the Social Negotiation of  Technology Happen?

Frequent questions that came up at the presentations given by the teams that produced the commissioned works were: Does this idea scale?   Does it close the loop in that you get answers to the questions asked?  How does the conversation gain agency?  And where does the social negotiation of technology happen?  (These last two questions were asked by Mike Kuniavsky at Mark Shepard’s presentation at Conflux: “Sentient City Survival Kit.” – see picture above)  I think it is fair to say that these questions for the most part remain unanswered. But Toward the Sentient city was alive with ideas and practical examples about ways we can explore these questions more deeply.

Usman Haque in response to the question, “Does this experiment scale?,” replied:

“it would, but at an individual level because it has to remain at the individual level because it is about the individual in relationship to the wider social context as opposed to building a forest to offset a city it is about each individual making choices of their own about what they do and  having some kind of knowledge about the effect they are having on other people because most of the time we are quite complacent – we are able to do whatever we want because we are not necessarily aware how our intrusions effect both human and non-human neighbors….’

So how does this close the loop?  Usman explains that one of the key aspects for him is that if you do take home a plant you become part of a system in which you are no longer anonymous and if a plant is threatened (plants get three lives) you have the opportunity to email the person in the system who has threatened your plant.  Usman noted that one of the interesting things that happened in the context of the exhibition, where there was a single unit, was that 90% of the time people switched it on to selfish mode – presumably because they were anonymous.  Another aspect of Natural Fuse that raises interesting questions is that as more people decide to join the network the risk of a plant being harmed by any particular individual’s selfishness lessens.  As Martijn de Waal, in his response that unpacks some of the deeper philosophical, epistemological, and ethical questions that Natural Fuse addresses, observes:

“The concept of a commons thus assumes cooperation and mutual accommodation. Could Sentient Technology play a role in the allocation of limited resources between citizens? Could it lead to the emergence of some sort of peer-to-peer governance model, that could prevent overusage of scarce resources?”

New Aesthetics of Distributed Participation

The works of, “Toward the Sentient City” point to possibilities for a new aesthetics of distributed participation in which users and system are no longer separated but instead “develop joint forms of observing and knowing that neither [...] is capable on its own.” (quote from upcoming, Situated Technologies Pamphlets 6: Micro Public Places, Marc Bohlen and Hans Frei).  Natural Fuse and Amphibious Architecture examine the new transactional realities of the Sentient City.

But there are many questions left unanswered.  We know a lot about the power of generativity from the internet (see Zittrain)-  the ur “architecture of participation.” As Zittrain points out, the “generativity” of the internet is “the engine that has catapaulted the internet from backwater to ubiquity.” Tim O’Reilly coined the phrase, “architecture of participation,” to “describe the nature of systems that are designed for user contribution,” such that “participants extend the reach/increase the value of the system.”  But as Tim O’Reilly put it in his recent talk, “State of the Internet Operating System:”

“Web 2.0 is about finding meaning in user-generated data, and turning that meaning into real-time user facing services.  “Web Squared” takes that same concept to real-time sensor data.”

We know little yet about what constitutes generativity for the “outernet,” particularly for the kind of  hybrid social networks that Natural Fuse and Amphibious Architecture present.  Social Networks that connect people and place, humans and non humans, challenge dichotomies of man and nature, and machine and user in new and unexpected ways.

At the moment, the internet is going through a metamorphosis with the emergence of real time technologies like XMPP, PubHubSubBub and Google Wave and the coming of age of mobile computing.   While these shifts were not investigated specifically in any of the commissioned works I think all the works  begged the question,  What is a common platform for social interaction in the “outernet,” or sentient city?  I was not entirely satisfied, from this point of view, with a web interface for Natural Fuse or SMS as a mobile interface for Amphibious Architecture.

David P. Reed points to the relationship between social mobility what he describes as the 3rd cloud  and the need for a common platform (see David Reed – Social Mobility and the 3rd Cloud. Hat tip to @srenan for pointing me to David’s presentation).

Slides above are from David P. Reed’s presentation,  Social Mobility and the 3rd Cloud

What is an architecture of participation for mobile, social interaction? This is something I am very interested in.

Recently I began a project with a small group of augmented reality developers and enthusiasts to use Google Wave Federation Protocol as a transport system for open distributed, social augmented experiences (lots more to come on this soon – you can see the back story in my posts here and here).  Wave has introduced an open federated architecture of participation that combines asynchronous & synchronous data,  bringing  together the advantages of real-time communication with the persistent hosting of collaborative data (like wikis).

Augmented Reality puts who you are, where you are, and what you are doing center stage, and is an interface for “communications embedded in context” and “enabled by identity” – two key qualities of what David P. Reed calls the 3rd cloud.  An open, distributed framework for augmented reality could create  an interconnected sense of AR, one that fuses augmentation, data overlays, and varied media with location/time/place and crucially, social networking.  Such an interface would open up many possibilities for the new transactional realities that could integrate real-time cloud based data with a human perspective and social networking.  I am using the term, transactional realities  to suggest an extension into social augmented experiences of  what, Di-Ann Eisnor, Platial, describes as,  “transactional cartography” – “the movement from map providing entertainment/information to map as enabling action” (see Human as Sensors).

We have only just got a glimpse of  how real time technologies and “communications embedded in context” will transform social interaction and our cities.  This post on Writing as Real-Time Performance that looks at the Google Wave playback feature is a brilliant example of how real time technology turns familiar practices like writing inside out, and catapaults us into new time trajectories. And, if you haven’t already seen Matt Jones of BERG’s, brilliant look at, “All the time in the world” - from the “soft time” and “squishy time” of  cell phone culture, to their anticedents in real-time computing, go now!  Also see Dan Hill’s work on “time based notation,” and Tom Carden’s work for mysociety.org

Transactional Realities Between the “Asynchronous City” and the “Synchronous Internet of  Things”

Out of Toward the Sentient City’s five commissioned works, only Trash Track focused on the “synchronized Internet of Things.” Trash Track asks what can we learn from the aggregated data streams of “smart” trash about the infamous path of trash from cities of privilege to rivers of want,  rather than exploring the the particular transactional realities of a social network that linked people with their trash

The goals of Trash Track were, Assaf Biderman explained during his presentation:

“to learn about the removal chain, to see if knowing more could promote behavioral change, and investigate if smart tagging could one day lead to 100% recycling.”

The team from SENSEable City Laboratory, MIT included – Carlo Ratti: Director, Assaf Biderman: Associate Director, Rex Britter: Advisor, Stephen Miles: Advisor, Kristian Kloeckl Project Leader, Musstanser Tinauli, E Roon Kang, Alan Anderson, Avid Boustani, Natalia Duque Ciceri, Lorenzo Davolli, Samantha Earl, Lewis Girod, Sarabjit Kaur, Armin Linke, Eugenio Morello, Sarah Neilson, Giovanni de Niederhausern, Jill Passano, Renato Rinaldi, Francisca Rojas, Louis Sirota, Malima Wolf.

However, Assaf,  in his presentation, presented another project from SENSEable City Laboratory in partnership with the City of Copenhagen, The Copenhagen Wheel.  This project seems to work brilliantly at the intersection of the “asynchronous city” (Bleeker and Nova) and the “synchronized internet of things”  The “smart” wheel – a low cost, open source, human electric hybrid is:

“an electric bicycle wheel that can be easily retrofitted into any regular bicycle and location and environmental sensors which are powered by the bike wheel and in turn provide data for a variety of applications.”

This project, that aims to promote urban sustainability through smart biking, opens up many possibilities for a bottom up architecture of participation for the sentient city (see video here).

Mark Shepard describes something he calls “propagative  urbanism:”

“a way of thinking about shaping the experience of urban space in terms of a bottom-up, participatory approach to the evolution of cities.”

And, in the most recent pamphlet in the Situated Technologies pamphlets series, #5 “Asynchonicity Design Fictions for Asynchronous Urban Computing,” Julian Bleeker and Nicolas Nova invert an emphasis in the so-called “real-time database enabled city” with its synchronized Internet of Things….  and speculate on the existence of an “asynchronous city.”  They “forecast situated technologies based on weak signals that show the importance of time on human perspectives.”  They ask:

“why, besides ‘operational efficiency,’ would we want a ubiquitously computed environment?  What are the measures of ‘better’ that we want to count as meaningful?”

They explain:

..we are trying to think through what “urbanwares might be – urban operating systems – if they were less about synchronization, top-down construction and connected channels of information and databases and so forth, and more about asynchronized, decentralized things.  Software, data, time out of alignment, incongruities, tiles and imbrications of the geographic, spatial parameters into a delicious kind of lively peasant’s stew.”

One takeaway, perhaps, from Toward the Sentient City is that it’s at the intersection of  the  “asynchronous city”  and the  “real-time database enabled city” where many new transactional realities of the sentient city will arise.

I first started talking to Robert Rice, CEO of Neogence Enterprises, Chairman of the AR Consortium, in 2008.  Robert was already actively working on creating the world’s first global augmented reality network.  But it took a few months before what Robert had said to me about impending explosion of  augmented reality into our lives really sunk in – “this is going to be much bigger than the Web!,” he extolled.

By January, 2009 I was convinced and I posted my first interview with Robert, “Is it OMG Finally for Augmented Reality?..” As I mentioned in the intro, I had recently tried out Wikitude and Nathan Freitas’s grafitti app on the streets of New York City and I was impressed.  Now, 7 months later, Augmented Reality has  not disappointed and there is an explosion of new applications, and the arrival of some of first commercial and practical toolsets, SDKs, and APIs for aspiring developers.

For more on this see my previous post, Augmented Reality’s Growth is Exponential: Ogmento – “Reality Reinvented,” talking with Ori Inbar, which is an introduction to my series of interviews with the key players in augmented reality and founding members of the ARConsortium – Int13, Metaio, Mobilizy, Neogence Enterprises, Ogmento, SPRXmobile, Tonchidot, and Total Immersion.

As I mentioned before, Maarten Lens-FitzGerald of SPRXmobile told me the other day that my first Interview with Robert Rice, in January of this year, was a key inspiration for SPRXmobile to get started on the development of Layar – a Mobile Augmented Reality Browser. Much more on Layar and Wikitude – world browser in my upcoming interviews with Maarten Lens-FitzGerald and Mark A. M. Kramer, respectively.

Recently, both Layar and Wikitude earned a mention in the white paper by Tim O’Reilly and John Battelle, Web Squared: Web 2.0 Five Years On. Web Squared is essential reading not only because it covers the underlying technological shifts of “Web Meets World,” which augmented reality is a vital part of;  but, crucially, Web Squared focuses on how there is a new opportunity for us all:

“The new direction for the Web, its collision course with the physical world, opens enormous new possibilities for business, and enormous new possibilities to make a difference on the world’s most pressing problems.”

I am currently working on a post on Green Tech AR which is one of the areas augmented reality can play an important role “in solving the world’s most pressing problems.” Augmented Reality has a lot to offer Green Tech development.  As Gavin Starks of AMEE said at Euro Foo in 2006, “climate change would be much easier to solve if you could see CO2.”

But really useful Green Tech AR requires still hard to do markerless object recognition (going beyond feature tracking and modified marker recognition), and a tight alignment of media/graphics with physical objects, in addition to a quite a high level of instrumentation of the physical world.  And for Green Tech AR to really shine, we are going to need innovators like Robert Rice who are working on, and solving, multiple really hard problems like:

“privacy, media persistence, spam, creating UI conventions, security, tagging and annotation standards, contextual search, intelligent agents, seamless integration and access of external sensors or data sources, telecom fragmentation, privilege and trust systems, and a variety of others.”

Recently Robert Rice presented at MoMo Amsterdam. Here is a drawing of him in action (picture below from wilgengebroed‘s Flickr Stream).

In his Twitter feed Robert Rice ( @RobertRice ) Robert reminds us: “By the way folks, what you see out there now as “augmented reality” is not what it is going to be in two years.”   Robert plans to show the first public demo of his “platform for platforms” at  ISMAR 2009.

Robert is writing up a series of White Papers currently.  I got a preview of the first, “The Future of Mobile – Ubiquitous Computing and Augmented Reality.”  Robert points out, “AR through the lens of the mobile industry and ubiquitous computing is almost overwhelming compared to AR as marker based marketing campaign.”

I asked Robert, “What are the key take-aways for investors interested in the augmented reality field at the moment:

“First, Mobile AR is going to be bigger than the web. Second, it is going to affect nearly every industry and aspect of life. Third, the emerging sector needs aggressive investment with long term returns. Get rich quick start ups in this space will blow through money and ultimately fail. We need smart VCs to jump in now and do it right. Fourth, AR has the potential to create a few hundred thousand jobs and entirely new professions. You want to kick start the economy or relive the golden days of 1990s innovation? Mobile AR is it.

Don’t be misguided by the gimmicky marketing applications now. Look ahead, and pay attention to what the visionaries are talking about right now. Find the right idea, help build the team, fund them, and then sit back and watch the world change. Also, AR has long term implications for smart cities, green tech, education, entertainment, and global industry. This is serious business, but it has to be done right. I’m more than happy to talk to any venture capitalist, angel investor, or company executive that wants to get a handle on what is out there, what is coming, and what the potential is. Understanding these is the first step to leveraging them for a competitive edge and building a new industry. Lastly, AR is not the same as last decade’s VR.”

Talking with Robert Rice

Picture of Robert Rice at MoMo from Guido van Nispen‘s Flickr Stream

Tish Shute: So perhaps we better start with an update on state of play with Neogence?

Robert Rice: Neogence is doing well actually. We don’t talk much about the fact that we are still a small startup and we face a lot of the usual obstacles related to that and being a small team. Fundraising has been extra difficult, mostly because people are just now beginning to see the potential in AR, but that is still colored by perceptions based on a lot of the gimmicky AR ad campaigns out there. Still, it is better than it was two years ago the idea of an AR startup was a bit of a joke to a lot of VCs we talked to. However, we do have an agreement from a new venture fund in Europe (which we can’t talk about yet) for our first round of funding, but we don’t expect to close that for several months.

If all goes well, we hope to debut our first public demo at ISMAR 2009 in Orlando to select individuals and a few press folks. We might release a few viral videos before then that are conceptual and about what we are building in the long run, but that depends on how things go over the next several weeks.

We are also very active in looking for and building strategic partnerships and relationships with other companies, and this is not restricted to the augmented reality or mobile sector. As I have said before, we are looking at this as a long term business venture and the industry as something that will be bigger than the web itself within ten years. We are doing typical contract work and custom AR solutions to keep the cash flow going and build up the corporate resume a bit. So, if you want something done, and better than the stuff you are seeing now with all of the generic “look at our brand in AR with markers and a webcam” you should definitely give us a call.

Tish Shute: Just to clarify because most of the recent press has been about browser type AR like Wikitude and Layar which are not in the purist sense AR ‘cos they do not have graphics tightly linked to physical world. Neogence, if I am correct, is focused on building a true AR platform in the sense I just described?

Robert Rice: Hrm, I have argued with a few others about the actual definition of AR. Some people prefer a narrow and limiting view (3D overlaid on video), but I think in terms of the market and the end-user, it is better to have a wider definition. In that sense, AR is purely the blend of real and virtual, with or without full 3D overlaid on video. If we go with that, then Wikitude, Layar, Sekai, NRU, and others all fit into the AR definition.

Anyway, you are correct. We are building a true platform for AR, and this is quite different from what others are marketing as AR browser “platforms.”

There are a few problems with the “AR Browsers” approach that no one seems to be noticing. One is that they are all trying to get people to build new applications for their browsers, when they should be trying to get people to create content that they can share and browse.

Second, someone using Layar is not going to see anything that is designed for Sekai or Wikitude.

Third the experiences are generally for one user. While I love all of these guys and think each of the teams has some real talent on it, the model is flawed until someone using Wikitude can see the same thing that someone using Layar or Sekai camera is seeing (provided they are in the same physical location).

While we are working on our own client side technologies that we hope will be useful and integrated with every mobile device and AR browser out there, our core focus is on connecting everything and everyone together, and facilitating the growth of the industry with the tools to create content, applications, and so forth. We want to solve the really difficult technical problems (some of which most people haven’t even considered yet, because of the perspective they are looking at the potential of AR with), and make it easy for everyone else to do the cool stuff. We want to be the facilitators.

If you really want an idea of where we are going or some of what has inspired us, you have GOT to read Dream Park, Rainbows End, and The Diamond Age. If you have heard me speak anywhere or read my blog, you know that I am continually suggesting these and others.

Anyway, short answer, yes, we are building a true platform for ubiquitous mobile augmented reality, and we are absolutely the first to be doing so. I hope to demo some of this in October at ISMAR, with a full commercial launch next year (10/10/10 at 1010am Hehe, seriously). We will probably launch a website soon for people to start signing up and building a community now (especially if you want in on the beta testing of the whole kibosh).

Tish: So just to clarify,  how will Neogence’s approach differ and fit into the  growing world of Augmented Reality tools that we have now, e.g.,  ARTookit, Imagination, Unifeye?

Robert: I guess you could say that we are trying to build the infrastructure for the global augmented reality network. This could be viewed as a service, or even a platform for platforms. If Neogence does its job right, anything you create using ARtoolkit, Unifeye, or Imagination would be applications you could ultimately link to, integrate with, or deploy on or through, what we are building, and not be tied to a specific set of hardware, browser, or walled garden.

Tish: You mention Neogence is going to provide a platform for platforms. Without knowing the details that sounds like a lot of centralization which prompts the inevitable question: “Who owns the data?” Do you think other AR applications or providers would resist a “Platform for Platforms?” I know the potential centralization power of Google Wave has already got people talking about these issues (one of the comments in my recent blog post was about how Google Wave protocol may be interesting for a least some parts of augmented reality communication).

Robert: It really depends on perception and how we end up building it. We aren’t talking about creating a closed system. As far as who owns the data, it depends on what data we are talking about. For the most part, I think that if the end-user creates something, they should own it and have control over it. They should also be able to do what they want with it, independent of everything else.

This is one thing that proponents of the smart cloud and the thin/dumb client don’t like to talk about. It sounds great on paper, but when you start thinking about it, all that does is strip away power from the end user. Case in point…Amazon recently wiped every copy of George Orwell’s 1984 from all Kindle devices. They claimed they didn’t have rights to distribute/publish it and it was available on accident. The scary thing though, is that they literally went into every kindle out there, found copies, and deleted them.

How would you like it if Microsoft suddenly decided to delete every copy of Microsoft Office? Or every file that had a .doc extension? That is a huge violation…we feel like we own what is on our computers. But with the whole cloud thing, your data is at the mercy of whoever is running the cloud servers. No privacy, no ownership, no control. And if the system breaks, all you will have is a pretty dumb device that can’t do much on its own. Now, that isn’t to say that the technical merits and benefits of a cloud model aren’t worth pursuing, they are.

But I think there needs to be some hybrid model. Don’t dumb down my computer or my smart phone, let’s keep pushing how much these devices can do. We should take full advantage of centralized and distributed systems, but in a hybrid mashup sense. That is what we are pursuing with our AR platform, while trying to protect ownership and intellectual property rights of the end user.

Tish: Earlier today I was telling you how impressed I was by Google Wave – it is quite mind blowing to experience massively multiplayer real time interaction on what will be an open internet wide platform – Wave is breaking new ground here and more than one person has mentioned its potential role in AR to me (see the comments to my recent post on Ogmento).

I know you are a strong advocate of this kind of real time shared experience being part of AR.  But we are only just beginning to see it emerge via Wave on the existing web – what will it take to have this kind of real time shared experience in AR!  We got briefly into the thick client, thin client, cloud versus P2P discussions – what is your approach to delivering a massively shared real time experience that is like Wave not confined to a walled garden?

Robert: I’m not a fan of any of those models as being stand alone or mutually exclusive. Again, the hybrid model with the best of both worlds is key. In the early stages of the emerging industry, you are likely to see some walled gardens (or perhaps a walled garden of walled gardens…).

No one knows how things are going to turn out in the next five to ten years and few people are thinking about it actively. For us though, I favor Alan Kay’s quote (pardon the paraphrasing): “To accurately predict the future, invent it”. That’s what we are doing. In the short term, there will be plenty of experimentation in the industry and a lot of model testing.

Tish: Do you think though Wave protocols might be useful as at least part of the picture for AR standards?  As you point out open standards and open protocols are going to be vital for shared experiences of AR.  Is it important to build off existing protocols to get the ball rolling and what do you see as being the important early protocols for AR?

Robert: I think for now, we will use a lot of existing protocols for communications and whatnot, as well as the usual standards for things like 3D models, animation, and so forth. This is only natural. However, as the industry and technology evolves, we will need entirely new ones. As far as I know there is no existing market standard for anything like the Holographic Doctor from Star Trek Voyager, and that type of thing is definitely in the pipeline for the future (sooner than you would think).

Tish: All the excitement at the arrival of the browser like mobile reality developments has been really great – I feel people are getting a taste for what it means to compute with anyone/anything, anywhere and and anytime.

Wikitude started the ball rolling. And with Wikitude.me it is the first to support user generated content. Now there is Layar, Sekai Camera also. But as you mentioned to me in an earlier chat, with Layar and Wikitude opening up “their are probably half dozen other apps coming out in short order with similar functionality (even the AR twitter thing has some similarities).”

What has been most exciting to you about these developments up to this point? What will these apps/platforms need to do to stand out in a crowd.  Up to now, these browser like AR experiences do nothing with close by objects. Do you see “world browsers” with near object recognition coming out in the near future. Could Wikitude do this with an integration of SRengine or Imagination?

Robert: Yes, Wikitude or Layar could do this (integrate with something else for “near” AR) and it would be a step in the right direction. Tagging things in the real world is the basic functionality that will grow from text tags to photos, videos, 3D objects, and all sorts of other types of data and meta data. This gets really fun when that data is generated by the object itself. First is just giving people the ability to tag something and share that tag with their friends, everything else grows from that. This sort of functionality is probably the most exciting in terms of near future advancement.

However, I think the idea of a stand-alone browser platform is a bit awkward…unless you also consider firefox a website browser platform. After all, you can create widgets (applications) for it. Anyway, the point is having access to the same data…if you put three people in a room, one for each browser, they should see and experience the same content, although the interface might be different (based on what browser and of course which hardware they are using). This means there needs to be some communication between whatever servers they are storing their data on (meaning, user tags) and some standard for how those tags are created.

Of course, if all they are doing is grabbing the GPS coordinates of the nearest subway station and telling you how far it is and in what direction, then they should all be able to see the same thing, regardless of the platform. But then, that isn’t really interesting is it? I could get the same info on a laptop with google maps.

This is part of the problem right now though…no one seems to be thinking about the bigger picture much. All of the effort is either on making the next cool ad campaign for a car or a movie, or creating a tool to tell you where the nearest thingamajig is, but in a really cool fashion on a mobile device.

No one is talking much about filtering data, privilege systems, standards, third party tools, interoperability, and so on. There is also little conversation about where hardware is going. Right now everyone is developing software based on what hardware is available. This needs to change where hardware is being developed to take advantage of new software coming out (this happened in the PC industry a while back and growth accelerated dramatically).

These are some of the reasons why I led the effort to start the AR Consortium. We brought CEOs from 8 different AR companies and startups together to start talking about these issues. We are still getting organized and have plans to expand the membership to other companies, but we want to do this right and we aren’t rushing things. The important thing is that we have started and there is at least a line of communication open now, where there wasn’t before.

I would expect to see the early movers expanding what they offer very soon, and they will probably lead the way in the short term. Definitely keep an eye on the companies involved in the AR Consortium. There are lots of very smart and motivated people there, and they are far ahead of all the experimental dabbling in AR we are beginning to see on youtube, twitter, and elsewhere.

Tish: When we had a discussion about what were the basics for an AR platform and an AR browser earlier, you talked about the difference between tools, a platform, and a AR browser – like Wikitude and Layar which should be about features/functionality e.g. to create treasure hunts AR geocaching, invisible AR yellow sticky notes you can leave at restaurants you don’t like, etc. Also you noted it should let you explore (browse) multiple formats, and open content content for AR – any data, information, or media that is linked to something in the real world and the visualization/interaction with the same.

Wikitude is a stepping stone to a true browser by your definition. But are we also seeing what you would define as an AR platform emerging – Unifeye, Wikitude (you can recap your definition if you like too)?

I think Wikitude hopes to provide the lego blocks for  augmented reality readers, browsers, applications, tools, and  platforms?

Robert: I expect some segmentation among the various AR companies that are out now, as they find their individual strengths and focus on them. Some will emphasize the client software (the browser), others will develop robust tools for creating content, SDKs/APIs will advance and facilitate rapid development of applications, etc. Neogence is ultimately working on the glue in the middle that ties everything together, makes it massively multiuser, persistent, and ubiquitous. Things like Unity3D have the potential to fill a need in the middleware space.

Tish: I know Blair McIntyre (see my interview with Blair here) and others are using Unity3D as an AR client, Could Unity3D become increasingly important?

Robert: It has the potential to become a favored middleware for providing the rendering layer. It already works nicely in regular browsers, and on several mobile platforms. Why code all the graphics rendering stuff from scratch when you can just license something and extend its features with AR functionality?

Tish: Now to ask your own question back to you! There seems to be a lot of reason to think that, eventually, there will be the kind of access to the iphone video API that augmented reality really requires and by that I mean more than we will get with OS 3.1 which is rumored to deliver only about half of what we really need for AR on the iphone – “not truly useful when you want to align video. with graphics.”  So:

“The iphone…future or failure? Seemingly anti-developer stance regarding augmented reality, and only a sliver of the global market share. Are we letting the short term glitz of Apple and the iPhone fad pull us in the wrong direction? Shouldnt we be focusing on symbian devices that have the lion’s share of the market? or should we be looking more at either other OSs (winmobile, android) or not at all and trying to create a new platform that is more MID and less smart phone with a hardware partner?”

Robert: Apple and the iphone are a bit problematic right now. There is no way I can go to a venture capitalist (at least in North America) and say hey we are building awesome AR applications for winmobile or symbian…they would either laugh or they simply wouldn’t get it. There is this false perception that the iphone is the ultimate mobile device, it is the sexiest, and the only thing that people want. Everyone wants a demo on the iphone, the media is mostly interested in iphone developments, and the apple fanatic market could give a fig about other devices. Other devices may have a larger market share or even better hardware, but we have to focus on the iphone right now at least in the demo stage to get any market attention and traction worth the time and effort.

In the future though, unless Apple changes its stance with their SDK and APIs, and starts adding hardware that is key for mobile AR (beyond what is there now), the market will move on without them. This is a really easy decision to make given Apple’s draconian policies and the fact that their percentage of the global market is miniscule. The smart companies are looking at the whole picture and not putting all of their eggs in the Apple basket.

Of course, once the wearable displays are commercially viable everything changes. Wearable computers with small screens or even no screens are going to be what everyone wants. The interface will go from handheld touch screens to virtual holographic interfaces that you interact with using your bare hands.

So for now, (the immediate short term), its all about the iphone. Taking mobile ubiquitous AR to the global market and building for the future will be based on something else. Hardware risks becoming a commodity or a closed platform. Do you really want to buy the Apple iGlasses and only see AR content that is compatible, where your best friend has a pair of WinGlasses and sees something entirely different? No. The hardware, and the client software (what people are calling the ar browser now) will become common and it won’t matter what brand you use, they will all be accessing the same content.

But at least for the forseeable future, we are building software for specific hardware, and the sexiest mobile on the block is the iphone. The second someone comes out with something much better and the paradigm shifts (software driving hardware instead of vice versa) everything changes.

Tish: How is the quest for sexy AR eyewear going.  I know we were checking out the Japanese eyewear with Adam Johnson from Genkii just now.  For the Neogence project – as you are going for a fully developed model of AR doesn’t this necessitate going beyond the iphone and getting the hardware companies moving on the eyewear?

Robert: The guys making wearable displays really need to get off the pot and stop paying lip service to mobile AR. If they don’t do something quick, I,  and others, are going to be scouring the planet looking for someone capable of building the lightweight stylish wearable displays with transparent lenses we are begging for. We aren’t going to be waiting around for hardware anymore. The AR Pandora’s box has been opened. I should note that many of us (AR Consortium members) have had less than pleasant experiences or communications with the half dozen companies or so that are making wearable displays. Either their visual design is terrible, the materials feel flimsy, the field of view is limited, or the companies are preoccupied with other business and government contracts. Any attention to the growing AR market is an afterthought and in a few cases condescending. AR is going to be a billion dollar industry in a very short time, and these guys are just leaving money on the table. If they were smart, they would be begging the CEOs from the AR Consortium to fly out to their offices and collaborate on building a pair of wicked sick glasses. The smart phone manufacturers should be doing the same thing, but I have to say that they at least seem to have some ambition and zeal to create better devices, so I can’t really complain too much there.

Anyway, to answer the rest of your question, we have to assume that the hardware guys, especially regarding the eyewear, is going to take a long time to develop and release the things we need for the ultimate AR experience. So, our goal is to start building things now for what is available. That means scaling things down and handicapping what AR can do, so it works on the “sexy” iphone. The important thing though is to start creating applications -now- so when the glasses are commercially available, there will be a wealth of content for people to access and use on day one.

As long as Apple isn’t playing nice, it is going to hurt everyone. Is it any surprise that they shut down Google Voice? There is a huge opportunity for someone to step up and leapfrog the rest of the industry. Give us the hardware and we will create amazing software for it. Don’t compete with the iphone, surpass it.

Tish: What is the state of play of current AR technology and toolkits?

Robert: The current crop of AR technology and toolkits is absolutely critical for this stage of the industry, and everyone should be leveraging it as much as possible. I talk down marker and image based tracking a lot, but I also like to point out that it is the necessary baseline that the industry is going to be built on. The problem is that there is only so much you can do with marker driven apps, and as creative people and marketing types start conceptualizing about all sorts of cool stuff for the future, they risk setting the expectations too high. It is one thing to show someone the future, it is another to say this is the future and its happening right now. This is why I cringe everytime I see a conceptual video presented as “our product DOES this” instead of “our product WILL DO this.” Something that simple can still cause the butterfly effect of raising expectations too high and contribute to overhyping.

Tish: One of the things that seems very exciting about the new Ogmento partnership is that experienced content producers  Brad Foxhoven and Brian Selzer from Blockade are now taking a leading role in AR.  What are the most exciting directions for content that you see emerging for AR in the next 12 months?

Robert: Virtual (well, augmented) pets, and multiuser mobile AR games (2-4 people) are probably going to lead in the next 12 months for content. Easy, accessible, engaging.

Tish: And are you at Neogence also involved in content partnerships?

Robert: Yes, we are in the process of finalizing some content partnerships with an eye for long term relationships. We are specifically looking for partners that want to find substantive ways to leverage AR technology, and not use it as a superficial gimmick or attraction that wears off after five minutes. I’m still cringing over the Proctor & Gamble Always campaign with AR.

Tish: So back to your observation about some of the tricky problems re creating a true global massively multiuser, ubiquitous, mobile AR platform – what are some of the main obstacles to this mission in our view? (aside from getting investment!)

Robert: Trying to explain it to people. The technical problems we can handle or have already solved. But trying to communicate what exactly we are doing is still tough. Not because it is overly complicated, but rather because it is so new and different. People are having a hard time grasping augmented reality beyond marker/webcam.

Tish: Which AR tools are most important right now?

Robert: Content is critical right now to show what the technology is capable of and to continue building the presence of augmented reality in the public mind the big benefit to integrated / unified platforms now is speed of development for content. I think that the flash artoolkit = papervision is rocking the planet right now. It is accessible, easy to learn, and lets people create something very quickly. More tools and middleware are coming out and this increases options for designers and developers.

Tish: What are your favorite papervision apps?

Robert: Hrm, I don’t have a favorite papervision app just yet, although I think the tech is solid. I expect to see a lot of stuff built on that platform in the near future. Especially as more ad agencies get on the bandwagon and start telling their IT guys to learn how to program flash so they can make something. Have you seen www.ronaldchevalier.com Not so much for the actual AR stuff, but because the whole thing is just brilliant. Its exactly like some cult figure spiritual guru would do with AR. I wish I had thought of it first actually. This is probably one of the best -seamless- implementations of AR in marketing where it fits…it isn’t just jammed in there for the sake of saying they used AR.

Tish: Do you think Apple is going open the iphone to the full potential of augmented reality anytime soon – a lot of expectations have been raised?

Robert: Apple is like that guy has a party at his house and owns this really awesome state of the art home theater in his basement, but makes everyone watch a movie in the living room on a regular TV with a VCR.

They need to get over themselves and quit being a wet blanket. Otherwise, we are taking the beer and pizza we brought, and going to someone else’s house. Sorry, the Apple thing is a bit of a sore point with me.

Tish: But will people leave all that candy and soda at the appstore?

Robert: I tell you what though, there is an opportunity for certain mobile phone manufacturers to give me a call and start talking to Neogence and the other members of the Consortium. We have some ideas and specs that could have a radical impact on the mobile market and stuff the IPhone in a box. Hint hint.

Tish: So what is your vision for the ARconsortium.  I know it kicked off with a letter to Apple about the video API.  What is the next step? There was a lot of hope that this year would be big for MIDs but this really hasn’t happened yet – do you think there is hope for a MID take off despite the lousy economy?)

Robert: MIDs? No, not yet. smart phones are too lucrative and too hot. It isn’t time yet for the MID to go mainstream. For that to happen, there needs to be a driving need (cough ubiquitous AR cough)

The AR consortium is mostly an informal affiliation. I expect that representatives from each member will probably meet at every significant conference to catch up over drinks. We are also going to be planning for our own members conference at least once a year. That will happen after we expand the membership though.

The main idea behind the consortium though was to open up a channel of communication between the CEOs so we could work together on standards, solving problems, collaborating, forming some partnerships, and using the collective to bang on the doors of companies like Apple and others. There is power in a group.

Tish: You mentioned there is a whole long conversation we can have about getting the eyewear.  As you point out true AR eyewear changes everything.  Can give a little road map of where this has to go?

Robert: There are essentially four or five main approaches, depending on whether or not you make the lenses special or if they are just plain. You would normally want them to be plain so people with prescription lenses wouldn’t have problems and would have the option to switch them out. Some types use a more prismatic approach for top down projection, or a corner piece mounts lasers and bounces them off the lens into the eye.  Another approach is embedding OLEDs or something else into the lenses themselves.

I really like the Lumus approach, but their product design isn’t quite there yet. If the wearables don’t look cool, people won’t use them. To be honest, if I had the money, I’d probably ask the Art Lebedev guys to design them based on someone else’s optical engineering. They designed the optimus maximus old keyboard…   brilliant industrial designers, loaded with engineers too. If these guys couldn’t build the glasses and make them look damn bad ass, I’d be shocked. Heck, I bet they could build the next gen MID while they were at it.

Tish: Getting the hardware innovation and software innovation feeding into each other would be really great.

Robert: Absolutely.

Tish: That would push the eyewear forward too wouldn’t it?

Robert: All it takes is one, and then the competitive landscape would fire right up.

Tish: What applications would the accurate gps enable?

Robert: Everything. for example, you know exactly where the phone is and where it is facing, that means you can put it on a table and hit a button, then move it somewhere else and do the same thing in a few minutes, you have a nearly accurate “mental” model of the whole place now you go back and start dropping virtual flower pots everywhere.

This is one area where I think the smart phone guys are missing the boat and taking the cheap route. It is possible to have very accurate GPS (down to a six inch area) with better chips and firmware, but it is cheaper to stick in old tech. Most apps today dont need that hyper accuracy, so they aren’t bothering. Mobile AR though, thats a different story.

With that level of accuracy, you would know exactly where the mobile device is, so all you would need to know is the direction it is facing (orientation), and you could solve one of the problems with registering exactly where 3D objects and augmented media is (it is more complicated than I am describing it, but we don’t need to get into that much detail here). You wouldn’t need markers anymore.

Tish: Isn’t Wikitude doing this with Wikitude.me their tagging app.?

Robert: Not really. That type of approach is on a very large scale using the accelerometers compass and GPS to determine where you are and what is in the distance. They (and others like Layar) don’t handle “near” AR. They effectively poll your GPS and then check a database to see what is nearby and what degree/distance it is and then they draw a representation on the screen. They don’t even need a mobile device’s camera at all.

Even if they did things up close, its still based on finding landmarks or on things that are broadcasting their location. For example, if they were standing near me, they might get “robert, 37 degrees, 15 meters away” but they wouldn’t be tracking me exactly as I walk around or have the ability to overlay graphics on ME.

Tish: I retweeted your #ar marketing using ARToolkit + flash (markers/webcams) = Photoshop pagecurl <six months. Bad design kills innovation. I know you like Dr Chevalier though!  What are some of the other AR marketing projects that you like. What would you like to see in terms of innovation in the next 6 months?

Robert: The marker/webcam approach is already becoming overused and cliche (tremendously fast). Older readers will remember the ubiquitous photoshop page curl that adorned nearly every website and graphic on the internet back in the day. It was horrible. Yes, the Dr. Chevalier stuff cracks me up.

I want to see some big companies or ad agencies really try to do something different with AR, preferably mobile. Take some risks, do something different. Don’t follow the crowd. Innovation? I want to see some wearable displays with transparent lenses, I want a mobile device specifically designed for ubiquitous AR, I want to see some experimenting with AR in the green tech sector, and I’d like to see someone get that GiFi wireless technology from that researcher in Australia and jam it into a smart mobile. I would also like my flying car and lunar vacation now, thank you. It is almost 2010 and no one has found that black obelisk yet.

Tish: So a few closing thoughts! What do you see as the next big thing? Hopes for the ar consortium?  Biggest bstacle for commercial AR?  And what is the coolest thing you have seen this year?!

Robert: The next big thing is what I’m working on hahaha. I hope the AR Consortium will grow and be the active catalyst in making AR mainstream, practical, and world changing.

The biggest obstacle is making sure that the right funding finds the right developers to develop the right technology and create kick ass applications.

The coolest thing I’ve seen this year would probably be the facade projection stuff (see below): Now, imagine that, but without the projector. Thats part of what I envision for AR in the future.