I am super excited to be speaking at SXSW Interactive 2012, as part of Will Wright’s Stupid Fun Club, on “A Lifestyle with a Gaming Sense.” Michael Trice just did a post on our session for SXSW.com, Screen Burn Panels at the Palmer Presents Will Wright’s Stupid Fun Club. The photos of Will Wright, Tish Shute (me!) and Peter Swearengen are by Anya Zavarzina. Thank you Anya for such great photos!

I have been too busy to blog much lately, but there is a lot to unpack in future posts in my quote in Michael’s SXSW post!

“Really we’ve entered a new era where the world has become a platform for storytelling and the goal is to turn everyday life into an opportunity for play, relatedness, and new forms of autonomy and fun. We’ve now come to a point where software has moved out of the computer and into the world. Rather than viewing this process in terms we’ve already grown out of, like gamification, we view this as an opportunity to explore everyday activities as possibility spaces.”

For the complete post, including Peter Swearengen of Stupid Fun Club on StoryMaker, see here http://sxsw.com/node/9969

Here  is the “world premier” of  the  ARWave demo – also see on youtube here.  Below are the slides from our panel: The Next Wave of AR: Exploring Social Augmented Experiences, with Anselm Hook presenting on “10 reasons Why Ar Not a Flash in the Pan,”  Jeremy Hight, spoke on “Augmenting the Map,”  Sophia Parafina presented on “Citizen Sensor,” and I presented the “ARWave” project.  These slides are also posted on the Where 2.0 site.  Joe Lamantia was unable to come from Europe due to last minute obstacles, but you can check out his slide deck here. The videos of the Where 2.0 keynotes are up, and I highly recommend watching what went on at this watershed event.

We got so much positive feedback for the ARWave project at Where 2.0 that I am very excited about the next steps, and about our session at WhereCamp tomorrow.

WhereCamp will be at Google, Maxwell Tech Talk and the buzz continues!  The latest from WhereCamp is:  WE ARE OVER-SUBSCRIBED. HOWEVER PLEASE SIGN UP ANYWAY. THE EVENT TAKES PLACE OVER TWO DAYS. IF WE REALLY HAVE TO TURN PEOPLE AWAY ON SATURDAY YOU CAN STILL COME ON SUNDAY.  Hope to see you there!

“The internet eats everything it touches,” write Brady Forrest and Nathan Torkington, O’Reilly Media, Inc., in their must read 2006 companion essay The State of Where 2.0 (PDF).  Now in 2010 that statement is more true than ever.

Last week,  I talked to Brady about what we can look forward to at Where 2.0, 2010,  and what he thinks will be the “internet eating” trends emerging this year.  Brady is uniquely positioned to get a glimpse of things to come.  His job for O’Reilly Media is tracking changes in technology and organizing large scale events, including Where 2.0 which he chairs, and Web 2.0 Expo in San Francisco and NYC which he co-chairs.  Brady also runs Ignite, and previously worked at Microsoft on Live Search.  And, when not doing his day job, he participates in such Uber Geek activities as Steve the Robot H.E.AI.D – A Human Energized Artificial Intelligence Device…with lasers and generative sound, (click on pic above or see video here).  Look out for Steve the Robot H.E.AI.D, at Augmented Reality Event, June 2nd and 3rd, Santa Clara, CA,  and a presentation from Brady.

As Vernor Vinge pointed out in his intro to ISMAR 2009 – the “possibilities are both scary and wondrous” as “the physical world becomes much more like a software construct.”  Brady Forrest has taken a lead role, since 2004 – when “‘local search’ was interesting but not yet real,” in shaping this transformation.

Where 2.0, together with WhereCamp (this year at Google) constitutes WhereWeek – a crucible for emerging trends in web mapping platforms, and location based technologies.  This year augmented reality, proximity-based social networking, local search, and the rapidly maturing field of Crisis Management are in the  mix along with the huge and long established GIS industry which has moved rapidly into the Where 2.0 space.

But what business models will oxygenate the system is still a key question – one Brady discusses in the interview below.  Certainly, the usefulness of location based analysis, mapping, new interfaces, and bringing this data to every application is clear.

Crisis management is center stage this year Jeffrey Johnson (Open Solutions Group), John Crowley (Star-Tides), Schuyler Erle (Entropy Free LLC) who will present on, Haiti: CrisisMapping the Earthquake.  And Chris Vein & Tim O’Reilly will “discuss how cities and application developers will benefit from open data and what these programs will look like in the future”  in the plenary City Data.

Mobile social, proximity- based social networking, which may soon emerge as a challenger to web based social networks, and augmented reality are the sexy rockstars of  the Where 2.0′s 2010 showcase of potentially disruptive technologies.  Augmented Reality has had a breakthrough year, and this is reflected in its strong showing on the Where 2.0 schedule.  But, as Brady notes, AR awaits the killer app, that will drive it to the next level  Of course, we hope to unveil that at are2010!

At Where 2.0, I am presenting on The Next Wave of AR: Exploring Social Augmented Experiences panel.  We will look at how social augmented experiences will be key to the next wave of mobile augmented reality.  Mike Liebhold, in a complementary presentation, looks at Truly Open AR. If you have been reading Ugotrade, you already know I am an advocate for an open, distributed, real time communications framework for AR – see ARWave.  Wave Federation Protocol is an open fast, compact, federated, communications protocol that is a dream come true for AR.  And, I would hazard a guess that in 2010, real time communications plus location will become oxygen.

But also key to the next wave of AR, as I discussed with Anselm Hook in this post on Visual Search, Augmented Reality and a Social Commons for the Physical World Platform, will be a view constructed through complex “hybrid tracking and sensor fusion techniques” (Jarell Pair), cooperating cloud data services, powerful search and computer vision algorithms, and apps that learn by context accumulation.”

And as Brady notes in the interview below,  a key step forward would be “to take advantage of your location, but it doesn’t need to have been mapped before.”

For some interesting news on the mapping front (and a discount code for Where 2.0 for Radar readers) see Brady’s post, Flickr Photos in Google Street View. These kind of human built maps have the potential to develop into “photo-based positioning systems” that could create new opportunities for augmented reality.  Brady asks:

“how often the Flickr photos get updated, where else these Flickr photos are going to show up in Google’s services (Google Goggles perhaps?) and will they show up in new search partner Bing? I am doubly curious if Facebook will ever let its photos be used in a similar way.”

Lior Ron of Google Goggles will be at Where 2.0 to tell us all about, Looking into Google Goggles.  And if you want to learn more about how our view of the physical world will be ” rooted in powerful computing, pervasive connectivity, and the cloud” don’t miss this one.  I will be there.  And I very much hope there is a Q and A with this session.

During our conversation (see the full conversation below) Brady gave me his short list for breakthroughs that he sees as having big significance in 2010:

“Well, I think Google Goggles is one of the most exciting things to me.  Having access to a visual search…having someone actually release a visual search engine in that way, to consumers, I think is huge.  You know, you see stuff like that in the labs. But I don’t see it… it’s rare to see it out.

I think Android is huge.  And the way Google is pushing hardware to show off the platform; so the Nexus One being another example and the fact that it’s breaking free from the carriers.  Because I think when we get away from the carriers we are able to see more innovation, it’s what’s going to allow people or developers and companies to really innovate.

And I think Twitter adding geo-location to their APIs and buying MixerLabs is a huge move. I think Twitter may end up becoming the end-all be-all of location services. They are going to be updated constantly by people; they are going to have a really good grasp, real-time, of what is happening in any one place, at least based on the people.

And then with the addition of the MixerLabs data, they’re going to have more datasets at their ready, as well as any data that they start to collect from the clients themselves, like from TweetDeck.

So there are global clients that are updating Twitter.  I think those are some of the most exciting things.  And again, just to come back to Yelp, I think Yelp’s Monocle is also pretty significant, just because it’s an AR [augmented reality] app that’s being pushed into consumers’ hands.

And we’ll see how useful they find it.”

Talking With Brady Forrest

Pic above from WhereCamp 2009, Brady Forrest, facing camera, checks out Mark Powell’s Food Genome Project.  Check it out here – it just woke up!

Tish Shute: So last year when you were interviewed by Michael Calore for WebMonkey before Where 2.0 you said, “Location is no longer a differentiator it’s going to become oxygen.” And after attending Where Week 2009, I agreed with you and wrote about it here.  But, in what ways did this prediction exceed expectations, and what ways were you disappointed now as we get close to Where 2.0, 2010?

Brady Forrest: Well, it exceeded expectations in that there are now five different mobile OS’s where you can load on third party applications that active users’ locations that can then be shared out.

And so, what it is making is the possibility of real-time social location aware applications.  And this is something that hasn’t truly been possible in years past. Looking back three years ago when the iPhone launched, it was the first major phone, especially in the US, to be location aware.  And a year later, the Apps Store launched, giving developers full access to location, which previously had been held onto very, very, incredibly tightly by the carriers.

And now, a year and a half later, you have Android, you have Palm Pre, you have Blackberry working on their SDK to make it better, but it still is there.  You have Windows Mobile working on their SDK.  And, you know, who knows?  Maybe even BREW will get into the mix.

And AT&T is opening up their own interactive store.  And so, AT&T and Verizon and all their smart phones may now be looking at BREW.

Tish Shute: Right. It was very exciting last year at Where 2.0, where we had all these new toolsets announced and then the iphone being location aware. What were the best implementations of these new capabilities that became available in 2009, do you think?  What, in your view, was the most creative, surprising and disruptive?

Brady Forrest: Well, I am a huge fan of Yelp Monocle. I think, you know, that is just a real life example of using Augmented Reality.  You are on a street.  You have got a bunch of restaurants.  You have got a bunch of businesses.  And just to be able to swing through and look for people…I mean and look for ratings and reviews.

They have just started to institute check in, so you will be able to know where your friends are and where your friends have gone.  And that type of real-time, incredibly useful data is what will make augmented reality a standard part of the landscape.

I think it is that type of data, more so than, say, reference data, that will make people want to have all the possible sensors.  So, what do you need for that?  You need a camera.  You need a compass for orientation.  You need a GPS or, at least, a decent location service.  And then you need a screen where you can actually see the data, and then you need an Internet connection.

So it is not like any phone can handle this.  And so, you are going to need those killer apps to actually drive people to the type of phones that can support this.  I don’t think AR is quite there yet.

Tish Shute: I agree, for true AR you need more that compass, camera, and GPS.  There are some missing pieces for the real deal experience – and not just a pair of sexy AR spec.  As you mention, hybrid tracking and sensor fusion techniques that can combine computer vision technology with  compass and GPS are vital.  We need the compass.  We need the GPS.  We definitely need the camera!  But we need this combined with computer vision techniques to get the tracking, mapping and registration for true AR, or even to deliver a stable experience with the post-it/geonote AR that we see emerging with Layar, Wikitude, and others. At the moment we need to put together the tools for a true AR hyper-local experience.

And, of course, another aspect of this is the kind of physical hyper-links that applications like Google Goggles are building.

Do you have a speaker from Google Goggles at Where 2.0.  I would be absolutely fascinated to hear more about their road map?

Brady Forrest: I was loading Google Goggles onto the program yesterday.

Tish Shute: Oh, you did?  Oh, fantastic. And you have Lior Ron speaking!

Brady Forrest: It is actually possible it is not up on the website, but I talked to them and got them to agree to do a talk on it.

Tish Shute: I very much want to hear more about their road map.  Google Goggle’s is a very, very significant step towards the physical internet and this integration of computer vision with sensor fusion techniques necessary for true AR.

Brady Forrest: I mean that combination with Computer Vision is going to be incredibly valuable, because,  and then the other issue you have there is like is it on the client,  or is it on the server?  And right now, Google Goggles is definitely on the server, and that is not fast enough in real-time AR.  So that is like more of a 10 blue links IO interface.

Tish Shute: And also, they haven’t got an open API, have they?

Brady Forrest: No, not yet.

Tish Shute: Maybe they will announce that.  Can you nudge them?  For true AR,  we need to move forward in several areas – of course, there is the mediating device issues, like access to the video buffers in the iphone, and the development of cool AR eye wear would be nirvana!

But my recent obsession has been working on a real-time communications infrastructure for AR, because that is quite doable now, yet we don’t really have that real-time infrastructure, i.e. a real-time mobile social utility that is really up to the real time requirements of AR [see more about this here and on ARWave wiki].

But we certainly don’t have the integration of computer vision and sensor techniques, and the access to the big image databases we need, let alone the clients we need to put it all together either!

Brady Forrest: Google has done work to help out the community with their support of Open CV.

It is based out of Willow Garage, but I believe that Google has done quite a bit of work on it.

Tish Shute: Could you talk a bit more about Open CV?

Brady Forrest: O’Reilly has  a 500 page book on it.  It came out of the Darpa Project, or the Darpa Contest, where unmanned vehicles are raced.  And that has since become, at least in my mind, the primary computer vision library that people work with.

I actually used it…or, one of the teammates did, on our project we did this summer.  We implemented an Open CV pretty quickly that detected where people were, and then we would play music based on that.

Uber Geek Meeting from ShellyShelly’s photostream
Tish Shute: Is that your Burning Man project? Do you have a link for that, and some pictures, video?

Brady Forrest: Yeah.  Heaid.com.  Human Enhanced Artificial Intelligence Dancing.

Tish Shute: Thank you! This year the augmented reality story has been fairly basic – relying on basic sensors, compass, gps, accelerometers.  But it has also been an exciting year because  we hadn’t even had  smart phones with the camera, and GPS, and compass before this.

But now, the big adventure is to hook this all these sensor fusion techniques up with computer vision so that we can actually do reverse positioning for example from photos from what we are looking at, right?

Brady Forrest: Yeah, and start to use it in a more ad-hoc manner so that as you are traveling around, yes, it will take advantage of your location, but it doesn’t need to have been mapped before.

Tish Shute: Right – moving from mapping to context awareness.  Could you give like a quick explanation of what you did in your Burning Man project and how that relates to this kind of,  ad-hoc, on the fly, beginning to know what you are looking at without it having been mapped before, that is fascinating.

Brady Forrest: Sure.  So we mounted a camera about 30 feet off the ground.  And as people would move underneath or dance, they would move from block to block.  And we had kind of created kind of bitmap of the area underneath and set up different sound zones.  So as people moved from zone to zone, it would play different music.

And we used Maxim FP to handle the computer vision, although it has Open CV library to handle the computer vision part and to handle determining which of the audio to fire off.  And then, also, we had a laser that would play at the same time.

And then we used Ableton Live, which is a very popular DJ software to actually handle the music.  So as someone moved from, say, square A to square B, it would fire off various MIDI signals and Ableton would interpret that.  And each person who went in, up to…well, theoretically, up to 4- 8 people.  But because of how small the stage was and how the sounds are played, realistically, more like 4-6 people.

Each person had there own set of sound.

Pic from extramatic‘s Flickr stream here

Tish Shute: Wow! Awesome.

Brady Forrest: We would be able to detect different people, assign them a sound, or a set of sounds, so, like bass, drums, vocals.  And then we would have clips that played well together that were 3-5 seconds in length.

Tish Shute: At what distance could you detect people?

Brady Forrest: We had a 22 foot area underneath the camera.  That was mostly based on what the lens could capture.

Tish Shute: OMG I love this!  This is really the next step for augmented realities – not just attaching reference data to the world but exploring new shared “cosensual realities” (see Anselm Hook’s interview part 2 upcoming).

I am very interested in how in something you talk about a lot in your “State of Where 2.0″ essay, about lifestyle coming first for a potentially disruptive technology, not commercial considerations.  I still have to post the second half to my interview with  Anselm Hook but Anselm has some brilliant ideas in this area.  He is working on a project called Angel, where part of the vision is for people to actually find what they need without explicitly having to ask for it having to ask for it.

And this brings me to something that is very, to me, noticeable about Where 2.0 this year, and very exciting.  This is that location aware technology and crisis management basically has matured, hasn’t it?  We are beginning to see really useful stuff in this area now.

What is different this year that has brought crisis management and location aware technology together, a world in crisis?

Brady Forrest: Well, I think the primary thing that has brought all these technologies together is Haiti.  Without Haiti…A lot of times, future crises benefit from the current one, because people put in a lot of work.  And so, there is new infrastructure being laid with things such as Ushahidi, which is an open source platform for tracking…well, originally for tracking election violence in, but now is being used to track people and their locations and food requests in Haiti.

Also, Haiti did not have solid, accessible, good maps at the time of the of the earthquake.  And there have been two volunteer projects that have sprung up to help with that.  One being headed by the Open StreetMap Wood Foundation and many volunteers.  And then the other, Google Map Maker. And in both cases the activity around Haiti on these programs went up exponentially…or, I don’t know about exponentially, but a lot.  In the case of Map Maker, it was up 100 times and was the most worked on country for that week.  And one of the most downloaded for that week.

Tish Shute: Yes the work being done in CrisisCamps around the country is very encouraging.

Brady Forrest: And then also, you know, not just Ushahidi or Open Street Map, but also the People Finder which had open API so that different organizations could share their data, thus learning from Katrina.  There are all these different pieces of technology will be used in the future and hopefully be able to save more lives.  I didn’t see…there are iPhones apps that were released.  But I’m not aware of any Android apps.  I’m not aware of any AR apps.

Tish Shute: We don’t have smart phones devices distributed widely enough for them to be appropriate, do we, in a lot of areas where crisis strikes.

Brady Forrest: Yeah and there was criticism that they shouldn’t have been on iPhone.  You know, that iPhones were a waste of time. Because they aren’t…a lot of on the ground agencies aren’t going to have iPhones.  However, a lot of people who are going from the States will, and if the apps are there, then people will start to have them.

But relatively speaking, an iPhone is not that expensive.

Tish Shute: One thing I noticed and actually I discussed this in the second half of the interview I did with Anselm which I am getting ready to post.  But one of the aspects of the crisis filter was having people working as curators looking at messages coming out of Haiti, and while integrating the streams that would be useful is still probably a challenge, many curators will be on iPhones because they are based in the US.

We need to work across all platforms probably.

Brady Forrest: Yes.  Patrick Meier of Ushahidi, who runs Crisis Mappers, he ran a 24/7 emergency room. It was out of the Fletcher School in Boston.

They had volunteers all over the States and Canada.  They had volunteers in Vancouver that were translating Creole messages in under ten minutes.

Tish Shute: Yes and another point that is interesting in terms of the reconstruction and rebuilding of  Haiti is  the whole idea of leap frogging, and the idea that you can really… there’s always, as we’ve seen in other parts of the world, opportunity, when you miss pieces of basic infrastructure, to skip a whole stage and go onto the next one, like how virtual banking took off in Africa because of the absence of brick and mortar infrastructure.

Brady Forrest: To skip to a topic that been in my head, I’m just so bummed that the iPad does not have a camera.

Tish Shute: I was bummed is barely the word I would use.  Particularly as we had just been planning our ground breaking AR/next generation ebook in the days leading up to the announcement!

I suppose there is the hope they’re going to put it in the next one.  But I suppose the play for conventional content delivery is so big that everything else is trivial in comparison – especially in seems jump starting the emerging augmented reality industry!

So we might get thrown a camera and compass in the next round but will we get access to the video buffers?  AR enthusiasts may have to live on table scraps from Apple a bit longer it seems.

But what blows my mind is why hasn’t the iTouch got a camera, been AR enabled?  AR gaming would get an enormous boost from that alone. My son loves even the simple minded AR games available now on the iphone, and he loves iphone games – he has 110 games downloaded!

Brady Forrest: Ridiculous.  Yeah.  I don’t know what they don’t like about cameras.  And I plan on getting an iPad, but because of the limitations I plan on using it for base content and will probably get the bottom line model. I can’t imagine…I don’t know.

Tish Shute: It is very interesting, who actually puts together the big enabling mediating device for AR is still an open question, isn’t it?  I mean, that’s the truth; we have sort of mediating devices but we don’t have the magic brew yet do we?

Brady Forrest: No. Not yet.

Tish Shute: Good enough in some ways, and certainly a start but not quite the real deal.  For me, Where 2.0 this year covers the groundwork for true AR, mobile social proximity-based social networking, visual search, computer vision and sensor fusion techniques….   And because all these things have a chicken and egg relationship laying the groundwork is basically as important as having the mediating device otherwise you can’t do interesting things when we get the mediating device, right?

Is this the year we get the magic brew for AR, i.e., the business model, the killer app, and the mediating device?

Brady Forrest: This is not the year.

Tish Shute: Then I should ask you. Are you in the Goggles camp? That is do you think AR needs eyewear to go mainstream?

Brady Forrest: I think this may be where we get…we start to see what is going to be the killer app that gets people to buy the hardware that will support AR.  You see what I mean?  And then from there the apps will come out and the hardware will advance in that direction.

I don’t think AR has made that leap yet.  It hasn’t, to use almost a cliché, it hasn’t crossed the chasm yet and it hasn’t proven that it will.  Because I don’t know if…I think it’s difficult to tell right now.  Is it going to be games?  Is it going to be data layers? What is going to drive people to an AR device, especially one fully dedicated to it?

Tish Shute: I think in terms of AR games taking off a bit of help from the mediating device e.g. access to the iphone video buffers would probably be enough to stoke up AR games into being a hot commodity.  But in terms of AR data layers  going mainstream, we need some of the other players in the location space to put together the magic brew on the business model, don’t we?

Brady Forrest: That’s why I’m so curious though…that’s why I gave Yelp their own talk.  They are…Those guys are gang busters, they’re a consumer company, very consumer facing website.  They’ve got amazing data stores.  They do a lot of interesting stuff with their data.  And I don’t think people always give them the geek credit they deserve.

Tish Shute: You began Where 2.0 back in 2004, when as you point out, “‘local search’ was interesting but not yet real” and you have always stressed something that’s proven to be absolutely true which is lifestyle before commerce, right?  And that if location based services were going to be big it was because they meant something in terms of our lifestyle, not just because they told us where to get another good burger.  Right?

I think there’s been a lot of breakthrough in that area this year in terms of what location based services and proximity based social networks are to us now, how they’re changing our lifestyle.  What do you see as the breakthroughs for in 2009 and what are you hoping for in 2010?

Brady Forrest: Well, I think Google Goggles is one of the most exciting things to me.  Having access to a visual search…having someone actually release a visual search engine in that way, to consumers, I think is huge.  You know, you see stuff like that in the labs. But I don’t see it… it’s rare to see it out.

I think Android is huge.  And the way Google is pushing hardware to show off the platform; so the Nexus One being another example and the fact that it’s breaking free from the carriers. Because I think when get away from the carriers we are able to see more innovation, it’s what’s going to allow people or developers and companies to really innovate.

And I think Twitter adding geo-location to their APIs and buying MixerLabs is a huge move. I think Twitter may end up becoming the end-all be-all of location services. They are going to be updated constantly by people; they are going to have a really good grasp, real-time, of what is happening in any one place, at least based on the people.

And then with the addition of the MixerLabs data, they’re going to have more datasets at their ready. As well as any data that they start to collect from the clients themselves, like from TweetDeck.

So there are global clients that are updating Twitter. I think those are some of the most exciting things. And again, just to come back to Yelp, I think Yelp’s Monocle is also pretty significant, just because it’s an AR app that’s being pushed into consumers’ hands.

And we’ll see how useful they find it.

Tish Shute: Gary Gale, Yahoo! Inc., is going to talk on overcoming the business, social, and technological hurdles so we can reach the long promised [Laughs] Hyperlocal Nirvana. I think you’ve outlined some of these obstacles in relation to  AR, where there are obstacles are in terms of mediating device, and bringing all the pieces together including computer vision techniques in order to have an AR view. That’s the AR side of it. But the layer below that, which is the layer where actual location based apps that are beginning to go mainstream now,  are these presenting successful business models for location-based services.

So in short, in your view, what are the big hurdles to Hyperlocal Nirvana before we get to AR, even just for these location-based services?

Brady Forrest: Well, how do you make money?

Tish Shute: Yeah, to put it bluntly. I like John Battelle’s way of putting it [laughs] how do we oxygenate the system!

Brady Forrest: So are location-based services something that you can make money in the long-term? Nokia bought NavTec for $8 billion. And then two years later, they’re giving it away free as part of Ovi Maps.

Tish Shute: Right.

Brady Forrest: I’m assuming that that’s actually part of the plan.  And that although their hand may have been forced by Google with their release of Turn-By-Turn that…but it’s still got to be a hard nut to swallow that this huge investment in location ends up becoming a loss leader to sell more phones.

So, can you make money through subscriptions, through selling apps? And I think that is still being proven. The other one is, can you use advertising? And it’s kind of scary to see that Apple is restricting the use of advertisers to use location.

It came out yesterday or two days ago that advertisers cannot use location, or app developers cannot use location for ads. They can only use location to show something interesting or useful to their customers.

And there’s a lot of speculation that it’s because Apple wants to control the location-based ads that go on the iPhone.

Tish Shute: Yes. I heard a strange rumor.  Actually its an un-strange rumor, a likely rumor in fact,  that Apple and MS are getting together to replace some of the Google aspects of the iPhone like search and maps?

Brady Forrest: Yes, …. Microsoft employees get 10% off at the Apple store. There’s a longstanding relationship between those two companies.

And Android is definitely more of a competitive threat than Windows Mobile is.  And it’s well-known what the relationship between PCs and Macs are. So I don’t think…I don’t find that to be that surprising of a rumor.  I do wonder if it would hurt the iPhone, but it doesn’t surprise me that they would consider it.

Tish Shute: I do know, certainly from the AR point of view, Microsoft has recently hired some of the key researchers, including Georg Klein. And they are looking for more people in the image recognition area so it seems currently MS is going to be making a bigger push not just with PhotoSynth, but with image ID.

So it could be a pretty powerful combo between the iPhone, and Microsoft – they have some of the key computer vision research that would be needed for full AR.

Brady Forrest: Oh, yeah. Microsoft has amazing research depth. They’ve got an amazing team.

Tish Shute: But it is a bit of a mystery to me why Microsoft haven’t done more with Photosynth.  As I noted in my previous post, Nokia’s ImageSpace is beginning to do what many thought Microsoft would do with photosynth two years ago.  And “photo-based positioning systems” -  3d models of the environment to cover every possible angle, and then software that can work out in reverse based on a picture precisely where you are and where your facing could be hugely important to AR.  But that brings me to another mystery why haven’t we seen more from Nokia in this space yet – the N900 doesn’t have a compass?

Brady Forrest: Yeah, I don’t know why Nokia hasn’t made more of a space for themselves in these things. They did a lot of early work in these areas. I think they are trying to…my guess is that they’re trying to restructure themselves. They made some pretty big changes on the web-Ovi made its own division. And they’ve been doing a lot of location-based acquisitions: Places, Gate 5 several years ago, Gossler, just the past six months. And so I think that’s really been their focus…and the research team.

And a large company, since they haven’t found a business model, which is what we’ve been discussing here, they are hesitant to launch it, or to…they don’t really know if this is a business that they need to launch, or if this is an app that they should have there out for fun.

Tish Shute: Yeah. And that’s back to the oxygenation of the system and location.  We really still have some work to do to with the business models

Final question!  At the core of many of today’s business model is the idea of hoarding data – that’s an underpinning.

But ultimately, for open AR, we want a situation where we can really share data so that we don’t really have the data all locked inside one particular browser or app. The current crop of AR browsers aren’t really browsers in the sense that we understand a browser on the web today, because the data’s locked inside each service, Wikitude, Layar, Acrossair etc.

I have become very interested with Federation as a model for solving this, so that we can begin to have an opportunity to build consensual relations around data, sometimes sharing, sometimes not. Federation is my big dream at the moment.  And now we even have something to work with in the Wave Federation Protocol. But how do we get from here to there, where we really have a federated world of data for AR and location-based services? But you think people need to solve the question of business models first?

Brady Forrest: I think people need…I think one potential is ads; so serving up content.  And by ads, I also mean coupons, meals, the Foursquare…. what it looks like Foursquare’s going to do, featured content, which is Layar’s.

So we need to see, is that the way we’re going to sell these? The other is to have the best viewer, which in some ways is a race in selling that, but that’s potentially a race to the bottom, price-wise.

Tish Shute: Right. Do you think Google Wave Federation Protocol has a chance of taking off and changing the game for real-time communications, federation, real-time…

Brady Forrest: Quite possibly with the real-time. I think they need to work on the UI.

Tish Shute: Oh dear we can’t discuss the Wave UI right at the end of the interview – of course I believe it would do better in an AR view!   I know you have to go  now but I have to say Google Wave not standardizing the client/server interface – so we could seem some new UIs for Wave [we are working with PygoWave for ARWave because of this], and  iPad’s lack of camera were two huge disappointments in recent months.

Brady Forrest: Yeah. It’s [the Wave client] is very difficult to use.

Tish Shute: But the Wave Federation Protocol is an open fast, compact protocol that is a dream come true for AR.  Open, distributed, real time communications is a very big enabler for AR.  I would hazard a guess that in 2010 real time communications plus location becomes oxygen.

Visual search is heating up, and with it a key stage of turning the physical world into a platform is underway as images become hyperlinks to the world in applications like Google Goggles, Point and Find, and SnapTell – see this post by Katie Boehret.  And while there may be no truly game changing augmented reality goggles for a while, make no mistake, key aspects of our augmented view, factors that will have a lot to do with what we will actually see when an augmented vision of the world is a commonplace, are already in the works.  And, as Anselm Hook (pic above from @caseorganic’s flickr) notes:

“There is a real risk of our augmented reality world being owned by interests which are not our own. There is a real question of when you hold up that AR goggle, what are you going to see?”

Cooperating services, e.g., Google Earth, Maps, Streetview, Google Goggles, and leader in local search like Yelp (see here) would have an enormous ability to filter and control a mobile, social, context aware view of the physical world, and Google themselves see an ethical quandary.

“A Google spokesperson says this app has the ability to use facial recognition with Goggles, but hasn’t launched this feature because it hasn’t been built into an app that would provide real value for users. The spokesperson also cites “some important transparency and consumer-choice issues we need to think through” (quote from Wall Street Journal Column by Katie Boehret).

Anselm Hook and Paige Saez, with great prescience, have been advocating a social commons for the placemarks and imagemarks to our physical world platform through a number of pioneering projects, including imagewiki.   I have interviewed both Anselm and Paige (upcoming) in depth, recently.  My talk with Anselm was nearly three hours long!  So I am publishing the transcript in two parts.

Understanding what it means to have a social commons for  our physical world platform, and augmented reality, are key questions for all of us to think about, but especially important for those of us involved in the emerging industry of augmented reality.

Anselm notes :

“The placemarks and imagemarks in our reality are about to undergo that same politicization and ownership that already affects DNS and content. Creative Commons, Electronic Frontier Foundation and other organizations try to protect our social commons. When an image becomes a kind of hyperlink – there’s really a question of what it will resolve to. Will your heads up display of McDonalds show tasty treats at low prices or will it show alternative nearby places where you can get a local, organic, healthy meal quickly? Clearly there’s about to be a huge ownership battle for the emerging imageDNS”

The mobile internet is moving beyond the internet in your pocket phase of mobility with mobile, social, proximity-based, context aware networks like FourSquare, Gowalla, Brightkite and GraffitiGeo (see Smart Data Collective) likely, soon, to start to take precedence over other forms of social network.

Regardless of the timeline for true augmented reality – 3D images & graphics tightly registered to the physical world,  proximity-based social networking and real time search are already taking us into a hyper-local mode and the realm of augmented reality which is “inherently about who you are, where you are, what you are doing, and what is around you” (Robert Rice – see here). The ground is being prepared for augmented reality now.

If you have been reading Ugotrade, you will know I have been actively involved in developing  an open, distributed AR platform/mobile social interaction utility for geolocated data based on the Wave Federation Protocol – AR Wave a.k.a Muku – “crest of a wave” (see my posts here, here and here for more on this project, and the AR Wave Wiki here).  Federation is, I believe, one vital aspect to developing a social commons for augmented reality and the physical world platform.

Also, a bit of news, I am co-chairing the upcoming Augmented Reality Event (are2010) with Ori Inbar of Games Alfresco and Ogmento, whurley.  Sean Lowery, Prospera, is the event organizer, and are2010 has the support of the AR Consortium.   The are2010 web site is live and there is an Open Call For Speakers.  You can submit your proposals and demos for one of the three tracks, business, technology, or production on the web site here.

Bruce Sterling “prophet” of  augmented reality and more, “will deliver the most anticipated Augmented Reality keynote of the year.”

It didn’t surprise me when Anselm mentioned that Bruce Sterling was a key influence for his work on the geospatial web and augmented reality.  Anselm explained:

“I’d seen a talk by Bruce Sterling at an event called Planetwork [May, 2000]. And that event was, for me, a turning point where I decided to focus full time on exactly what I cared about instead of doing things that were kind of similar to what I cared about. So, his influences is a pretty significant one to me at that exact moment.”

For more see viridiandesign.org -  seems it is time for a “Neo-Viridian,” revival!

This post by Bruce Sterling on Pachube Feeds, and Thomas Wrobel’s prototype design for open distributed augmented reality on IRC, were key inspirations for me when I began thinking about the potential of Google Wave Federation protocol for augmented reality.  I had been exploring Pachube and deeply interested in the vision of Usman Haque, but I had a real aha moment when I read this :

“(((Extra credit for eager ubicomp hackers: combine this [pachube feeds] with Googlewave, then describe it in microsyntax. Hello, 2015!)))”

I think the AR Wave group will earn the extra credit and more very soon!  Davide Carnovale, need2revolt, and Thomas Wrobel have been leading the coding charge, and there will be a very early AR Wave demo soon, perhaps as soon as the Feb 16th ARNY Meetup. 

Open access to the creation of view that will eventually find its way into AR goggles, will depend not only on the power of  an open distributed platform for collaboration like the AR Wave project.  Our augmented reality view will be constructed through complex “hybrid tracking and sensor fusion techniques” (Jarell Pair), cooperating cloud data services, powerful search and computer vision algorithms, and apps that learn by context accumulation will drive our augmented experiences, and at the moment, these kind of resources, at least at scale, are for the most part in private hands.

In the interview below, Anselm’s discusses  how trust filters, and being able to publicly permission your searches so that other people can respond and so that people can reach out to you, and the democratization of data in general, are even more of a concern with augmented reality and hyper local search. The task of understanding what it means to have  a social commons for the outernet remains an open, and pressing question.

Anselm explains (see full interview below):

“as we move towards a physical internet where there’s no clicking and there’s no interface and the computer’s just telling you what it thinks you’re looking at, translating, you know, an image of a billboard to the name of the rock star who’s on that billboard, or translating the list of ingredients on a can of soup to the source outlets where it thinks that, those ingredients came from. When you have that kind of automated mediation, the question of trust definitely arises.

And we haven’t seen the Clay Shirkys or the Larry Lessigs of the world start to talk about this yet.  Although I suspect that in the next four or five years that the zero click interface will become the primary interface, that we’ll have…we’ll come to assume that what we see with the extra enhanced data we get projected onto our view is the truth. Yet, at the same time, there is just no structure or mechanism even being considered for a democratic ownership of it.”

Augmented Reality will emerge through sensor fusion techniques & cooperating cloud services

In 2010, sensor fusion techniques, computer vision technology in conjunction with GPS and compass data will create data linking that can enable the kind of augmented reality that has been the stuff of imagination for nearly four decades (see Jarrell Pair’s post).

Putting stuff in the world in 3D is of course key to the original vision of augmented reality, and one of its biggest challenges.  Augmented reality is going to be implicated in a real time mapping of the world at an unprecedented scale and granularity.  We have barely an inkling of the implications of this now.

Anselm and Paige have been working in the heart of the social cartography movement for nearly a decade.  The vision and experience of this community is vital to understanding how augmented reality and the world as a physical platform can evolve into something that benefits people and allows them “to have a better understanding of the opportunities around them.”

We have been hacking maps for millenia –  “from conceptual story mapping, to colloquial mapping in European development and the cartographic renaissance created by the global voyages and rediscovery of Ptolemy’s maps” (Andrew Turner).  And, recently, initiatives on a public-provided GIS, like OpenGeo, have led the way toward more open, interoperable, geospatial data.

Mapping takes on a new an crucial role to augmented reality.  Nokia’s ImageSpace is beginning to do what many thought Microsoft would do with photosynth two years ago.

And, if we see these kind of projects developed into a “photo-based positioning systems” -  “3d models of the environment to cover every possible angle, and then software that can work out in reverse based on a picture precisely where you are and where your facing” (Thomas Wrobel), we would find augmented reality leap forward over night.

It is time to take very seriously the vast opportunities and potential pitfalls of an augmented world.

“when you are mediating the translation layer between the image and the data, then there is an opportunity for you to control it, and that opportunity is hard to resist.  It is hard to choose not to own that opportunity. It is an advertising opportunity. It is a revenue opportunity. It is a chance to send a message and a tone.

I know that Google and companies like that are keenly aware of the kinds of roles they don’t want to hold, but it is sometimes seductive to think about them. And I am afraid that we, as a community, need to assert an ownership, kind of a commons, over how computers will translate what they see to information that we perceive.”

There are some initiatives emerging.  Tonchidot (who closed on $4 million of VC for augmented reality last December) has helped create the AR Commons in Japan.  CFO of Tonchidot, Ken Inoue explained in an interview with me in September 2009.

“We feel that public data, such as landmarks, government facilities, and public transport should be shared. We see an AR world where people can readily and easily access information by just seeing – quick, easy, and efficient.  And because of this ease and intuitiveness, children, the elderly and handicapped will surely benefit.  AR could help create a safer society.  Warnings, alerts, and safety information could save lives and avoid disasters.  These are what we, and AR Commons would like to tackle in the not so distant future.”

But the task of building a social commons for the physical world platform has only just begun.

Interview with Anselm Hook

photo from Anselm’s Flickr stream here

Tish Shute: We first met last year at Wherecamp. The start of 2009 was I think the “OMG finally” moment for augmented reality and in less than a year AR, at least in proto forms, AR is breaking into the mainstream now! You are one of the founding visionaries/philosophers/hackers of the geo web and you have been thinking about geo web and AR for a long time – all the way back to the legendary Head Map Manifesto, and before.  Mostly recently you led the way in the very successful ARDevCamp in Mountain View. Could you start by telling me a little bit about the history of your pioneering work with geolocated data?

Anselm Hook: I am a long time Geo fanatic. I’m really interested in social cartography and what some people call public-provided GIS, that’s some language that people use. Anyway, my personal interest, when I talk to people who are non-technical (and it’s been a long term interest in the way I phrase it) is that I want to help people see through walls. So, the goal is very simple. I want people to have a better understanding of opportunities around them, the landscape around them. I always get frustrated when people make bad decisions because of a lack of information, especially when it’s related to their community and related to their environment. But, plainly put, I really just want “to help people see through walls”. It’s a very simple goal.

Tish Shute: I know you worked on Platial, which is really one of my favorite social mapping applications. It really broke new ground. What was the history of that? How did you get involved with Platial?

Anselm Hook: That’s an interesting question. It actually started at around 2000 when I saw Bruce Sterling talk. I had been writing video games for many years, and I was quite good at it, and I enjoyed it. But, the reasons I was doing it diverged from why the industry was doing it. I was making video games because I like to make shared spaces for my friends to play in and to share experience. I really enjoyed making shared environments. I worked on BBS’s and my friends and I were always making these collaborative shared environments.

Once the video game industry kind of started to take off, I started to do high performance, 3D interactive video games and making compelling shared spaces, and it was a lot of fun. But, the frustration for me was that there was a huge industry growing around it and became very commercial. Although it paid well, it started to diverge from my values which were more centered around community environments, and shared understanding.

Tish Shute: Yes very rapidly, the big games kind of devolved from the social aspects and became more and more into single player really, didn’t they?

Anselm Hook: It was the way, actually, because even though often you were in a many player world, you weren’t collaborating, everything else became just a target.  I liked the idea of deep collaboration that calls the kind of playful space you see in IRC, or in the real world, where people are solving real world problems.

And I grew up in the Rockies, and I was always had a lot of access to the outside. So, I saw shared spaces and collaboration as a way to protect our environment. [ To step back ] I think people used different metrics for measuring their choices in the world and many people have a value system centered around minimization of harm: making sure that the people are not hurt. But, my value system is different. I personally believe that protecting the planet is more important: to maximize biodiversity. I feel like protecting people around me comes from protecting the ecosystems they live in.

Tish Shute: That’s interesting, isn’t it, because the history of Keyhole was really that, wasn’t it.  Keyhole later became Google Earth, but I mean it began out of a project to look at what was going on in the ecosystem over Africa at that time, didn’t it?

Anselm Hook: Yes, in fact many people’s projects are stemming from an environmental concern. Mikel Maron’s works for example – he’s doing Map Kiberia, and he also worked on OpenStreetMaps.

Tish Shute: Map Kiberia – that is the new project?

Anselm Hook: Oh, yes his project is called Map Kiberia. He’s mapping a city in Africa.
[For more see Map Kiberia’s YouTube Channel – photo below from ricajimarie ]

Tish Shute: Right, great!

Anselm Hook: When I started to look at GIS and mapping I started to meet people who had a very similar background. What happened to me is I kind of stepped away from games around the year 2000. I’d seen a talk by Bruce Sterling at an event called PlaNetwork. And that event was, for me, a turning point where I decided to focus full time on exactly what I cared about instead of doing things that were kind of similar to what I cared about. So, his influences is a pretty significant one to me at that exact moment.

[For more see viridiandesign.org – seems that it is time for a “Neo-Viridian,” revival.]

Tish Shute: It’s interesting because now your paths are crossing again with augmented reality. You are on the same wavelength again.

Anselm Hook: It’s funny, actually, I’ve had a couple of brief overlaps in that way.  Well, so in 2000 I went to see this talk and I did a small project called — well, I called it SpinnyGlobe. What I did is I mapped protests from a number of websites onto a globe to show the level of community opposition to the pending war in Iraq. It was the first time there had been a protest before a war. So, it was very interesting to me. [ See http://hook.org/headmap ]

Tish Shute: That’s really fascinating. Do you have any pictures of that you could send me?

photo from anselm’s flickrstream

Tish Shute: Yes, I’ll definitely look SpinnyGlobe up. It sounds very interesting.  One of the aspects of your work on geo-located data projects like this and Platial is that you really started to develop this idea of a culture of place, about how people make place. This was the wake up call to me regarding the power of networks combined with geo-data.

We are hoping to extend this idea into augmented reality with the an open distributed platform for AR so that we can collaboratively map our worlds from the perspective of who we are, where we are, and what we are doing.  I know you’ve just done some work recently in augmented reality.  I know you put the code up already.

By the way, I love the way you take your philosophy into the way you make code – the practice of making some code, trying some things out, making it all public and publishing your findings, you know, your comments on that experience.  Perhaps you could recap sort of how you picked up recently on the state of play with augmented reality and what aspects you looked at, and what came out of that experience?

Anselm Hook: So, it’s a very simple trajectory. Coming out of the work I had done, Platial, among other projects and I started to just look at the hyper-local and I suddenly realize that even those services weren’t really speaking to living, and how to really see and solve local problems. What was missing was a sense of context.

The map doesn’t know how you’re feeling, it doesn’t know if you’re in a hurry, it doesn’t know what you want, it’s very static. Even the web maps are very static. And augmented reality for me I started to recognize as a combination of — well — it’s probably collision of many forces, many forces that we’re all a part of. We’ve also didn’t realize that the real-time web is really important, it’s part of what AR is about.

We have all started to realize that the context is important. You know, your personal disposition, your needs, if you want to be interrupted or not. That is the kind of thing that the ubiquitous computing crowd has talked about. We started to recognize that there are sensors everywhere, and the ambient sensing communities talked about that. So what is funny for me about augmented reality is I started realizing it is just a collision of many other trends into something bigger.

Everything else we thought was a separate thing is actually just part of this thing. Even things like Google Maps or mapping systems we think are so great are really just kind of almost an aspect of a hyper-local view. You actually don’t really care what is happening 10 blocks away or 100 blocks away. If you could satisfy those same interests and needs within a single block, one block away, you would probably be really happy. You really just want to satisfy needs and interests, find ways to contribute, or get yourself fed, or whatever it is you want. And AR seemed to be the playground to really explore the human condition.

Tish Shute: Anyway, I think one of the things that has been very amazing this year is we to have the good mediating devices that, for the first time, give us compasses, GPS, and accelerometers. But one sort of missing pieces with AR at the moment is [tracking, mapping, and registration] – the kind of things colloquial mappings of the world could be of great help with.

We have seen mapping coming out of the Flickr data, e.g., the University of Washington, put the maps together from the geo-tagged Flickr photos. Now if we could have that linked up with AR, then we have the kind of mapping we need to kind of really hook the geo-data onto the world in a way that goes beyond…you know, what compass and GPS can really deliver is pretty minimal at the moment.

Anselm Hook: There is a real risk of our augmented reality world being owned by interests which are not our own. There is a real question of when you hold up that AR goggle, what are you going to see? Are you going to see corporate advertising? Are you going to see your friends’ comments or criticisms? It is going to be an Iran or a democracy, right? It is unclear.

Right now there are some disturbing trends I have noticed. I am a big fan of Google Goggles. I think it is a great project. But when you are mediating the translation layer between the image and the data, then there is an opportunity for you to control it, and that opportunity is hard to resist. It is hard to choose not to own that opportunity. It is an advertising opportunity. It is a revenue opportunity. It is a chance to send a message and a tone.

I know that Google and companies like that are keenly aware of the kinds of roles they don’t want to hold, but it is sometimes seductive to think about them. And I am afraid that we, as a community, need to assert an ownership, kind of a commons, over how computers will translate what they see to information that we perceive.

Tish Shute: Yes. And this is how we met, again, recently [over the project to create an open, distributed platform for AR using the Wave Federation Protocol]…

This is something I feel really deeply is that, you know, basically we need the physical internet to be as open as, as the, as the internet, as the end-to-end internet has been. Or more so, actually, because the end-to-end internet has seen the trend has been to walled gardens.  Basically Facebook became enormous, an enormous walled garden which, I think, was despite, our predictions about them, [walled gardens] are the social experience really on the web.  It’s very much in walled gardens still and I, and I really feel that with the physical internet, we need to make great efforts not for it not just to be a series of small pockets of privately funded walled gardens.

There needs to be some kind of communications infrastructure that keeps it open so that was when I got interested in looking at the Wave Federation Protocol because it was a real time, you know, an open real time protocol that could possibly be a basis for that. But I think the point you’ve talked to just now, the mapping of the world and who has the “goggles”, i.e., the image data, image databases, that make the world meaningful is really, that’s still a, it’s still a BIG question [i.e. who controls the view?].

When I saw ImageWiki, [I realized] that is a piece that is vital for, for augmented reality. We need to have a huge social effort to be involved in this,  linking in and creating the  physical internet, in creating the image hyperlinks that will make that meaningful.

Anselm Hook: I think that’s a great point. The search interface, the kind of Internet that we’re used to, the way we talk to the network now, is fundamentally open end to end. Yes, you can have your oligarchies inside of it, as we see with Facebook, but you can always start your own venture up and you can do a search on something, and you can find that, that website and you can join it or you can put up your own webpage and people can find it.

The translation layer, the idea of text search and the ability to discovery power and the serendipity and the openness of that discovery, it’s pretty open right now. We do have some serious boundaries of language, which is one of the reasons I was working at the Meedan.org [hybrid distributed, natural language translation] for a couple of years, trying to bridge that issue.

But here, as we move towards a physical internet where there’s no clicking and there’s no interface and the computer’s just telling you what it thinks you’re looking at, translating, you know, an image of a billboard to the name of the rock star who’s on that billboard, or translating the list of ingredients on a can of soup to the source outlets where it thinks that, those ingredients came from. When you have that kind of automated mediation, the question of trust definitely arises.

And we haven’t seen the Clay Shirkys or the Larry Lessigs of the world start to talk about this yet.  Although I suspect that in the next four or five years that the zero click interface will become the primary interface, that we’ll have…we’ll come to assume that what we see with the extra enhanced data we get projected onto our view is the truth. Yet, at the same time, there is just no structure or mechanism even being considered for a democratic ownership of it.

We have with DNS, for example, the idea that you can register the domain name and people can search for it, and find it, and go to it. There’s no such thing as an Image DNS, or an image translation to DNS right now. What does it mean when everything is just “magic”, when there’s no way for you to be a part of the conversation, where you’re just a consumer of what people tell you, or of what one company right now, tells you, is reality? That’s a real concern.

Tish Shute: This, to me is the most important question at the moment. I mean, it’s the big one and it’s the place to put energy if you love the Internet [and what it can now become] right. You’ve got to put a lot of energy into this because this [a democratized view of the physical world as a platform] won’t just happen, because there’s a lot of momentum already for it to be heavily privatized, partly because, one reason is, some of the computer vision algorithms that, say, make sense of things like the geotag photographs are not open.  I mean, for example, the beautiful maps that have been made from the University of Washington [from Flickr geotagged photo sets], that isn’t in the public domain.

Anselm Hook: Right. Tish, and in fact you’re referring to [with the maps from the Flickr photos] to ordinary maps and the fact we’ve already seen that maps lie, we’ve already, seen how much maps are reflecting a certain truth that becomes the normative truth. Google maps reflects roads, because this is roads and cars, right? Only recently have they thought about buses and walking. So the normative view that people assume is the reality, is showing off you know Starbucks, and roads, and cars, that becomes the default, those prejudices are just assumed, you know, the truth. But they’re not the truth at all.

I was talking to a friend of mine in Montreal, [Renee Sieber], and she said that their Indian portage routes are a bridge across land and water, they don’t think of a piece of land and a piece of water as being different things, they think of them as one thing: a route. It’s already a different kind of language we can’t even reflect it.

So not only is there this kind of formal, anthropological lie, in a sense, but there’s this way that we deceive ourselves because of our own prejudices.

Tish Shute: Yes I agree and that’s why I think when I saw some of the things you had written on the ImageWiki point clearly to the need to create a social commons. We need a social commons for the real-time physical internet, we need it for the image hyperlinks that make sense of that.

And it’s a complicated thing in a sense, though, because we don’t actually have a good distributed infrastructure for AR yet, and I found exploring AR Wave, that at last we have the suggestion of an open, federated protocol for real-time communication – the wave federation protocol. [Real time communications is a very important part of AR].  It isn’t an actuality yet where lots of people are able to use it, set up their own servers, and there’s not a standard all the way through  [there is not a standard for how data is sent between the client and the server].

But Wave Federation Protocol does make possible truly distributed social AR.  I started thinking when I saw ImageWiki that to bring ImageWiki together with the social collaborative power of distributed AR.  This really would be the basis of creating a social commons for augmented reality and the physical world as a platform – the start of a bottom up with deep social collaboration on how we create augmented reality colloquial maps that can inform a hyper-local of the world.

Anselm Hook: Yes. When Paige Saez, John Wiseman, and myself, and a few other folks… You know, Benjamin Foote, Marlin Pohlmann, and a couple other people started to play with this, we quickly found that… We started to realize, “Oh, this kind of thing will be at least as popular as IRC. There will be at least as many people doing this as chatting in little virtual spaces. There’ll be at least as many people decorating the world with augmented reality markup, and maybe using the real world as a kind of barcode for translating what you’re looking at into an artifact, a digital artifact.

And that the size of that space was going to be huge, basically. Maybe not quite as commodifiable as Twitter, but certainly very energetic.

Many of the projects we did were just kind of looking at these kinds of issues sort of from an artistic, technical, and political point of view. We weren’t so much posing complete solutions, but simply using a praxis to explore the idea with an implementation, as a foundation for this discussion. So I think we sort of opened that can of worms for sure.

Tish Shute: Did you actually set up ImageWiki to be working as a location based app yet?

Anselm Hook: It is a location based app. It collects your longitude, latitude, and the image and stores it. And then it uses that as a way to translate that image to anything else. It could be a piece of text or a URL.

Tish Shute: So there is a smartphone app, but you didn’t take it as far as an AR app yet?

Anselm Hook: No. We didn’t do a heads-up view. There are apps on the iPhone store that do that, but they don’t do the brute force image recognition that we were using. We used a third party off the shelf algorithm that we found on Wikipedia and downloaded the source code, and threw it on the server. And John Wiseman in LA wrote the scalable database backend so that we could scale the actual…

Tish Shute: So how did you set the iphone app up to work?

Anselm Hook: The iPhone side was very simple. You take a picture of something and it tells you what it is. That is all it did. We would take the location, but the client side, the iPhone side, just rendered, returned to you…It said, “Someone said that this picture of a barking dog is an advertisement for a local band.”

Tish Shute: Right. So basically it was a geo-tagged?

Anslem Hook: Yes. We are just collecting the geo information. Actually, there were a whole lot of technical challenges. The whole idea of ImageWiki is actually kind of beyond our technical ability for a small team like us. It really does take a team, a group like Google, to do this kind of thing in a scalable way.

Tish Shute: Why is that?

Anslem Hook: There are two sides. There is the curating the images. I think that is the job of groups like us – open source groups who can curate images that are owned by the community. And then the searching side, the algorithm side, where you are actually matching the fingerprint of one image to images in your database, that takes a much more…that is much more industrial.  We get both sides, ours is not a scalable solution. It is mostly…proving that it could be done was important.

Tish Shute: In terms of hooking Imagewiki up to the collaborative possibilities of AR Wave wouldn’t federation pose some interesting possibilities for scaling search algorithms and all that?

Anselm Hook: Yes. And what is funny also, incidentally, is that, nevertheless, we did look for some financial support for it, but we couldn’t…we just didn’t find the investors to scale it. Now, other companies like SnapTell took a shot at it. And they have an app in the iPhone store where you can point at a beer bottle and get back the name of the beer bottle.

The classic example everyone uses is a book. Amazon has all the image jackets of all their books. You can point SnapTell at almost any book and get back links to buy that at Amazon, the price of the book, and user comments on the book. So they are treating Amazon as the canonical voice of the book, for better or worse. That is the state of the art so far, up until Google Goggles came out a little while ago, which actually blows it out of the water. But, that is where we are now.

Tish Shute: Right. But the point you raise about how when something like Amazon comes canonical of what is book, right, this is the whole point, isn’t it?

Anselm Hook: Is Amazon truth? It’s not bad. Jeff Bezos seems like a nice guy, but, you know.

Tish Shute: And this is the point of having these open infrastructures for this.  And this should be obvious in a way, but it comes back to the thing about what made the Internet great was the fact that even though as you note, you get an oligarchy like Facebook, but people always could just go off and do something else, right? Because the fundamental infrastructure was basically open and designed to be available for everyone. And many people have championed that and fought for it hard [to maintain this openness] haven’t they? They have devoted their lives to keeping it that way, even if the oligarchies have done their thing.

Anselm Hook: Yes. There are really some things that are underneath all of this that haven’t been solved yet.

One is that the trust in social networks has not been built yet, so we can’t do peer based recommendations very well. We can’t filter noise by peers. Twitter kind of is moving there, but I don’t just want to listen to my Twitter friends. I want to listen to my friends of friends. If I am getting truth from somebody, I want to get that truth from people my friends say that they trust.

Then the second problem is that there is a search business. My friend Ed Bice, who owns Meedan, always says that a search itself, a search request, is an opportunity to make…is a publishing moment. It is an opportunity to say what you think. In the real world, if you are just hanging out with humans and you look somewhere, other people might look at your gaze and they might look at what you are looking at. Your gaze itself is a public act.

Gaze is a soft act, but it is one that is visible. With Google, the gaze of four billion people is invisible. We don’t what people are looking at, there is no opportunity to participate. Let me give you a real example.  I have taken a image of something of the bust of figure or a statue.  Why can’t the museum in Cairo look at my request and tell me oh yeah that is Tutankhamen, or that is Nefertiti right? Why can’t they have a chance to participate in the search and respond to me?

Right now the the only person that responds is Google when I do a search. We need to invert the search pyramid and open up search, so that search is a democratic act, so that you can publicly permission your searches so that other people can respond and so that people can reach out to you, not just you having to do a dialogue.

The common example of this.. and we see this everywhere: I am looking for a slice of pizza right, now I am hungry I want some pizza. I have to ask Google, look find twelve websites, call twelve phone numbers, and talk to each of the twelve stores, and ask them are they open late, is the food organic, is the food in any good, do my friends like it.

Whereas what I should be able to do is just say it’s a search moment and I am interested in pizza. If those pizza places my criteria like you know my friend’s like them and they are organic, they are open, then that pizza place can call me. I have the money why should I do the search? So the whole business of search, the whole structure of search is predicated around a revenue model, but its a really short-sighted revenue model, its not a brokerage.

Search isn’t search, search is hand waving.  These should be moments for us to have a discourse. So problem we are seeing in AR with communication of the right information is actually underneath AR, at the level of the whole infrastructure.

Search needs to be inverted, trust filters need to be built. We need to democratically own our data institutions.  We don’t right now.  That will be more of a concern, especially with AR.

Tish Shute: Yes, especially with AR, which is this why got all excited about federation.  Do you think federation has the potential, an opportunity to create [the new infrastructure you describe?]

Anselm Hook: Absolutely,  its absolutely what we must do. It is much harder to do. It is absolutely critical.

Tish Shute: And why is it much harder to do? Could you explain that?

Anselm Hook: Well, it’s very easy for a bunch of hackers to build a service that you log into and fetch some data, it’s a single thing. They don’t have to talk anybody, they can use their own protocols, they can hack it, it’s a big black box, behind the scenes. There’s running back and forth in a giant Chinese room delivering manuscripts and scrolls to you. Whatever is behind the black box, you don’t care, it just works.  But when you federate, you need to actually publish and have standards, and then you’re talk about semantic, everyone starts getting really excited and wave some hands. It becomes a disaster. It’s, at least, another power order, more difficult than DIY, build it yourself.

Tish Shute: So, in terms of what Google Wave have done with their approach to federation, what do you think have been their achievements and what do you think is their obstacles? What do you think are the failings of the Wave? Because it’s the first big public major player backed approach to something federated, isn’t it? In real time.

Anselm Hook: Yes. I think the most important non-federated service on the planet today is Twitter.  Identi.ca it’s not getting any traction with respect to Twitter. [ Even though ] Identi.ca is a federated version of Twitter and is very good. [ Identica is now Status.net ] . So, we see already there that small players aren’t being competitive. Then look at other services like IRC. IRC is the secret backbone of the Net. All the open source projects, all the teams, all the people that work on opensource projects are all on IRC. It’s the only way they get anything done.

With Google Wave, and the protocols underneath Google Wave, we see an attempt to build a similar kind of real time, but distributed protocol. I think it’s the right direction. I think, people should pick up the offering and make their own servers. I think that protocol is really great, I think the fact that is compressed, its high performance, it is small, real-time of blobs of data flying around, all exactly the way it should be done. It is getting close to this kind of rewrite of the Internet that people keep talking about, because, you know, the net protocols are so bad, it is starting to treat the idea of intermittent exchanges being more transitory, volatile, and not heavy.

….to be continued.  Part 2 coming soon!

Click on the image below or here to watch this presentation and others from Momo13