Visual search is heating up, and with it a key stage of turning the physical world into a platform is underway as images become hyperlinks to the world in applications like Google Goggles, Point and Find, and SnapTell – see this post by Katie Boehret.  And while there may be no truly game changing augmented reality goggles for a while, make no mistake, key aspects of our augmented view, factors that will have a lot to do with what we will actually see when an augmented vision of the world is a commonplace, are already in the works.  And, as Anselm Hook (pic above from @caseorganic’s flickr) notes:

“There is a real risk of our augmented reality world being owned by interests which are not our own. There is a real question of when you hold up that AR goggle, what are you going to see?”

Cooperating services, e.g., Google Earth, Maps, Streetview, Google Goggles, and leader in local search like Yelp (see here) would have an enormous ability to filter and control a mobile, social, context aware view of the physical world, and Google themselves see an ethical quandary.

“A Google spokesperson says this app has the ability to use facial recognition with Goggles, but hasn’t launched this feature because it hasn’t been built into an app that would provide real value for users. The spokesperson also cites “some important transparency and consumer-choice issues we need to think through” (quote from Wall Street Journal Column by Katie Boehret).

Anselm Hook and Paige Saez, with great prescience, have been advocating a social commons for the placemarks and imagemarks to our physical world platform through a number of pioneering projects, including imagewiki.   I have interviewed both Anselm and Paige (upcoming) in depth, recently.  My talk with Anselm was nearly three hours long!  So I am publishing the transcript in two parts.

Understanding what it means to have a social commons for  our physical world platform, and augmented reality, are key questions for all of us to think about, but especially important for those of us involved in the emerging industry of augmented reality.

Anselm notes :

“The placemarks and imagemarks in our reality are about to undergo that same politicization and ownership that already affects DNS and content. Creative Commons, Electronic Frontier Foundation and other organizations try to protect our social commons. When an image becomes a kind of hyperlink – there’s really a question of what it will resolve to. Will your heads up display of McDonalds show tasty treats at low prices or will it show alternative nearby places where you can get a local, organic, healthy meal quickly? Clearly there’s about to be a huge ownership battle for the emerging imageDNS”

The mobile internet is moving beyond the internet in your pocket phase of mobility with mobile, social, proximity-based, context aware networks like FourSquare, Gowalla, Brightkite and GraffitiGeo (see Smart Data Collective) likely, soon, to start to take precedence over other forms of social network.

Regardless of the timeline for true augmented reality – 3D images & graphics tightly registered to the physical world,  proximity-based social networking and real time search are already taking us into a hyper-local mode and the realm of augmented reality which is “inherently about who you are, where you are, what you are doing, and what is around you” (Robert Rice – see here). The ground is being prepared for augmented reality now.

If you have been reading Ugotrade, you will know I have been actively involved in developing  an open, distributed AR platform/mobile social interaction utility for geolocated data based on the Wave Federation Protocol – AR Wave a.k.a Muku – “crest of a wave” (see my posts here, here and here for more on this project, and the AR Wave Wiki here).  Federation is, I believe, one vital aspect to developing a social commons for augmented reality and the physical world platform.

Also, a bit of news, I am co-chairing the upcoming Augmented Reality Event (are2010) with Ori Inbar of Games Alfresco and Ogmento, whurley.  Sean Lowery, Prospera, is the event organizer, and are2010 has the support of the AR Consortium.   The are2010 web site is live and there is an Open Call For Speakers.  You can submit your proposals and demos for one of the three tracks, business, technology, or production on the web site here.

Bruce Sterling “prophet” of  augmented reality and more, “will deliver the most anticipated Augmented Reality keynote of the year.”

It didn’t surprise me when Anselm mentioned that Bruce Sterling was a key influence for his work on the geospatial web and augmented reality.  Anselm explained:

“I’d seen a talk by Bruce Sterling at an event called Planetwork [May, 2000]. And that event was, for me, a turning point where I decided to focus full time on exactly what I cared about instead of doing things that were kind of similar to what I cared about. So, his influences is a pretty significant one to me at that exact moment.”

For more see viridiandesign.org -  seems it is time for a “Neo-Viridian,” revival!

This post by Bruce Sterling on Pachube Feeds, and Thomas Wrobel’s prototype design for open distributed augmented reality on IRC, were key inspirations for me when I began thinking about the potential of Google Wave Federation protocol for augmented reality.  I had been exploring Pachube and deeply interested in the vision of Usman Haque, but I had a real aha moment when I read this :

“(((Extra credit for eager ubicomp hackers: combine this [pachube feeds] with Googlewave, then describe it in microsyntax. Hello, 2015!)))”

I think the AR Wave group will earn the extra credit and more very soon!  Davide Carnovale, need2revolt, and Thomas Wrobel have been leading the coding charge, and there will be a very early AR Wave demo soon, perhaps as soon as the Feb 16th ARNY Meetup. 

Open access to the creation of view that will eventually find its way into AR goggles, will depend not only on the power of  an open distributed platform for collaboration like the AR Wave project.  Our augmented reality view will be constructed through complex “hybrid tracking and sensor fusion techniques” (Jarell Pair), cooperating cloud data services, powerful search and computer vision algorithms, and apps that learn by context accumulation will drive our augmented experiences, and at the moment, these kind of resources, at least at scale, are for the most part in private hands.

In the interview below, Anselm’s discusses  how trust filters, and being able to publicly permission your searches so that other people can respond and so that people can reach out to you, and the democratization of data in general, are even more of a concern with augmented reality and hyper local search. The task of understanding what it means to have  a social commons for the outernet remains an open, and pressing question.

Anselm explains (see full interview below):

“as we move towards a physical internet where there’s no clicking and there’s no interface and the computer’s just telling you what it thinks you’re looking at, translating, you know, an image of a billboard to the name of the rock star who’s on that billboard, or translating the list of ingredients on a can of soup to the source outlets where it thinks that, those ingredients came from. When you have that kind of automated mediation, the question of trust definitely arises.

And we haven’t seen the Clay Shirkys or the Larry Lessigs of the world start to talk about this yet.  Although I suspect that in the next four or five years that the zero click interface will become the primary interface, that we’ll have…we’ll come to assume that what we see with the extra enhanced data we get projected onto our view is the truth. Yet, at the same time, there is just no structure or mechanism even being considered for a democratic ownership of it.”

Augmented Reality will emerge through sensor fusion techniques & cooperating cloud services

In 2010, sensor fusion techniques, computer vision technology in conjunction with GPS and compass data will create data linking that can enable the kind of augmented reality that has been the stuff of imagination for nearly four decades (see Jarrell Pair’s post).

Putting stuff in the world in 3D is of course key to the original vision of augmented reality, and one of its biggest challenges.  Augmented reality is going to be implicated in a real time mapping of the world at an unprecedented scale and granularity.  We have barely an inkling of the implications of this now.

Anselm and Paige have been working in the heart of the social cartography movement for nearly a decade.  The vision and experience of this community is vital to understanding how augmented reality and the world as a physical platform can evolve into something that benefits people and allows them “to have a better understanding of the opportunities around them.”

We have been hacking maps for millenia –  “from conceptual story mapping, to colloquial mapping in European development and the cartographic renaissance created by the global voyages and rediscovery of Ptolemy’s maps” (Andrew Turner).  And, recently, initiatives on a public-provided GIS, like OpenGeo, have led the way toward more open, interoperable, geospatial data.

Mapping takes on a new an crucial role to augmented reality.  Nokia’s ImageSpace is beginning to do what many thought Microsoft would do with photosynth two years ago.

And, if we see these kind of projects developed into a “photo-based positioning systems” -  “3d models of the environment to cover every possible angle, and then software that can work out in reverse based on a picture precisely where you are and where your facing” (Thomas Wrobel), we would find augmented reality leap forward over night.

It is time to take very seriously the vast opportunities and potential pitfalls of an augmented world.

“when you are mediating the translation layer between the image and the data, then there is an opportunity for you to control it, and that opportunity is hard to resist.  It is hard to choose not to own that opportunity. It is an advertising opportunity. It is a revenue opportunity. It is a chance to send a message and a tone.

I know that Google and companies like that are keenly aware of the kinds of roles they don’t want to hold, but it is sometimes seductive to think about them. And I am afraid that we, as a community, need to assert an ownership, kind of a commons, over how computers will translate what they see to information that we perceive.”

There are some initiatives emerging.  Tonchidot (who closed on $4 million of VC for augmented reality last December) has helped create the AR Commons in Japan.  CFO of Tonchidot, Ken Inoue explained in an interview with me in September 2009.

“We feel that public data, such as landmarks, government facilities, and public transport should be shared. We see an AR world where people can readily and easily access information by just seeing – quick, easy, and efficient.  And because of this ease and intuitiveness, children, the elderly and handicapped will surely benefit.  AR could help create a safer society.  Warnings, alerts, and safety information could save lives and avoid disasters.  These are what we, and AR Commons would like to tackle in the not so distant future.”

But the task of building a social commons for the physical world platform has only just begun.

Interview with Anselm Hook

photo from Anselm’s Flickr stream here

Tish Shute: We first met last year at Wherecamp. The start of 2009 was I think the “OMG finally” moment for augmented reality and in less than a year AR, at least in proto forms, AR is breaking into the mainstream now! You are one of the founding visionaries/philosophers/hackers of the geo web and you have been thinking about geo web and AR for a long time – all the way back to the legendary Head Map Manifesto, and before.  Mostly recently you led the way in the very successful ARDevCamp in Mountain View. Could you start by telling me a little bit about the history of your pioneering work with geolocated data?

Anselm Hook: I am a long time Geo fanatic. I’m really interested in social cartography and what some people call public-provided GIS, that’s some language that people use. Anyway, my personal interest, when I talk to people who are non-technical (and it’s been a long term interest in the way I phrase it) is that I want to help people see through walls. So, the goal is very simple. I want people to have a better understanding of opportunities around them, the landscape around them. I always get frustrated when people make bad decisions because of a lack of information, especially when it’s related to their community and related to their environment. But, plainly put, I really just want “to help people see through walls”. It’s a very simple goal.

Tish Shute: I know you worked on Platial, which is really one of my favorite social mapping applications. It really broke new ground. What was the history of that? How did you get involved with Platial?

Anselm Hook: That’s an interesting question. It actually started at around 2000 when I saw Bruce Sterling talk. I had been writing video games for many years, and I was quite good at it, and I enjoyed it. But, the reasons I was doing it diverged from why the industry was doing it. I was making video games because I like to make shared spaces for my friends to play in and to share experience. I really enjoyed making shared environments. I worked on BBS’s and my friends and I were always making these collaborative shared environments.

Once the video game industry kind of started to take off, I started to do high performance, 3D interactive video games and making compelling shared spaces, and it was a lot of fun. But, the frustration for me was that there was a huge industry growing around it and became very commercial. Although it paid well, it started to diverge from my values which were more centered around community environments, and shared understanding.

Tish Shute: Yes very rapidly, the big games kind of devolved from the social aspects and became more and more into single player really, didn’t they?

Anselm Hook: It was the way, actually, because even though often you were in a many player world, you weren’t collaborating, everything else became just a target.  I liked the idea of deep collaboration that calls the kind of playful space you see in IRC, or in the real world, where people are solving real world problems.

And I grew up in the Rockies, and I was always had a lot of access to the outside. So, I saw shared spaces and collaboration as a way to protect our environment. [ To step back ] I think people used different metrics for measuring their choices in the world and many people have a value system centered around minimization of harm: making sure that the people are not hurt. But, my value system is different. I personally believe that protecting the planet is more important: to maximize biodiversity. I feel like protecting people around me comes from protecting the ecosystems they live in.

Tish Shute: That’s interesting, isn’t it, because the history of Keyhole was really that, wasn’t it.  Keyhole later became Google Earth, but I mean it began out of a project to look at what was going on in the ecosystem over Africa at that time, didn’t it?

Anselm Hook: Yes, in fact many people’s projects are stemming from an environmental concern. Mikel Maron’s works for example – he’s doing Map Kiberia, and he also worked on OpenStreetMaps.

Tish Shute: Map Kiberia – that is the new project?

Anselm Hook: Oh, yes his project is called Map Kiberia. He’s mapping a city in Africa.
[For more see Map Kiberia’s YouTube Channel – photo below from ricajimarie ]

Tish Shute: Right, great!

Anselm Hook: When I started to look at GIS and mapping I started to meet people who had a very similar background. What happened to me is I kind of stepped away from games around the year 2000. I’d seen a talk by Bruce Sterling at an event called PlaNetwork. And that event was, for me, a turning point where I decided to focus full time on exactly what I cared about instead of doing things that were kind of similar to what I cared about. So, his influences is a pretty significant one to me at that exact moment.

[For more see viridiandesign.org – seems that it is time for a “Neo-Viridian,” revival.]

Tish Shute: It’s interesting because now your paths are crossing again with augmented reality. You are on the same wavelength again.

Anselm Hook: It’s funny, actually, I’ve had a couple of brief overlaps in that way.  Well, so in 2000 I went to see this talk and I did a small project called — well, I called it SpinnyGlobe. What I did is I mapped protests from a number of websites onto a globe to show the level of community opposition to the pending war in Iraq. It was the first time there had been a protest before a war. So, it was very interesting to me. [ See http://hook.org/headmap ]

Tish Shute: That’s really fascinating. Do you have any pictures of that you could send me?

photo from anselm’s flickrstream

Tish Shute: Yes, I’ll definitely look SpinnyGlobe up. It sounds very interesting.  One of the aspects of your work on geo-located data projects like this and Platial is that you really started to develop this idea of a culture of place, about how people make place. This was the wake up call to me regarding the power of networks combined with geo-data.

We are hoping to extend this idea into augmented reality with the an open distributed platform for AR so that we can collaboratively map our worlds from the perspective of who we are, where we are, and what we are doing.  I know you’ve just done some work recently in augmented reality.  I know you put the code up already.

By the way, I love the way you take your philosophy into the way you make code – the practice of making some code, trying some things out, making it all public and publishing your findings, you know, your comments on that experience.  Perhaps you could recap sort of how you picked up recently on the state of play with augmented reality and what aspects you looked at, and what came out of that experience?

Anselm Hook: So, it’s a very simple trajectory. Coming out of the work I had done, Platial, among other projects and I started to just look at the hyper-local and I suddenly realize that even those services weren’t really speaking to living, and how to really see and solve local problems. What was missing was a sense of context.

The map doesn’t know how you’re feeling, it doesn’t know if you’re in a hurry, it doesn’t know what you want, it’s very static. Even the web maps are very static. And augmented reality for me I started to recognize as a combination of — well — it’s probably collision of many forces, many forces that we’re all a part of. We’ve also didn’t realize that the real-time web is really important, it’s part of what AR is about.

We have all started to realize that the context is important. You know, your personal disposition, your needs, if you want to be interrupted or not. That is the kind of thing that the ubiquitous computing crowd has talked about. We started to recognize that there are sensors everywhere, and the ambient sensing communities talked about that. So what is funny for me about augmented reality is I started realizing it is just a collision of many other trends into something bigger.

Everything else we thought was a separate thing is actually just part of this thing. Even things like Google Maps or mapping systems we think are so great are really just kind of almost an aspect of a hyper-local view. You actually don’t really care what is happening 10 blocks away or 100 blocks away. If you could satisfy those same interests and needs within a single block, one block away, you would probably be really happy. You really just want to satisfy needs and interests, find ways to contribute, or get yourself fed, or whatever it is you want. And AR seemed to be the playground to really explore the human condition.

Tish Shute: Anyway, I think one of the things that has been very amazing this year is we to have the good mediating devices that, for the first time, give us compasses, GPS, and accelerometers. But one sort of missing pieces with AR at the moment is [tracking, mapping, and registration] – the kind of things colloquial mappings of the world could be of great help with.

We have seen mapping coming out of the Flickr data, e.g., the University of Washington, put the maps together from the geo-tagged Flickr photos. Now if we could have that linked up with AR, then we have the kind of mapping we need to kind of really hook the geo-data onto the world in a way that goes beyond…you know, what compass and GPS can really deliver is pretty minimal at the moment.

Anselm Hook: There is a real risk of our augmented reality world being owned by interests which are not our own. There is a real question of when you hold up that AR goggle, what are you going to see? Are you going to see corporate advertising? Are you going to see your friends’ comments or criticisms? It is going to be an Iran or a democracy, right? It is unclear.

Right now there are some disturbing trends I have noticed. I am a big fan of Google Goggles. I think it is a great project. But when you are mediating the translation layer between the image and the data, then there is an opportunity for you to control it, and that opportunity is hard to resist. It is hard to choose not to own that opportunity. It is an advertising opportunity. It is a revenue opportunity. It is a chance to send a message and a tone.

I know that Google and companies like that are keenly aware of the kinds of roles they don’t want to hold, but it is sometimes seductive to think about them. And I am afraid that we, as a community, need to assert an ownership, kind of a commons, over how computers will translate what they see to information that we perceive.

Tish Shute: Yes. And this is how we met, again, recently [over the project to create an open, distributed platform for AR using the Wave Federation Protocol]…

This is something I feel really deeply is that, you know, basically we need the physical internet to be as open as, as the, as the internet, as the end-to-end internet has been. Or more so, actually, because the end-to-end internet has seen the trend has been to walled gardens.  Basically Facebook became enormous, an enormous walled garden which, I think, was despite, our predictions about them, [walled gardens] are the social experience really on the web.  It’s very much in walled gardens still and I, and I really feel that with the physical internet, we need to make great efforts not for it not just to be a series of small pockets of privately funded walled gardens.

There needs to be some kind of communications infrastructure that keeps it open so that was when I got interested in looking at the Wave Federation Protocol because it was a real time, you know, an open real time protocol that could possibly be a basis for that. But I think the point you’ve talked to just now, the mapping of the world and who has the “goggles”, i.e., the image data, image databases, that make the world meaningful is really, that’s still a, it’s still a BIG question [i.e. who controls the view?].

When I saw ImageWiki, [I realized] that is a piece that is vital for, for augmented reality. We need to have a huge social effort to be involved in this,  linking in and creating the  physical internet, in creating the image hyperlinks that will make that meaningful.

Anselm Hook: I think that’s a great point. The search interface, the kind of Internet that we’re used to, the way we talk to the network now, is fundamentally open end to end. Yes, you can have your oligarchies inside of it, as we see with Facebook, but you can always start your own venture up and you can do a search on something, and you can find that, that website and you can join it or you can put up your own webpage and people can find it.

The translation layer, the idea of text search and the ability to discovery power and the serendipity and the openness of that discovery, it’s pretty open right now. We do have some serious boundaries of language, which is one of the reasons I was working at the Meedan.org [hybrid distributed, natural language translation] for a couple of years, trying to bridge that issue.

But here, as we move towards a physical internet where there’s no clicking and there’s no interface and the computer’s just telling you what it thinks you’re looking at, translating, you know, an image of a billboard to the name of the rock star who’s on that billboard, or translating the list of ingredients on a can of soup to the source outlets where it thinks that, those ingredients came from. When you have that kind of automated mediation, the question of trust definitely arises.

And we haven’t seen the Clay Shirkys or the Larry Lessigs of the world start to talk about this yet.  Although I suspect that in the next four or five years that the zero click interface will become the primary interface, that we’ll have…we’ll come to assume that what we see with the extra enhanced data we get projected onto our view is the truth. Yet, at the same time, there is just no structure or mechanism even being considered for a democratic ownership of it.

We have with DNS, for example, the idea that you can register the domain name and people can search for it, and find it, and go to it. There’s no such thing as an Image DNS, or an image translation to DNS right now. What does it mean when everything is just “magic”, when there’s no way for you to be a part of the conversation, where you’re just a consumer of what people tell you, or of what one company right now, tells you, is reality? That’s a real concern.

Tish Shute: This, to me is the most important question at the moment. I mean, it’s the big one and it’s the place to put energy if you love the Internet [and what it can now become] right. You’ve got to put a lot of energy into this because this [a democratized view of the physical world as a platform] won’t just happen, because there’s a lot of momentum already for it to be heavily privatized, partly because, one reason is, some of the computer vision algorithms that, say, make sense of things like the geotag photographs are not open.  I mean, for example, the beautiful maps that have been made from the University of Washington [from Flickr geotagged photo sets], that isn’t in the public domain.

Anselm Hook: Right. Tish, and in fact you’re referring to [with the maps from the Flickr photos] to ordinary maps and the fact we’ve already seen that maps lie, we’ve already, seen how much maps are reflecting a certain truth that becomes the normative truth. Google maps reflects roads, because this is roads and cars, right? Only recently have they thought about buses and walking. So the normative view that people assume is the reality, is showing off you know Starbucks, and roads, and cars, that becomes the default, those prejudices are just assumed, you know, the truth. But they’re not the truth at all.

I was talking to a friend of mine in Montreal, [Renee Sieber], and she said that their Indian portage routes are a bridge across land and water, they don’t think of a piece of land and a piece of water as being different things, they think of them as one thing: a route. It’s already a different kind of language we can’t even reflect it.

So not only is there this kind of formal, anthropological lie, in a sense, but there’s this way that we deceive ourselves because of our own prejudices.

Tish Shute: Yes I agree and that’s why I think when I saw some of the things you had written on the ImageWiki point clearly to the need to create a social commons. We need a social commons for the real-time physical internet, we need it for the image hyperlinks that make sense of that.

And it’s a complicated thing in a sense, though, because we don’t actually have a good distributed infrastructure for AR yet, and I found exploring AR Wave, that at last we have the suggestion of an open, federated protocol for real-time communication – the wave federation protocol. [Real time communications is a very important part of AR].  It isn’t an actuality yet where lots of people are able to use it, set up their own servers, and there’s not a standard all the way through  [there is not a standard for how data is sent between the client and the server].

But Wave Federation Protocol does make possible truly distributed social AR.  I started thinking when I saw ImageWiki that to bring ImageWiki together with the social collaborative power of distributed AR.  This really would be the basis of creating a social commons for augmented reality and the physical world as a platform – the start of a bottom up with deep social collaboration on how we create augmented reality colloquial maps that can inform a hyper-local of the world.

Anselm Hook: Yes. When Paige Saez, John Wiseman, and myself, and a few other folks… You know, Benjamin Foote, Marlin Pohlmann, and a couple other people started to play with this, we quickly found that… We started to realize, “Oh, this kind of thing will be at least as popular as IRC. There will be at least as many people doing this as chatting in little virtual spaces. There’ll be at least as many people decorating the world with augmented reality markup, and maybe using the real world as a kind of barcode for translating what you’re looking at into an artifact, a digital artifact.

And that the size of that space was going to be huge, basically. Maybe not quite as commodifiable as Twitter, but certainly very energetic.

Many of the projects we did were just kind of looking at these kinds of issues sort of from an artistic, technical, and political point of view. We weren’t so much posing complete solutions, but simply using a praxis to explore the idea with an implementation, as a foundation for this discussion. So I think we sort of opened that can of worms for sure.

Tish Shute: Did you actually set up ImageWiki to be working as a location based app yet?

Anselm Hook: It is a location based app. It collects your longitude, latitude, and the image and stores it. And then it uses that as a way to translate that image to anything else. It could be a piece of text or a URL.

Tish Shute: So there is a smartphone app, but you didn’t take it as far as an AR app yet?

Anselm Hook: No. We didn’t do a heads-up view. There are apps on the iPhone store that do that, but they don’t do the brute force image recognition that we were using. We used a third party off the shelf algorithm that we found on Wikipedia and downloaded the source code, and threw it on the server. And John Wiseman in LA wrote the scalable database backend so that we could scale the actual…

Tish Shute: So how did you set the iphone app up to work?

Anselm Hook: The iPhone side was very simple. You take a picture of something and it tells you what it is. That is all it did. We would take the location, but the client side, the iPhone side, just rendered, returned to you…It said, “Someone said that this picture of a barking dog is an advertisement for a local band.”

Tish Shute: Right. So basically it was a geo-tagged?

Anslem Hook: Yes. We are just collecting the geo information. Actually, there were a whole lot of technical challenges. The whole idea of ImageWiki is actually kind of beyond our technical ability for a small team like us. It really does take a team, a group like Google, to do this kind of thing in a scalable way.

Tish Shute: Why is that?

Anslem Hook: There are two sides. There is the curating the images. I think that is the job of groups like us – open source groups who can curate images that are owned by the community. And then the searching side, the algorithm side, where you are actually matching the fingerprint of one image to images in your database, that takes a much more…that is much more industrial.  We get both sides, ours is not a scalable solution. It is mostly…proving that it could be done was important.

Tish Shute: In terms of hooking Imagewiki up to the collaborative possibilities of AR Wave wouldn’t federation pose some interesting possibilities for scaling search algorithms and all that?

Anselm Hook: Yes. And what is funny also, incidentally, is that, nevertheless, we did look for some financial support for it, but we couldn’t…we just didn’t find the investors to scale it. Now, other companies like SnapTell took a shot at it. And they have an app in the iPhone store where you can point at a beer bottle and get back the name of the beer bottle.

The classic example everyone uses is a book. Amazon has all the image jackets of all their books. You can point SnapTell at almost any book and get back links to buy that at Amazon, the price of the book, and user comments on the book. So they are treating Amazon as the canonical voice of the book, for better or worse. That is the state of the art so far, up until Google Goggles came out a little while ago, which actually blows it out of the water. But, that is where we are now.

Tish Shute: Right. But the point you raise about how when something like Amazon comes canonical of what is book, right, this is the whole point, isn’t it?

Anselm Hook: Is Amazon truth? It’s not bad. Jeff Bezos seems like a nice guy, but, you know.

Tish Shute: And this is the point of having these open infrastructures for this.  And this should be obvious in a way, but it comes back to the thing about what made the Internet great was the fact that even though as you note, you get an oligarchy like Facebook, but people always could just go off and do something else, right? Because the fundamental infrastructure was basically open and designed to be available for everyone. And many people have championed that and fought for it hard [to maintain this openness] haven’t they? They have devoted their lives to keeping it that way, even if the oligarchies have done their thing.

Anselm Hook: Yes. There are really some things that are underneath all of this that haven’t been solved yet.

One is that the trust in social networks has not been built yet, so we can’t do peer based recommendations very well. We can’t filter noise by peers. Twitter kind of is moving there, but I don’t just want to listen to my Twitter friends. I want to listen to my friends of friends. If I am getting truth from somebody, I want to get that truth from people my friends say that they trust.

Then the second problem is that there is a search business. My friend Ed Bice, who owns Meedan, always says that a search itself, a search request, is an opportunity to make…is a publishing moment. It is an opportunity to say what you think. In the real world, if you are just hanging out with humans and you look somewhere, other people might look at your gaze and they might look at what you are looking at. Your gaze itself is a public act.

Gaze is a soft act, but it is one that is visible. With Google, the gaze of four billion people is invisible. We don’t what people are looking at, there is no opportunity to participate. Let me give you a real example.  I have taken a image of something of the bust of figure or a statue.  Why can’t the museum in Cairo look at my request and tell me oh yeah that is Tutankhamen, or that is Nefertiti right? Why can’t they have a chance to participate in the search and respond to me?

Right now the the only person that responds is Google when I do a search. We need to invert the search pyramid and open up search, so that search is a democratic act, so that you can publicly permission your searches so that other people can respond and so that people can reach out to you, not just you having to do a dialogue.

The common example of this.. and we see this everywhere: I am looking for a slice of pizza right, now I am hungry I want some pizza. I have to ask Google, look find twelve websites, call twelve phone numbers, and talk to each of the twelve stores, and ask them are they open late, is the food organic, is the food in any good, do my friends like it.

Whereas what I should be able to do is just say it’s a search moment and I am interested in pizza. If those pizza places my criteria like you know my friend’s like them and they are organic, they are open, then that pizza place can call me. I have the money why should I do the search? So the whole business of search, the whole structure of search is predicated around a revenue model, but its a really short-sighted revenue model, its not a brokerage.

Search isn’t search, search is hand waving.  These should be moments for us to have a discourse. So problem we are seeing in AR with communication of the right information is actually underneath AR, at the level of the whole infrastructure.

Search needs to be inverted, trust filters need to be built. We need to democratically own our data institutions.  We don’t right now.  That will be more of a concern, especially with AR.

Tish Shute: Yes, especially with AR, which is this why got all excited about federation.  Do you think federation has the potential, an opportunity to create [the new infrastructure you describe?]

Anselm Hook: Absolutely,  its absolutely what we must do. It is much harder to do. It is absolutely critical.

Tish Shute: And why is it much harder to do? Could you explain that?

Anselm Hook: Well, it’s very easy for a bunch of hackers to build a service that you log into and fetch some data, it’s a single thing. They don’t have to talk anybody, they can use their own protocols, they can hack it, it’s a big black box, behind the scenes. There’s running back and forth in a giant Chinese room delivering manuscripts and scrolls to you. Whatever is behind the black box, you don’t care, it just works.  But when you federate, you need to actually publish and have standards, and then you’re talk about semantic, everyone starts getting really excited and wave some hands. It becomes a disaster. It’s, at least, another power order, more difficult than DIY, build it yourself.

Tish Shute: So, in terms of what Google Wave have done with their approach to federation, what do you think have been their achievements and what do you think is their obstacles? What do you think are the failings of the Wave? Because it’s the first big public major player backed approach to something federated, isn’t it? In real time.

Anselm Hook: Yes. I think the most important non-federated service on the planet today is Twitter.  Identi.ca it’s not getting any traction with respect to Twitter. [ Even though ] Identi.ca is a federated version of Twitter and is very good. [ Identica is now Status.net ] . So, we see already there that small players aren’t being competitive. Then look at other services like IRC. IRC is the secret backbone of the Net. All the open source projects, all the teams, all the people that work on opensource projects are all on IRC. It’s the only way they get anything done.

With Google Wave, and the protocols underneath Google Wave, we see an attempt to build a similar kind of real time, but distributed protocol. I think it’s the right direction. I think, people should pick up the offering and make their own servers. I think that protocol is really great, I think the fact that is compressed, its high performance, it is small, real-time of blobs of data flying around, all exactly the way it should be done. It is getting close to this kind of rewrite of the Internet that people keep talking about, because, you know, the net protocols are so bad, it is starting to treat the idea of intermittent exchanges being more transitory, volatile, and not heavy.

….to be continued.  Part 2 coming soon!

Om Malik just wrote that he was surprised by the number of location-aware mobile services being launched in Austin, Texas, SXSW Festival this year in his post, “SXSW, Location Awareness Is The New Black.” I am not surprised.  The Mobile Meet Social, Tech Meetup in New York City in February was abuzz with new location aware applications and platforms for location based services.

I have been fortunate to attend a number of the key conferences this year. I am just back from ETech 2009 brimming with ideas and interviews for posts, and I am just catching up with publishing my interviews from Tools of Change 2009!  At TOC I did an interesting interview with Chris Brogan on how location awareness will change social media.  The web is flowing out into the world and questions of  who YOU are, WHERE you are, WHAT you are doing, WHAT is around you are becoming central to the rapidly emerging internetworked world.  TOC explored the digital future of books and heralded the transition of books into “everyware,” (see my interview with Adam Greenfield author of “Everyware: The Dawning of the Age of Ubiquitous Computing”).

There were many highlights for me at TOC 2009, including Nick Bilton’s keynote, “The Narrative is Changing: Sensors, Social Editors and the New Storytelling.” Nick developed some of these ideas further at ETech where I had the pleasure of playing Werewolf with Nick, Jane McGonigal, and many other ETech presenters and attendees. Nick is seated to the right of Jane (center) – click to enlarge. Jane is demonstrating the attitude of a werewolf to noobs to the game like me. Nick Bilton by the way is an accomplished werewolf (in case one day you find yourself as an innocent villager, seer, or healer defending yourself against him), for more photos of Werewolf at ETech see my Flickr stream.    Also see this excellent write up of Nick Bilton’s talk at ETech, “Sensors, Smart Content, and the Future of News,” by Richard MacManus.

Peter Brantley’s, “Literature as a (Web) Service,” talk at TOC also elaborated a theme, “literature as a service,” that was developed more broadly at ETech where Mike Kuniavsky of Thingm brilliantly presented on things as services (much more coming on this soon!).  Brantley, at TOC, concluded with an invocation for the future of machine parsed books.  And Bob Stein’s, “A Book is a Place”, Tim O’Reilly’s “Reasons to be Excited,” and Google’s  Jon Orwant’s update on Google’s quest for the holy grail, “converting images to original intent XML,” should not be missed (you can watch the videos here). Enjoying the serendipity particular to conferences,  I had a very inspiring long lunch conversation with  Ben Vershbow about “Books as Spimes” – more on this too soon!

Tim O’Reilly’s talks were inspiring at Tools of Change and ETech. His finger is on the pulse as always. Tim O’Reilly’s big vision is richly informed by  his own lived engagement with tools of change. Tim does not merely narrate how Twitter can facilitate publisher/curator relationships with authors and content, he is part of the evolution of Twitter as a publisher’s medium, through his own creative use. Also see these two key posts by O’Reilly Radar’s Joshua Michele-Ross on Forbes, The Rise of the Social Nervous System and a follow up by Tim,  The Social Nervous System Has More Than One Sense.

Tim’s  keynote for ETech is perhaps one of the best invocations and elaborations of a theme he has been developing in recent months – “work on stuff that matters” (see this excellent summary in Venture Beat).

In  my post, “Towards a Newer Urbanism: Talking Cities, Networks, and Publics with Adam Greenfield,” Adam makes some interesting comments on “the networked book” and  “unbooks.” Adam’s upcoming book, The City Is Here For You To Use,  is evolving as somewhat of an unbook.  Unbooks are, he writes:

“a container for long-form ideas appropriate to an internetworked age” and an approach that “can usefully harness the dynamic and responsive nature of discourse on the Web. At the same time, you preserve the things books are really good at: coherence, authorial voice and intent.”

Unbooks draw on lessons from the “open source” approach in software – version control, open-endedness to inform a collaborative development of books (this is an approach Usman Haque has taken to environments and cities – Urban Versioning System, for more see my interview with Usman here).

It seems fiiting that as I began writing this post an email came in from my friend Steve Fagin inviting me to The Last Book Project in LA, April 26th, 2009 – save the date.  Steve, is a brilliant artist, director, and impresario (please check out one of his early films, The Machine That killed Bad People, 1989 which was how I first became aquainted with his work).  Steve writes:

“Our vain glory is the attempt to resurrect the medieval illuminated manuscript through the invocation of our current alchemy, the new technologies, to conjure a future the past in reverse.”

The Last Book project’s illustrious team  include’s China’s “best bad girl novelist,” Mian Mian, as the reader of the Last Book.

The mobile phone is leading the charge into ubiquitious computing (although more immersive forms of experience will not be far behind (see my earlier post here). Note in Japan, “half the top selling books are WRITTEN on mobile phones.” While the “total spimy revolution isn’t here yet” – see “What Bruce Sterling Actually Said About Web 2.0 at Webstock 09,” we are blowing holes in “the spider’s web” of the end to end internet.

Since the NewYorkTech Meeetup – Mobile Meets Social, I have been thinking about the question: “How do timeliness and location-independence affect social media?”

This is a core question as David Oliver pointed out to me after the meetup (interview with David and Nathan Freitas upcoming).  David and Nathan are the principles of  Oliver + Coady a company focusing on Mobile/Social/Architecture  – also see Nathan’s blog for more.

At Tools of Change, I had the opportunity to talk with the social media guru and uber blogger Chris Brogan.

You can catch up on Chris’ TOC workshop, “Blogging and Social Media,” -  here (also see my Posterous here, “Smart Phones the Gateway Toy in Everyone’s Pocket”).

Interview with Chris Brogan

Tish Shute: I have been thinking about the question that David Oliver (principle of Oliver + Coady with Natahan Freitas) put to me: “How do timeliness and location-independence affect social media?” Mobility, is the key as David points out, NOT mobile as in a desktop in your hand, but timeliness (you do things when you need them) and location independence (you do things where you need them.

Chris Brogan: There’s a theory that I’ve been working on and I’ve typed a little bit into my blog from time to time where I call it ‘the secrets of the annotated world’. What I’m saying is, where I’ve experienced differences in Brightkite as applied to the iphone. When I first ever used Brightkite, I didn’t like it. The app didn’t do anything for me on the desktop. But when the iphone app came about, because it sucked my location off of my iphone, it said “can I tell people where you are?” And I’d say, “Why yes.” So what I’d do with it wasn’t so much talk to my friends and have a location element to it.  What I would do is I’d talk of the location. You know, “I’m at the Roger Smith hotel on 47th and Lex and they have a nice quaint little bar.”  And I would note, “try the pear fused gin.”

So what I’ve done is I’ve left a note in time and also in space. So now if another Brightkite user, and this is an example but I think this extrapolates to other places.. and I’ll think about a clearing house of space. What I think is that another Brightkite user who now comes somewhere near that space and who says what information, who said anything nearby where I am right now, will see I was at the Roger Smith Hotel, I had the pear fused gin it’s delicious. Oh! I like gin maybe I’ll try that. I think I’m helping put signposts up in space and time such that other people will come and read these glyphs that nobody else can see. And that’s sort of the imagery I have.

I think there’s an opportunity with this. One I think there’s an opportunity for this data to port across more places. I understand that in the data, if you own the customers then you own the future. But, if you don’t do it then somebody like Google will do it and I think that what will happen is Google maps will own this idea of you can add notes to a place. Google Local does this right now on the desktop,  so if I look up my local pizza place I now have would you like to write a review about it?  Would you add more info to the story?  And I thought, “wow this is getting there, except I still can’t see it nicely on my phone,” but now Google Locate arrives and now maybe I can.

So I think what it adds to social isn’t … Dodgeball tried awhile back and theirs is more like where are you, where are you, where are you. This is great if you’re a very mobile tribe, not everyone is. But from time to time we wander through different pieces of space and we would run across this information that would tie us more to the space. And so I’m passionate about how do I annotate and how do I maybe do more equipping. It’s interesting to me that Yelp as a restaraunt review product doesn’t have input from an iphone.  So in lieu of, I mean it lets me ask what reviews are up on the site, but in lieu of that  I can then put my restaurant reviews in Brightkite and then anyone can come find them based on time and space not based on a fixed site.

Tish Shute: I suppose it is hardly surprising (if disappointing to me) that some of the early location based services are trying to get mindshare by picking up on the glue celebrities give to mass culture. At the last New York Tech Meetup, OMGICU demoed a rather terrifying new pre-launch location based “participatory celebrity gossip application” which seems to combine all the worst features of social media with celebrity stalking, plus a narrative to change the notion of celebrity itself by “turning D listers into A listers.”

Chris Brogan: It raises a really interesting question because I’m willing to divulge my information freely. I’m not willing to divulge my family’s information. I don’t ever point out where they are in the world. One criticism that people have had when I use a product like Brightkite, is they’re saying you’re identifying that you’re not at your home. But I’m also telling people I’m not at conference. There’s lots of things, it’s not Brightkite telling them that. Its everything I do, it’s my blog. I’m not afraid of that. But I am interested in the negative impact of people doing that OMGICU thing because maybe I don’t want people to know I’m in Wyoming working with a client who maybe doesn’t want the world to know I’m working with them. It might be under non-disclosure. Or I might not want people to know which town exactly  I live in or where exactly my house is.

There’s some beautiful things that can be done on location, Wired had a big piece in their magazine about some location things. even that doesn’t get to the crux of the things I would do with it. Look at my analogy of annotating, I’ll put tags in the air that you’ll come and stumble across. Why can’t I find caches of these – like geocaching – why can’t I find caches of data that only exist in place. Why can’t I find data that’s … now I’m talking more like Gibson and Neuromancer.  But why can’t I find pods of data that can only be accessed by being in a locale.

Tish Shute: Have you seen Wikitude?  Something like Wikitude would be really great with some social tools.

Chris Brogan: Anywhere where it is possible to have all those hands raise up the load it is so much better because things get done faster, they get done more effectively. I am never going to get over to Williamsburg in Brooklyn and hang out there.  But if someone else was doing it I could always get the mapping, and I would know where I would want to hang out if I did. So I think these tools cry for this. The hint of this was in the Batman movie when he used eveyone’s phone to make a picture. The technology was obviously more vivid for a film sense.  But that is not unlike what we are doing with things like Twitter.

Tish Shute: One of the things I really like about Twitter is a a sense that i can drop in and out of the stream.  Is Twitter is the best current example of timeliness and social media?

Chris Brogan: I think that Twitter’s a great example, Twitter filters the noise out. If you search twitter, there’s a trending topics thing, that tells us now here are the top 6 trends going on at the moment. And so you can see when something surfaces that matters to you. Or you can search and try to aggregate.  So for example when the Australia fires story was breaking for a little while the only news you got was just links back to other peoples news. Then what we were searching for there was specifically this gentleman named Paul Mooney, and he was searching for first hand accounts. That was harder to get through twitter. But it’s almost like calling it out. It’s imagined in a game sense, we’re all in a big room, it’s almost like Marco Polo in the water. Maybe there’s 10 people and there’s one of us blind and he has to send out a ping, and all 10 ACK that they’re there somewhere. Well I think that the social meets presence type of things deal with, can you bubble that information up to me so that I hear it through the din and can you direct me to things that I shouldn’t normally have access to, that through my own senses cannot find.

Tish Shute: Without underestimating the value of social media  (as a blogger I owe much to social media), do you think though we need to develop different and more nuanced social tools as the web reaches out further into who YOU are, WHERE you are, WHAT you are doing, WHAT is around you?

Chris Brogan: There’s a lot to be said about the whole friends quote unquote equation  But I don’t understand, I mean believe me in the business sense I do, but I do not understand the amnesia that happens. When I go over and jump on any of those new services that you just showed me, I am now forced to go refind my friends. So the very first thing I need to do is cure my amnesia. Oh we’re friends here too? Oh I like that we’re friends. Why not here? Like open social but not like data portability. I need to carry my network with me. And another sort of future trend is that I also want private areas where it’s sort of the velvet rope social network, if you will. Right now we’re in the media area here. If we were doing this out in the hallway, then I’d get however many folk from that session. If we weren’t hitting the afternoon sessions, it’s say “Hey, great speech whatever or horrible speech I think you’re terrible.” We’d be bothered. So I think as we join the stalkerati, and as we know where everyone is all the time, we now often need a sort of “hush mode” that lets us go off and do some things in sort of small clusters.

The one number we’re up against is Dunbar’s number of 150 people. I’m fascinated by that because on twitter I have 40 thousand people, and I follow 36 thousand. I clearly don’t know what 36,000 people are doing. So what I’ve done instead, I’ve sort of wired a phone network. So I can ring you because I have this access to you, but I don’t always listen in on every conversation. Instead I dip in when I need to and otherwise I’ll wait until you come in. I use Tweet Deck like you do. I have one call in for @replies, I have one call in for my last name because sometimes people don’t form it right. And then direct messages and then the regular stream. The regular stream, I’ll see stuff go by and I’ll comment on as, but let’s just do quick math – 40 thousand people 1 percent of that is 400 people. 1 tweet each per day for 1 percent of my people would fill that screen for some time. So I can’t field all your conversations. But what I can do, as I demonstrated in my presentation, I use search a lot and I fall onto those conversations and then talk to people that way. So I’m using Twitter differently than most. And then the goal is to extrapolate that out until we find that there’s a bunch of journalists and people talking about journalism talking on this thing called journchat which is a Twitter group of people, and the only way to do that is to follow a tag. The beauty of such an option is in the way that the web could be TV, and still hasn’t in a meaningful way, in that I can deliver very exactly what someone wants to them. We don’t see the parity of that yet in a lot of spaces, and we haven’t really found the perfect ways to monetize the effort that’s there.

Clearly this is the magic. We were talking about this more than a decade ago. But it never came out the way we said. You were talking about the forward thinking part of it. I was in wireless technology for years. It just never was really there fast enough, and the reason was, of course, because the telco’s don’t really want to innovate. There’s no motivation to. They’re making money as it is. Telecoms adopted when VOIP looked like it was going to crush them. And so they absorbed it. and so they may or may not absorb location as a service. They might absorb presence as a service.

That’s my speech tomorrow. I stole most of my idea from Jeff Holver’s line. Jeff Holver said young people today bring their own dial tones with them. That means if you think of our US election, one person won and the other person lost. and the other person who lost was ringing everybody’s land line home phone numbers. And the person who won was on facebook and twitter and myspace and text messaging and all those sources. So dial tone as we understand it to make a telephone call is nothing like it used to be. When you call a home that’s where you call to ask a question. But that’s not, I mean I have a cell phone, my wife has a cell phone, there’s a land line but we never pick up the land line because it’s usually some calling about a bill we didn’t pay. So we listen to the message, pay the bill and we’re done.

Tish Shute: I have to ask you this as you are THE expert on the business of social media, how will people make money out these new services?

Chris Brogan: Honestly I’m really surprised that more of these things aren’t willing to try a subscription model. At the volume I use twitter, I mentioned if I do one tweet a day it goes out to 40000 people. So every time I do that it writes 40K records in 40K different accounts if you think of it as a database. Twitter technically should charge me. Maybe there’s other services they can offer in sort of a premium model, so there’s a free to premium kind of a plan. That’s how companies can make the money. I don’t know when we stopped feeling like we should pay for software.

I don’t know that we know just yet but I don’t think that there’s a business model for the phone except for subscription. So if twitter is to be the new phone it’s a subscription product. But you know, the iphone store, there’s people selling 99 cent to 5 dollar product and it’s going well. I’ve bought 6 apps now.

I think as we’re defining the etiquette and as we’re starting to understand how social platforms in general do and don’t work. I think what we’re finding is that people are having to relearn their business communications skills, not their marketing or their sales skills, because the old ways that we were trying to market were getting more and more frenetic, and getting more pushy, and more and more the walls came up and no one wanted to pay attention anymore. That’s where we are as a group in the western world for the most part. So now what we’re doing is we’re rediscovering relationships. I’ve talked to people from Tyson Foods. Why should I ever care about a company that sells chicken nuggets, except that I have a small child. So I’m finding conversations like that all the time. I think that’s what’s going to happen. I think there’s sales there. I think the problem is that the numbers are a little more wiley.  And I think that the numbers are a little more hops down the stream instead of direct. I can argue till the cows come home that you buying a post card and shipping out to a pile of people isn’t worth the money. But you at least can say well I got 45 sales out of those 5 thousand post cards that cost me this much. I got 45 sales and that numbers higher than this number, we did well.

Tish: I can see you are being called to your next meeting. Thank you so much for taking the time to answer my questions!