Shortly after Augmented Reality Event – are2010, I talked with Bruce Sterling on skype and in gdocs about his experience there.  I am posting the conversation in two parts to make it a more blog friendly length!

The picture above is the Auggie Award for the best AR demo (above) designed by Sigal Arad Inbar.  It was won by YDreams! See, Ivan Franco recounts the team’s ARE 2010 experience, and winning the event’s first-ever Auggie Award, for more.   The video below was shot at the YDreams booth by Bruce Sterling.

“The Hotness” – YDreams rocking it at ARE2010 from brucesflickr

Rudy Rucker, who was hanging out with Bruce Sterling, captured the are2010 buzz and some great images in his post, Augmented Reality, Painting, Twitter. As Rudy put it:

“AR is hoping to be a next big thing, a cozier and more commerce-driven cousin of the old VR, or virtual reality.”

Bruce Sterling’s opening key note is up, ARE 2010 Keynote by Bruce Sterling: Bake a Big Pie!, and also the ARE 2010 Keynote by Will Wright: Brilliant Inspiration for the Augmented Reality Community with more videos from are2010 on the way.  One must read post on are2010 is Chris Cameron’s post, Augmented Reality’s Next Steps: Sitting Down with the Titans of AR.

Talking with Bruce Sterling, Part 1

The Auggie panel, Bruce Sterling, Jesse Schell, and Mark Billinghurst inspect the award.

Tish Shute: In your keynote at the 9am of the augmented reality industry you asked some questions of the are2010 audience: “What’s the mission statement?  You’re the world’s first pure play experience designers, except that user experience it’s mostly futuristic hot air.  But run with that, right?  What are your tactical steps?  You should get dressed, have a coffee, have a to-do list.”

How much of that did you see going on in the next two days?

Bruce Sterling: Well, I wasn’t privy to any of the business discussions.  I didn’t think it was an accident that this standard AR enabled tag thing came up from Bruno Uzzan, Total Immersion.  That seemed to me to be a useful thing.  I was always interested in the Augmented Reality Consortium.  It struck me as remarkable that there was this group of people who clearly all knew one another and it had some kind of game plan.  I applaud them for that, because these are not the 1980’s.  [laughs]  You know, it’s just a different world for young startup companies.

Tish Shute: I think you’re right. There seem to be some VC conversations going on, we don’t know what went on in the meetings, but it was noticeable in the atmosphere of excitement, and remarked on by a few people. So I think that kind of was definitely going on.

And, of course, I was so busy I never even got to see the expo properly! You said you wanted to be surprised.

Did anyone surprise you in any of the talks, in any of the expo?

AR used as interfaces for devices

Italian augmented robot from SEAC02 from brucesflickr

Bruce Sterling: I have to say I was a little bit surprised to see Andrea Carignano demoing a robot. I happen to know him because he’s here in Torino. He’s the guy that came out of Fiat and went into AR. I am not a particularly huge robot fan, but I think it’s of great interest that AR is used as interfaces for devices, as opposed to the Jesse Schell idea that AR is all about a “man with the X-ray eyes.”

My suspicion is that a lot of surprises will come out of mashups of AR.

Tish Shute: I didn’t get to see Andrea’s robot.  So what did it do?

Bruce Sterling:  It’s basically a sister device to that little helicopter that those Parrot AR Drone guys were doing.  It’s a little autonomous robot and it runs around with a webcam on it.  You can place video into the acquisition stream coming off the robot.  You can play a game, and blow away imaginary monsters or whatever.

Tish Shute: It’s interesting, because did you notice Will Wright and Patrick O’Shaughnessey, Patched Reality, spend some time hacking the Parrot AR drone in the hallway?  Did you come across them?

Bruce Sterling: Rudy was there with them.  You know, I didn’t want to watch Will Wright hack a robot.

[laughter]

Tish Shute: They seemed to be having fun even though as it turned out the power supply was dead.

Bruce Sterling:  I’m sure Will enjoyed that.  As a game designer, you want to go out and get your hands dirty with a plastic gizmo.

[laughter]

My Swiss Army knife can’t get through airport security, so I really don’t want to strip anything down.  But yeah, what else did I see that was of particular interest?  I was pretty happy about the Korean guys because they are a difficult group to get close to.

AR companies are like mini-global micro-startups.  They’re “glocal”.

“Korean elegance at the Zenitum booth” – from brucesflickr

Tish Shute: Zenitum.  What did you like from Zenitum.  They were one of our sponsors, along with Qualcomm.

Bruce Sterling:  I know that Seoul is like the number one center for augmented reality discussion.  But it’s  difficult to get behind the scenes as a journalist there and  track what’s going on in Korea.  I’m fine with Italian “realtà aumentata.”   And I feel like I’ve got a handle on French “réalité augmentée.”   The Germans were not hard to find, and the Dutch all speak English!  But the Koreans, and whoever the hell it is in Kuala Lumpur…  I have no idea what’s going in Kuala Lumpur, and only the vaguest idea of what’s transpiring in Singapore!  But I know that people there are paying a coherent interest.

So the Koreans show up, and they had some relatively predictable anime style 3D avatar conversion stuff.  But they had a really nice display space.

“Anime figures become three-d smartphone animated avatars,” from brucesflickr

Tish Shute: Ah, So Zenitum created a hot spot at the exhibition?

Bruce Sterling:  Yeah.  The Koreans had  IKEA furniture and some nifty little woven baskets.  They’d really classed up their presentation.  Most Koreans in tech tend to be kind of muscular.  The Koreans are not known for their refined presentations.  On the contrary, they tend to undersell everybody else.  But I don’t know, maybe they’ve been hanging out with Samsung and upgrading their design chops. [laughs]

Tish Shute:  Did you take some photos you could send me?

Bruce Sterling:  I took a few, but  I don’t consider myself a photographer.  They’re all up on my Flickr set. It was interesting to see so many people from so many different nations in such a collegial atmosphere.

Tish Shute: Yes – there were many different countries represented at are2010

Bruce Sterling:  It’s the beginning… and so global at such a young stage.

Tish Shute: Yes. As you said, it was 9 AM, so everyone was actually super excited to be gathered together from across the globe to start a new day together.  As you mentioned, there was a very warm affirmative vibe – everyone sharing a passion.

Bruce Sterling:   They have an online commonality. They seem to be aware of one another’s work through the Internet.

Clearly they had all heard about one another.  That’s a departure from earlier models of tech startup, where you usually have like three hippies in a local garage.  Now you’ve got German-American-Korean outfits like Metaio, and Total Immersion has a Russian affiliate.  They’re inherently multinational, both inside the company and out.

Tish Shute: It was the multinational garage, wasn’t it?

Bruce Sterling:  Yeah.  AR companies are like mini-global micro-startups.  They’re “glocal.” There’s something quite new to me about that.  I don’t find it’s shocking, because in Europe today it’s common to find startup teams who are multinational.  But to see such intense globalism at such an early stage of an industry is really different.

Tish Shute: Yes it made for a fun atmosphere?  It was wonderful running into Iguchi Takahito, Tonchidot.  You have a great rapport with each other despite the language barrier?

Bruce Sterling:  Yeah.  That guy from Tonchidot, he’s very charismatic.  He’s punchy.  That’s reflected in the very strong graphic design from his company.

Tish Shute: Using minimal English to make the case for Sekai No Camera at the Auggies, Iguchi Takahito still got through to the audience.

Bruce Sterling:  Well, his visuals were good.

What AR means for artistic practice…

Picture of Eric Gradman’s Cloud Mirror, from James Alliban post ARE2010 – Augmented Reality utopia in Silicon Valley – see for more on the are2010 ARt Gala

Tish Shute: So before I move on to wider themes, I’m going to wrap up on some of the different aspects of the conference.  I was chairing the technology track but you were more free roaming, was there anything that went on in the sort of hallway discussions and the presentation rooms that struck you?

Bruce Sterling:  Well, I did get collared by artists.   They really wanted to talk to me.  We got into some serious discussions on  what AR means for artistic practice.  How you can do this and reach that, how can one sharpen up one’s presentation?  I mean, they really wanted some art criticism.

Tish Shute: That’s very interesting.  Did you come up with anything that you hadn’t been thinking about already through the conversations?

Bruce Sterling: I’ve seen augmented reality installations before, and I certainly know many electronic artists.  But I don’t know.  People in the AR art space, they are looking for guidance and trying to find fellow spirits.  In their own way, they have the same pioneer spirit as the business people.

Helen Papagiannis shows Iguchi Takahito, Tonchidot, her AR Wonder Turner, an exquisite corpse inspired installation

Tish Shute: Yeah, it’s interesting, because we wanted the art gala to be even bigger, but it turns out, because of the logistics of putting up art in a conference space is fabulously expensive, because it has to be all installed and hung…

Bruce Sterling:  I’m keenly aware of that.  At Share Festival in Turin we bring in six installations, and it’s very heavy work.  It really takes a lot of logistics.  It was like a Battle of the Bands.  It’s like doing a rock concert.

Tish Shute: One of the installations I was really sad to not have there was Uber geeks’  “Steve” H.E.AI.D installation that Brady Forrest & Co. took to Burning Man.

So I was very happy that we actually did get the number of artists we did.

Bruce Sterling:  Well, there aren’t a million AR artists in the world, so it’s hard to judge.   I didn’t see many business people rushing up to have me critique their business plans.

Tish Shute: [laughs]  They were all in the meeting rooms.

Bruce Sterling:  Maybe it’s for the best.

VC and AR Startup Action

The Zenitum Booth, are2010, photo from Chris Cameron’s Flickr stream

Tish Shute: Do you know that why your talk started a few moments late is because we had 50 people who arrived from the Silicon Valley neighborhood I guess!

Bruce Sterling:  Did they not preregister?

Tish Shute: No. They all stood in the line for the same day registration!

Bruce Sterling: It ‘ll be interesting to see what transpires there, if there is a little wave of startup action.  God knows they need some place to put their money, because the VC scene in the US is pretty much moribund.

Tish Shute: Ogmento is the first US AR Games startup to get VC, I think.  I think there was some VC action at are2010 for sure.  And Qualcomm obviously seems to have commercialization plans for their AR technology, and to be scouting talent  and ways to deliver new AR experiences.

Jay Wright, Qualcomm presents Joe Dunn, e23 Games, winner of the are2010 StartUp Launch Pad with a check

Bruce Sterling:  Some  people don’t need venture capital.  I mean, Google Goggles isn’t going to be hurting for VC money, obviously [ see Chris Cameron’s RWW post, Google Goggles Coming Soon to iPhone] .  AR may come up through other methods, like people allying themselves with Hollywood, or peeling off of advertising companies.   There’s a lot of outfits who might conceivably want in-house AR skills.  Then when people set up a specialty AR shop,  they  peel off the list of clients.  I don’t know.  Those old days  of Silicon Valley venture capital seem like a lost world.

Tish Shute: Yes.  I, again, didn’t see anything really of the business tracks and production tracks.  Did you get back and forth between the tracks?

Bruce Sterling:  I went to the Hollywood tracks.  I mean, to the extent that I could.

Is Hollywood stirring? Who’s going to have the first breakout AR property?

Tish Shute: So what did you see from…Is Hollywood stirring?  Is it waking up?  I mean I know Kent Demaine, Oooii,  and Brad Foxhoven, Ogmento, spoke about the Hollywood AR scene.

Bruce Sterling:  There were guys there from LA who were sort of saying, look…they are aware of us, but they just want AR to promote their properties to some particular niche.  They realize that AR is potentially a mass medium and that you could do some real AR entertainment.  So they were batting around some ideas as to where that might happen.  Like, could it come out of a console gaming scene?  Who’s going to have the first breakout AR property?  A popular hit AR property, as opposed to like a neat way to sell shoes, or whatever.   Really, anybody’s guess is as good as theirs or mine.  But at least they were actively guessing.

Tish Shute: I know the breaking the fourth wall discussion has been going on for a while and now the question is, whether AR is going to take down the fourth wall and bring interactive storytelling into the mainstream.  Did you hear any of that?

Bruce Sterling:  Well, I always shy away from discussions of that kind because I don’t think there’s any “final thing.”  Practically everything that AR is involved in right now is  a transitional technology. Also, because I am a storyteller, I get alarmed whenever people in technology start saying, “Oh well, it’s all about telling stories.”  Because obviously it isn’t.

People can tell stories perfectly well orally, and absolutely nobody does that.  AR is not at all about telling stories.  It’s about a great many other things, such as user bases, niche audiences,  media saturation, urban informatics,  convergence culture, and the language of digital media.   I could list these factors until the world looks level. It’s really becoming pretty chaotic.  As I was saying in my speech, AR companies are media startups who almost never use the old-fashioned word “media.”

Tish Shute: Oh, that’s interesting.  Yes.  So why do you think that has happened that way?

Bruce Sterling:  Well, it’s because they are trying to do a different thing than media does.  I mean, they are trying to “augment reality.”  They don’t want you to know that you are using a medium.  They don’t want you to realize that you’re watching computer animation overlaid on some video acquisition stream.  That would defeat the whole point of AR.  It’s entirely different from an analog medium like television, where you turn on the television and there’s a constant stream of station identification alerts.   That’s like: “Don’t touch that dial!  You’re on channel 13!  Stay with us!”  Then it’s like, “And now a few words from our friendly sponsors!”  That medium was engineered to keep your eyeballs locked to a single stream that they’re feeding you.

In AR, it’s much more participative, more geolocative.  I’m not particularly interested in station-identification branding from my AR provider. What I really want to see is the interactivity of the augments they’re bringing to me.  It’s like  FlickR, the photo sharing site. You don’t have any TV-style splash page for FlickR.  “Hi! We’re FlickR! FlickR, bringing your photos to you!” No, FlickR is all about “you, you, you,” your photos, your tags, your friends, your activity around you.   It’s immediately trying to be very participative.

Tish Shute: Will Wright got to that point, didn’t he. He was trying to move us into an idea of blended reality. That the game is about the world, not about the dragons or the overlays per se.

Bruce Sterling: Right. I think that’s true. But see, the world isn’t a medium. A medium is something like this interview,  where I’m connecting to you and there’s a video Skype channel between us.  Whereas AR is more about spatial 3-D,  about 3-dimensional impositions.  Pieces of media: sound, vision, information visualization, tags, floating tags, air tags, icons, arrows, warning signs, warning sounds, tactility, whatever, being brought into the environment around us.

That’s why it’s properly called “augmented reality” instead of just augmented media.   If you call your work “augmented media,” you’re really in trouble. Because if it’s all about augmenting somebody else’s media, why doesn’t that medium just buy you, and augment their own selves?    If you think that way, instead of augmenting the world, you’ll just be a modest little plug-in for old-school media.

The World as the Platform

Blaise Aguera y Arcas, Microsoft, Santa Clara, are2010, photo from Chris Cameron’s Flickr stream

Tish Shute: Yes, which is why Blaise so generously gave the technical underpinning  for augmenting reality in his tech talk – about the trellis and the grapes,  he really explained how the world can become a platform for augmented reality.

Bruce Sterling: I wish I could have seen that. I did not see Blaise’s speech.

Tish Shute: We’re going to put the videos up in better quality.  People in the front row have put it up on the web already.  He really went into some of the challenges of mapping for augmented reality.

Bruce Sterling: His visual-mapping technique is important.  Registration is super important for AR.

Tish Shute: I think it was a really generous talk actually because he went step by step on how we will do this.

Bruce Sterling: I rather imagine that Microsoft has patented those steps.

Tish Shute: Oh, yes, I guess so!

Bruce Sterling: I could be wrong. Maybe they’ll open-source it. You never know.

Tish Shute: You never know. Because the world as a platform isn’t something one company can own, or go it on their own to exploit.

Bruce Sterling: I expect there to be a thorny path, but sometimes I’m surprised. Sometimes people really do try to fertilize the tech field in the hope of getting a good corn crop before they start fighting.

Tish Shute: We’ll I keep hearing that we may even see the unlikely marriage of Apple and Microsoft  – maybe wishful thinking, but there are motivations beyond AR for this unlikely match, and certainly between them these titans have what it takes to realize the grand visions of AR ? [laughs] But who knows…

Bruce Sterling: Well, yeah, it depends on where the thing catches fire.

Tish Shute: Yes. You mean whether AR catches fire in the form of  AR and mapping..

Bruce Sterling: It’s hard to say, but I’m convinced now that there’s more going on than I once thought. I thought that Bruno Uzzan made a very good speech for his company when he talked about how he worked on AR for eleven years.  Eleven years is no flash in the pan.   He has his long list of clients and successful applications. I thought he was right in his impatience with the press for not catching on. It’s gone on for quite awhile. The mere fact that you’re not aware of it, doesn’t mean it doesn’t exist.

The Illusive AR eyewear

My are2010co-chair, Ori Inbar, CEO and co-founder of the hottest new AR game development start-up, Ogmento, donning his goggles to open are2010 – picture from Chris Cameron’s Flickr stream

Tish Shute: Yes. So, the other theme you brought up in your opening keynote and I would be interested to know if anything you saw at are2010 changed your view is the illusive AR eyewear, and  if we actually got AR Goggles that worked they would bring AR’s gothic sister, VR, back from the grave right? [laughs]

Bruce Sterling: Right.

Tish Shute: It took quite a lot of work, but we pulled together a six-company HMD panel, right?

Bruce Sterling: Yeah. I was impressed to see so many of them there.  And I was chagrined to see how prototype-like all their gadgets were. But that doesn’t surprise me, because if any of those head-mounts were remotely working, they would be hyped out the wazoo. Everybody’s been waiting for them and hoping for the best. They’re obviously not ready for prime time. [laughs] Maybe in certain limited applications. Like maybe a diving mask. [laughs]

Tish Shute: No, I think what was nice though they got inspired and they all got together on the last day. I saw them having a meeting about standards. They got inspired to actually work together.

Bruce Sterling: Yeah, well, unless they’re going to invent mechanical eyeballs that those machines can fit onto, it’s going to be tough. OK, I’m a skeptic, but I’m prepared to be surprised. I’m also a skeptic in Artificial Intelligence, but as soon as they bring me an AI that can write a decent novel, I’m going to get it and review that book. [laughs]

Tish Shute: It’s interesting. Re AI, I’m totally in agreement with you. In terms of the way computers turned out, it wasn’t AI per se that they turned out to be good for, not in the way everyone had dreamed of it, rather it was the harvesting of human intelligence that turned out to be the big thing. But what is interesting is that despite all of that, AI or machine learning, as it is now called, permeates our whole society now from the stock market to how many businesses make many of their decisions.

Bruce Sterling: Well, there’s a lot of so-called collective intelligence.  But Marvin Minsky-style hard AI, no way. Alan Turing-style AI, forget about that.

Tish Shute: Yeah. So, that’s an interesting comparison with the HMDs.

Bruce Sterling: People stretch the definitions.  It’s like, well, my car engine is Artificial Intelligence. Yeah, so is your wall transistor. No, I don’t really think so.

And AR is a similarly big tent. I mean, Uzzan had to admit that he had denied that AR was AR, unless it was using his favorite technology. And he felt embarrassed to be rubbing shoulders with people who put AR into cell phones. And I can understand his feeling there, because, gee whiz, that’s certainly not what AR pioneers had in mind. But he had to admit he’d become more ecumenical about it. Obviously, they’re  there and doing business like gangbusters. You can’t very well ignore success, right?

I had a similar feeling about the goggles. Obviously, the goggles would be great, should they work. But if they did work, I rather think virtual reality would come very strongly to the fore.  You’d see people doing all kinds of elaborate immersive-style stuff.   A truly immersive technology doesn’t need to “augment” much of anything.

Tish Shute: Yeah, you’re right.

Social Augmented Experiences

Bruce Sterling: I think many of the most interesting AI aspects are not personal in the way goggles are.  They’re not about guys walking around with personal tech. They’re about big, communal, social-media experiences, like stage shows, and urban informatics, things where large numbers of people can interact with the same augmented reality. The projection mapping, which I go on and on about. Augmented public spectacles.

Tish Shute: Yeah, projection’s our best example of a social augmented experience right now because we are yet to have an easy way to do networked social augmented experiences easily – but that is of course the thrust of my interest in ARWave [see the slides for my presentation, AR Wave:  Federation, Game Dynamics, Semantic Search, Mobile Social Communications here].

Bruce Sterling: I think of Edison’s early days, when he wanted to sell movies to people for a nickel a clip.  You had to bend over and put your eyes on this visor and turn this crank. That coin-op device was easy for Edison to monetize, as opposed to getting a bunch of people to sit in theater seats. But people laugh at movies when they’re together in the seats.   Cinema is a more social, involving experience in a crowd situation.

Tish Shute: But it started with them, didn’t it, Hollywood – the movie biz? Basically Nickelodeons, right?

Bruce Sterling: That’s right. They were Nickelodeons. They were a lot like the goggles because they isolated the user.

Tish Shute: Yeah, that’s a really important point that the goggles are not Nirvana because of this question of whether they actually detract from the social augmented experience and blended realities, by drawing us into VR experiences?

Bruce Sterling: I’m tempted to claim that they’re more a VR technology than an AR technology.

Tish Shute: That’s a very interesting point because…

[thunder]

Tish Shute: Wow! What was that?

Bruce Sterling: Thunder storm.

Tish Shute: Oh, my God, how very Gothic! [laughs]

Bruce Sterling: It can get pretty loud up here in the mountains.

Tish Shute: Oh, you live in the mountains, better still!

Bruce Sterling:  Torino is in the foothills. This is Piemonte. So the Apennines are over there. The Alps are over here. We do get some rather spectacularly unstable weather.

Tish Shute: It sounded like a bomb to my NYC ears. [laughs]

Bruce Sterling: Yeah, it didn’t hit the building, but it was maybe half a kilometer away. I saw the flash.

Tish Shute: Oh, you did?   Well, I hope you don’t lose your power midstream here.    I was really happy to hear of that connection between Rudy Rucker and Layar  [Rudy was touched when Maarten Lens-FizgGerald from Layar said that he met the Layar co-founder at a Rudy Rucker lecture].

Bruce Sterling: That was very fun, yes.

Tish Shute: Wasn’t that wonderful? What was that experience like going around the conference with Rudy?

Bruce Sterling: Well, you know, Rudy’s very into graphics. He’s a mathematician, so he understands the underpinnings of this stuff. But he’s a skeptic. He thinks they’re kid toys. He’s not a gamer. He’s a good old-fashioned computer-science hacker. So he wanted to tell me all about his new eighth-order, fifth-dimensional fractals. He showed me a great many of them. They’re psychedelic. Rudy’s fractals are considerably trippier than most apps that help you find a barber or a train station. [laughs] Rudy really is a visionary. He’s into some very weird stuff.

Gamer Guys at are2010

Brad Foxhoven, Chief Marketing Officer, Co-Founder, Ogmento at are2010

Tish Shute: At are2010 there was a lot of discussion about how game dynamics and AR are going to intersect, right? Anything that you saw of interest there?

Bruce Sterling: Well, obviously, there are gamer guys there. Ori’s a gamer. The gamer guys are getting some money. The big buzz right now in gaming is, of course, social gaming.  Farmville has kicked everybody’s ass because it’s not even a game and yet it has more users than the entire gaming industry.

Tish Shute: I know, right! [laughs]

Bruce Sterling: Obviously that’s kind of humiliating. For a long time, I’ve seen people trying to do giant multiuser games on cell phones. It’s difficult to do because the interface on cell phones is crap, right? People aren’t going to run around responding to SMSs.

I can imagine people running around with little Wii-style bats that have audio and visuals on them. It makes a very large native AR game seem more plausible.

Tish Shute: Yes. that would be cool!

Bruce Sterling: Again, it’s not very gamelike to use those little fiduciary markers.

Tish Shute: No.

Bruce Sterling: Moving little cardboard chips, around like with card games…. It would be pretty easy to set up a little AR chess game.  Star Trek style hologram chess pieces,  and so forth. But it’s just cumbersome.

Tish Shute: And also, from what we’ve seen from things like Foursquare, the proximity based social gaming doesn’t have to offer very much [a crown badge, a mayorship] to get some mind share.. the social is the primary game dynamic…

Bruce Sterling:  I’ve seen a lot of different philosophies of gaming over the years. Who’s to say that Second Life doesn’t have the best idea? They built a little scene and then slammed their gate shut behind them.  But at least they’ve got a really nicely-paying little cult stuck in there. It’s different. And it’s manageable and it’s really theirs, theirs, theirs.  They don’t have to call in outside experts to try and run the monster.   They haven’t blown it up to the scale of Yahoo! where they’ve lost control of the enterprise, and gone into a tailspin of management overhead. Second Life has a very intense, almost a cultish atmosphere among the player-slash-developers.

Tish Shute: One thing that helped them was the thing they were always criticized, that the barrier of entry was so high. But once they got people they never left, right?

Bruce Sterling:  That’s not a bug, that’s a feature.

Tish Shute: One of the best features!

Bruce Sterling: Yeah, it’s like being in Mensa. Why don’t you lower your barriers to entry and get in some interesting stupid people?

Tish Shute: [laughs]

Bruce Sterling: In Mensa, we’d rather sit here making puns about neutrinos and fourth-order quadratic equations. [laughs] OK, that’s a business model, if that’s what you want.

The Man With the X-Ray Eyes!

Jesse Schell’s during his keynote, “Seeing,” at are2010, picture from Chris Cameron’s Flickr stream

Tish Shute: Ok!  Now to unpack the man with the x-ray eyes idea, Jesse Schell’s keynote theme.  This is a root metaphor for AR – making the invisible visible, seeing through walls. To me. I think you kind of wrote the book on this because all my ideas on what radical transparency might be come from you – your idea of Amazon.org is key to how I understand this..

Bruce Sterling: Oh, really? That’s funny.   I was touched that Jesse brought up that famous Corman film, because I was a judge in a fantasy film conference in Trieste earlier this year.  And Roger Corman was there.  He was the guest of honor.   “X: the Man with the X-ray Eyes” was one of the films shown during the conference, and I saw it.  I even had dinner with Roger Corman.  I had never met him before, so that was quite amusing.  The difficulty with a film of that kind is that what we science fiction writers call a “House of Cards Ending.”  In that story structure,  you ramp the thing up until the protagonist sees God, and then he has to be destroyed by the falling pillars of the temple.  That’s a classic science fiction structure:  like Frankenstein.  For the sake of the drama, Corman evades the issue of what’s really going on. For instance, let’s just suppose “the Man with the X-ray eyes” is not in fact a psychopath.  Let’s say he gets a grant from the Department of Health and Human Services, and he acts like a real scientist, not a stock B-movie “mad scientist.” So he has, like, backup guys, and some placebos, and a large group of people to test it on, trusted colleagues, and so forth.  You wouldn’t get any of that movie’s wild activity out of that.  What you would get is like a 5% improvement to people’s vision.

Then, in a year, there would be a 10% improvement in people’s vision.  There would be a  classic industrial story.  A rising star, you know, a cash cow.   Real tech isn’t done by a single guy as a divine curse.  It’s created by classic  tech startup culture.  So a runaway technology really behaves in the way that personal computers do.

Tish Shute: The things that get me all Utopian and happy about this are the ideas like those you first outlined with the notion of Amazon.org.

Bruce Sterling:  It would be easy to do an entirely different kind of film than “Man with the X-ray Eyes.”  Something much less B movie,  much less pat.  I mean, at the end of the film,  he destroys his own hardware and blinds himself.  Why?  For what rational reason would he do that?  Why doesn’t anybody else know the big secret of what he’s doing?  Why aren’t there Koreans doing it?  Why aren’t there Austrians doing it?  Why aren’t there Italians doing it?  Why?  AR doesn’t behave like that.  It’s not one lone guy with magic eye drops.  It’s entire teams of people that have been working on stuff for 17 years.  They all approach it in different ways.

Now, they are going to get scandals in AR.  I can guarantee you that.  They are going to get into  hot water eventually.  At least some people will surely come out and accuse them of being Roger Corman B movie monsters.  But unless they accidentally discover atomic fission or destroy the Gulf of Mexico with an oil spill [laughs], I don’t think they’re going to be particularly badly off!   The trouble I imagine  for AR people is very typical new media trouble.  It’s like movies being accused of corrupting our morals, or comic books being accused of leading to violence, or Google being accused of making us stupid and warping our brains.

I’m not an alarmist in that sense, but at least I’m concerned about real threats.  Roger Corman is a B-movie director who’s trying to sew up his lost plot ends by destroying his hero and his hardware. That’s not very plausible. It’s a nice science fiction movie device, but technology isn’t a movie.

Tish Shute: Yes. Well, the other thing that you always remind us of with AR is not to be saying it’s going to be this glorious moment when it’s no longer gimmickey, no longer pop culture. You always emphasize that’s actually part of what’s good about it.

Bruce Sterling: It’s not an accident that practically everybody in that audience knew about Roger Corman.  Nobody looked surprised; not the Austrians, not the Koreans. They were all like: “Oh, yes! Roger Corman!  Love him!”

Tish Shute: There were so many Rudy Rucker fans. Were you watching Twitter? People like Eric Gradman were succumbing to fanboyz moments..

Bruce Sterling: “Yeah. Rudy Rucker, he’s the best.”

“Rudy Rucker gripping an Augmented Reality shoe” from brucesflickr

Tish Shute: [laughs]  I noticed you inspired him to join Twitter..

Bruce Sterling: Well, I’ve got 8,000 followers and, obviously, a lot of them are Rudy’s fans.  Of course he’s going to be gang-rushed on Twitter. That’s not really any more surprising than two motorcycle stunt guys at the same attraction. And I’m a big fan of his Rudy’s blog.   He’s always got interesting things to say.

Tish Shute: Yes. AR does seem to bring out some of the coolest smartest people!  This morning I had breakfast with Joshua Kauffman in Central Park.  He is an advisor and entrepreneur working on design in the public sphere.  I was feeling rather brain dead and jet lagged.  I told Joshua I was wondering how to get the cottonwool out of my brains for this interview and he suggested,  the All Souls College one-word question interview!  Have you ever heard of that? – although apparently they recently scrapped it.

Bruce Sterling: Well, I’ve heard of All Souls College there in Oxford. What was their interview question?

Tish Shute: They used to use only one word, so they would only give you one word. It’s not a question. Basically, they throw out the word and then you had to spin off from there.

Bruce Sterling: You’re supposed to free-associate on a single word?

Tish Shute: I guess so. I hadn’t heard about it, but Joshua suggested it.

Bruce Sterling:  Well, it’s possible..

Tish Shute: Joshua came up with some good words..

Bruce Sterling: Right.

Tish Shute: We were talking about these proximity-based social work networks like Foursquare and Gowalla and how they may influence the emergence of social augmented experiences.

So Joshua’s suggestion for the first word was “territorialization” e.g. how do these new mobile social experiences like Foursquare,  and the observation that actually rather than breaking down territorialization, which would be a good thing, tend to support territorialization…but perhaps new forms of territorialization?

Bruce Sterling: Yeah, they’re re-intensifying it in a very odd, electronic fashion.

Tish Shute: Yes.

Bruce Sterling: I have noticed that.  It’s not true of stuff like projection mapping or the webcam fiduciary display stuff. But with the handheld stuff, and especially the urban informatic stuff, it really can’t help but take on a local flavor. Layar is like “Augmented Dutch Reality.”

And TonchiDot really is “Augmented Japanese Reality.” It’s hard to imagine a Layar interface going gangbusters at Tokyo.  Whereas the TonchiDot interface, which is very clearly influenced by Anime and cartoon graphics…. Maybe it could find some niche of hipsters in Amsterdam hash bars…

…to be continued in Part 2

“The internet eats everything it touches,” write Brady Forrest and Nathan Torkington, O’Reilly Media, Inc., in their must read 2006 companion essay The State of Where 2.0 (PDF).  Now in 2010 that statement is more true than ever.

Last week,  I talked to Brady about what we can look forward to at Where 2.0, 2010,  and what he thinks will be the “internet eating” trends emerging this year.  Brady is uniquely positioned to get a glimpse of things to come.  His job for O’Reilly Media is tracking changes in technology and organizing large scale events, including Where 2.0 which he chairs, and Web 2.0 Expo in San Francisco and NYC which he co-chairs.  Brady also runs Ignite, and previously worked at Microsoft on Live Search.  And, when not doing his day job, he participates in such Uber Geek activities as Steve the Robot H.E.AI.D – A Human Energized Artificial Intelligence Device…with lasers and generative sound, (click on pic above or see video here).  Look out for Steve the Robot H.E.AI.D, at Augmented Reality Event, June 2nd and 3rd, Santa Clara, CA,  and a presentation from Brady.

As Vernor Vinge pointed out in his intro to ISMAR 2009 – the “possibilities are both scary and wondrous” as “the physical world becomes much more like a software construct.”  Brady Forrest has taken a lead role, since 2004 – when “‘local search’ was interesting but not yet real,” in shaping this transformation.

Where 2.0, together with WhereCamp (this year at Google) constitutes WhereWeek – a crucible for emerging trends in web mapping platforms, and location based technologies.  This year augmented reality, proximity-based social networking, local search, and the rapidly maturing field of Crisis Management are in the  mix along with the huge and long established GIS industry which has moved rapidly into the Where 2.0 space.

But what business models will oxygenate the system is still a key question – one Brady discusses in the interview below.  Certainly, the usefulness of location based analysis, mapping, new interfaces, and bringing this data to every application is clear.

Crisis management is center stage this year Jeffrey Johnson (Open Solutions Group), John Crowley (Star-Tides), Schuyler Erle (Entropy Free LLC) who will present on, Haiti: CrisisMapping the Earthquake.  And Chris Vein & Tim O’Reilly will “discuss how cities and application developers will benefit from open data and what these programs will look like in the future”  in the plenary City Data.

Mobile social, proximity- based social networking, which may soon emerge as a challenger to web based social networks, and augmented reality are the sexy rockstars of  the Where 2.0′s 2010 showcase of potentially disruptive technologies.  Augmented Reality has had a breakthrough year, and this is reflected in its strong showing on the Where 2.0 schedule.  But, as Brady notes, AR awaits the killer app, that will drive it to the next level  Of course, we hope to unveil that at are2010!

At Where 2.0, I am presenting on The Next Wave of AR: Exploring Social Augmented Experiences panel.  We will look at how social augmented experiences will be key to the next wave of mobile augmented reality.  Mike Liebhold, in a complementary presentation, looks at Truly Open AR. If you have been reading Ugotrade, you already know I am an advocate for an open, distributed, real time communications framework for AR – see ARWave.  Wave Federation Protocol is an open fast, compact, federated, communications protocol that is a dream come true for AR.  And, I would hazard a guess that in 2010, real time communications plus location will become oxygen.

But also key to the next wave of AR, as I discussed with Anselm Hook in this post on Visual Search, Augmented Reality and a Social Commons for the Physical World Platform, will be a view constructed through complex “hybrid tracking and sensor fusion techniques” (Jarell Pair), cooperating cloud data services, powerful search and computer vision algorithms, and apps that learn by context accumulation.”

And as Brady notes in the interview below,  a key step forward would be “to take advantage of your location, but it doesn’t need to have been mapped before.”

For some interesting news on the mapping front (and a discount code for Where 2.0 for Radar readers) see Brady’s post, Flickr Photos in Google Street View. These kind of human built maps have the potential to develop into “photo-based positioning systems” that could create new opportunities for augmented reality.  Brady asks:

“how often the Flickr photos get updated, where else these Flickr photos are going to show up in Google’s services (Google Goggles perhaps?) and will they show up in new search partner Bing? I am doubly curious if Facebook will ever let its photos be used in a similar way.”

Lior Ron of Google Goggles will be at Where 2.0 to tell us all about, Looking into Google Goggles.  And if you want to learn more about how our view of the physical world will be ” rooted in powerful computing, pervasive connectivity, and the cloud” don’t miss this one.  I will be there.  And I very much hope there is a Q and A with this session.

During our conversation (see the full conversation below) Brady gave me his short list for breakthroughs that he sees as having big significance in 2010:

“Well, I think Google Goggles is one of the most exciting things to me.  Having access to a visual search…having someone actually release a visual search engine in that way, to consumers, I think is huge.  You know, you see stuff like that in the labs. But I don’t see it… it’s rare to see it out.

I think Android is huge.  And the way Google is pushing hardware to show off the platform; so the Nexus One being another example and the fact that it’s breaking free from the carriers.  Because I think when we get away from the carriers we are able to see more innovation, it’s what’s going to allow people or developers and companies to really innovate.

And I think Twitter adding geo-location to their APIs and buying MixerLabs is a huge move. I think Twitter may end up becoming the end-all be-all of location services. They are going to be updated constantly by people; they are going to have a really good grasp, real-time, of what is happening in any one place, at least based on the people.

And then with the addition of the MixerLabs data, they’re going to have more datasets at their ready, as well as any data that they start to collect from the clients themselves, like from TweetDeck.

So there are global clients that are updating Twitter.  I think those are some of the most exciting things.  And again, just to come back to Yelp, I think Yelp’s Monocle is also pretty significant, just because it’s an AR [augmented reality] app that’s being pushed into consumers’ hands.

And we’ll see how useful they find it.”

Talking With Brady Forrest

Pic above from WhereCamp 2009, Brady Forrest, facing camera, checks out Mark Powell’s Food Genome Project.  Check it out here – it just woke up!

Tish Shute: So last year when you were interviewed by Michael Calore for WebMonkey before Where 2.0 you said, “Location is no longer a differentiator it’s going to become oxygen.” And after attending Where Week 2009, I agreed with you and wrote about it here.  But, in what ways did this prediction exceed expectations, and what ways were you disappointed now as we get close to Where 2.0, 2010?

Brady Forrest: Well, it exceeded expectations in that there are now five different mobile OS’s where you can load on third party applications that active users’ locations that can then be shared out.

And so, what it is making is the possibility of real-time social location aware applications.  And this is something that hasn’t truly been possible in years past. Looking back three years ago when the iPhone launched, it was the first major phone, especially in the US, to be location aware.  And a year later, the Apps Store launched, giving developers full access to location, which previously had been held onto very, very, incredibly tightly by the carriers.

And now, a year and a half later, you have Android, you have Palm Pre, you have Blackberry working on their SDK to make it better, but it still is there.  You have Windows Mobile working on their SDK.  And, you know, who knows?  Maybe even BREW will get into the mix.

And AT&T is opening up their own interactive store.  And so, AT&T and Verizon and all their smart phones may now be looking at BREW.

Tish Shute: Right. It was very exciting last year at Where 2.0, where we had all these new toolsets announced and then the iphone being location aware. What were the best implementations of these new capabilities that became available in 2009, do you think?  What, in your view, was the most creative, surprising and disruptive?

Brady Forrest: Well, I am a huge fan of Yelp Monocle. I think, you know, that is just a real life example of using Augmented Reality.  You are on a street.  You have got a bunch of restaurants.  You have got a bunch of businesses.  And just to be able to swing through and look for people…I mean and look for ratings and reviews.

They have just started to institute check in, so you will be able to know where your friends are and where your friends have gone.  And that type of real-time, incredibly useful data is what will make augmented reality a standard part of the landscape.

I think it is that type of data, more so than, say, reference data, that will make people want to have all the possible sensors.  So, what do you need for that?  You need a camera.  You need a compass for orientation.  You need a GPS or, at least, a decent location service.  And then you need a screen where you can actually see the data, and then you need an Internet connection.

So it is not like any phone can handle this.  And so, you are going to need those killer apps to actually drive people to the type of phones that can support this.  I don’t think AR is quite there yet.

Tish Shute: I agree, for true AR you need more that compass, camera, and GPS.  There are some missing pieces for the real deal experience – and not just a pair of sexy AR spec.  As you mention, hybrid tracking and sensor fusion techniques that can combine computer vision technology with  compass and GPS are vital.  We need the compass.  We need the GPS.  We definitely need the camera!  But we need this combined with computer vision techniques to get the tracking, mapping and registration for true AR, or even to deliver a stable experience with the post-it/geonote AR that we see emerging with Layar, Wikitude, and others. At the moment we need to put together the tools for a true AR hyper-local experience.

And, of course, another aspect of this is the kind of physical hyper-links that applications like Google Goggles are building.

Do you have a speaker from Google Goggles at Where 2.0.  I would be absolutely fascinated to hear more about their road map?

Brady Forrest: I was loading Google Goggles onto the program yesterday.

Tish Shute: Oh, you did?  Oh, fantastic. And you have Lior Ron speaking!

Brady Forrest: It is actually possible it is not up on the website, but I talked to them and got them to agree to do a talk on it.

Tish Shute: I very much want to hear more about their road map.  Google Goggle’s is a very, very significant step towards the physical internet and this integration of computer vision with sensor fusion techniques necessary for true AR.

Brady Forrest: I mean that combination with Computer Vision is going to be incredibly valuable, because,  and then the other issue you have there is like is it on the client,  or is it on the server?  And right now, Google Goggles is definitely on the server, and that is not fast enough in real-time AR.  So that is like more of a 10 blue links IO interface.

Tish Shute: And also, they haven’t got an open API, have they?

Brady Forrest: No, not yet.

Tish Shute: Maybe they will announce that.  Can you nudge them?  For true AR,  we need to move forward in several areas – of course, there is the mediating device issues, like access to the video buffers in the iphone, and the development of cool AR eye wear would be nirvana!

But my recent obsession has been working on a real-time communications infrastructure for AR, because that is quite doable now, yet we don’t really have that real-time infrastructure, i.e. a real-time mobile social utility that is really up to the real time requirements of AR [see more about this here and on ARWave wiki].

But we certainly don’t have the integration of computer vision and sensor techniques, and the access to the big image databases we need, let alone the clients we need to put it all together either!

Brady Forrest: Google has done work to help out the community with their support of Open CV.

It is based out of Willow Garage, but I believe that Google has done quite a bit of work on it.

Tish Shute: Could you talk a bit more about Open CV?

Brady Forrest: O’Reilly has  a 500 page book on it.  It came out of the Darpa Project, or the Darpa Contest, where unmanned vehicles are raced.  And that has since become, at least in my mind, the primary computer vision library that people work with.

I actually used it…or, one of the teammates did, on our project we did this summer.  We implemented an Open CV pretty quickly that detected where people were, and then we would play music based on that.

Uber Geek Meeting from ShellyShelly’s photostream
Tish Shute: Is that your Burning Man project? Do you have a link for that, and some pictures, video?

Brady Forrest: Yeah.  Heaid.com.  Human Enhanced Artificial Intelligence Dancing.

Tish Shute: Thank you! This year the augmented reality story has been fairly basic – relying on basic sensors, compass, gps, accelerometers.  But it has also been an exciting year because  we hadn’t even had  smart phones with the camera, and GPS, and compass before this.

But now, the big adventure is to hook this all these sensor fusion techniques up with computer vision so that we can actually do reverse positioning for example from photos from what we are looking at, right?

Brady Forrest: Yeah, and start to use it in a more ad-hoc manner so that as you are traveling around, yes, it will take advantage of your location, but it doesn’t need to have been mapped before.

Tish Shute: Right – moving from mapping to context awareness.  Could you give like a quick explanation of what you did in your Burning Man project and how that relates to this kind of,  ad-hoc, on the fly, beginning to know what you are looking at without it having been mapped before, that is fascinating.

Brady Forrest: Sure.  So we mounted a camera about 30 feet off the ground.  And as people would move underneath or dance, they would move from block to block.  And we had kind of created kind of bitmap of the area underneath and set up different sound zones.  So as people moved from zone to zone, it would play different music.

And we used Maxim FP to handle the computer vision, although it has Open CV library to handle the computer vision part and to handle determining which of the audio to fire off.  And then, also, we had a laser that would play at the same time.

And then we used Ableton Live, which is a very popular DJ software to actually handle the music.  So as someone moved from, say, square A to square B, it would fire off various MIDI signals and Ableton would interpret that.  And each person who went in, up to…well, theoretically, up to 4- 8 people.  But because of how small the stage was and how the sounds are played, realistically, more like 4-6 people.

Each person had there own set of sound.

Pic from extramatic‘s Flickr stream here

Tish Shute: Wow! Awesome.

Brady Forrest: We would be able to detect different people, assign them a sound, or a set of sounds, so, like bass, drums, vocals.  And then we would have clips that played well together that were 3-5 seconds in length.

Tish Shute: At what distance could you detect people?

Brady Forrest: We had a 22 foot area underneath the camera.  That was mostly based on what the lens could capture.

Tish Shute: OMG I love this!  This is really the next step for augmented realities – not just attaching reference data to the world but exploring new shared “cosensual realities” (see Anselm Hook’s interview part 2 upcoming).

I am very interested in how in something you talk about a lot in your “State of Where 2.0″ essay, about lifestyle coming first for a potentially disruptive technology, not commercial considerations.  I still have to post the second half to my interview with  Anselm Hook but Anselm has some brilliant ideas in this area.  He is working on a project called Angel, where part of the vision is for people to actually find what they need without explicitly having to ask for it having to ask for it.

And this brings me to something that is very, to me, noticeable about Where 2.0 this year, and very exciting.  This is that location aware technology and crisis management basically has matured, hasn’t it?  We are beginning to see really useful stuff in this area now.

What is different this year that has brought crisis management and location aware technology together, a world in crisis?

Brady Forrest: Well, I think the primary thing that has brought all these technologies together is Haiti.  Without Haiti…A lot of times, future crises benefit from the current one, because people put in a lot of work.  And so, there is new infrastructure being laid with things such as Ushahidi, which is an open source platform for tracking…well, originally for tracking election violence in, but now is being used to track people and their locations and food requests in Haiti.

Also, Haiti did not have solid, accessible, good maps at the time of the of the earthquake.  And there have been two volunteer projects that have sprung up to help with that.  One being headed by the Open StreetMap Wood Foundation and many volunteers.  And then the other, Google Map Maker. And in both cases the activity around Haiti on these programs went up exponentially…or, I don’t know about exponentially, but a lot.  In the case of Map Maker, it was up 100 times and was the most worked on country for that week.  And one of the most downloaded for that week.

Tish Shute: Yes the work being done in CrisisCamps around the country is very encouraging.

Brady Forrest: And then also, you know, not just Ushahidi or Open Street Map, but also the People Finder which had open API so that different organizations could share their data, thus learning from Katrina.  There are all these different pieces of technology will be used in the future and hopefully be able to save more lives.  I didn’t see…there are iPhones apps that were released.  But I’m not aware of any Android apps.  I’m not aware of any AR apps.

Tish Shute: We don’t have smart phones devices distributed widely enough for them to be appropriate, do we, in a lot of areas where crisis strikes.

Brady Forrest: Yeah and there was criticism that they shouldn’t have been on iPhone.  You know, that iPhones were a waste of time. Because they aren’t…a lot of on the ground agencies aren’t going to have iPhones.  However, a lot of people who are going from the States will, and if the apps are there, then people will start to have them.

But relatively speaking, an iPhone is not that expensive.

Tish Shute: One thing I noticed and actually I discussed this in the second half of the interview I did with Anselm which I am getting ready to post.  But one of the aspects of the crisis filter was having people working as curators looking at messages coming out of Haiti, and while integrating the streams that would be useful is still probably a challenge, many curators will be on iPhones because they are based in the US.

We need to work across all platforms probably.

Brady Forrest: Yes.  Patrick Meier of Ushahidi, who runs Crisis Mappers, he ran a 24/7 emergency room. It was out of the Fletcher School in Boston.

They had volunteers all over the States and Canada.  They had volunteers in Vancouver that were translating Creole messages in under ten minutes.

Tish Shute: Yes and another point that is interesting in terms of the reconstruction and rebuilding of  Haiti is  the whole idea of leap frogging, and the idea that you can really… there’s always, as we’ve seen in other parts of the world, opportunity, when you miss pieces of basic infrastructure, to skip a whole stage and go onto the next one, like how virtual banking took off in Africa because of the absence of brick and mortar infrastructure.

Brady Forrest: To skip to a topic that been in my head, I’m just so bummed that the iPad does not have a camera.

Tish Shute: I was bummed is barely the word I would use.  Particularly as we had just been planning our ground breaking AR/next generation ebook in the days leading up to the announcement!

I suppose there is the hope they’re going to put it in the next one.  But I suppose the play for conventional content delivery is so big that everything else is trivial in comparison – especially in seems jump starting the emerging augmented reality industry!

So we might get thrown a camera and compass in the next round but will we get access to the video buffers?  AR enthusiasts may have to live on table scraps from Apple a bit longer it seems.

But what blows my mind is why hasn’t the iTouch got a camera, been AR enabled?  AR gaming would get an enormous boost from that alone. My son loves even the simple minded AR games available now on the iphone, and he loves iphone games – he has 110 games downloaded!

Brady Forrest: Ridiculous.  Yeah.  I don’t know what they don’t like about cameras.  And I plan on getting an iPad, but because of the limitations I plan on using it for base content and will probably get the bottom line model. I can’t imagine…I don’t know.

Tish Shute: It is very interesting, who actually puts together the big enabling mediating device for AR is still an open question, isn’t it?  I mean, that’s the truth; we have sort of mediating devices but we don’t have the magic brew yet do we?

Brady Forrest: No. Not yet.

Tish Shute: Good enough in some ways, and certainly a start but not quite the real deal.  For me, Where 2.0 this year covers the groundwork for true AR, mobile social proximity-based social networking, visual search, computer vision and sensor fusion techniques….   And because all these things have a chicken and egg relationship laying the groundwork is basically as important as having the mediating device otherwise you can’t do interesting things when we get the mediating device, right?

Is this the year we get the magic brew for AR, i.e., the business model, the killer app, and the mediating device?

Brady Forrest: This is not the year.

Tish Shute: Then I should ask you. Are you in the Goggles camp? That is do you think AR needs eyewear to go mainstream?

Brady Forrest: I think this may be where we get…we start to see what is going to be the killer app that gets people to buy the hardware that will support AR.  You see what I mean?  And then from there the apps will come out and the hardware will advance in that direction.

I don’t think AR has made that leap yet.  It hasn’t, to use almost a cliché, it hasn’t crossed the chasm yet and it hasn’t proven that it will.  Because I don’t know if…I think it’s difficult to tell right now.  Is it going to be games?  Is it going to be data layers? What is going to drive people to an AR device, especially one fully dedicated to it?

Tish Shute: I think in terms of AR games taking off a bit of help from the mediating device e.g. access to the iphone video buffers would probably be enough to stoke up AR games into being a hot commodity.  But in terms of AR data layers  going mainstream, we need some of the other players in the location space to put together the magic brew on the business model, don’t we?

Brady Forrest: That’s why I’m so curious though…that’s why I gave Yelp their own talk.  They are…Those guys are gang busters, they’re a consumer company, very consumer facing website.  They’ve got amazing data stores.  They do a lot of interesting stuff with their data.  And I don’t think people always give them the geek credit they deserve.

Tish Shute: You began Where 2.0 back in 2004, when as you point out, “‘local search’ was interesting but not yet real” and you have always stressed something that’s proven to be absolutely true which is lifestyle before commerce, right?  And that if location based services were going to be big it was because they meant something in terms of our lifestyle, not just because they told us where to get another good burger.  Right?

I think there’s been a lot of breakthrough in that area this year in terms of what location based services and proximity based social networks are to us now, how they’re changing our lifestyle.  What do you see as the breakthroughs for in 2009 and what are you hoping for in 2010?

Brady Forrest: Well, I think Google Goggles is one of the most exciting things to me.  Having access to a visual search…having someone actually release a visual search engine in that way, to consumers, I think is huge.  You know, you see stuff like that in the labs. But I don’t see it… it’s rare to see it out.

I think Android is huge.  And the way Google is pushing hardware to show off the platform; so the Nexus One being another example and the fact that it’s breaking free from the carriers. Because I think when get away from the carriers we are able to see more innovation, it’s what’s going to allow people or developers and companies to really innovate.

And I think Twitter adding geo-location to their APIs and buying MixerLabs is a huge move. I think Twitter may end up becoming the end-all be-all of location services. They are going to be updated constantly by people; they are going to have a really good grasp, real-time, of what is happening in any one place, at least based on the people.

And then with the addition of the MixerLabs data, they’re going to have more datasets at their ready. As well as any data that they start to collect from the clients themselves, like from TweetDeck.

So there are global clients that are updating Twitter. I think those are some of the most exciting things. And again, just to come back to Yelp, I think Yelp’s Monocle is also pretty significant, just because it’s an AR app that’s being pushed into consumers’ hands.

And we’ll see how useful they find it.

Tish Shute: Gary Gale, Yahoo! Inc., is going to talk on overcoming the business, social, and technological hurdles so we can reach the long promised [Laughs] Hyperlocal Nirvana. I think you’ve outlined some of these obstacles in relation to  AR, where there are obstacles are in terms of mediating device, and bringing all the pieces together including computer vision techniques in order to have an AR view. That’s the AR side of it. But the layer below that, which is the layer where actual location based apps that are beginning to go mainstream now,  are these presenting successful business models for location-based services.

So in short, in your view, what are the big hurdles to Hyperlocal Nirvana before we get to AR, even just for these location-based services?

Brady Forrest: Well, how do you make money?

Tish Shute: Yeah, to put it bluntly. I like John Battelle’s way of putting it [laughs] how do we oxygenate the system!

Brady Forrest: So are location-based services something that you can make money in the long-term? Nokia bought NavTec for $8 billion. And then two years later, they’re giving it away free as part of Ovi Maps.

Tish Shute: Right.

Brady Forrest: I’m assuming that that’s actually part of the plan.  And that although their hand may have been forced by Google with their release of Turn-By-Turn that…but it’s still got to be a hard nut to swallow that this huge investment in location ends up becoming a loss leader to sell more phones.

So, can you make money through subscriptions, through selling apps? And I think that is still being proven. The other one is, can you use advertising? And it’s kind of scary to see that Apple is restricting the use of advertisers to use location.

It came out yesterday or two days ago that advertisers cannot use location, or app developers cannot use location for ads. They can only use location to show something interesting or useful to their customers.

And there’s a lot of speculation that it’s because Apple wants to control the location-based ads that go on the iPhone.

Tish Shute: Yes. I heard a strange rumor.  Actually its an un-strange rumor, a likely rumor in fact,  that Apple and MS are getting together to replace some of the Google aspects of the iPhone like search and maps?

Brady Forrest: Yes, …. Microsoft employees get 10% off at the Apple store. There’s a longstanding relationship between those two companies.

And Android is definitely more of a competitive threat than Windows Mobile is.  And it’s well-known what the relationship between PCs and Macs are. So I don’t think…I don’t find that to be that surprising of a rumor.  I do wonder if it would hurt the iPhone, but it doesn’t surprise me that they would consider it.

Tish Shute: I do know, certainly from the AR point of view, Microsoft has recently hired some of the key researchers, including Georg Klein. And they are looking for more people in the image recognition area so it seems currently MS is going to be making a bigger push not just with PhotoSynth, but with image ID.

So it could be a pretty powerful combo between the iPhone, and Microsoft – they have some of the key computer vision research that would be needed for full AR.

Brady Forrest: Oh, yeah. Microsoft has amazing research depth. They’ve got an amazing team.

Tish Shute: But it is a bit of a mystery to me why Microsoft haven’t done more with Photosynth.  As I noted in my previous post, Nokia’s ImageSpace is beginning to do what many thought Microsoft would do with photosynth two years ago.  And “photo-based positioning systems” -  3d models of the environment to cover every possible angle, and then software that can work out in reverse based on a picture precisely where you are and where your facing could be hugely important to AR.  But that brings me to another mystery why haven’t we seen more from Nokia in this space yet – the N900 doesn’t have a compass?

Brady Forrest: Yeah, I don’t know why Nokia hasn’t made more of a space for themselves in these things. They did a lot of early work in these areas. I think they are trying to…my guess is that they’re trying to restructure themselves. They made some pretty big changes on the web-Ovi made its own division. And they’ve been doing a lot of location-based acquisitions: Places, Gate 5 several years ago, Gossler, just the past six months. And so I think that’s really been their focus…and the research team.

And a large company, since they haven’t found a business model, which is what we’ve been discussing here, they are hesitant to launch it, or to…they don’t really know if this is a business that they need to launch, or if this is an app that they should have there out for fun.

Tish Shute: Yeah. And that’s back to the oxygenation of the system and location.  We really still have some work to do to with the business models

Final question!  At the core of many of today’s business model is the idea of hoarding data – that’s an underpinning.

But ultimately, for open AR, we want a situation where we can really share data so that we don’t really have the data all locked inside one particular browser or app. The current crop of AR browsers aren’t really browsers in the sense that we understand a browser on the web today, because the data’s locked inside each service, Wikitude, Layar, Acrossair etc.

I have become very interested with Federation as a model for solving this, so that we can begin to have an opportunity to build consensual relations around data, sometimes sharing, sometimes not. Federation is my big dream at the moment.  And now we even have something to work with in the Wave Federation Protocol. But how do we get from here to there, where we really have a federated world of data for AR and location-based services? But you think people need to solve the question of business models first?

Brady Forrest: I think people need…I think one potential is ads; so serving up content.  And by ads, I also mean coupons, meals, the Foursquare…. what it looks like Foursquare’s going to do, featured content, which is Layar’s.

So we need to see, is that the way we’re going to sell these? The other is to have the best viewer, which in some ways is a race in selling that, but that’s potentially a race to the bottom, price-wise.

Tish Shute: Right. Do you think Google Wave Federation Protocol has a chance of taking off and changing the game for real-time communications, federation, real-time…

Brady Forrest: Quite possibly with the real-time. I think they need to work on the UI.

Tish Shute: Oh dear we can’t discuss the Wave UI right at the end of the interview – of course I believe it would do better in an AR view!   I know you have to go  now but I have to say Google Wave not standardizing the client/server interface – so we could seem some new UIs for Wave [we are working with PygoWave for ARWave because of this], and  iPad’s lack of camera were two huge disappointments in recent months.

Brady Forrest: Yeah. It’s [the Wave client] is very difficult to use.

Tish Shute: But the Wave Federation Protocol is an open fast, compact protocol that is a dream come true for AR.  Open, distributed, real time communications is a very big enabler for AR.  I would hazard a guess that in 2010 real time communications plus location becomes oxygen.