Planetary from Bloom Studio, Inc. on Vimeo.

It is just over a week until Augmented Reality Event, and I know there are a lot of people, including me (full disclosure I am co-chair and co-founder) who are totally psyched to see what unfolds there this year.   Bruce Sterling, Vernor Vinge, Blaise Aguera Y Arcas,  Jaron Lanier, Will Wright, Marco Tempest and Frank Cooper will join 107 speakers from 76 augmented reality companies on a single stage (see my previous post) to tell a momentous story of a technology of our time (also see my previous post here).

As Bruce Sterling points out, Augmented Reality is “truly a child of the twenty-teens, a genuine digital native,” and one visible indication that:

..the Internet really could look like a “legacy.” The Legacy Internet as an old-fashioned, dusty, desk-based place best left to archivists and librarians, while the action is out on the streets (see the full interview below).


(photo by Jasmina Tesanovic)

Opening this post is a video of Ben Cerveny’s Planetary app, which “turns your music into a universe,” and enchants all who try it.  Planetary shot into #3 on the Top Ten Free ipad app list soon after its release.

Ben Cerveny’s talk at Augmented Reality Event will be one of the must attend talks (see the full schedule for Augmented Reality Event here, and note my discount code for Augmented Reality Event, TISH295, is still good, if you want to register).

Planetary, while it is not an AR experience,  points the way for AR to take us out of the old-fashioned, “Legacy Internet.”

“Planetary is just the sort of science fiction experience you expect when using an object from the future like iPad,” developer Bloom Studio writes on the app’s iTunes page. ( from Mark Brown’s Wired post).

In his interview on cnet Daniel Terdiman, Ben describes how popular computing will evolve beyond those, “dusty, desk-based place best left to archivists and librarians,” (Bruce Sterling).

Ben points out:

“The tablet is a total disruption of how we understand popular computing. The next era of experiences will be driven by visceral gesture-based input, and rich fluid responsiveness in native graphics contexts. I see the potential for Bloom to help define a “killer pattern” for application design. Because apps have been deconstructed into discrete tasks that flow across devices….”

Bruce Sterling had some interesting comments on the Bloom app:

I’m a big fan of Ben and his good works in infoviz — and urban informatics, too. I admit I’m not sure the I entirely need the metaphor of a solar system in order to play a few Texas blues tracks. But I could be persuaded. Ben Cerveny is a significant thinker and a very well-spoken guy.

I’m not saying that happens tomorrow; I’m just saying, why not? Why not aspire to that?

I too am a huge fan of  The Bloom team, Ben Cerveny, Tom Carden, and Jesper Sparre Andersen (also see my post here about Fizz, the Bloom team’s app used by The Locker Project for their Strata demo).  And, if you haven’t already heard about The Locker Project and Telehash – get on it!  This is one of the most important projects of our time – an infrastructure for a better future!

Interview with Bruce Sterling by Tish Shute and Ori Inbar

Tish Shute: As you so memorably put it, “AR is a technovisionary dream come true – something really rare, and you have to be really patient for those….”

What is best and worst, in your view, about the way Augmented Reality technovisionary dream is coming true and emerging to flourish in the wild?

Bruce Sterling: The best part is that AR is truly happening and is a lot of fun, and the worst part is that it’s happening in a Depression. If AR had broken loose in the dotcom days when cash flew around like soap bubbles, man, that would have been psychedelic.

AR that is even more of-our-time than “social media.” AR has arisen directly from modern technical factors that just didn’t use to exist. It’s made from shiny new parts, and is truly a child of the twenty-teens, a genuine digital native. It’s a little kid and it has to walk before it can run, but it’s great to see it walking.

Tish Shute: As Jesse Schell pointed out last year at ARE2010, “The whole point of AR is to see things from a different point of view…How can there be a more powerful art form than one that actually changes what you see?” What do you feel will be the most impactful application of AR in people’s everyday lives?

Bruce Sterling: I’m all for impact, but it’s pretty clear that the people who would weep for joy to have Augmented Reality are people whose reality is already damaged. People who need reality augmented as a prosthetic, in other words, so that they can achieve an “everyday life.” This is like the impactful but underappreciated role of the Internet in the lives of people who’ve been shut-in. If you’re laid-up in a hospital bed, a laptop is a revolution in convalescence.

But that kind of “impact” doesn’t sound too exciting or too profitable. My guess would be that the biggest arena for “impactful AR” would be augmenting cityscapes for foreign people who can’t speak the local language, can’t read the signs, and lack time to learn the local reality. Imagine, say, the Brazilian overlay for Moscow. You could show up, read your native Brazilian overlay of that city, do your business, eat, sleep, buy, leave, and scarcely “be in Moscow” at all. Constructed right, the AR Brazilian Moscow might even be a better Moscow — a Moscow that Russians themselves would pay to visit.

Tish Shute: You pointed out last year, in your opening keynote for ARE2010, that less immersive forms of AR have their own merits. We are still not seeing much “head mounted display weirdness” yet, but many other forms of AR are emerging – mobile, webcam, projected video, sonic augmented reality, even sticky light. You noted, practically everything that AR is involved in is a transitional technology. But since you spoke last year at ARE2010, which of these transitional technologies have shown the most promise for AR?

Bruce Sterling: It’s got to be handsets. Smartphones. The stats there are just amazing. The smartphone biz makes the personal computer business look like a Victorian railroad. When I read a guy like Tomi Ahonen, who talks about transitioning out of the old-fashioned “Legacy Internet,” that idea is startling. But AR is one visible indication that the Internet really could look like a “legacy.” The Legacy Internet as an old-fashioned, dusty, desk-based place best left to archivists and librarians, while the action is out on the streets.

Tish Shute: This year we have seen gestural interfaces go mainstream. What are the most interesting directions for gestural interfaces that you have seen emerge in recent months?

Bruce Sterling: To me, the most “interesting” part is seeing people do gestural stuff in public. William Gibson, my fellow author, observes that cellphones have stolen the gestural language of cigarettes. There’s lots of fidgeting, box tapping, ash-swiping, slipping boxes in and out of pockets… People quickly learn to do that without thinking twice, and they forget how weird it looks. It’s “design dissolving in behavior,” as Adam Greenfield puts it.

The gestural hack scene for the Kinect has been amazing. It’s like watching 1950s Beatnik dancing go mainstream.

Tish Shute: You have observed that Augmented Reality is Glocal which not only gives us different flavors of augmented experience but is “a departure from earlier models of tech startups, where you usually have like three hippies in a local garage. Now you’ve got German-American-Korean outfits like Metaio, and Total Immersion has a Russian affiliate. They’re inherently multinational, both inside the company and out.” What flavors of glocalness do you hope/expect to see at Augmented Reality Event this year.

Bruce Sterling: I’d be pretty happy to see some AR input from Brazil, India, and South Africa. I seem to be picking up a lot of followers in my Twitter stream from those locales. If I saw some Augmented Bollywood Reality, that would pretty much make my day.

Ori Inbar: What sessions will you go to at ARE this year? Who do you want to meet at ARE 2011?

Bruce Sterling: I make it my business to hang out with artists, but I’m hoping to drill down more on the technical aspects. For instance, where exactly are the bottlenecks in building animated augments? It looks like we’re about a sneeze away from jamming some crude Hanna-Barbera cartoons into real spaces. But the devil is in the details there.

Ori Inbar: Your commentary about the evolution of the AR industry over the years had significant focus on style. Is the AR industry dressed to kill yet? Any glimpses of promise in that direction?

Bruce Sterling: I’m not “pro-style” in every possible aspect of life, but as an Augmented Reality critic, it’s clear to me that if you claim to “augment” reality, then you should work hard to augment it — struggle to make it better. Otherwise you might as well call yourself “Defaced Reality,” or even “3D Spam.” When I see that kind of crudity and carelessness in AR, I’m gonna call people out on it. I know there will be the AR equivalent of cheesy billboards and gang graffiti, but I never much cared for those, either.

The industry’s videos have improved radically in the past year and a half. It used to be all about “look at my grainy, shaky handheld video of my cool new AR hack,” but nowadays the biz has really pulled its socks up.

If AR is about “experience design,” as I think it basically is, then eventually, as a matter of intellectual consistency and professional pride, everything you create will be considered part of “the experience.” That’s the industry’s way forward — that’s what it would do if it was grown-up.

AR people already look better than most similar geeks in the gaming business, and some day, I really do believe that augmentation people will become glamorous. They won’t be supermodels, but they’ll be about as chic as, say, professional set designers. Because AR is set design, in a way; it’s real-time interactive set-design for three-D spaces.

Ori Inbar: In the Layar Launch in 2009 you said “it’s the dawn of AR…”, at ARE 2010, you followed up on the theme saying “it’s 9am in the AR industry.” What time is it now?

Bruce Sterling: I’d be guessing it’s around 9:30 AM, but come on, that’s just a metaphor! ARE we all gonna blow off at 4:30 PM and have a beer, or is AR one of those cruel tech startups where nobody ever gets a personal life?

Ori Inbar: Are you reading any new fictional literature about AR that inspires you? And/or What interesting design fictions for AR have you come across recently?

Bruce Sterling: Well, I’m always interested in creative people who just plain make stuff up. Because that’s why I commonly do myself. The stuff that “inspires” me is usually stuff that I just didn’t expect to see. But when I don’t expect it, that usually means I wasn’t paying enough attention. I plan to pay a lot of attention to AR this year.

I’m not sure it makes a lot of sense to write fiction nowadays “about AR,” because it’s no longer a fictional topic. It’s become like writing fiction “about cinema.” You can write good fiction about someone who works in cinema, but not fiction about cinema itself. AR is not sci-fi “Augmented Reality” any more, it’s become a real-world phenomenon, a new industry of real augmentation.

With that said, I must remark that I sit up straight whenever I see Marco Tempest do stuff. Magicians are all about mystery and wonder. You wouldn’t see a magician, say, using AR to work an assembly line, or re-order library books, or find a pizza joint in Barcelona. And that’s great. Marco is always gonna do something freaky and out-there, and even though he’s a tech magician, it’s never about the tech first. It’s always about his ingenuity in finding new ways to employ new tools in creating a magical experience for his audience.

Marco’s not an entrepreneur, he’s not gonna revolutionize people’s daily lives or invent Web 4.0, but even if AR becomes “old hat” some day, it’s never going to be old hat when he’s doing it. The guy is a pro, and I’m quite the fan.

Magic Projection Live @ TEDxTokyo 2010 from Marco Tempest on Vimeo.

Social Augmented Experiences leveraging geoawareness and human and machine intelligence to create real time information brokerages, combined with an augmented reality view, can create a new opportunities to reimagine our relationships with each other and our environment.

This Summer, I have been on a blogging hiatus, which has meant I haven’t been sharing as frequently and, unfortunately, the second half of two conversations I had earlier this year, both of which have much influence my thinking on social augmented reality, have languished in private mode -  part 2 of my talk with Bruce Sterling (see Interview with Bruce Sterling, Part I: At the 9am of the Augmented Reality Industry, are2010, and part 2 of my conversation with Anselm Hook - Visual Search, Augmented Reality and a Social Commons for the Physical World Platform: Interview with Anselm Hook, Part 1. Time to get caught up on some blogging!  The lightly edited transcript of Part 2 of my conversation with Bruce Sterling is posted in full below.

Bruce Sterling has been blogging all the key developments in augmented reality (amongst other topics of interest!) on his Wired Blog, and he brought my attention to Boskoi the Ushahidi based app for Android phones, augmented foraging pictured in use above – for more pics see fightthegooglejugend.

Augmented Reality and Real Time Information Brokerages

Picture above is the path the “nomads” took through the Westhaven cryptoforest with Pieter Bol,co-auteur of the book Biological Globalisation and Theun Karelse of Urban Edibles Amsterdam “who presented his ‘augmented foraging’ app Boskoi.“  For more see, The Cryptoforests of Utrecht and, Westraven Psychogeography, 6 June 2010. Note: Cryptoforests: 1) Urban forests hidden from view 2) Urban fallows that might or might not be considered as forests 3) Gardens gone wild)

My interest in the Ushahidi family of ideas was already fired up by a conversation with Anselm Hook early this year.  We discussed a number of Ushahidi related projects, Swift, Crisis Filter and Anselm’s project Angel, Augmented Reality, and my own keen interest in an open, real time, distributed platform for augmented reality – ARWave.

The Ushahidi platform and the related project Swift has pioneered the real time brokerage of information with people acting in curatorial roles or matchmaking roles coevolving with machine assisted matching to connect wants to haves.  Ushahidi uses multiple gateways including SMS, and Twitter.  But the Ushahidi family of ideas is extremely interesting when combined with augmented reality and suggests many new possibilities for social augmented experiences, as Anselm pointed out, for human to human communications, human to civilization communication, and human to environment communications (e.g., perhaps, how machine intelligence can help bridge the difference in time scale that Kate Hartman explores in her, Research for Glacier-Human Communication Techniques).

Ushahidi, which means “testimony” in Swahili, is a website that was initially developed to map reports of violence in Kenya after the post-election fallout at the beginning of 2008. It is now an open platform with a wide range of applications and growing developer community.  See What is the Ushahidi Platform? from Ushahidi on Vimeo.

Swift - a project that emerged from the Ushahidi dev community, is a human sensor/real-time brokerage for dealing with emergencies, enabling the filtering and verification of real-time data from channels such as Twitter, SMS, Email and RSS feeds.

Boskoi – augmented foraging is the first app,  I have seen, to begin linking Ushahidi with augmented reality – although I don’t think there is a full augmented view for Boskoi developed yet?

“The whole point of AR is to see things from a different point of view…”

Click to enlarge poster from upcoming ARWave demo at Software Freedom Day – for more see below

I am often asked what augmented reality brings to the table with respect to location based social networking, which is on the verge of going mainstream in smart phone apps like Four Square. While the first part to my answer is usually to explain what is unique to augmented reality.

As Bo Begole notes, the full vision of AR requires machine perception technologies to detect the identity and physical configuration of objects relative to each other to accurately project information alongside/overlaid with a physical object (see this post on the PARC Blog by Bo Begole on the difference between AR and ubiquitous computing – thank you Rouli for bringing my attention to this).

But it is only in recent months that we have begun to see the kind of tools that make this possible become freely available to developers – see my interview with Jay Wright of Qualcomm here.   Also see this post on Bundler: Structure from Motion for Unordered Image Collections an open source system that allows the creation of 3D point clouds from unordered image collections, e.g. internet image collections.  We now have many tools available to move mobile augmented reality beyond the recent crop of apps relying on GPS and compass alone for positioning into a new era of vision assisted AR apps that will increasingly bring the full vision of AR into our daily lives.

Further, the integration of visual search applications like Google Goggles and Kooaba which can detect the identity of particular objects will add another vital tool to machine perception technologies enabling AR “checkins” on potentially anything in the physical world around us, and more fuel to the Gamepocalypse (e.g. it would be easy to turn every trash can in the city into a basketball hoop as we discussed at the ARNY meetup last month).  And soon, the Pandora’s Box of  facial recognition (Google Goggles have the capability though it is not released to the public yet) will open up.

Jesse Schell described the importance of AR in a nutshell in his keynote for are2010:

“The whole point of AR is to see things from a different point of view…How can there be a more powerful art form than one that actually changes what you see?”

But how AR matures as a social experience will be the key to Jesse’s suggestion that:

“Augmented Reality will be one of the things that fundamentally define the 21st century”

There are many interesting forms of AR that are not reliant on a tight registration between media and physical objects – several are put forward by Bruce in the convo below.  And, it is likely we will see AR eyewear as an occasional useful accessory to a smart phone long before we have the sexy, affordable augmented reality eyewear worn that we wear throughout the day.   These speech to text glasses would be a very useful and viable accessory to a smart phone right now for the hearing impaired.

For the moment, as Bruce notes, some of the most interesting and useful augmented experiences to date have not been in the cell phone space:

“There are other aspects of AR besides the cell phone space. There’s Total Immersion’s big display screens. There’s the web-based fiduciary stuff. And there’s projection mapping. And then there’s experience design just for people who need their reality augmented for whatever personal or social reason.”

On of my favorite social AR experiences is this SMS Slingshot.

But I have been excited for a long while about the intersection of mobile social augmented reality, real time communications, and ubiquitous computing see Total Immersion and the “Transfigured City:” Shared Augmented Realities, the “Web Squared Era,” and Google Wave.  And I have described in many places why I think real time, open, distributed communications for AR are so important to developing social augmented experiences – see the slides for my talk at Augmented Reality Event here, here and here for starters.

ARWave at Software Freedom Day 2010, September 18th 2010

Thomas Wrobel and Bertine van Hovell will demo the first ARWave Android client at Software Freedom Day this weekend!

A number of people have asked me, (including Bruce), What will be the future of ARWave now that Google Wave is no longer a stand alone application?  Yes, the recently announced release of Wave in a Box (see here and here) is very exciting for the ARWave team.

The ARWave Android client is the first open AR client built on an open, real time, distributed platform -  based on a server that anyone can download and set up, currently the “FedOne” server but Wave in a Box, hopefully, will be even easier to deploy.  Wave in a Box seems perfect for ARWave’s needs – for more see the WiaB Google Group here.  And for more information on the ARWave client -  click to enlarge the poster below, see the ARWave concept video here, and for more, and how to get involved see arwave.org.   Props to Thomas Wrobel and Bertine van Hovell (posters below from demo for Software Freedom Day), Mark Evin, Davide Carnovale, and Markus Strickler, for all their hard and brilliant work on ARWave.  Also to JCPT the open Android 3D engine that has saved a lot of work!

click to enlarge slide

Social Augmented Experiences that Matter

My ideas on the future of social augmented experience have been deeply informed by the the conversations I had with Bruce Sterling and Anselm Hook this year.

Bruce Sterling notes in the conversation below, location based social apps like, Four Square, are interesting because they are not “urban geography like Google’s satellite stare from above,” but rather “groups of citizens are doing portraits of their own region.” Augmented Reality, with its of lauded power to make the invisible visible is, of course, is the ideal tool for “citizen portraits”  to the next level.  Cory Doctorow described to me three years ago (see here) an “inverse surveillance society,” enabled by an augmented view  – “where all the data from the positional and temporal characteristics of all the objects that we own were in aggregate visible and available so that we can mix and match them remix them understand them and have more agency in the world.”

It is very cool to go back to reread this conversation now that it is becoming possible to build the kinds of apps Cory described, and Bruce Sterling envisioned in Shaping Things (see Amazon.org  page 111).

click to enlarge

My  conversation with Bruce earlier this summer (see below) took place on the heels of are2010 – Augmented Reality Event.   See the video of Bruce’s keynote, “Bake a BigPie,” here,  and the final keynote, “Seeing,” by Jesse Schell (see video here) in which Jesse riffed on AR and the man with the X-ray eyes.  Both these awesome talks are still fresh in my mind.  Bruce noted how we should pay attention to augmentations for people and situations that could really use some augmentation… and not get too fixated on the coming of AR Goggles.  He elaborated on this in our conversation (again full transcript below):

“Well, it’s a matter of deciding whose reality it is that you’re trying to augment. I’m not trying to be a bleeding heart about it, but obviously there are people in our society right now with reality that could really use some augmentation. They are mostly disadvantaged people. They are vision impaired, or maybe they have autism. They might be senile and just can’t remember where they put their shoes. These are people who could really use some help, right?”

So, start with people who really need sensory or cognitive help. Before you turn our geeks into Superman, why don’t you try turning some people who are harmed into more functional individuals? Then you’ll be able to learn how to do that. Then maybe you can ramp it up to these Nietzschian heights of the superb Man With the X-ray Eyes. Whatever.”

What will make AR interesting and useful long before and long after we see the full vision of AR eyewear manifest is its social aspects.  Bruce points out:

“My argument would be that if you want people to be more sensitive toward certain, say, issues and problems, it’s better to find the people who are already sensitive to those issues and problems, and give them a bigger stake in your augmentation system.”

“Say that I am really worried about public health. Well, if I have a lot of nurses that are using my system, people who are aware of my issues, then I could be walking around and I’ll see a lot more tags saying, “This is where he got food poisoning!” “In this shooting gallery, many people have caught AIDS!” Or, you know, “Tuberculosis has been spotted over here in this building.”

At that point, I could simply share their knowledge and get some social intelligence. As opposed to trying to amp the basements of my little hacker-mind and drag stuff up that’s escaped my conscious attention.”

Finding new ways to broker information – bring together needs with haves and different participants, empowered and disempowered  is., as Anselm discussed with me, one way to change our view of human to human, human to environment and human to civilization communication (particularly in light of this  “sobering account of how open data is used against the poor in Bangalore” that as @timoreilly noted recently OpenData Empowering the Empowered).

The key idea in a crisis filter, Anselm noted,  was to break up the participants into different kinds, to connects wants with haves:

“There are people who are in  situation.  We call them citizens.  And then there are reporters, people who report situations back to Twitter.  And then there are curators, people that canvas Twitter looking for important Tweets.  And then there are first responders, people who take the curating collection of responses and then act on them.”

This kind of brokerage between people acting in a curatorial role or matchmaking role with each other can be extended into and coevolve with machine assisted matching as Anselm explains.

It is also a vital part of creating social augmented experiences that matter.

One of Anselm Hook’s projects, which is called Angel is the the most radical expression of connecting wants with haves in that the idea is that “you have a situation, you broadcast that situation, and help magically appears.  You don’t even sign up for a service.  You just get help …

As Anselm explains this is the same idea of a brokerage for dealing with emergencies, but applied to the long tail of crisis response.  As Anselm describes it:

“I am interested in personal crisis.  ‘I lost my cat.  Help.  I can’t find where my kid is.  I am out of gas.  I have a flat tire.  My house is on fire.  My aunt is trapped in the bedroom.’  The kind of personal crisis that is just as important, but is not enough to get a national movement to help you…

I will publish this conversation with Anselm in full in an upcoming post.

Zorop – an ARG for World Peace

If you want to be part of a really exciting experiment to reimagine our relationships with each other and can be in San Jose this weekend, I highly recommend exploring this “rabbit hole”.

Thank you Gene Becker, Lightning Laboratories and Ubistudio for sending me this invite:

“Ken Eklund (@writerguygames) is developing a wonderful game for the 01SJ Biennial called ZOROP, aimed at creating World Peace(!). Some of you might know Ken from his work on the amazing ARGs EVOKE and World Without Oil. Anyway Ken, along with his collaborator Annette Mees, are furiously working to get ZOROP ready to go for the Sept 17th premiere at 01SJ.

Are you intrigued? I thought so, and here are your next steps down the rabbit hole:

> Check out http://zorop.org to learn about the game

> Follow @ZoropPrime to watch it unfold: http://twitter.com/zoropprime

> ‘Like’ ZOROP on FB for a different view: http://www.facebook.com/pages/Zorop/141140772593618

> Become one with the game; consider volunteering as a Zoropathian: curious@zorop.org

> Head down to San Jose on the 17th, play the game, and ride the ZOROP Mexican Party Bus. Seriously.”

Interview with Bruce Sterling

Click on image above to see video clip from from brucesflickr

[Note the first part of this interview is here and I broke in anticipation of Part 2 just as I started experimenting with an idea Joshua Kauffman – an advisor and entrepreneur working on design in the public sphere gave me for an interview technique – the All Souls College one-word question interview.  Although apparently they recently scrapped it and I am not very good to sticking to a single word!]

Tish Shute: We were talking about these proximity-based social work networks like Foursquare and Gowalla and how they may influence the emergence of social augmented experiences.

So Joshua’s suggestion for the first word was “territorialization” e.g. how do these new mobile social experiences like Foursquare, and the observation that actually rather than breaking down territorialization – which would be a good thing, tend to support territorialization…

Bruce Sterling: Yeah, they’re re-intensifying it in a very odd, electronic fashion.

Tish Shute: Yes.

Bruce Sterling: It’s not true of projection mapping or the webcam fiduciary display stuff. But with the handheld stuff, and especially the urban informatic stuff, it really can’t help but take on a local flavor. Layar is like “Augmented Dutch Reality.”

And TonchiDot is “Augmented Japanese Reality.” It’s hard to imagine a Layar interface going gangbusters at Tokyo. Whereas the TonchiDot interface, which is so clearly influenced by Anime and cartoon graphics…. Maybe it could find some niche of hipsters in Amsterdam hash bars…

Stuff that’s socially generated by people on the ground, as with Foursquare and Gowalla, is bound to take on a regional influence. Right? It’s like the New York hipsters who were early adopters of Foursquare. They’re not mapping New York! They’re mapping Hipster New York.

It’s all about Williamsburg and places where 24-year-olds go to drink… They found a demographic niche there. These guys are building the service for them. They’re people who are willing to work for Foursquare for free, because they want to wear the little king hat.

Tish Shute: I got the far far away badge ‘cos I live on the Upper West Side!

Bruce Sterling: But that’s not urban geography, right? I mean, that’s not like Google’s satellite stare from above. That’s a group of citizens doing a portrait of their own region. You’re going to see interesting things happen because, of course, people who use Foursquare elsewhere are going to check into New York, and they’re going to look at the “New York Foursquare.” They’re going to be aliens who interact with Foursquare people in New York and annotate what they’re seeing.

Tish Shute: Oh! Yes. Good point.

Bruce Sterling: That Foursquare community has a certain émigré soul. It’s different from the normal émigré soul of simple tourists on New York. So you’re friend there is right about the territorialization.

Tish Shute: Yes, Joshua Kauffman is a smart guy! Yes I am interested to see what interesting kinds of deterritorializations proximity based social networks and the hyperlocal view of augmented reality might bring, not just the new territorializations.

Bruce Sterling: It’s not the intense kind of territorialization, like gangs putting down graffiti markers and beating people up. It’s an inherent regional character that comes with using peer production to build your database.

Tish Shute: We were discussing whether AR could break down the walls between people – people who share the same physical space but actually inhabit different territories even if they are sitting on the table next to you.

Bruce Sterling: You know, I just wrote an article for my Italian magazine column. I think I mentioned this to you – a report about ARE 2010. I titled it, “Chicks Dig Augmented Reality.”

Tish Shute: [laughs]

Bruce Sterling: There is a very heavy social element to AR, and a phone based element. So the question is: Why would a woman wear a fiducial marker? Like our Metaio speaker at ARE2010 who had a fiducial marker on her lapel pin.

Tish Shute: Right. Lisa!

Bruce Sterling: Why would a woman go out in public with her Facebook profile on her body?

Tish Shute: Well I can think of some reasons…

Bruce Sterling: So that men will approach her, of course.

Tish Shute: Yes the core of all successful social networks is always a form of dating app.

Bruce Sterling: You do it a social icebreaker. It’s like: I’m a woman, I’m sitting here alone, and you can sort of glide by and, you know, take a snap of me. Then you retreat and have a beer with your friends and you work up the courage, and then you come and say, “So! Susan! I understand you like bicycling! And, boy, me too!” Right?

Tish Shute: There are all kinds of social barriers between people in cities that AR might be helpful in breaking down. An extreme example is the dilemma you actually quite often face as a New Yorker as you walk around a city. There are people asleep on the pavement and you don’t know if they’re dead or alive.

Bruce Sterling: Right.

Tish Shute: And you sort of like have this awful ethical dilemma of like, “Am I walking by someone I should be shaking by the shoulder, right, to wake them up so they don’t die, right?”

Bruce Sterling: Yes.

Tish Shute: You said in your keynote that we should pay attention to augmentations for people and situations that could really use some augmentation..

Bruce Sterling: Right. There actually is such an app in Britain right now. I posted about it: two Augmented Reality schemes for rubbish and hobos.

Tish Shute: Right. Yes I saw that!

Bruce Sterling: “Any sufficiently advanced technology is indistinguishable from garbage and hobos.” You don’t need to personally find out whether this hobo is worth your help. What you need is a good way to report the hobo to a hobo check-up service. They come in, and they look on their own database or supply a database to you, or a facial recognition unit, whatever. The service says: “Oh, well. That’s Fred. He’s a paranoid schizophrenic. He always sleeps in that alley. Let him be.”

The same goes for the rubbish — although I don’t want to compare rubbish to hobos. In fact, people do go out with their AR kits and take pictures of abandoned garbage bags and broken glass. They upload them with geolocated tags for the local garbage guys. Guys who are sitting around doing pretty much nothing because they don’t know where the rubbish is.

And they will come out and get the rubbish! I mean, they just deputize guys to go out and follow these alerts. Right?

But nobody predicted — least of all me — that you were going to have a high-tech Augmented Reality system that consisted of removing rubbish and derelicts. Right? But rubbish and derelicts always go profoundly under-reported. It’s just hard to get people’s attention.

But it’s very easy to set up a system so that, if you get ten reports on the same piece of rubbish, that’s going to work its way to the top of the stack. That’s why I was trying to get AR people away from the romance of the hottest app for the shiniest machine. More toward a design stance that’s more user-centric.

Where are the actual problems about stuff that we perceive? Stuff we can’t do anything about? Or people whose mechanisms of perceptions are harmed. They could be doing good work, being more participative, if they didn’t, basically, walk around without their glasses on.

Tish Shute: Well this leads well into the second word, Joshua suggested was interesting spring board – sensitivity.

On the one hand we can do these things for people who maybe need the augmentation because they have difficulty with one or another sense, e.g., their eyes are not functioning, or their ears are not functioning. But on the other hand, we can’t cross the social bridge to communicate with people who are temporarily disempowered in relation to the rest of society e.g. hobos and people who sleep on the streets of New York City.  And even though Augmented Reality could potentially be helpful it can even be more disempowering to the already disempowered.

Bruce Sterling: Right.

Tish Shute: But re “sensitivity” – does augmentation increase or decrease our sensitivity? This is a problem that Will Wright brought up [see video of Will Wright’s keynote at are2010], e.g, the problem of parking HUDs getting in the way of your intuitive parallel parking skills. The Lexus that takes driving control from you when you look back, ‘cos it knows that you’re looking at the road, and it starts to brake. Right?

Bruce Sterling: Right.

Tish Shute: The fact that the problem with technology is that it makes us less sensitive, right, augmentations sometimes get in our way?

Bruce Sterling: I suppose that’s true. But I’ve heard that said about practically every medium. Especially television.

Everybody wants to blame machinery for their lack of morality. It’s hard to top something like the Kitty Genovese killing in New York. This sort of legendary New York horror story from the 1960s. A woman is stabbed to death in public, no one does anything.

Tish Shute: Right.

Bruce Sterling: I don’t think that our media is making us any less humane or more callous.

Tish Shute: All right. Oh no! I see what you’re saying. Perhaps I misrepresented what Will was suggesting by putting it that way. The question is perhaps more how do we get the sensitivity into the technology. Human bodies are fantastically sensitive and sensory.

Bruce Sterling: Right.

Tish Shute: And we have these like sensitivities. For instance, How could augmentations of reality be like a blush ? You definitely want an interaction that’s not just this data being pushed at you. But what is the data that counts, right? Will shows a slide often of an iceberg with the tip of the iceberg which is the conscious mind.

Bruce Sterling: Oh, I see. Yeah.

Tish Shute: And underneath it is all the preconscious stuff that really counts, right? Any thoughts on that?

Bruce Sterling: I did take interest in that. Will has obviously been spending a lot of time studying cognition.

Tish Shute: Yes.

Bruce Sterling: I’m not convinced that AR has got a lot to do with that. There is certainly a trend there. There are a lot of people who want to do body hacks and brain hacks. I can imagine AR being used for that purpose, but it seems like a niche application. What is the point of our accessing even more stuff that’s outside of our consciousness?

Tish Shute: One of the things he is talking about is game dynamics, is it? The role of the imagination in play. For example, he shows the high dynamic range photos that make the world magical. Something you want to engage with playfully. This he points out increases a sense of agency because you are encouraged to engage and to play with the world.

Bruce Sterling: Well, I’m a literary guy. Italo Calvino did a lot of writing about this. He talked about the classics of literature. Why do we read the classics? Calvino said we do not read, but reread the classics. And the reason we do that is that, at first, we read a classic book and we think, “Boy, this book is really good.” Then, five years later, we read it again and we think, “Boy, this is a really good book, and it’s got so much more in it than I thought it had when I was 18.” Then we read it again at 28, and it’s like, “OK, now I really seem to understand this book, and it means something to me now that I didn’t know when I was 18 and 25.”

What you are doing through that access is learning something about yourself. So Will is arguing is what I really need is like a better augmentation. So that I can go in there and sop up the book all at once. I can grab every cultural nuance in it, instead of the stuff that’s sliding past me because I’m 18 and kind of young and hasty. Maybe I could have certain words and phrases helpfully underlined, that are like, “OK, well, this part is problematic for you.” In some sense, that’s not allowing me to be 18.

I’m never going to have the experience of my own maturation against this text, because I’ve devoured it all in one gulp.

My argument would be that if you want people to be more sensitive toward certain, say, issues and problems, it’s better to find the people who are already sensitive to those issues and problems, and give them a bigger stake in your augmentation system.

Tish Shute: Yes the social augmented experiences are going to be the most valuable.

Bruce Sterling: Say that I am really worried about public health. Well, if I have a lot of nurses that are using my system, people who are aware of my issues, then I could be walking around and I’ll see a lot more tags saying, “This is where he got food poisoning!” “In this shooting gallery, many people have caught AIDS!” Or, you know, “Tuberculosis has been spotted over here in this building.”

At that point, I could simply share their knowledge and get some social intelligence. As opposed to trying to amp the basements of my little hacker-mind and drag stuff up that’s escaped my conscious attention.

Tish Shute: Interesting that seems to bring us to another kind of repetitive theme in AR, the people tend to pigeon hole it as “merely” a visual interface. But actually, it’s the intersection, isn’t it, of social intelligence and augmentation.

Bruce Sterling: Well, it depends entirely on how you design the system. If I’ve got a military augmented reality, I would expect that to be mostly about urban fighting. It’s going to be about kicking in a door and shooting terrorists. If I pull that helmet off my head and put that on the head of an emergency worker or a cop, I’m going to get a militarized cop or a militarized emergency worker.

Tish Shute: Well the histories of the two great mass media of the twentieth century – TV and the atomic bomb were intertwined, and I suppose the evolution of ubiquitous media, augmented reality and urban warfare is already intertwined too.  So how can we encourage augmented realities to move beyond military roots that is common to much technology and into more peaceful urban realities?

Bruce Sterling: Well, it’s a matter of deciding whose reality it is that you’re trying to augment. I’m not trying to be a bleeding heart about it, but obviously there are people in our society right now with reality that could really use some augmentation. They are mostly disadvantaged people. They are vision impaired, or maybe they have autism. They might be senile and just can’t remember where they put their shoes. These are people who could really use some help, right?

So, start with people who really need sensory or cognitive help. Before you turn our geeks into Superman, why don’t you try turning some people who are harmed into more functional individuals? Then you’ll be able to learn how to do that. Then maybe you can ramp it up to these Nietzschian heights of the superb Man With the X-ray Eyes. Whatever.

Tish Shute: Did you notice that a couple of apps actually like TagWhat have apps geared towards people with disabilities – I haven’t had a chance to check it out.

Bruce Sterling: I’m sorry, I wasn’t looking at their tags.

Tish Shute: I was discussing this with Joshua who mentioned Zachary Lieberman’s Eye Writer, which is for people with locked-in syndrome. Do you know that?

Bruce Sterling: Sure. And people appreciate that because the poor guy, he’s laid up with Lou Gehrig’s Disease. Now they’ve given him a way out. AR is like a spark of new hope that gives his life meaning. What’s wrong with that?

Tish Shute: Yeah. And Tim Byrne using Sixth Sense for Autism is interesting.

Bruce Sterling: Let’s consider it the other way. Let’s say this graffiti writer there, instead of him being sick and weak, let’s say he’s an athlete. So I want to make him into a super-human graffiti writer. I want him to run around graffiti-tagging the entire town before dawn. Is that a good idea? Do we need that? Super human, super taggers? What if he’s going to spray up stencils of Nietszche? I kinda wonder whether the game is worth the candle.

Tish Shute: Yes I suppose it is not a great social scenario to be always augmenting the lives of the elites! Hmm, the third single word interview question is “homophily,” and earlier were you’re saying that we’ve kinda got to accept this is very much part of AR – as how it works, because hyperlocal experiences gets created by local communities – that up to know have tended to be homophilies.

Bruce Sterling: Well, I think that’s easily handled with some design thinking. You’ve got to do some user observation and show some sympathy with the user, and to be aware that you’re designing for the user and you’re not designing for yourself.

In a field as young as this, it’s mostly geeks building cool stuff for geeks. In a lot of ways, it’s a “can you top this” contest. That’s OK, but it’s not good design to be your own client all the time. It’s like writing novels to amuse yourself, or sitting on the porch singing the blues on your own guitar with only yourself to hear.

Tish Shute: What will it take for AR mature out of this “geeks building cool stuff for geeks” phase do you think?

Bruce Sterling: It’s necessary to master some of the tools first. I think of the way the web has developed over the years. When the World Wide Web first appeared, it was just for physicists, and was all line commands and quite unstable and difficult. Then you got usability studies, and things like Ajax and so forth. It’s a very painstaking thing.

We’re not best at building interfaces for the best computer scientists. Web 2.0 was built from things like watching people cry while they were trying to fill out insurance forms. “Well, why are you so upset?”

“Well, I got to the end of the webpage, and then it said I took too long, and it cut me off and now I have to start all over!” Jesse James Garrett, right? Benefactor of mankind.

If you’re experienced, you think: “Why don’t I build a little module here, and kind of move the form over here, then I’ll periodically update it with some asynchronous Java and XTML.” And people are like, “Gee, how odd.” But that really works for real people. It comes from studying what people want to do. Whereas, the current AR approach to a problem like the insurance form would be like, “I will give you the ability to record the entire insurance form, and it will flash before your eyes!” OK great, that’s a cool hack, but I don’t really need X-Ray Eyes to fill out my insurance form. What I need is a more user friendly interface.

Tish Shute: Well it seems like we are moving into the terrain of Joshua’s fifth word “ventilation,” – if I understand it rightly – it is at least partially the antidote to territorialization because it’s this idea that a place needs air so we come out of our hermetically sealed boxes of the way we relate to a place and what kind of augmentation would bring more oxygen to that space.

There was an interesting moment in the Auggies because when Maarten Lens-FitzGerald presented the guerrilla shopping Layar and basically Mark Billinghurst and Jessie Schell who spoke first didn’t seem too impressed. They didn’t want to walk to shopping – that was what web shopping did, it saved us from walking to shop… but I felt, to me you picked up on something which might have some bearing on “ventilation” in that this AR shopping Layar was kind of squatting Prada – a favela chic AR shopping thing?

Bruce Sterling: I wasn’t sure if I was interpreting what Maarten had in mind by that. But I think Maarten sees his structure accurately as an experience thing rather than a mapping thing. I think he’s proudest of things like the Berlin Wall app on Layar, as opposed to Layars that help you go get a hamburger. It’s like…so when Layar inserts parasitic augmented shopping over other people’s real shopping? That was rather a subversive thing.

I think the key there is that his client is called “Hostage T-shirts,” right? I mean it’s actually kind of a transgressive little hippy T-shirt store that Layar can dump anywhere in the world. Layered right over, say, Versace and Prada. I don’t know what becomes of that effort. And I’m not sure about the term “ventilation,” because that’s a term of art I haven’t heard much.

Tish Shute: Maybe it’s like in a cafe. Ventilation would mean we were able to communicate with all these different categories of people that we normally would be unable to connect to, even though we might be sitting only a few feet apart.

Bruce Sterling: So it means ventilation in the bottles of our homophilies.

That’s not a personal problem for me. I commonly live in foreign cities and, you know, and spend a helluva lot of time talking to strangers at conferences. So I don’t think I’d have that particular tight little social island problem.

Tish Shute: Of the three judges at the Auggies, you seemed most enthusiastic about the Layar entry.

Bruce Sterling: It may be they’re not as familiar with the business models of locative AR as I am, and as Maarten is. It was kind of a subtle in-joke he was making about Layar’s own business model there.

Tish Shute: How do you explain that?

Bruce Sterling: Well, you know, Layar’s in the business of selling software to make mapping and urban structures into ecommerce.

The ideal way to do that obviously would be to move the richest customers into the most expensive shops in the most rapid way possible. Or at least distribute them in the directions they want to go, a la Google. Whereas this app that Maarten was talking about puts big barnacles in the way that are selling punk t-shirts.

Tish Shute: Right! Right!

Bruce Sterling: The Dutch are a bit subtle in their humor. I rather imagine there’s a lot of discussion in Layar’s inner circle about exactly what they want developers to do with their platform. They’re going to have considerable political difficulty deciding who can have a Layar key and how you discipline people when they start doing weird stuff. “The Oakland Medical Marijuana layar.”

Tish Shute: Well, finding nudists is one of the top layars at the moment.

Bruce Sterling: You know, obviously so. And finding narcotics in Amsterdam, or a prostitution layer. I warned them nine months ago this was bound to happen. I’m sure they’re aware of it. I don’t think Layar wants Google’s style of cool, technocratic detachment.

Tish Shute: But that’s pretty difficult to do in current augmented reality because we don’t have all the mathematical voodoo for full on AR search yet, do we?

Bruce Sterling: Well, you can hire it out. Somebody’s going to do it, if they get interested enough. There’s Nokia-Yahoo. Nokia-Yahoo! just did a big corporate deal…involving Nokia’s mapping system and Yahoo’s localization. So the Nokia-Yahoo! mash-up is called Nooo! Or could be called Yahno. Yakia! Unfortunately ridiculous names.

Tish Shute: It’s interesting because you mentioned the spiders’ mating problem at Google. They’ve got all the pieces to make this kind of level of AR obviously right now. But they actually haven’t done it yet.

Bruce Sterling: There must be at least some discussion in Google, but the same goes for Microsoft. I’m frankly baffled by Microsoft, because it’s just full of insanely brilliant people. What the hell are they doing in there? Name one serious innovation that’s come out of their labs in five years. They make Integral Research look dynamic. It’s really kind of sad.

Tish Shute: It’s a very curious situation with AR though, because AR more than any new technology relies on these big hordes of data particularly for the mapping, right? And only the big four have the data – although we are beginning to see upstarts, Earth Mine, Simple Geo… Did you get a chance to meet Di-Ann Eisnor from Waze – real-time maps and traffic information based on the wisdom of the crowd.  Waze is a very interesting project that is a potential giant killer.

Bruce Sterling: No, I didn’t talk to them. I’ve seen people speculate that Earthmine and Apple are going to make an allegiance. I guess if you’re thinking that urban informatic mapping is a super big thing for AR, that must be true. But I’m not convinced that’s necessarily the case. People have pointed out that you can just use Google Maps, and you don’t have to walk around with a little visor. There are other aspects of AR besides the cell phone space. There’s Total Immersion’s big display screens. There’s the web-based fiduciary stuff. And there’s projection mapping. And then there’s experience design just for people who need their reality augmented for whatever personal or social reason. [dog barking]

Tish Shute: Right. Oh, I’m in the middle… My son’s come. What a good hair cut!

Bruce Sterling: Hi, there.

Tish’s Son: Hi.

Bruce Sterling: How’s it going, sir? Good to see you…

Tish’s Son: Good.

Tish: [laughs]

Bruce Sterling: Yeah. Nice looking shirt. I like the back of it.

Tish Shute: That’s from the American Shaolin Temple. [laughs]

Bruce Sterling: All right. Awesome. Kung Fu geek shirt.

Tish Shute: Yup he is a bit of Kung Fu Geek. He and his dad did an iPhone app on it for Yu-Gi-Oh, for Yu-Gi-Oh scoring.

Bruce Sterling: Awesome. Plenty of Pokémon-style combat in Yu-Gi-Oh.

Tish Shute: Yeah. Well, it’s interesting because you’ve talked about this aspect. That all of this, the Pokémon aspect of AR hasn’t kicked in yet. But it’s obviously a match made in heaven to some degree, isn’t it?

Bruce Sterling: One would think so, yeah. The whole little kid gaming thing. What does that have to do with Google or Bing? You don’t need a massive database for stuff like that.

Tish Shute: Yeah, you’re right. But good tracking, mapping and registration requires a lot of mapping…

Bruce Sterling: Well, our current tracking, mapping and registration requires that. Maybe there’s some other way to hack it that we don’t know about yet.

Tish Shute: That’s a very interesting point. We always have to stretch the way we think about mapping… perhaps its a real-time understanding of the location you’re in…perhaps the map is being negotiated through several social processes?

Bruce Sterling: There are maps, and then there are maps. There’s a kind of artillery map where you need to know the precise location of target spaces. And then there’s the kind of social map where I’m really looking for the IN-N-OUT Burger where my sister went last Tuesday. That’s a different system.

Tish Shute: And I think AR, at the moment, we’re getting the most out of the social maps certainly. And the other [machine perception technologies to detect the identity and physical configuration of objects relative to each other to accurately project information alongside/overlaid with a physical object] is still kind of the big dream, isn’t it?

Bruce Sterling: They say that men never ask for directions and women never read maps. Clearly, the genders have different ways of navigating the world. Who’s to say what manner of augmenting our experiences is hottest? I’m not convinced that today’s rather rigid geolocativity is really what our society wants from that particular service. Maybe what we want is something more folksy. Some useful nudge in the right direction as opposed to grids with 200 meters here and instructions to turn such-and-such.

Besides, there’s other hacks we haven’t considered. We’re very dependent on GPS, but just suppose all those satellites are blown out of the sky in a solar storm. Would we really want to give up mapping? Wouldn’t we just come up with some other nifty hack? Radio beacons, let’s just say. Atomic clock timers in towns. Or maybe just little QR codes on lampposts that give you the exact location of that lamppost, and just click the thing and have it calculate where you are.

Tish Shute: Yes the NextHope OpenAMD project had a clever way of triangulating location indoors.

Bruce Sterling: Well, GPS is there and people all want to use it. It’s got good API, so of course you want to. And the guys who are good at doing it are real geolocative freaks. But the mere fact that we can do it this way, and that you can make it pay, doesn’t mean that it’s the ultimate way to provide that service to people. It’s like saying that Egyptian hieroglyphics must be the greatest way to write, because we’ve got a lot of them and they’re hard to learn. What if somebody comes along with an alphabet? It’s going to be a little embarrassing.

Tish Shute: Yeah, that’s a very good point. Now, this is a more simple ordinary question about the event. YDreams went off the map in the Auggie voting, and walked away with The Auggies. No one doubted that that was the most…

Bruce Sterling: I don’t know. I thought those Occipital guys with the panoramic painting…. That was hairy. I would have been tempted to give them the prize myself, actually.

Tish Shute: And what did you like best about that? Because I agree. I love Occipital.

click to enlarge

Bruce Sterling: I thought it was a more technically difficult stunt than the hand registration thing. Using a hand as a 3-D cursor is hot, but not like painting a panorama in 3-D in real time. That was an impressive technical feat.

Tish Shute: And they hinted at the 2.1.1 AR, more AR version of that. What do you see coming out of that as possibilities?

Bruce Sterling: Well, I’d heard of YDreams, so I wasn’t stunned. But I’d never heard of those guys. I wonder what else the heck they’ve got in the attic.

Tish Shute: very cool stuff…

Bruce Sterling: Well, more power to them. But clearly YDreams was the popular favorite. And who couldn’t like it? It was just so AR.

Tish Shute: It was so AR and so gorgeous.

Bruce Sterling: It was pretty, actually.  Except for their ugly menu button and poor font choice.

Tish Shute: Oh, yes. You didn’t like that, did you? [laughs] But with the Occipital panorama, what do you see the next stage of that?

Bruce Sterling: Well, obviously quicker and faster. Quicker and faster and more accurate in a network. Let’s just say I’m in New York and you’re in New York and I’m calling you for help. And you say where are you? I just whirl around like this and I mail it to you on a Google Wave. And you whirl around like that, and then we compare the two panoramas and do an instant triangulation. And you say: I’m over here on this red dot of your screen.

Tish Shute: Yeah, exactly.

Bruce Sterling: We’re navigating with panoramas by having two connected panoramas and considering the difference.

Tish Shute: Yeah, very interesting…

Bruce Sterling: Not shabby, right?

Tish Shute: Not shabby at all.

Bruce Sterling: If you could do it in real time.

Tish Shute: Then the other thing I missed because I was going to meet Will was I missed the Launch Pad competition. Did you catch that?

Bruce Sterling: I didn’t see it either. I thought of another app though.

Tish Shute: Oh!

Bruce Sterling: You’ve got a panorama maker in your home office, and it just scans the office 24 hours 365 and tags anything that moves, right? OK, where’s the clipboard?  At 8:15 it was over here. Now it’s vanished. Now another object is viewed over here. So, logically, ping, you hit it with a sticky light and there it is, right?

Tish Shute: Oh, that’s cool also knowing what has changed in any environment would be a big enabler for a lot of AR visions.

Bruce Sterling: I’m sure there are many other things you could do with panoramas.

Tish Shute: My jet lag is beginning to kick in big time – so many ideas to pursue from are2010 – those panoramas are very exciting though.

Bruce Sterling: Oh, well, it’s all right. We can augment reality! I’ve got three heads and six hands!

Shortly after Augmented Reality Event – are2010, I talked with Bruce Sterling on skype and in gdocs about his experience there.  I am posting the conversation in two parts to make it a more blog friendly length!

The picture above is the Auggie Award for the best AR demo (above) designed by Sigal Arad Inbar.  It was won by YDreams! See, Ivan Franco recounts the team’s ARE 2010 experience, and winning the event’s first-ever Auggie Award, for more.   The video below was shot at the YDreams booth by Bruce Sterling.

“The Hotness” – YDreams rocking it at ARE2010 from brucesflickr

Rudy Rucker, who was hanging out with Bruce Sterling, captured the are2010 buzz and some great images in his post, Augmented Reality, Painting, Twitter. As Rudy put it:

“AR is hoping to be a next big thing, a cozier and more commerce-driven cousin of the old VR, or virtual reality.”

Bruce Sterling’s opening key note is up, ARE 2010 Keynote by Bruce Sterling: Bake a Big Pie!, and also the ARE 2010 Keynote by Will Wright: Brilliant Inspiration for the Augmented Reality Community with more videos from are2010 on the way.  One must read post on are2010 is Chris Cameron’s post, Augmented Reality’s Next Steps: Sitting Down with the Titans of AR.

Talking with Bruce Sterling, Part 1

The Auggie panel, Bruce Sterling, Jesse Schell, and Mark Billinghurst inspect the award.

Tish Shute: In your keynote at the 9am of the augmented reality industry you asked some questions of the are2010 audience: “What’s the mission statement?  You’re the world’s first pure play experience designers, except that user experience it’s mostly futuristic hot air.  But run with that, right?  What are your tactical steps?  You should get dressed, have a coffee, have a to-do list.”

How much of that did you see going on in the next two days?

Bruce Sterling: Well, I wasn’t privy to any of the business discussions.  I didn’t think it was an accident that this standard AR enabled tag thing came up from Bruno Uzzan, Total Immersion.  That seemed to me to be a useful thing.  I was always interested in the Augmented Reality Consortium.  It struck me as remarkable that there was this group of people who clearly all knew one another and it had some kind of game plan.  I applaud them for that, because these are not the 1980’s.  [laughs]  You know, it’s just a different world for young startup companies.

Tish Shute: I think you’re right. There seem to be some VC conversations going on, we don’t know what went on in the meetings, but it was noticeable in the atmosphere of excitement, and remarked on by a few people. So I think that kind of was definitely going on.

And, of course, I was so busy I never even got to see the expo properly! You said you wanted to be surprised.

Did anyone surprise you in any of the talks, in any of the expo?

AR used as interfaces for devices

Italian augmented robot from SEAC02 from brucesflickr

Bruce Sterling: I have to say I was a little bit surprised to see Andrea Carignano demoing a robot. I happen to know him because he’s here in Torino. He’s the guy that came out of Fiat and went into AR. I am not a particularly huge robot fan, but I think it’s of great interest that AR is used as interfaces for devices, as opposed to the Jesse Schell idea that AR is all about a “man with the X-ray eyes.”

My suspicion is that a lot of surprises will come out of mashups of AR.

Tish Shute: I didn’t get to see Andrea’s robot.  So what did it do?

Bruce Sterling:  It’s basically a sister device to that little helicopter that those Parrot AR Drone guys were doing.  It’s a little autonomous robot and it runs around with a webcam on it.  You can place video into the acquisition stream coming off the robot.  You can play a game, and blow away imaginary monsters or whatever.

Tish Shute: It’s interesting, because did you notice Will Wright and Patrick O’Shaughnessey, Patched Reality, spend some time hacking the Parrot AR drone in the hallway?  Did you come across them?

Bruce Sterling: Rudy was there with them.  You know, I didn’t want to watch Will Wright hack a robot.

[laughter]

Tish Shute: They seemed to be having fun even though as it turned out the power supply was dead.

Bruce Sterling:  I’m sure Will enjoyed that.  As a game designer, you want to go out and get your hands dirty with a plastic gizmo.

[laughter]

My Swiss Army knife can’t get through airport security, so I really don’t want to strip anything down.  But yeah, what else did I see that was of particular interest?  I was pretty happy about the Korean guys because they are a difficult group to get close to.

AR companies are like mini-global micro-startups.  They’re “glocal”.

“Korean elegance at the Zenitum booth” – from brucesflickr

Tish Shute: Zenitum.  What did you like from Zenitum.  They were one of our sponsors, along with Qualcomm.

Bruce Sterling:  I know that Seoul is like the number one center for augmented reality discussion.  But it’s  difficult to get behind the scenes as a journalist there and  track what’s going on in Korea.  I’m fine with Italian “realtà aumentata.”   And I feel like I’ve got a handle on French “réalité augmentée.”   The Germans were not hard to find, and the Dutch all speak English!  But the Koreans, and whoever the hell it is in Kuala Lumpur…  I have no idea what’s going in Kuala Lumpur, and only the vaguest idea of what’s transpiring in Singapore!  But I know that people there are paying a coherent interest.

So the Koreans show up, and they had some relatively predictable anime style 3D avatar conversion stuff.  But they had a really nice display space.

“Anime figures become three-d smartphone animated avatars,” from brucesflickr

Tish Shute: Ah, So Zenitum created a hot spot at the exhibition?

Bruce Sterling:  Yeah.  The Koreans had  IKEA furniture and some nifty little woven baskets.  They’d really classed up their presentation.  Most Koreans in tech tend to be kind of muscular.  The Koreans are not known for their refined presentations.  On the contrary, they tend to undersell everybody else.  But I don’t know, maybe they’ve been hanging out with Samsung and upgrading their design chops. [laughs]

Tish Shute:  Did you take some photos you could send me?

Bruce Sterling:  I took a few, but  I don’t consider myself a photographer.  They’re all up on my Flickr set. It was interesting to see so many people from so many different nations in such a collegial atmosphere.

Tish Shute: Yes – there were many different countries represented at are2010

Bruce Sterling:  It’s the beginning… and so global at such a young stage.

Tish Shute: Yes. As you said, it was 9 AM, so everyone was actually super excited to be gathered together from across the globe to start a new day together.  As you mentioned, there was a very warm affirmative vibe – everyone sharing a passion.

Bruce Sterling:   They have an online commonality. They seem to be aware of one another’s work through the Internet.

Clearly they had all heard about one another.  That’s a departure from earlier models of tech startup, where you usually have like three hippies in a local garage.  Now you’ve got German-American-Korean outfits like Metaio, and Total Immersion has a Russian affiliate.  They’re inherently multinational, both inside the company and out.

Tish Shute: It was the multinational garage, wasn’t it?

Bruce Sterling:  Yeah.  AR companies are like mini-global micro-startups.  They’re “glocal.” There’s something quite new to me about that.  I don’t find it’s shocking, because in Europe today it’s common to find startup teams who are multinational.  But to see such intense globalism at such an early stage of an industry is really different.

Tish Shute: Yes it made for a fun atmosphere?  It was wonderful running into Iguchi Takahito, Tonchidot.  You have a great rapport with each other despite the language barrier?

Bruce Sterling:  Yeah.  That guy from Tonchidot, he’s very charismatic.  He’s punchy.  That’s reflected in the very strong graphic design from his company.

Tish Shute: Using minimal English to make the case for Sekai No Camera at the Auggies, Iguchi Takahito still got through to the audience.

Bruce Sterling:  Well, his visuals were good.

What AR means for artistic practice…

Picture of Eric Gradman’s Cloud Mirror, from James Alliban post ARE2010 – Augmented Reality utopia in Silicon Valley – see for more on the are2010 ARt Gala

Tish Shute: So before I move on to wider themes, I’m going to wrap up on some of the different aspects of the conference.  I was chairing the technology track but you were more free roaming, was there anything that went on in the sort of hallway discussions and the presentation rooms that struck you?

Bruce Sterling:  Well, I did get collared by artists.   They really wanted to talk to me.  We got into some serious discussions on  what AR means for artistic practice.  How you can do this and reach that, how can one sharpen up one’s presentation?  I mean, they really wanted some art criticism.

Tish Shute: That’s very interesting.  Did you come up with anything that you hadn’t been thinking about already through the conversations?

Bruce Sterling: I’ve seen augmented reality installations before, and I certainly know many electronic artists.  But I don’t know.  People in the AR art space, they are looking for guidance and trying to find fellow spirits.  In their own way, they have the same pioneer spirit as the business people.

Helen Papagiannis shows Iguchi Takahito, Tonchidot, her AR Wonder Turner, an exquisite corpse inspired installation

Tish Shute: Yeah, it’s interesting, because we wanted the art gala to be even bigger, but it turns out, because of the logistics of putting up art in a conference space is fabulously expensive, because it has to be all installed and hung…

Bruce Sterling:  I’m keenly aware of that.  At Share Festival in Turin we bring in six installations, and it’s very heavy work.  It really takes a lot of logistics.  It was like a Battle of the Bands.  It’s like doing a rock concert.

Tish Shute: One of the installations I was really sad to not have there was Uber geeks’  “Steve” H.E.AI.D installation that Brady Forrest & Co. took to Burning Man.

So I was very happy that we actually did get the number of artists we did.

Bruce Sterling:  Well, there aren’t a million AR artists in the world, so it’s hard to judge.   I didn’t see many business people rushing up to have me critique their business plans.

Tish Shute: [laughs]  They were all in the meeting rooms.

Bruce Sterling:  Maybe it’s for the best.

VC and AR Startup Action

The Zenitum Booth, are2010, photo from Chris Cameron’s Flickr stream

Tish Shute: Do you know that why your talk started a few moments late is because we had 50 people who arrived from the Silicon Valley neighborhood I guess!

Bruce Sterling:  Did they not preregister?

Tish Shute: No. They all stood in the line for the same day registration!

Bruce Sterling: It ‘ll be interesting to see what transpires there, if there is a little wave of startup action.  God knows they need some place to put their money, because the VC scene in the US is pretty much moribund.

Tish Shute: Ogmento is the first US AR Games startup to get VC, I think.  I think there was some VC action at are2010 for sure.  And Qualcomm obviously seems to have commercialization plans for their AR technology, and to be scouting talent  and ways to deliver new AR experiences.

Jay Wright, Qualcomm presents Joe Dunn, e23 Games, winner of the are2010 StartUp Launch Pad with a check

Bruce Sterling:  Some  people don’t need venture capital.  I mean, Google Goggles isn’t going to be hurting for VC money, obviously [ see Chris Cameron’s RWW post, Google Goggles Coming Soon to iPhone] .  AR may come up through other methods, like people allying themselves with Hollywood, or peeling off of advertising companies.   There’s a lot of outfits who might conceivably want in-house AR skills.  Then when people set up a specialty AR shop,  they  peel off the list of clients.  I don’t know.  Those old days  of Silicon Valley venture capital seem like a lost world.

Tish Shute: Yes.  I, again, didn’t see anything really of the business tracks and production tracks.  Did you get back and forth between the tracks?

Bruce Sterling:  I went to the Hollywood tracks.  I mean, to the extent that I could.

Is Hollywood stirring? Who’s going to have the first breakout AR property?

Tish Shute: So what did you see from…Is Hollywood stirring?  Is it waking up?  I mean I know Kent Demaine, Oooii,  and Brad Foxhoven, Ogmento, spoke about the Hollywood AR scene.

Bruce Sterling:  There were guys there from LA who were sort of saying, look…they are aware of us, but they just want AR to promote their properties to some particular niche.  They realize that AR is potentially a mass medium and that you could do some real AR entertainment.  So they were batting around some ideas as to where that might happen.  Like, could it come out of a console gaming scene?  Who’s going to have the first breakout AR property?  A popular hit AR property, as opposed to like a neat way to sell shoes, or whatever.   Really, anybody’s guess is as good as theirs or mine.  But at least they were actively guessing.

Tish Shute: I know the breaking the fourth wall discussion has been going on for a while and now the question is, whether AR is going to take down the fourth wall and bring interactive storytelling into the mainstream.  Did you hear any of that?

Bruce Sterling:  Well, I always shy away from discussions of that kind because I don’t think there’s any “final thing.”  Practically everything that AR is involved in right now is  a transitional technology. Also, because I am a storyteller, I get alarmed whenever people in technology start saying, “Oh well, it’s all about telling stories.”  Because obviously it isn’t.

People can tell stories perfectly well orally, and absolutely nobody does that.  AR is not at all about telling stories.  It’s about a great many other things, such as user bases, niche audiences,  media saturation, urban informatics,  convergence culture, and the language of digital media.   I could list these factors until the world looks level. It’s really becoming pretty chaotic.  As I was saying in my speech, AR companies are media startups who almost never use the old-fashioned word “media.”

Tish Shute: Oh, that’s interesting.  Yes.  So why do you think that has happened that way?

Bruce Sterling:  Well, it’s because they are trying to do a different thing than media does.  I mean, they are trying to “augment reality.”  They don’t want you to know that you are using a medium.  They don’t want you to realize that you’re watching computer animation overlaid on some video acquisition stream.  That would defeat the whole point of AR.  It’s entirely different from an analog medium like television, where you turn on the television and there’s a constant stream of station identification alerts.   That’s like: “Don’t touch that dial!  You’re on channel 13!  Stay with us!”  Then it’s like, “And now a few words from our friendly sponsors!”  That medium was engineered to keep your eyeballs locked to a single stream that they’re feeding you.

In AR, it’s much more participative, more geolocative.  I’m not particularly interested in station-identification branding from my AR provider. What I really want to see is the interactivity of the augments they’re bringing to me.  It’s like  FlickR, the photo sharing site. You don’t have any TV-style splash page for FlickR.  “Hi! We’re FlickR! FlickR, bringing your photos to you!” No, FlickR is all about “you, you, you,” your photos, your tags, your friends, your activity around you.   It’s immediately trying to be very participative.

Tish Shute: Will Wright got to that point, didn’t he. He was trying to move us into an idea of blended reality. That the game is about the world, not about the dragons or the overlays per se.

Bruce Sterling: Right. I think that’s true. But see, the world isn’t a medium. A medium is something like this interview,  where I’m connecting to you and there’s a video Skype channel between us.  Whereas AR is more about spatial 3-D,  about 3-dimensional impositions.  Pieces of media: sound, vision, information visualization, tags, floating tags, air tags, icons, arrows, warning signs, warning sounds, tactility, whatever, being brought into the environment around us.

That’s why it’s properly called “augmented reality” instead of just augmented media.   If you call your work “augmented media,” you’re really in trouble. Because if it’s all about augmenting somebody else’s media, why doesn’t that medium just buy you, and augment their own selves?    If you think that way, instead of augmenting the world, you’ll just be a modest little plug-in for old-school media.

The World as the Platform

Blaise Aguera y Arcas, Microsoft, Santa Clara, are2010, photo from Chris Cameron’s Flickr stream

Tish Shute: Yes, which is why Blaise so generously gave the technical underpinning  for augmenting reality in his tech talk – about the trellis and the grapes,  he really explained how the world can become a platform for augmented reality.

Bruce Sterling: I wish I could have seen that. I did not see Blaise’s speech.

Tish Shute: We’re going to put the videos up in better quality.  People in the front row have put it up on the web already.  He really went into some of the challenges of mapping for augmented reality.

Bruce Sterling: His visual-mapping technique is important.  Registration is super important for AR.

Tish Shute: I think it was a really generous talk actually because he went step by step on how we will do this.

Bruce Sterling: I rather imagine that Microsoft has patented those steps.

Tish Shute: Oh, yes, I guess so!

Bruce Sterling: I could be wrong. Maybe they’ll open-source it. You never know.

Tish Shute: You never know. Because the world as a platform isn’t something one company can own, or go it on their own to exploit.

Bruce Sterling: I expect there to be a thorny path, but sometimes I’m surprised. Sometimes people really do try to fertilize the tech field in the hope of getting a good corn crop before they start fighting.

Tish Shute: We’ll I keep hearing that we may even see the unlikely marriage of Apple and Microsoft  – maybe wishful thinking, but there are motivations beyond AR for this unlikely match, and certainly between them these titans have what it takes to realize the grand visions of AR ? [laughs] But who knows…

Bruce Sterling: Well, yeah, it depends on where the thing catches fire.

Tish Shute: Yes. You mean whether AR catches fire in the form of  AR and mapping..

Bruce Sterling: It’s hard to say, but I’m convinced now that there’s more going on than I once thought. I thought that Bruno Uzzan made a very good speech for his company when he talked about how he worked on AR for eleven years.  Eleven years is no flash in the pan.   He has his long list of clients and successful applications. I thought he was right in his impatience with the press for not catching on. It’s gone on for quite awhile. The mere fact that you’re not aware of it, doesn’t mean it doesn’t exist.

The Illusive AR eyewear

My are2010co-chair, Ori Inbar, CEO and co-founder of the hottest new AR game development start-up, Ogmento, donning his goggles to open are2010 – picture from Chris Cameron’s Flickr stream

Tish Shute: Yes. So, the other theme you brought up in your opening keynote and I would be interested to know if anything you saw at are2010 changed your view is the illusive AR eyewear, and  if we actually got AR Goggles that worked they would bring AR’s gothic sister, VR, back from the grave right? [laughs]

Bruce Sterling: Right.

Tish Shute: It took quite a lot of work, but we pulled together a six-company HMD panel, right?

Bruce Sterling: Yeah. I was impressed to see so many of them there.  And I was chagrined to see how prototype-like all their gadgets were. But that doesn’t surprise me, because if any of those head-mounts were remotely working, they would be hyped out the wazoo. Everybody’s been waiting for them and hoping for the best. They’re obviously not ready for prime time. [laughs] Maybe in certain limited applications. Like maybe a diving mask. [laughs]

Tish Shute: No, I think what was nice though they got inspired and they all got together on the last day. I saw them having a meeting about standards. They got inspired to actually work together.

Bruce Sterling: Yeah, well, unless they’re going to invent mechanical eyeballs that those machines can fit onto, it’s going to be tough. OK, I’m a skeptic, but I’m prepared to be surprised. I’m also a skeptic in Artificial Intelligence, but as soon as they bring me an AI that can write a decent novel, I’m going to get it and review that book. [laughs]

Tish Shute: It’s interesting. Re AI, I’m totally in agreement with you. In terms of the way computers turned out, it wasn’t AI per se that they turned out to be good for, not in the way everyone had dreamed of it, rather it was the harvesting of human intelligence that turned out to be the big thing. But what is interesting is that despite all of that, AI or machine learning, as it is now called, permeates our whole society now from the stock market to how many businesses make many of their decisions.

Bruce Sterling: Well, there’s a lot of so-called collective intelligence.  But Marvin Minsky-style hard AI, no way. Alan Turing-style AI, forget about that.

Tish Shute: Yeah. So, that’s an interesting comparison with the HMDs.

Bruce Sterling: People stretch the definitions.  It’s like, well, my car engine is Artificial Intelligence. Yeah, so is your wall transistor. No, I don’t really think so.

And AR is a similarly big tent. I mean, Uzzan had to admit that he had denied that AR was AR, unless it was using his favorite technology. And he felt embarrassed to be rubbing shoulders with people who put AR into cell phones. And I can understand his feeling there, because, gee whiz, that’s certainly not what AR pioneers had in mind. But he had to admit he’d become more ecumenical about it. Obviously, they’re  there and doing business like gangbusters. You can’t very well ignore success, right?

I had a similar feeling about the goggles. Obviously, the goggles would be great, should they work. But if they did work, I rather think virtual reality would come very strongly to the fore.  You’d see people doing all kinds of elaborate immersive-style stuff.   A truly immersive technology doesn’t need to “augment” much of anything.

Tish Shute: Yeah, you’re right.

Social Augmented Experiences

Bruce Sterling: I think many of the most interesting AI aspects are not personal in the way goggles are.  They’re not about guys walking around with personal tech. They’re about big, communal, social-media experiences, like stage shows, and urban informatics, things where large numbers of people can interact with the same augmented reality. The projection mapping, which I go on and on about. Augmented public spectacles.

Tish Shute: Yeah, projection’s our best example of a social augmented experience right now because we are yet to have an easy way to do networked social augmented experiences easily – but that is of course the thrust of my interest in ARWave [see the slides for my presentation, AR Wave:  Federation, Game Dynamics, Semantic Search, Mobile Social Communications here].

Bruce Sterling: I think of Edison’s early days, when he wanted to sell movies to people for a nickel a clip.  You had to bend over and put your eyes on this visor and turn this crank. That coin-op device was easy for Edison to monetize, as opposed to getting a bunch of people to sit in theater seats. But people laugh at movies when they’re together in the seats.   Cinema is a more social, involving experience in a crowd situation.

Tish Shute: But it started with them, didn’t it, Hollywood – the movie biz? Basically Nickelodeons, right?

Bruce Sterling: That’s right. They were Nickelodeons. They were a lot like the goggles because they isolated the user.

Tish Shute: Yeah, that’s a really important point that the goggles are not Nirvana because of this question of whether they actually detract from the social augmented experience and blended realities, by drawing us into VR experiences?

Bruce Sterling: I’m tempted to claim that they’re more a VR technology than an AR technology.

Tish Shute: That’s a very interesting point because…

[thunder]

Tish Shute: Wow! What was that?

Bruce Sterling: Thunder storm.

Tish Shute: Oh, my God, how very Gothic! [laughs]

Bruce Sterling: It can get pretty loud up here in the mountains.

Tish Shute: Oh, you live in the mountains, better still!

Bruce Sterling:  Torino is in the foothills. This is Piemonte. So the Apennines are over there. The Alps are over here. We do get some rather spectacularly unstable weather.

Tish Shute: It sounded like a bomb to my NYC ears. [laughs]

Bruce Sterling: Yeah, it didn’t hit the building, but it was maybe half a kilometer away. I saw the flash.

Tish Shute: Oh, you did?   Well, I hope you don’t lose your power midstream here.    I was really happy to hear of that connection between Rudy Rucker and Layar  [Rudy was touched when Maarten Lens-FizgGerald from Layar said that he met the Layar co-founder at a Rudy Rucker lecture].

Bruce Sterling: That was very fun, yes.

Tish Shute: Wasn’t that wonderful? What was that experience like going around the conference with Rudy?

Bruce Sterling: Well, you know, Rudy’s very into graphics. He’s a mathematician, so he understands the underpinnings of this stuff. But he’s a skeptic. He thinks they’re kid toys. He’s not a gamer. He’s a good old-fashioned computer-science hacker. So he wanted to tell me all about his new eighth-order, fifth-dimensional fractals. He showed me a great many of them. They’re psychedelic. Rudy’s fractals are considerably trippier than most apps that help you find a barber or a train station. [laughs] Rudy really is a visionary. He’s into some very weird stuff.

Gamer Guys at are2010

Brad Foxhoven, Chief Marketing Officer, Co-Founder, Ogmento at are2010

Tish Shute: At are2010 there was a lot of discussion about how game dynamics and AR are going to intersect, right? Anything that you saw of interest there?

Bruce Sterling: Well, obviously, there are gamer guys there. Ori’s a gamer. The gamer guys are getting some money. The big buzz right now in gaming is, of course, social gaming.  Farmville has kicked everybody’s ass because it’s not even a game and yet it has more users than the entire gaming industry.

Tish Shute: I know, right! [laughs]

Bruce Sterling: Obviously that’s kind of humiliating. For a long time, I’ve seen people trying to do giant multiuser games on cell phones. It’s difficult to do because the interface on cell phones is crap, right? People aren’t going to run around responding to SMSs.

I can imagine people running around with little Wii-style bats that have audio and visuals on them. It makes a very large native AR game seem more plausible.

Tish Shute: Yes. that would be cool!

Bruce Sterling: Again, it’s not very gamelike to use those little fiduciary markers.

Tish Shute: No.

Bruce Sterling: Moving little cardboard chips, around like with card games…. It would be pretty easy to set up a little AR chess game.  Star Trek style hologram chess pieces,  and so forth. But it’s just cumbersome.

Tish Shute: And also, from what we’ve seen from things like Foursquare, the proximity based social gaming doesn’t have to offer very much [a crown badge, a mayorship] to get some mind share.. the social is the primary game dynamic…

Bruce Sterling:  I’ve seen a lot of different philosophies of gaming over the years. Who’s to say that Second Life doesn’t have the best idea? They built a little scene and then slammed their gate shut behind them.  But at least they’ve got a really nicely-paying little cult stuck in there. It’s different. And it’s manageable and it’s really theirs, theirs, theirs.  They don’t have to call in outside experts to try and run the monster.   They haven’t blown it up to the scale of Yahoo! where they’ve lost control of the enterprise, and gone into a tailspin of management overhead. Second Life has a very intense, almost a cultish atmosphere among the player-slash-developers.

Tish Shute: One thing that helped them was the thing they were always criticized, that the barrier of entry was so high. But once they got people they never left, right?

Bruce Sterling:  That’s not a bug, that’s a feature.

Tish Shute: One of the best features!

Bruce Sterling: Yeah, it’s like being in Mensa. Why don’t you lower your barriers to entry and get in some interesting stupid people?

Tish Shute: [laughs]

Bruce Sterling: In Mensa, we’d rather sit here making puns about neutrinos and fourth-order quadratic equations. [laughs] OK, that’s a business model, if that’s what you want.

The Man With the X-Ray Eyes!

Jesse Schell’s during his keynote, “Seeing,” at are2010, picture from Chris Cameron’s Flickr stream

Tish Shute: Ok!  Now to unpack the man with the x-ray eyes idea, Jesse Schell’s keynote theme.  This is a root metaphor for AR – making the invisible visible, seeing through walls. To me. I think you kind of wrote the book on this because all my ideas on what radical transparency might be come from you – your idea of Amazon.org is key to how I understand this..

Bruce Sterling: Oh, really? That’s funny.   I was touched that Jesse brought up that famous Corman film, because I was a judge in a fantasy film conference in Trieste earlier this year.  And Roger Corman was there.  He was the guest of honor.   “X: the Man with the X-ray Eyes” was one of the films shown during the conference, and I saw it.  I even had dinner with Roger Corman.  I had never met him before, so that was quite amusing.  The difficulty with a film of that kind is that what we science fiction writers call a “House of Cards Ending.”  In that story structure,  you ramp the thing up until the protagonist sees God, and then he has to be destroyed by the falling pillars of the temple.  That’s a classic science fiction structure:  like Frankenstein.  For the sake of the drama, Corman evades the issue of what’s really going on. For instance, let’s just suppose “the Man with the X-ray eyes” is not in fact a psychopath.  Let’s say he gets a grant from the Department of Health and Human Services, and he acts like a real scientist, not a stock B-movie “mad scientist.” So he has, like, backup guys, and some placebos, and a large group of people to test it on, trusted colleagues, and so forth.  You wouldn’t get any of that movie’s wild activity out of that.  What you would get is like a 5% improvement to people’s vision.

Then, in a year, there would be a 10% improvement in people’s vision.  There would be a  classic industrial story.  A rising star, you know, a cash cow.   Real tech isn’t done by a single guy as a divine curse.  It’s created by classic  tech startup culture.  So a runaway technology really behaves in the way that personal computers do.

Tish Shute: The things that get me all Utopian and happy about this are the ideas like those you first outlined with the notion of Amazon.org.

Bruce Sterling:  It would be easy to do an entirely different kind of film than “Man with the X-ray Eyes.”  Something much less B movie,  much less pat.  I mean, at the end of the film,  he destroys his own hardware and blinds himself.  Why?  For what rational reason would he do that?  Why doesn’t anybody else know the big secret of what he’s doing?  Why aren’t there Koreans doing it?  Why aren’t there Austrians doing it?  Why aren’t there Italians doing it?  Why?  AR doesn’t behave like that.  It’s not one lone guy with magic eye drops.  It’s entire teams of people that have been working on stuff for 17 years.  They all approach it in different ways.

Now, they are going to get scandals in AR.  I can guarantee you that.  They are going to get into  hot water eventually.  At least some people will surely come out and accuse them of being Roger Corman B movie monsters.  But unless they accidentally discover atomic fission or destroy the Gulf of Mexico with an oil spill [laughs], I don’t think they’re going to be particularly badly off!   The trouble I imagine  for AR people is very typical new media trouble.  It’s like movies being accused of corrupting our morals, or comic books being accused of leading to violence, or Google being accused of making us stupid and warping our brains.

I’m not an alarmist in that sense, but at least I’m concerned about real threats.  Roger Corman is a B-movie director who’s trying to sew up his lost plot ends by destroying his hero and his hardware. That’s not very plausible. It’s a nice science fiction movie device, but technology isn’t a movie.

Tish Shute: Yes. Well, the other thing that you always remind us of with AR is not to be saying it’s going to be this glorious moment when it’s no longer gimmickey, no longer pop culture. You always emphasize that’s actually part of what’s good about it.

Bruce Sterling: It’s not an accident that practically everybody in that audience knew about Roger Corman.  Nobody looked surprised; not the Austrians, not the Koreans. They were all like: “Oh, yes! Roger Corman!  Love him!”

Tish Shute: There were so many Rudy Rucker fans. Were you watching Twitter? People like Eric Gradman were succumbing to fanboyz moments..

Bruce Sterling: “Yeah. Rudy Rucker, he’s the best.”

“Rudy Rucker gripping an Augmented Reality shoe” from brucesflickr

Tish Shute: [laughs]  I noticed you inspired him to join Twitter..

Bruce Sterling: Well, I’ve got 8,000 followers and, obviously, a lot of them are Rudy’s fans.  Of course he’s going to be gang-rushed on Twitter. That’s not really any more surprising than two motorcycle stunt guys at the same attraction. And I’m a big fan of his Rudy’s blog.   He’s always got interesting things to say.

Tish Shute: Yes. AR does seem to bring out some of the coolest smartest people!  This morning I had breakfast with Joshua Kauffman in Central Park.  He is an advisor and entrepreneur working on design in the public sphere.  I was feeling rather brain dead and jet lagged.  I told Joshua I was wondering how to get the cottonwool out of my brains for this interview and he suggested,  the All Souls College one-word question interview!  Have you ever heard of that? – although apparently they recently scrapped it.

Bruce Sterling: Well, I’ve heard of All Souls College there in Oxford. What was their interview question?

Tish Shute: They used to use only one word, so they would only give you one word. It’s not a question. Basically, they throw out the word and then you had to spin off from there.

Bruce Sterling: You’re supposed to free-associate on a single word?

Tish Shute: I guess so. I hadn’t heard about it, but Joshua suggested it.

Bruce Sterling:  Well, it’s possible..

Tish Shute: Joshua came up with some good words..

Bruce Sterling: Right.

Tish Shute: We were talking about these proximity-based social work networks like Foursquare and Gowalla and how they may influence the emergence of social augmented experiences.

So Joshua’s suggestion for the first word was “territorialization” e.g. how do these new mobile social experiences like Foursquare,  and the observation that actually rather than breaking down territorialization, which would be a good thing, tend to support territorialization…but perhaps new forms of territorialization?

Bruce Sterling: Yeah, they’re re-intensifying it in a very odd, electronic fashion.

Tish Shute: Yes.

Bruce Sterling: I have noticed that.  It’s not true of stuff like projection mapping or the webcam fiduciary display stuff. But with the handheld stuff, and especially the urban informatic stuff, it really can’t help but take on a local flavor. Layar is like “Augmented Dutch Reality.”

And TonchiDot really is “Augmented Japanese Reality.” It’s hard to imagine a Layar interface going gangbusters at Tokyo.  Whereas the TonchiDot interface, which is very clearly influenced by Anime and cartoon graphics…. Maybe it could find some niche of hipsters in Amsterdam hash bars…

…to be continued in Part 2

The Augmented Reality Event: Bruce Sterling’s keynote from Ori Inbar on Vimeo.

Bruce Sterling’s keynote (aka the prophet of the augmented reality industry) set the bar high on the opening day with a keynote address reminding us all of how awesome it is to be the “world’s first pure play experience designers,”  but also not to forget “whose reality needs to be augmented most,” and “to cut ourselves a space of our own.”

“It’s 9 am in the augmented reality industry…without vision people perish…it’s your chance to build a big pie before you start slicing it up…it’s time for you to get dressed…good luck to you, I’ll be watching you”

Wow! Augmented Reality Event was hotness.  I know, as one of the chairs, it is not like I am a neutral observer.  But, if you were there, and you didn’t pick up on the “warm, affirmative feeling,” as Bruce put it, and the optimism, fun, and cool uber geekery, of are2010, I guess I will have to hear from you in the comments ‘cos the feedback has been incredibly upbeat, so far.

Anyway,  I am still basking in that rare glow of a geekgasm at 9am – the augmented reality industry embracing lovers whose dreams are in still in full flight – no burst bubbles, betrayals, train wrecks, or tragedies in sight yet?  Although Roger Cormanesque horrors were threaded through  Jesse ‘the man with the x-ray specs” Schell’s  closing keynote, “Seeing.”  So keeping watching Ori’s Vimeo stream for Jesse’s keynote  -  it will be up soon and it’s quite a ride!

I leave for England tomorrow to celebrate my mother’s eightieth birthday, so this will be a brief post for now.  But I am looking forward to posting soon a long interview with Bruce Sterling on his experience of are2010.

Bruce Sterling hung out with with Rudy Rucker (see Rudy’s post and pics on are2010 here), attracting fans wherever they went, visiting the expo, going to sessions,  talking generously with all – their table always crowded.  Bruce said he hadn’t had so much fun in a while.  And, apparently, Rudy was touched when Maarten Lens-FizgGerald from Layar said that he met the Layar co-founder at a Rudy Rucker lecture. Awesome!

As Ori says here:

“are 2010 is over. It was a blast.  Many thanks to 400 AR enthusiasts who joined us for 2 days of AR goodness.

Special thanks to our 90 speakers from 40 augmented reality companies, our exhibitors, sponsors, and above all – for Qualcomm.”

People’s slides will be up on the are2010 web site soon. My presentation, AR Wave: Federation, Game Dynamics, and Mobile Social Communications, is already up on slideshare.

You can catch up on some of the highlights of are2010 in Rouli’s linkfest, and check out these pointers he gave to following the event for more.

• Chris Grayson’s twitter account
• Patched Reality’s Patrick O’Shaughnessey and the augmented citizen Dan Romescu are also doing their share on twitter.
• Sophia Parafina ( @spara ) live blogs the event using Google Wave.

I will have a mega post up soon.  But for now here are a few photos that will give you a taste of some of the #are2010 magic.

Bruce gets a t-shirt from tonchidot CEO, Iguchi Takahito ( @iguchi ), after checking out Helen Papagiannis’ AR art

If  an event with Bruce Sterling, Rudy Rucker, Jesse Schell and Blaise Aguera y Arcas there wasn’t already enough awesomeness, check out how Sophia Parafina, Locatively, and Patrick O’Shaughnessy of Patched reality got to hang out with Will Wright – hacking the Parrot AR Drone.  And then, when a dead power supply stopped that adventure in its tracks, Will played some of Patrick’s AR games and gave him feedback.  OMG!  Will’s only the most important game designer the whole world.  There is already some videos shot from the front row up of Jesse Schell’s, Blaise Aguera y Arcas’ and Will Wright’s keynotes up, but the hi res versions are coming soon!

Where 2.0 is going to be epic this year (see my interview with Brady Forrest here), and it is so exciting to be part of it.  Location technologies and augmented reality are annointed rulers now.  Time Magazine recognized augmented reality as one of its 10 Tech Trends for 2010 (for more see ReadWriteWeb).

The photo above is by Jeremy and Lisa Hight.  Jeremy Hight is an information designer, theorist and artist working in Augmented Reality and Locative Media.   His essay “Narrative Archaeology” was named one of the 4 primary texts in Locative Media.

Jeremy Hight will be part of our panel: The Next Wave of AR: Exploring Social Augmented Experiences, with Anselm Hook, Joe Lamantia, Sophia Parafina and myself. We will debut the video of the ARWave project demo that brings together augmented reality, geolocation, and wave federation (more details later in this post).  And, Jeremy will bring to our presentation some augmentations on his recent brilliant work and paper, “Writing Within the Map.”

Greg J. Smith’s points out in his in depth look at Jeremy’s work that it, “dovetails with some of the main points in Bruce Sterling’s recent atemporality keynote at Transmediale” – fortunately there is a transcription of Bruce’s keynote here.  What is so awesome about this dovetailing is that you can get a feel for the fun part of living in an, “atemporal network culture.”  And, if you want to really understand just how much locative media and augmented reality have changed us, you  might want to dig into these texts.

Bruce Sterling and Jeremy Hight, and members of the ARWave team, and a superb cast of augmented reality movers and shakers – including Will Wright and Jesse Schell, will be speaking at Augmented Reality Event in Santa Clara, June 2nd and 3rd.

But, this week, the AR community’s attention will be on the events at Where 2.0.   The keynote speakers will be streamed live, so if you are not fortunate enough to be there, tune in!

The Next Wave of AR: Exploring Social Augmented Experiences

On our panel, Jeremy Hight, Anselm Hook, Sophia Parafina, Joe Lamantia and I will cover some of the key social, cultural, technical and interactional questions for exploring social augmented experiences. There will be five lightning presentations, and an opportunity for questions from the audience, and a world premier of the ARWave demo!

1) “Augmenting the map as interface: AR and Locative Narratives” - Jeremy Hight

*Map augmentation of the historic route 66 can house an essay contest and publication globally but as embedded within that map augmentation instead of books or even web sites.

* A place on a map can be a graphic index and database to save and collect
the writing of that place with a graphic or textual search index.

*One can pop immersive visualizations of abandoned or lost buildings from map location in shared software and collectively augment (imagine channels within the lost core of detroit where one is memories and accounts tagged within parts in the immersive visualization while another is of poems and stories written by people moved by the place and its semiotics and story).

*The news stand is to be the map.

*New forms of literature will be born of mapping, spaces,augmentation and
new tools

The concept drawings below (click to enlarge) are a collaboration between Jeremy Hight and Paul Wehby, Senior Designer at LA County Museum of Art.

2) Anselm Hook will look at, “10 reasons why AR isn’t a flash in the pan,” and how, “AR can help us see the world we would like to have exist.”

Anselm notes, “So much of what we do is so fickle and I’m looking for ways to connect digital media work to deep values.”

3) Sophia Parafina will present on, “Social AR and Crisis Response”

“Augmented reality as a multi-party conversation.  Rather than being passive viewers of AR with a limited ability to  checkin to places and make annotations, current devices can broadcast sensor information that can be fused into an interactive stream. AR users can send and receive information, location, and sensor data from their mobile device.  The streams can be federated into a unique AR view composed by the user.

Entertainment and gaming are obvious applications, but it can also be applied to crisis situations such as the search and rescue operations in Haiti.  Efforts such as Mission 4636, the SMS translation service, could benefit from AR views.  The collaboration among the Mission 4636 volunteers was the key element  in their success for providing location and rapid translation to responders on the ground.

With an AR view, responders can send back their sensor information from their mobiles to provide contextual information to remote volunteers.  This extends the conversation between remote volunteers and on the ground responders and fosters collaboration which was a key element for the success of Mission 4636″

4) Joe Lamantia, an experience design and strategy consultant helping to define the interaction framework and scenarios behind ARWave, will discuss, “Design Principles For Social Augmented Experiences:”

“With the exotic mixed realities envisioned by futurists and science fiction writers seemingly around the corner, it is time to move beyond questions of technical feasibility to consider the value and impact of turning reality inside out for everyday social settings and experiences. Thanks to the inherently social nature of augmented reality, we can be sure the value and impact of many augmented experiences depends in large part on how effectively they integrate with the social dimensions of real-world settings, in real time.”

Joe will share, “eight guiding principles for designing experiences that engage naturally with the social dimension, and increase the value of augmented experiences.”

5) “ARWave – A demo and state of play,” from Tish Shute

I will have the awesome privilege, on our Where 2.0 panel, of showcasing ARWave.   We will  premier the ARWave demo which shows how ARWave has accomplished the basics of geolocating data on Wave Federation Protocol (and real time collaboration on this geolocated data).  If you’re interested in the ARWave project join the Mailing list, FAQ are here, and have a peek at the current state of development at Google Code, and the specification for an AR Blip.  We also have Waves for the project hosted on Google Wave.  You can join the general discussion here, and the technical side here.

The picture below is a screen shot from the demo video produced by core AR Wave developer and concept designer, Thomas Wrobel.

Click on the image to enlarge, and note: “The pink thing is from Dennou Coil. Its an anti-virus program (that literally chase’s down bugs and glitches and removes them).”

ARWave

In ARWave, stories or art are tied to place. And as Jeremy Hight writes:

“The possibility exists to take a part of an area and overlay a dystopia, a utopia, multiples of each of these, or even recreations of previous incarnations in the past. Writing and publication thus cannot only be of place, and form(s), but of selected augmentations of icons, streets, buildings and related texts on top of the map. These spaces can be built in real time and can be turned on and off as channels of augmentation that over time illustrate many faces of place in its present, past, possible futures,etc. with texts within these alternate spaces as commentary, as fused aesthetic analysis, or simply creative writing relevant to these charged and hybrid spaces.”

As Thomas notes, Jeremy Hight’s,  “idea of channels ties into the concept of waves = a layer, and people can have many layers on at once.”

This is different from the Layar concept of a layer or rather “layar.”

“We are not talking about layers in the classical map layer way of thinking, where you have a layer of all restaurants or a layer of all mountain peaks, etc.,” notes ARWave developer Markus Strickler.

Currently all geo location apps like Layar have to use their own servers, so users have to use different clients with different log ins to see data from different sources.  But because ARWave uses federation, we don’t depend on centralized infrastructure where the client of one company can only connect to the server of that company.  This opens up many exciting new possibilities for how people can decide to view and publish geolocated data.

With AR Wave, via one login, people can access the whole distributed network of servers (see diagrams below), and any content will be accessible to them. ARWave will make it easy for individuals, not just developers, to layer their environment – allowing the creation of augmented reality content to be as simple as contributing to a Wave.

“ARWave will enable individuals to publish easily to everyone….or just a few people,” Thomas notes:

“To ‘publish’ is also self publication and distribution in communities or like minded groups without the hard read of publication or rejection.” = publishing on a Wave. No one approves it, anyone can publish to communities, or their friends and family. Or even just personal publishing it for their own reference.”

But ARWave does not compete with existing AR Browsers.   On the contrary, AR browsers like Layar, Wikitude and others, could implement ARWave and use it to enhance their applications.

“Layar has a killer browser already,  ARWave would add social features. They can keep their “walled garden” of data and still join the federation of open data too ” (Thomas Wrobel)

Yup, that is the cool part of federation – you can have your cake and eat it too!

Sophia Parafina and I will be organizing a discussion session on ARWave and Federation at WhereCamp, right after Where 2.0, April 3rd and 4th, and Dan Peterson who is in leading the federation effort for Google Wave will join us.

The diagrams below illustrate how ARWave and federation can revolutionize the way we share our augmented realities.

Real Time Social Augmented Experiences

Another key aspect of ARWave is it’s near to real time update capabilities.  As Jeff Pulver pointed out in, “SXSW 2010: The days twitter became less relevant:”

“At South By Southwest 2010 (SXSW), a strange thing happened on the way to Austin. A community of twitter faithful shifted from sharing everything about everything on only twitter (and maybe Facebook) and changed their habits to rely on learning about what was happening and where things were happening by using foursquare and Gowalla instead. I’m sure there were other products and platforms being used including Loopt and GySPii but foursquare and Gowalla were the dominant platforms.”

Later Jeff wrote:

“There were times where I could feel the ebbs and the flows of the people move as different people checked into various locations. While most of this was felt locally in the place I was in, it also became apparent on the platforms when hundreds of people would rush to check in to a location. There were also times when it felt like I was chasing ghosts; These were the times I would go to a spot because a friend had checked into that spot only to discover they were no longer there.”

ARWave’s realtime collaborative capabilities are going to introduce some fascinating dynamics to “chasing ghosts,” as the ARWave framework gets integrated into services like foursquare – a project we have already begun to look at.

Augmented Reality Search

As I mention in my previous post, ARWave presents some fascinating possibilities for AR Search.  For example, one might do advanced searching within waves using SPARQL, which could then display in the form of a personal blip in your viewpoint (which in turn could be shared with others).  Linked data will be massively important in filtering and delivering useful info for augmented views (see my conversation with Will Wright about the problem of augmented reality overriding our very smart instincts and not being useless or worse as a result).

Anselm Hook, who I interviewed in depth recently about, Visual Search, Augmented Reality and a Social Commons for the Physical World Platform: Interview with Anselm Hook, has some very interesting thoughts on real time stuff, trading brokerages, and  the view within a single city block, which he elaborated on in the second half to this interview which is upcoming on Ugotrade soon!

The ARWave Developers

There are three people who unfortunately can’t join us at Where 2.0 –  the costs of travelling from Europe being an obstacle.  But as they have been developing the code for ARWave that will rock our augmented world, I asked them, in a Wave conversation, to give me a few comments about their interest in working on ARWave, and a pic and a short bio.   Also I should mention the work of the PyGoWave team whose incredibly fast work creating PyGoWave has given ARWave a rocket launch pad.  Also many thanks to the Wave community, see the Wave Federation Protocol documentation, Google’s Wave Server, RubyOnSails (Ruby On Rails based Wave server).

Davide Carnovale @need2revolt

“Imho, the coolest geolocated related thing is that we’re making a world where the info does not necessarily comes from an explicit search from the user, but comes also from the actual locaton you’re in. For instance, you can have special offers in stores like fourquare does, or your friends can leave geolocated notes for you that are triggered when you walk by.  We can have games based on the treasure hunt schema requiring you to actually go to specific location.

Other than this I can think about self-guided tours of the city, maybe user generated too, or for museums.

Naturally these are long term goals with some rl use cases.

As for my bio, there isn’t much to say… I got a first level degree in computer science and I’m taking the second (and last) level. I’ve developed with mobile agents, osgart/artoolkit, brain computer interfaces, linux kernel and that’s pretty much all…”

Thomas Wrobel

“If you are looking for specific advantages of using Wave I’d say;

* Federated – Letting creators tap into bigger userbase. Each new app or data layer will add to the “incentive” for users to join in. Google had some good stats a few months back as to how much a simple login screen can put people off using stuff.  By breaking that barrier it should make AR userbase’s grow.

* It deals with user accounts, permissions, and real-time updating without creators needing to make a new server standard themselves. It lowers barriers to development.

* As the clients, servers, and data can be made separately by different parties, its easier for developers to concentrate on just providing what they want. You want to just make content? No problem! You dont need to worry about doing anything else but that. It would become as easy as making a webpage (or easier!).

* Bots will allow the development of interactive AR games very easily. Just like modern version of IRC bots, the infrastructure does the heavy lifting, and interesting things can be done with just simple scripting.

* The idea is anyone will be able to make a layer onto the world, and people can mix, match and share their layers as they wish. Its not just the data that becomes interesting to see augmenting our world, but the combinations of data! For example, perhaps you could see the profits generated by different companies above their buildings, but also see how environmentally friendly they are at the same time. Or maybe see pollution levels against health-statistics.  Seeing combinations of geolocated data from different sources at the same time has many interesting possibilities both for scientific as well as casual (game/ map/ chat) use.

hmz..I could go on forever listing stuff here really…..

I guess if we are supposed to be forming a roadmap of significant/interesting things for ARWave?

* Example clients letting people make their own layers (waves) and add points to them.

* Letting people log in to different servers

* Servers federated together. (not our responsibility, but essential part of the roadmap).

* Anyone logged into any server can see data from anyone else that’s shared with them, regardless of where they are logged into

* 3D support, demonstrating various sorts of geolocated data.?

* Use of bots for example games?
—-
My Bio’s quite simple.
Studied 3D Animation in Portsmouth, UK.
Moved to the Netherlands, have since been working in creating ARG games, in the last year founded Lostagain (Lostagain.nl).”

Markus Strickler @kusako

“I think the main point behind ARWave is to go beyond simply displaying existing placemarks on top of a live camera view, towards a highly personalized, augmented world where everybody can edit and share localized information collaboratively and in real time. Wave provides the means to do this through its model of persistent real time conversations and adds even more by providing a way for personal agents (robots) to participate in these conversations.

As for my Bio: I’ve been developing Web applications for the last 15 years, hold a degree in Image Sciences and am currently working as a Java developer in Cologne, Germany.”

The video above, The Imawik commercial, is a collaboration between In The Can Productions and Paige Saez for Makerlab

“The Imawik (ImageWiki) is a visual search tool for mobile devices. It allows for the ability to turn images into physical hyperlinks, conflating visual culture with a community-editable universal namespace for images.”

Paige Saez is an artist, designer and researcher.  In 2007 she founded Makerlab with Anselm Hook, an arts and technology incubator focused on civic and environmental projects.

Paige and Anselm (see my interview with Anselm Hook here, Visual Search, Augmented Reality and a Social Commons for the Physical World Platform: Interview with Anselm Hook) have been asking a very important question:

“Who Will Own Our Augmented Future?”

But most importantly, they have been actually developing applications (again see my interview with Anselm for more background on this), to allow people to play with, hack and explore and create with the physical world platform, and to imagine new possibilities for physical hyperlinking and augmented realities.  This is pretty important stuff, and kudos to Paige and Anselm for beginning this work before the big players – Google Goggles, Point and Find, and SnapTell came hurtling into the field of visual search and physical hyperlinking  – see this demonstration of translation and optical character recognition in Google Goggle’s.  Also check out Jamey Graham’s (Ricoh Research) Ignite presentation at Tools of Change, 2010 – Visual Search: Connecting Newspapers, Magazines and Books to Digital Information without Barcodes, for more see ricohinnovations.com/betalabs/visualsearch.

We are only just beginning to get a glimpse of how contested the social commons of the physical world platform is going to be – see the Yelp controversy.

As Paige points out:

“The lens that you are actually looking through was as important as what you were looking at. And democratizing that lens became the most important thing that we could possibly do.”

I am in total agreement.  One reason I have so much enthusiasm for ARWave (note: if you are interested in following the developer conversations there are several public Waves) is I see this open framework playing an important role in the democratization of our augmented views, by creating an open, distributed, and universally accessible platform for augmented reality that will allow the creation of augmented reality content and games to be as simple as making an html page, or contributing to a wiki.

Federation, real time collaboration, linked data – ARBlips that contain metadata that is usable for semantic searches, and modified wave servers that can listen to and respond to  SPARQL HTTP requests properly (see Jason Kolb’s many interesting posts on XMPP and Wave).  These are just some of the reasons why ARWave could revolutionize augmented reality searches and more! (see my presentation at MoMo13 – video here)

For more on real time social augmented experiences see our panel, The Next Wave of AR: Exploring Social Augmented Experiences at Where2.0 2010, and don’t miss the Where2.0 conference which has been the crucible for the emergence of location technologies.

Augmented realities, proximity- based social networks, mapping & location aware technologies, sensors everywhere, linked data, and human psychology are on a collision course in what Jesse Schell calls the “Gamepocalypse”   See Jesse Schell’s Dice 2010 talk here, and check out his Gamepocalypse Now blog.  As Bruce Sterling’s notes in his post here:

*Another precious half hour out of your life.  However: if you’re into interaction design, ubiquity, social networking, and trendspotting, in the gaming biz or out of it, you’re gonna wanna do yourself a favor and listen to this.

And don’t forget to register now for Augmented Reality Event (ARE2010 in 2-3 June, 2010 – Santa Clara, CA).

Bruce Sterling, Will Wright, and Jesse Schell will be keynoting, and there is a totally awesome line up of AR innovators and industry leaders, including Paige and Anselm!

And:

You are in luck!

Here is a discount code for the first 100 folks to register to the event (before the end of March). Go to the registration page, type in code AR245 and “you’ll be asked to pay only $245 for 2 full days of AR goodness.”

“Watching AR prophet Bruce Sterling, and gaming legend Will Wright, visionary game designer Jesse Schell deliver keynotes for this price – is a magnificent steal.  And on top, participating in more than 30 talks by AR industry leaders will turn these $254 into your best investment of the year,” as Ori  put is so well on Games Alfresco!

If you want a preview of just how exciting it is to be involved in augmented reality right now check out Ori Inbar’s great round up on our latest monthly Augmented Reality Meetup NY (or as, Ori notes, we fondly like to call it ARNY.)  There is lots of video up now (much thanks to Chris Grayson, who  live streamed it).  Augmented Reality Magician, Marco Tempest, is an absolutely must see.  (developers note this is an awesome use of open Frameworks and OpenCV).   The video of the show includes a rare explanation of how it all works  – see here.

Talking with Paige Saez – “Software is candy now!”

Tish Shute: What interests me about ImageWiki is that you have thought about physical hyperlinking beyond the obvious of where to get your next good hamburger and beer, right?

Paige Saez: Right. It was interesting for me in just thinking about the two things. How do you design a tool to work in a way that people are getting value from it? And also, how do you make it work in a way where people can explore and hack it? I think the most interesting technologies, and this is probably something somebody else said sometime, are the ones that disappear, that we don’t see, instead we see through. They become just the intermediaries.  They don’t interfere with what we are trying to do.

It’s a struggle whenever you are developing a new way for people to get information or make something happen, because you are playing with magic a little bit. And you have to make it vanish the way a good magic trick makes an experience a magical one. But at the same time you also need to reveal just enough that you let people in and they can see how to change it and make it their own. That is the interesting tension for this space right now, the idea of augmented reality begins to lead the idea of a social commons for physical things. The Imagewiki project was a locus of just this tension. Tish you and I have previously discussed how difficult it was to even get people to understand the two concepts independently.

Tish Shute: Right, until recently most people hadn’t even heard the term augmented reality and I am not sure that a particularly high percentage of people would recognize it now despite the recent interest in smart phone apps.

Paige Saez: It’s very difficult to get people to understand the two concepts, and now you are adding in the third level of participation as well. So I don’t think it is impossible, but I do think it requires narrative. It is interesting that you were talking about the stories you heard this morning from the creatives at the event [Tish mentioned David Curcurito, Creative Director, Esquire gave an excellent presentation at Sobel Media event NYC] because it’s narrative and the attention to telling a story that help you walk through all of the ways you can understand how completely expansive this area is right now.

So I think we have to play with it, play with the space and the tools. I think we need to have an idea of what we want people to use the tool for, and we need to not only introduce them to the tool and the technology, but also introduce them to the concepts as well. So I see it as a three part process.

I’m really excited to be there with people, helping them do that. I think we need to do this face to face. I don’t think this can be only through a social network. The ImageWiki website is like one quarter of the entire picture, you know? The website is the resource center and the place where you can see people adding images, but what value is it to you to see an added image? It is more valuable for you to be interacting with the image or interacting with the object in the real world.

Designing for the experience of using the ImageWiki got very complicated very fast. I was trying to figure out the main thrust of the design for the UI for the ImageWiki and at a certain point I had to take a step back and say “Okay, this has to be good enough for now because we can lay it out and prototype as long as we want on the Web or mobile UI. What we need to be doing is going outside and actually aggregating and putting images into the database in order to see what exactly happens when we are adding.”  It’s not just like you are taking a picture of something and adding it to Flickr. Using the tool is very context specific and the information is context specific, and you can’t necessarily make that all happen at the exact same time. I think these are really fascinating spaces to be struggling in and I’m so glad to be working in this space.

Images by Chris Blow of unthinkingly.com

Tish Shute: Could you explain why we need ImageWiki? I mean I think I have ideas on this, but perhaps you can explain to me from you point of view why we need an ImageWiki, as opposed, to say, extending the image space of Wikimedia or something added on to Flickr.  I mean maybe something leveraging the geotagged photos sets and APIs we already have?

Paige Saez: Yes, definitely. It’s a really good question, I mean it really is. Like, do you need an entirely new place to be holding images outside of the places that we are already holding images? That’s a huge question; enormous. Especially when you take a look at the problems around that. Its’ exhausting for an end user. Who the heck wants to go and reload everything into yet another place, right?

Tish Shute: Right.

Paige Saez: Moreover, who is going to really bother? Another problem would be what happens to the existing datasets that people have already committed to? And then of course there is the problem of authority and explanations why….Gaining interest and authority in a space when nobody even understands why that space should exist in the first place. And those are just three, you know, off the top of my head problems with that idea.

And yet at the same time, I don’t actually know how else to go about thinking about the ImageWiki unless I think about it as it’s own thing. Then you start thinking about models of large independant image databases that exist already, examples of this from a product standpoint- references to consider. The Getty Foundation comes to mind. There are many other historical centers that have huge resources and images that are licensed out and used. So here we have a working example of people already doing this. But succesfully? I don’t know. We do have a ton of intellectual property rights and copyright issues and ownership and use issues with images currently. As a working artist these issues for me were a major red flag to consider. Working on the social commons for augmented reality starts paralleling issues found in digital rights management and intellectual property.

Tish Shute: But one good thing about Wikimedia, why I focused on Wikimedia, is Flickr and Wikimedia already use a creative commons licensing, right?

Paige Saez: Creative commons, you know they have their own resource center, too. But you know they haven’t been successful as great databases for images so far.

Tish Shute: What would you like to see that they don’t have? Like say maybe start with Wikimedia, right?

Paige Saez: There’s just still a lot of issues with how to encourage people to want to contribute. It’s hard to show the value to someone who doesn’t already understand the value for some reason. At least for me personally this is something I have run into frequently. I don’t know if it is necessarily what Wikimedia doesn’t have, I think it is a lack of understanding of what creative commons really means. And there is still a very strong sense of ownership and concern about creative property rights. Being paid to be creative is a tremendously difficult thing to do. People fear losing their livelihoods. They think this is possible. Is it? I dunno.

For example : Look at me, I take a photograph of something, I can sell that. And there’s a question about whether or not, as an artist, I want to have my photographs in a pool of images that is open and accessible when I could be making money on it instead. Now that is just an example. Me personally, I can see the value. But that is a common concern. The gist of the question being, ‘what value does it bring to give something away versus holding on to it?’ A hugely popular discussion right now.

This is the same crux of the problem we are dealing with when we talk about thinking about images in the social commons for the real world. It’s a conversation about ownership. It’s about, who does this belong to really? If I take a photograph of a Levi’s billboard, does that photograph belong to me or does it belong to Levi’s? We know the boundaries of that. But when the image becomes a living image, an image capable of transmutation; an image that provokes an action or hyperlinks to a product, experience, information….where are the boundaries in that?

Tish Shute: But how is ImageWiki handling that differently from Wikimedia, I suppose is my question.

Paige Saez: We haven’t solved the problem.

Tish Shute: Yes, I suppose it is not like we have fully solve the problem of a creative commons for images on the internet let alone the issues of a social commons for the real world! So neither one has solved the problem, right?

Paige Saez: Exactly. To be honest, it made my head spin. I realized we were building a web application and a mobile tool doing augmented reality, real time feedback on the world and suddenly we weren’t. Suddenly we were dealing with DNS and talking about physical hyperlinks and ownership and property. And basically at that point you just have to sit and really start looking at catching up on IP issues and figuring out how to deal with that space in a much more wholistic way. It became so important that we had to take a step back and go

“Oh my god I think we have really uncovered a real problem here.”

At the point when we were building out the tools we realized something was really going on with our project. Here we were thinking that this was just a beautiful experience of learning about the world around us. We really…Anselm and I both just really wanted this tool to exist. It was something that we both just really wanted to happen in the world, something that we felt really just thrilled to make. And we looked at and used it and realized that instead of it just being a beautiful experience, it was a fundamental shift in how we understood everything. That it impacted our world in the same way the Internet impacted our world. It was a fundamental shift in understanding. A sea-change.

So I put down the prototype and went back to researching, read a ton of books on IP and went and presented to friends, family, schoolmates and co-workers trying to explain the project and then the larger conceptual framework that had emerged from the project. I began using the metaphor of thinking about Magritte’s “Ceci n’est pas une pipe.” Thinking about a pipe that isn’t actually a pipe.

Tish Shute: Oh, yes!

Paige Saez: ..to try to help explain to people that the image that you see is actually not, you know, it’s not an image of a thing. It’s an image. And that image has a tone and that image has a voice, and that image was chosen. And there were decisions that were made through the interface of the camera, specific decisions that defined the view of what you were looking at. And that that wasn’t being acknowledged and that that was a fundamental part of what the ImageWiki was aiming to do. The lens that you are actually looking through was as important as what you were looking at. And democratizing that lens became the most important thing that we could possibly do.

Tish Shute: So the emphasis for you on ImageWiki was in fact the lens, even though you found obstacles to creating the interface, right?

Paige Saez: Yes. Definitely. That’s what I fell in love with first. I really wanted to be able to use my phone to learn about what kind of tree this was or to buy tickets for the band on the poster I just saw, or see a hidden secret. For me it was very much a story, a narrative experience that I just thought was magical. And that is how I fell in love with it, which is not where I ended up. Where I ended up was realizing it was a fundamental shift in not only my own understanding of how to use the world around me, but in our understanding of looking at the world.

Tish Shute: It would be pretty scary if an image DNS was basically in the hands of either one or very few people, right? I mean even ImageWiki would be stuck with this problem, that if you set up a bunch of servers, you are going to be holding a very, very large image database. I mean, whatever your motivation, right? I think at the minute that is why I am very into seeing everything through the lens of federation, I see that unless we have federation, these giant central, databases are inevitable aren’t they?

Paige Saez: Essentially, yes. I mean I wasn’t able to walk through it as quickly as that. It kind of just overwhelmed me. Looking back on it, it seems perfectly obvious. I was just like “Oh my god, what have we done? Like what is going on?” Particularly for me because so much of my life has been spent in art, it was really easy to immediately understand the connection between the view, the viewer, and what’s being viewed as all just different layers of ownership and understanding that it is a gaze. Right? We know that we are never able to look at something without passing judgment on it, but to see that become a part of the interface in a real-time fashion just blew my mind.

Tish Shute: Yes.

Paige Saez: I think you are right. Getty Images, Flickr images, no matter what you are always holding on to something and you have to be responsible for it. Right? So how do you deal with the responsibility but don’t take on too much ownership? Where is the boundary with that?

Tish Shute: And for me, the simple answer to that is loosely connected small parts, distributed systems and federation. Because there is only one way to be able to utilize these things is to have them distributed so that no one holds all the cards. Right?

Paige Saez: Definitely and I personally agree with you wholeheartedly. However, the idea of distributed power is a concept that most people just don’t know how to deal with.

Tish Shute: And it’s easier said than done because actually the root problems that you are talking about aren’t got rid through federation, because if someone really holds the, sort of, all the good image databases just because they have the potential to be federated, they may not choose to open them up on many levels.

Paige Saez: And even then you have to think about, sort of, like the next level of it, which is we want it to be all open and accessible, but everything is owned by somebody. Like, what really is public anymore, in general?

Tish Shute: And what is interesting though, regardless of what we speculate conceptually on this, we already set off down the road. I mean we have already several large…they are all in beta I suppose, Google Goggles, Point and Find, right? But we have applications that are beginning to implement this. They are beginning to implement search on it, and it is geo-located even if it’s not in an augmented view, right? So it is proximity based.

Paige Saez: Right, right. I mean maybe the solution is that if we follow that line of thinking then Flickr will be partnering with Google Goggles. And then my images would stay under my ownership through the authority of Flickr. And I would use Flickr as my place to add images and they would just be responsive via my devices via AR.

Tish Shute: That’s very interesting.

Paige Saez: Definitely I think so. It is also the shortest distance between things.

Tish Shute: Yes, and as Anselm kept pointing out, basically it is going to happen in the simplest way possible, really, regardless of the implications of that. But OK, getting back to ImageWiki. As you say neither Wikimedia nor Flickr were really designed to take this role, right?

Paige Saez: Right.

Tish Shute: With ImageWiki, you’ve had these ideas and a concern with the social implications of physical hyperlinking in your mind since it’s inception. Are there any design ideas you’ve come up with that you know, as opposed to sort of, as you say, connecting Flickr to Point and Find, or who knows, Google Goggles. How is ImageWiki going to be different, do you think? Is that a hard question at this point?

Paige Saez: It is, and it’s a great question, and it’s a question I really love to think about. I think we have to introduce the politics with the tools. It has to be acknowledged that it’s not just a place to hold information, that’s what I feel in my heart.

At the same time, is that too much for people to really grasp at one time? In my experience it really has been, so the design of the experience needs to allow for an understanding of the power of the tool and the level of authority that the tool offers, while not getting in the way of it; just using it. Because ultimately, at the end of the day, nobody will use anything if it isn’t valuable to them. And so I could talk for miles and miles and miles about how important it is that corporations don’t own all of the rights to all of the visual things in my life, right? For the rest of my life I could talk about that. The idea that advertising is dominating all of our views of anything in the world around us is horrifying. It doesn’t matter unless I can show somebody why it matters to them or how it affects them. It’s just that that is a tremendously difficult thing to explain through a user interface.

And I actually think that it’s great that tools like Google Goggles and Nokia Point and Find are here to do a lot of the hard work of showing people how it works. Recently somebody explained to me their experience of using Google Goggles. They went through this process of saying how the Google Goggles took a picture and then did this really complicated visual scanning thing over the image and it took a full minute.

And I said, “Well of course they did it that way.” And they said, “Well what do you mean?” I said, “Well, what they are really doing there when they are doing all these fancy graphics, is they are showing you how it works.” And even if it isn’t actually related at all to how it functionally works, algorithmically, that’s not the point. The point is that this gesture of the time taken to make it look like it’s scanning an image and going back and forth with pretty colors is giving people the time to process that as an experience. That’s a metaphor for what’s really happening. And these kinds of metaphors are crucial with user experience design. We have lots and lots of examples of them and how they work, and many of them aren’t necessary. Like you know, for example, the bar that shows you the time it’s taking for something to process.There is no relationship between that and reality. But it is really important.

Tish Shute: Yes those bars often have no relationship between the actual time..

Paige Saez: And that’s the thing. Like the idea of time versus our perceived understanding of time. Right? The length of time it takes for your Firefox browser to open and load your last 30 tabs, versus the reality of what’s actually happening. When you are doing that sort of research you are actually accessing millions and millions of places and points of interest all over the world, so we need more of that. We need more of the process shown. Anselm and I worked with a film maker named Karl Lind from In the Can Productions here in Portland to try and make a video about the ImageWiki. We made this little video and I can try to show it to you or send it to you if you want.

Tish Shute: One of the issues with this kind of visual search is that it is inherently dependent on large databases, regardless of where they are federated, are going to be very large. Right? I mean someone is going to have something big, and aggregated there. I suppose someone will figure out the challenges of federated search eventually but that is quite a big challenge!

So I suppose I am still trying to understand what ImageWiki can offer that we can’t get with any other existing service? How will their be a social commons and even a social contract for the world as a platform for computing and physical hyperlinks?

Eben Moglen brought up something when I talked to him about virtual worlds, he said we need code angels to let us know what was going on in the virtual space – who was gathering data and how, for example.

Paige Saez: Tell me more about that, I want to hear more about that.

Tish Shute: Eben suggested this metaphor for when I was asking him about privacy in virtual worlds. The fact that people just didn’t know that when they were pushing avatars around virtual worlds what metrics were being gathered on their behavior. And he basically said that what we need is code angels when we enter these spaces because having the rules of the game buried in a TOC was ridiculous.

Paige Saez: That is a really interesting idea.

Tish Shute: Maybe ImageWiki needs to be our code angel to navigate the augmented world. I mean that’s what I want to see it as. And when I hear you talk, what I hear is you talking in broad categories about what a code angel might be in the space of images and image links to the physical world. I mean that is what I hear from you.

Paige Saez: Yeah. No, I definitely agree with that. It is interesting. In that sense, it is kind of a protection layer. Is that what you are thinking?

Tish Shute: Yes, I suppose because we can’t be navigating a lot of complicated opt-ins and opt-outs just to get around our neighborhood safely (in terms of privacy (also see Eben Moglen’s definition of privacy here…) We will need a code angel that is sort of keeping up with you in real time!

Paige Saez: Right, right. I wonder how that would work in regards to images, though. That is a really interesting thing to try and put on an image. I guess why I am having such a hard time being specific about it, is I am just trying to work it in my head, thinking of a specific use case, like what would be an example of that?

Tish Shute: Well I suppose the example, and this is a crude one, is when you point your Google Goggles to the book jacket, the code angel, this is very crude, would say “You are right now drawing images from the Amazon database – they are collecting data such and such data from your search.

And then of course the ability to have crowd sourced tagging and corrections..

There was a wonderful book that came out last year on how we can have commercial intelligence -Dan Goleman’s new book: “Ecological Intelligence: How Knowing the Hidden Impacts of What We Buy Can Change Everything”…

how corporations various different stakeholders, including their customers will drive corporations to do the morally right thing because they will lose the commercial support of customers who won’t support them unless they are more green, fairer, do the things we would like them to do whatever that happens to be – physical hyperlinking and tagging I guess would be a big part of this.

Paige Saez: Sort of a transparency issue. And that almost becomes a page rank algorithm in and of itself. I mean now we are really talking about search more than anything, and what tool becomes the dominant search tool. Anselm and I talked a lot about one platform… I mean eventually we will have a unified platform. It will…No matter what, for the Internet and for physical objects and visual objects in the real world. It will just be a matter of, literally, who can find the best and most valuable, most relevant information on a thing. Currently we just have it very proprietary.

Tish Shute: Yes.

Paige Saez: That definitely won’t last. It just can’t, because of the exact problem that you are raising. And we already know too much about resources and information as they pertain to products for us to ever go back to a time where we are not considering other ways of getting information about it anyway. Right?

Like I have the same concerns nowadays when I look at fruit. I look at a piece of fruit in the store. I would never just assume that the person who put the sticker on that fruit, anymore, is the ultimate authority necessarily. I would always assume at this point I could go online and go find out more information about a company. Issues about like eco-footprint or how much toxicity, or pesticides or whatnot are now totally accessible already.

So I am thinking when you look at that piece of fruit and that sticker for Google, say what you are describing, do we just go immediately to the company’s website, or is it even more specific? Do we know that the sticker on that piece of fruit is going to tell us specific information about that? Or are we just getting back the nutritional resources, or are we getting a listing of all of the different options out of a page rank algorithm that shows us, “Well this is the website for the fruit. Here is the nutritional information. Here are the last 15 comments on it.” It’s basically just a basic search.

Have you heard of Good Search?

Tish Shute: you mean http://en.wikipedia.org/wiki/GoodSearch

Paige Saez: Right.

Tish Shute: A code angel interface would have to give you options, wouldn’t it on possible views available?

Paige Saez: Yes. You are then talking about filtering your view. Then it really gets really interesting, of course. I don’t even know if we have a choice in that. I think we are really kind of hitting a wall with who owns the space and the platform. Is it just a basic search because we are already familiar with search? If you had an option to choose, say, “I want to look at this apple sticker and I only want to get…programmatically only looking at my friend’s opinions of this company.”

Or I have a safety valve on it that only shows me certain information based on what the code angel knows about me, my preferences, my age, things like that. Then that gets really, really interesting, because we are trying to do all that work right now just with social media and the Internet. We are already overwhelmed with too much information. It is already past the point of comprehension. So to think that we would actually drill down even more specifics is very interesting.

Tish Shute: That was a point Anselm made about the fact that once you are into this mobile, just in time, one view kind of situation, it is quite different than the Internet where you can bring up all these different screens and go to another website.

Paige Saez: Well yes, mobile is a different level of engagement. Very contextual. Much less information. Much more about timeliness. I don’t want to look an apple and get back a Google search. Oh my God no. That’s the last thing I want. I would love to be able to look at an apple and my phone already knows exactly what I want, information-wise, to get back from that apple. But I don’t know. It’s all contextual and personal. So I think the code angle concept you are talking about is really interesting because you still need to think about who is the person that is adding or creating those level filters- is it you, a filtered friend network, an algorithm? How much work is too much work? Where do we draw the line? How much of this are we willing to let the machine do for us?

Tish Shute: Right.

Paige Saez: And then of course once you have those filters in place, you need control over them. You will need to dial them up and dial them down, be able to choose and add new ones, so on and so forth. It becomes very modal at that point. For example, I want to change my view: To walk into a grocery store and instead of finding out information, I’d want to see where the hidden Easter egg puzzles were that my friends left last week because we’re playing a game.

I’m still really attracted to the creative opportunities with the ImageWiki. I’m really attracted to changing this experience from being a one-to-one relationship (from Corporation to Consumer) to an open-ended relationship (From Person to Person). If I look at a book jacket, sure I can find out where to buy the book, but that’s boring. Who cares? I’d like to find out a link to a story or an adventure or a movie or something unthought-of before.

How do we build that in? How do we encourage serendipity? Mystery? I think the ImageWiki is the space for building that in, actually. Not how, that would be the one place, right? That’s my really big fear is that this relationship just stays one-to-one. Click an image of consumable object, get back objects retail value. How completely dull. We have to do better than this.

Additionally, what if I want to take a photograph of a book, an apple, or something and I don’t want to pull back data. Instead, I want to pull back music, or I want to pull back a video, or I want to pull back a song, or lyrics, or a story, or another image. It’s just a hyperlink at the end of the day, you know? That’s all we’re really doing. Hyperlinks can pull back so many different things.

Tish Shute: And that’s one of the reasons I’m into mobile social interaction utility building, because without that, if we don’t have that way to do that in mobile technology…that’s very available on the Internet, as we’ve seen, with Twitter. These applications are very easy to do on the Internet. They’re not easy to do natively in a mobile application..

hey, I’m just promoting AR Wave again. I should shut up.

Paige Saez: Oh, no. I think it’s a fascinating concept, I really do. I totally agree. As we’ve talked about it before, it’s amazing that marketing and advertising are helping push forward AR, and it’s great. It’s fantastic.

But it’s also the worst possible thing that could ever happen because it is such a singular way of looking at an overall ubiquitous computing experience. There are other ways.

The best experience I ever had was trying to explain to people about physical hyperlinks. I had to walk them through it. Good interactive isn’t something you present or show, it’s something you do. Nothing beats just walking around and showing people with a device or a tool or something else.

I mean, God forbid it always stays in our computers and our phones. I really hope we don’t have to be stuck living our entire lives with these horrible interfaces. But for the time being, we will. Having an AR app show you a puzzle, or a mystery, or a game, or an adventure is a magnificent experience, totally overwhelming, and people get it right away. There’s no question; they totally understand.

Tish Shute: Yes, I agree.

Paige Saez: You walk them through the experience with a physical hyperlink and then you say, “Here, I could use this device and I could show you where to buy this thing, or I could use this device and we could start playing a game.” Then everybody gets it.

Tish Shute: So then I have a question, because one of the things Anselm said to me when he wanted me to refer back to you is that he feels that the direction for ImageWiki should be perhaps to focus less on the technology and more on just the actual, I suppose, gathering of the images, how they’re going to be annotated, the metadata, right? But my question to him was, the problem if you do that, without the platform, there’s no experience or motivation for people to do that. Right? Is there?

Paige Saez: Yeah, I agree with you on that one. I’m curious what his…I think the reason why he wants to do that is he wants to be able to show people examples via the resources. Like to be able to show someone a library, essentially, which I think makes sense with some people. I definitely think that some audiences would really relate to that. For me, it doesn’t make sense because I’m just very experiential. I need to do it and I need to show other people how to do it and I need to grow that way. I think that at the end of the day, those are great ways to go about doing it. It’s just it’s a huge thing to do in either direction.

What Anselm’s really thinking on, I believe, is more about exemplifying how we read and understand images culturally. Then you’re really getting into Visual Studies and Critical Theory which is what I did for my Masters at PNCA. I worked on the ImageWiki while I was in grad school, it was something I was doing for fun. Independently of my studies, the project lead to issues on democracy and objects and property and I ended up right smack in the middle of what I was studying; the nature and cultural analysis of images Questions like, ‘what exactly do we get out of images?’ and how all these different things are happening in an image, and people get tons of totally different things out of an image depending on many factors.

The questions I began to ask myself got very philosophical. Questions like “Is this apple red? Is this apple red-orange? Is this a small apple? What’s my understanding of small versus your understanding of small?”

Because you supposed that you needed a text backup to the search, how would I be able to search for an apple? Because what if my understanding of apple is red and your understanding of apple is green. And so if I’m looking for a green apple, am I looking for the same green apple as you? It’s all semantics, sure. But at the same time, it gets bigger and bigger, and it’s fascinating.

Tish Shute: Google Goggles seem to work best on book jackets, basically.

Paige Saez: But book jackets are actually perfect for this. Book jackets are perfect for this problem, because book jackets are specifically designed art. So at the end of the day, we are still talking about creative works, artistic works, that have been designed as a communication tool. But that is not something that people can own. Creative works that are designed are a communication tool, with varying levels of skill to be sure, but still something anybody can do. What we need to do is we need to be using that language. We don’t need to be trying to reach as far as facial recognition. We need to develop our own logos, our own brand, our own…I mean not brand. Brand is a bad way of saying it. Another way of saying it would be like, just use it. Develop a visual language that we can use that is as effective and as well utilized as book jackets or the movie posters or something.

Tish Shute: What are some of the use cases for ImageWiki you would like to develop first?

Paige Saez: My dream…I have like four or five use cases that I want to see happen. One of them is I walk down the street and there is a new poster for my favorite band. And I can just go up to the poster and I use my device, whatever it looks like, and I download the latest album. It’s transactional. I am able to just plug in my headset and walk down the street and the transaction is done. I saw something I wanted. It was beautiful. I was able to get it and I was able to move on in my life. And that is totally possible.

Another one would be I walk down the street and there is a piece of graffiti. And I am able to use my device to find out who the artist was that made it and to give them props, and to point my other friends to the fact that the piece is there and it will most likely be there only for a short period of time- information retrieval and socialization.

Or, use my device to find an Easter egg, to find a narrative puzzle that ends up going on for weeks, and everybody is involved, and we are all playing this game together. Adventure-based, non-linear experiences. I want playfulness, not just purchases.

Tish Shute: Did you think of piggybacking on the Flickr API for geo-tagged photos as a way to work with those databases or not?

Paige Saez: Yeah, we definitely thought about that.

Tish Shute: And why did you decide not to, for any reason or…?

Paige Saez: Ultimately, we just…we were such a small group, we just had to tackle certain things at a certain time.

Tish Shute: Right. And you were so prescient, you were working slightly before we had the mediating devices, weren’t you? You were just before the mobile devices really got adequate for this.

Paige Saez: Yeah. We started on it…I believe it was January…No. December 2007. Basically, the iPhone had just launched like maybe six months prior or something like that.

Tish Shute: But not 3G and not 3GS, right?

Paige Saez: Not 3GS. It was the first generation iPhone. We built the ImageWiki before the App Store existed.

We knew that the App Store was coming out. And we knew that the App Store was going to be the biggest thing in the whole world. I remember getting into multiple fights with friends about how revolutionary the iPhone and the App Store were going to be and people thinking I was totally crazy; people just thinking I was absolutely nuts for being so excited about it.

It sucks that it is a closed proprietary system, but the App Store has done something for software that nothing has ever done in the whole world. Software is candy now. It’s candy. It is like when you are waiting at the grocery store at the checkout line and you are stuck behind somebody, and you have got all these little tchotchka’s, candy bars, magazines, nail-clippers and things. That is the equivalent of software now. It’s become an impulse buy, which is amazing. Nobody would ever have thought…that is actually revolutionary. That’s huge.

Tish Shute: Steven Feiner, who is one of the founding fathers of augmented reality said to me during a conversations at the ARNY meetup that one reason that augmented reality, despite the hype, is manifesting very differently from how virtual reality burst onto the tech scene is that it is about affordable apps on affordable readily available hardware.