The Missing Manual for The Future (or The Future: The Missing Manual)

O’Reilly Media, is famous for is producing  “missing manuals” for new technologies, but thinking of O’Reilly as just a publisher of books would be like saying Facebook is just a website (this came up in the discussion at Media Round Table at Web 2.0 Expo, NY, 2010).   In recent weeks, I managed to catch Tim O’Reilly at several events, Maker Faire, Web 2.0 Expo, Hadoop World, and the free webcast Tim did with John Battelle on The Battle for the Internet Economy (although Tim spoke several other times during this period!).

It occurred to me, as I immersed myself in the depth and breadth of innovation showcased and discussed at these events that Tim O’Reilly, and the O’Reilly team, are creating, The Missing Manual for the Future.

As Tim puts it, we are “changing the world by spreading the knowledge of innovators.” Tim uses a quote from William Gibson to illuminate what is at the heart of the O’Reilly project:

“The Future is here, it is just not evenly distributed yet.” (William Gibson).

But Tim O’Reilly makes another point about the future when he speaks.  The future unfolds unexpectedly – so we must invent for an unknown future not a known future, or as Alex Steffen put it so well in his post, Why Our Bright Green Futures Will Be Weirder Than We Think, – “The world we need is one we’ve never yet seen.” The magic of attending an O”Reilly event is that it gives you a chance to work on this koan in interesting ways, and to take more responsibility for how things turn out.

Tim O’Reilly also urges that we think more deeply about what we are doing.  His keynote for Hadoop World , NYC, billed as, “The Business of Data” turned towards “The Consequences of Living in a World of Data.” The 900 strong crowd at Hadoop World was probably one of the most savvy crowds in the world about the business of data, so this was a nice turn.

Web 2.0 Expo with the theme, Platforms for Growth, was a deep dive into the business of innovation.  Tim O’Reilly’s keynote at Web 2.0 Expo,  “Thinking Hard About The Future” (or rather “thinking a little bit creatively or differently about the future)  – see video here, developed the call he made at Web 2.0 Expo 2008, to “work on stuff that matters,” into a Four Cylinder Engine for Innovation.   The first of the four cylinders in the firing order is, “Having Fun!” But,  at Maker Faire, Web 2.0 Expo, and Hadoop World I got an inside look at the workings of all four cylinders, and there is more to come, I am sure, as the other O’Reilly events unfold over the coming months including,  Web 2.0 Summit, Strata (a new O’Reilly conference on The Business of Data), and Where 2.0, 2011.

In a free webcast, last week (recording here), previewing Web 2.0 Summit, John Battelle and Tim O’Reilly discussed the Points of Control Map which is developing into a fun and useful tool to examine a very serious topic, “The Battle for the Internet Economy,” and how the “increasingly direct conflicts between its major players” could effect “people, government and the future of technology innovation.”   In my previous post, Platforms for Growth and Points of Control for Augmented Reality, I had a great conversation with Chris Arkenberg using this map as a springboard.  More on Points of Control later in this post.

The Four Cylinders of Innovation

click to enlarge

From Jet Ponies to Jet Packs: The First Cylinder of Innovation – “Have Fun”

The “maker” energy and its spirit of play, and the courage to create, hack, reinvent and re-purpose everything and anything, is a quintessential example of the first cylinder of innovation firing big.  Many “maker” projects also go on to fire on all four cylinders.   But the Maker forte definitely is in the first cylinder zone (and safety third as some of the rides, including Jet Ponies, warned).  The photo opening this post by Marc de Vinck – for more pics see here, is of Tim riding The Jet Ponies at Maker Faire which took  the New York Hall of Science by storm in late September – see The New York Times coverage here.  The ride was “built by the dastardly danger-hackers at the Madagascar Institute.“ See this wonderful interview with Hackett on his work to design “our specific jets from a patent that was filed in 1960s by a Mr. Lockwood, for Valveless Pulse Jets.” Hackett points out:

“Louder than god, glowing white-hot and looking like the trombone of the Apocalypse, pulse jets are also really shitty, inefficient engines,”

But, he adds:

“I have always wanted a jetpack, and one of the reasons I learned to build these things was to further that goal.”

This grand vision behind the Jet Ponies is a key to firing, The Second Cylinder of Innovation,  “Hey, we can change the world!”

But Jet Ponies, as a stepping stone to jet packs, also really struck a chord for me as I have been devoting a lot of time lately to the emerging Augmented Reality industry, a technology which was lumped in the same category of sci fi chimera as jet packs until very recently.

Data is the Gasoline

“The faces are coming from the sky.  The locations are coming from the sky.  All these apps depend on something, somewhere up.  And that, to me, was always the heart of Web 2.0.  And I am so delighted that people are finally getting it.  Because for a long time, people thought, ‘Oh, Web 2.0, it’s about lightweight advertising supported in a web start up.’  So I went, ‘No, no, no. It’s about the fact that we’re building these giant database subsystems in the sky that are going to drive applications.’  And now, of course, the same application is on your PC, it’s on your phone, it’s on you iPad.  And clearly, the applications are just sort of an interface to something that is being driven from the cloud, and that is fabulous.  That’s the difference.  People get it now.” (Tim O’Reilly, said this as part of a response to the first questioner at the Media Round table Web 2.0 Expo)

Answering questions about the importance of “Having Fun” to innovation doesn’t look quite as fun as riding Jet Ponies! Photo above from luca.sartoni’s Flickr stream

“ the data that is generated by the sensors and the applications that use that data is going to be where people are going to be innovative.” (Tim O’Reilly)

During the Media Round Table, I had a chance to ask Tim more about the role of bottom up innovation in a world where big data is the gasoline for increasingly sophisticated engines – platforms integrating machine to machine intelligence and real time analytics.

Tish Shute: You brought up Maker Faire in your keynote, and again now.  I was there, which not many people in the audience were  [not too many hands went up when Tim asked during his keynote].  But I think one of the things that struck me was the jet ponies – they were just earthshaking to stand near.  They made the ground tremble; they made the world shake.  Yet, most of your keynote, and most of what’s on our minds here, at Web 2.0 Expo, is extracting intelligence from the big data [in the sky], and algorithmic intelligences are the jet engines of the internet.  And of course, not to be forgotten, as we are here in New York City, where the trading markets are creating the air we breathe  [although we probably don't realize it until we lose our mortgage or something] and these algorithmic economies or “robot casinos” as Kevin Slavin put it, are all about speed – it’s not just real-time, issues of latency are  so critical that co-location is key to winning the game of the markets.  [Kevin Slavin brilliantly unpacks this in his talk, "Loitering on the Motherboard." For more in this see my conversation with Kevin Slavin below].

So my question is, who’s making the jet ponies for the algorithmic economies in the sky that you just described?   How can we make a play from the bottom up?  I always feel Ushahidi is one of the jet ponies of the data algorithmic space [because of their great work to bring human and machine intelligence together to solve problems in crisis situations].  But who do you think is doing exciting work and how can we ensure that this powerful world of data and algorithmic intelligences does not become hidden in a closed black box [only really accessible to elite players like the NYC trading markets]?

Tim O’Reilly: “Well, I think there’s certainly a lot of interesting things happening in, say, the financial services that a lot of, kind of, the Internet folks are kind of blind to.  I think that there are companies like Next Jump which are really good with data and good with algorithms. But kind of speaking specifically to the maker side of this, that whole sensor enabled world which is going to produce data is in its infancy.  What we have that I think is so powerful right now is we have the first portable sensor platform.  I said in my talk the other day, you know, your phone has ears, it has eyes, it has a sense of where it is.  And these are all available to application developers. You know, you can compare, say, Dodgeball to Foursquare, you can see how different… Dodgeball is Foursquare in the tele-type era.  Foursquare is now possible because there are so many more capabilities on the phone.

And I think that we are going to see a lot of other areas that are revolutionized by the sensors in the device.  It could well be that some of them will come explicitly out of the maker kind of projects, or it could just be that make is sort of a proxy for them.  So yeah, Arduino is this great maker sensor platform, but hey, here’s a consumer sensor platform [holding up phone]. Maybe we vaulted past the maker stage already and we just didn’t know it.

And that’s not entirely true, because Arduino is building a whole economy of special purpose devices.  But it feels a little bit like the days when people rolling their own PCs coexisted with the rise of Dell, who was a kid in his college dorm room who made his own PCs and sold them on the net, but figured out how to scale it pretty quickly and get good at it. But there were still a lot of garage shops, you know, ‘I’ll make a PC and sell it to you’ people for probably a decade before there was really a clue that that was a commodity industry.  In fact, I do think the sensor platforms are going to become a commodity industry.  And the data that is generated by the sensors and the applications that use that data is going to be where people are going to be innovative.”

The internet operating system is a data operating system and it is happening in real time (Tim O’Reilly)

click to enlarge the image above  – a slide from Mike Olsen’s  (CEO of Cloudera) keynote at Hadoop World

Not only do we have a portable sensor platform in our pockets  but developers also have powerful platforms and tools to make sense of data that fuel our apps.   Opensource Hadoop makes available, to anyone with some data munching chops, the power to work with giant unstructured databases and do the kind of real time analytics previously only available to giants like Google.  Big players like Yahoo, Facebook, and Twitter use Hadoop (Jonathon Gray from Facebook noted they add 10TB a day).   But, as this great roundup of Hadoop World points out, while Hadoop gets the press for handling petabytes of data , Mike Olsen (CEO of Cloudera) noted, the fastest growing area of users are working with clusters smaller than 10TB and over half of the Hadoop clusters were under 10TB in size.

Four Square: A Platform for Growth with an ecosytem built on top of data that exists in the real world

As an augmented reality enthusiast it is not hard to guess that one of my favorite platforms for growth is Four Square.  See Dennis Crowley’s keynote at Web 2.0 Expo here.  The Four Square API has been available to developers since November 2009,  and there are already a number of  interesting applications, and there will be many more to come.  The screen shot above is of geopollster – see the gallery of Four Square apps here.

@dens tweeted recently  “Politics + @Foursquare = @GeoPollster” http://geopollster.com <- I love love love that people are using 4SQ to think about election tools

As Kati London pointed out in her keynote, Four Square is the “kind of augmented reality that is aimed at shifting or changing a person’s social reality, e.g. the mayor badges in Four Square that change my relationship to the people and the place I am in, and augment engagement and reputation through socially driven consumer tie ins.” We are already see augmented reality developers beginning to work with the Four Square API – see here, Foursquare + Augmented Reality + Virtual Graffiti = ARstreets.

As augmented reality development tools mature, Four Square will, increasingly, become an important platform for creative AR developers interested in integrating the power of this platform for augmented engagement and reputation with “device aided augmented reality that can shift visual experiences of situated geolocal experiences.” With the Qualcomm vision based augmented reality SDK now available for download, and Wave in a Box soon? to be released, and an ARWave client working on Android (almost!), I have been exploring the Four Square API in my non existent spare time!!

The Four Square API also offers some interesting possibilities for exploring games that take the complex economy of Four Square – not personal data but aggregates of behavior, as their subject matter (for more on this see my conversation with Kevin Slavin later in this post and in an upcoming post).

I took this picture of Dennis at Where 2.0, 2009 at the beginning of Four Square’s phenomenal growth (they are at 4 million plus users now).

Pachube (Patch-Bay): a web service for storing and sharing sensor, energy and environmental data

Eighteen months ago, I interviewed Usman Haque (architect and director, Haque Design + Research) and founder of Pachube – see Pachube, Patching the Planet.   Usman pointed me to this wonderful evocative image from T.R. Oke’s book, “Boundary Layer Climates” (original photo source Prof. L. E. Mount’s The Climatic Physiology of the Pig).  “It’s the same piglets, in the same box, but on the right hand side the temperature has been increased. This small change in how the space is “programmed” has dramatically changed the way the ‘inhabitants’ relate to each other and how they relate to their space.”

The Challenge of Connecting people and environments.

At Web 2.0 Expo, I got the opportunity to talk with Usman Haque again.   Pachube, is becoming an established platform now, Usman explained.  They have a development team of eleven and robust back end.  And, they will now be spending some more time on the front end, including a redesign of the website, making “it a lot easier to widgetize the entire website so that you will be able to take almost any element and embed that into your own website.” And, as Usman mentioned in his presentation, they are working on an augmented reality interface, Porthole, for facilities management and, “as a consumer-oriented application that extends the universe of Pachube data into the context of AR – a ‘porthole’ into Pachube’s data environments..  Usman is also contributing to the AR standards discussion and on the program committee now for the W3C group on augmented reality.

Click to enlarge the image above from Chris Burman’s paper for the W3C, Portholes and Plumbing: how AR erases boundaries between “physical” and “virtual”

Pachube, is sometimes described as the Facebook for Data or an analogy Usman prefers, a Twitter for Sensors.  At Web 2.0 Expo, I had an amazing opportunity to hear from Twitter and Facebook about their strategies as platforms for growth.  This gave me lots of fuel for questions about Pachube’s approach to developing their platform.  Simplicity was a theme that Facebook  and Twitter both affirmed as a key.  One of Pachube’s challenges will be to deliver ease of use, and the equivalent of Facebook’s “like” and  Twitter’s “follow” to gain mass appeal.

Here is a brief excerpt from my upcoming conversation with Usman:

Tish Shute: So as a platform you see Pachube as having more in common with Twitter – a Twitter for Sensors. In what ways is Pachube similar to Twitter?

Usman Haque: Well we are the Twitter of sensors, devices & machines in the sense that, really, the API that enables all this communication is important, much more so than the website itself. It is where, basically, most of the millions of our hits actually go, is to the backend. And we’ve now got dozens of applications built on top of the system, a little bit like Twitter’s applications; you know, all the apps are the important part.

But we are actually going to be doing some quite exciting things with API keys that we haven’t really spoken that much about in public. But we have come up with a pretty innovative solution to make almost every resource have granular privacy options on it, now discussed here.

At Hadoop World, Tim O’Reilly also raised some interesting broader questions that are very relevant to Pachube’s vision to “patch the planet”, e.g, the problem of digital identity in the age of sensors? (Smart phones already know their users by the way they walk!) And, “How should we think about privacy in a world where data can be triangulated?”

Usman talked about Pachube’s approach to both the technical aspects of how to build a massively scalable system, and the conceptual aspects of how people connect to each other, and what they might do with these new opportunities to connect environments and sensor data  (see my earlier talk with Usman, Pachube, Patching the Planet, for a detailed explanation of some of the concepts behind Pachube).

I look forward to posting this conversation. Pachube is growing, and Usman always goes beyond the familiar tropes of connecting human and machine intelligence.

2nd Cylinder of Innovation: “Hey Can We Change the World!”

The possibilities for reimagining of the role of data in healthcare produced some of the most powerful “Hey Can We Change the World” moments for me at both Web 2.0 Expo and Hadoop World.  The slide above is from Esther Dyson’s brilliant Ignite presentation, “What you can and can’t learn from your genes?” are here,   Tim O’Reilly also brought up the powerful role real time data analytics can play in improving healthcare in his Hadoop World Keynote.  Also see Alex Howard’s post, 10 Lessons for Gov 2.0 from Web 2.0 for some more great, “hey we can change the world moments” at Web 2.0 Expo.  The keynote from Lukas Biewald of CrowdFlower and Leila Chirayath Janah of Samasource (screen shot below) in particular, is a provocative exploration of the future of work in the new ecologies of human and machine intelligence.

Changing the World When Our Lives Are Increasingly Shaped by Forces Invisible To Us?

Click to enlarge

Mike Olsen, CEO of Cloudera, noted that “the largest area of data growth does not come from humans interacting with machines; rather, it’s from machines interacting with each other” (see here in Minor Technical Difficulties).   One of the most interesting presentations at Web 2.0 Expo was Kevin Slavin’s, “Loitering on the Motherboard,” which, as Tim O’Reilly pointed out in his keynote at Hadoop World, is a talk that raises all kinds of questions about a system where big players are gaming the data for their own ends.

Kevin Slavin, a founder of Area/Code, notes the operating system of our mortgage, life insurance, the operating system of currencies and gold is now governed by machine to machine intelligence and algorithimic economies outside of human cognitive processes.  The markets are now legible only to bots in an algorithmic arms race with bots surveilling bots, and throwing off false information in a bid for counter-surveillance.  He showed some slides of the eery but beautiful visualizations of traces of the trading bots created from the Nanex API.

The screenshot above is from the Nanex: Crop Circle of the Day – Quote Stuffing and Strange Sequences.  “The common theme with the charts shown on this page is they are all generated in code and are algorithmic. Some demonstrate bizarre price or size cycling, some demonstrate large burst of quotes in extremely short time frames and some will demonstrate both…” This one is a zoom of the NSDQ “Wild Thing.”  Wild price/size repeater from NSDQ running at 1,000 quotes per second, effecting the BBO along the way (I love the great names Nanex gives the different patterns and traces produced by the trading bots).

Nanex supplies a real-time data feed comprising trade and quote data for all US equity, option, and futures exchanges. They have archived this data since 2004 and have created and used numerous tools to “sift through the enormous dataset: approximately 2.5 trillion quotes and trades as of June 2010.” May 6th 2010 (day of the flash crash), had approximately 7.6 billion trade, quote, level 2, and depth records.

Kevin points out that our lives are being shaped by criteria invisible to us and the old hackneyed tropes of machine to machine intelligence such a robots reading HUDs in English are long worn out.  The latter point is, perhaps, something for us augmented reality geeks absorbed in ideas of “making the invisible visible” to chew on.

Changing a world shaped by forces that are, increasingly, invisible to us presents a huge challenge.

But I had the glimmer of a, “Hey Can We Change the World” moment, when I attended Kevin Slavin founder of Area/Code’s presentation and had a conversation with him after his talk.  Could games take these complex economies as their subject matter?  The economies of  Farmville and games like WoW are not opaque at all, and these are environments with complex economic behavior, “where you can actually have enough data to understand what it is” – “it’s not so much about personal data.  It’s more about, like, aggregate behaviors.” “Games that can really model those, and play with those, and take those as the subject the way that Monopoly takes Monopoly as a subject could be really interesting.” Kevin made many fascinating points – more to come on this topic.

Photo by James Duncan Davidson, of Kevin Slavin speaking at Web 2.0 Expo NY, 2010, from the O’Reilly Conferences Flickr stream

Here is the beginning of our conversation:

Talking With Kevin Slavin

Tish Shute: You began your talk today about visibility and where some of the algorithmic masters of disguise went to work, after they had solved the math behind stealth bombers.  I thought perhaps you were leading into ideas about a reverse surveillance society.

But you surprised me, as I felt you made visibility itself kind of a non-issue by the end of your presentation and that counter surveillance became basically a time and speed issue.  Now I am not sure quite how to imagine a counter-surveillance society, something I try to think about…

Kevin Slavin: Well, let’s see.  There’s a couple ways to think about it.  I think one point is just that when we talk about counter-surveillance, we usually locate that as something that comes from  the bottom up, something that comes from the population. Think about the way the plane spotters discovered the CIA black rendition flights.

I think in general, when people talk about counter surveillance, or sousveillance, they imagine it as an inversion of the traditional relationship between the people and the state.

But that’s what’s interesting. What’s happening now, is that there are forms of surveillance and counter-surveillance that are in play beyond any human perceptual horizons. These forms are at their most sophisticated in financial services, in the markets.

If you were a bot, and could read the market legibly (which humans cannot), what you would see, effectively, are bots that are surveilling bots. Then you have bots that are throwing off false information in a bid for counter-surveillance. Many of the bots are, themselves, surveilling other bots; each one of them is trying to figure out what all the other ones are going to do. In essence, it’s an algorithmic arms race, and game theory has become concrete, since the theories are code, the code is action, and the action affects, let’s say: your mortgage.

And so, basically what you have is you have this series of algorithms that are all looking to discern each other, while also trying to prevent themselves from being discerned. I think of the tunnels under the trenches in WWI, tunnels to surveil the trenches, and then, later, tunnels to surveil the tunnels. There’s a few examples of this kind of thing.  But It’s especially strange when it’s computer code, and at the magnitude we’re seeing today.

All of it, as noted in the talk, accounting for 70% of all the trades in the market. 70% of the market trades are never touched by human hands or even seen by human eyes; they don’t move through a conventional cognitive process.  And that’s why you get things like the Credit Suisse algorithm, it was buying, selling 200,000 shares of stocks to itself over and over and over again. It was a bug and it slowed the market to a crawl.

Credit Suisse was fined, in essence, for failing to control an algorithm. Maybe that’s the first time an algorithm was treated like a human, in a way. As if the algorithm broke the law, and Credit Suisse was responsible for letting it do so. For me, that feels like a threshold event.

It’s not that humans never made mistakes when trading on the market. But when algorithms err, they err with magnitude.

The idea that we now have bugs in the United States market economy is really worth looking at.  If Apple can’t keep code bugs from the most simple iPhone apps in a closed and regulated ecosystem, I’m pretty certain we’ll have a lot more Credit Suisse type bugs in the future.

And that will be pretty interesting. There will be viruses, and the operating system they will operate on will be the operating system of the United States. The operating system of your pension, your house, your life insurance. The operating system of currencies and gold.

Tish Shute: I was hard-pressed by the end of your talk to think of like, “Well, what would be the equivalent of, sort of a people’s uprising to create a better fairer society in this kind of world where, really, the things that affect the key aspects of lives most are going on beyond human perception at an algorithmic level?”  But you made a pretty radical suggestion at the end…

Kevin Slavin: Well I think increasingly the markets have become delaminated from anything meaningful. First from goods, then from fundamentals, and now finally from homo sapiens. So that’s hard to fight.

It’s the race towards abstraction that makes it impossible to simply “resist.” The latest version in the long series of fiscal catastrophes was based on Wall Street finding goods that could be rolled up and sold with false valuations, but goods that would take a long time to fail. Mortgages are handy like that. It’s the tradition of extending the abstraction as long as possible, until finally the bill arrives and the banks fail. I don’t know if that’s something to rise up against or not. It’s like a rally against evil.

But really, I think the point is that it won’t be the people that rise up. It will be the financial services themselves that rise up. They’ll just detach completely.

That was harder to do with cotton or with wheat, with simple futures; they keep financial services tied to the ground.  So what we’re doing is creating increasingly complex financial instruments that are further and further removed from anything you can touch.  Like the way a mortgage is abstract. But, of course, the bottom line is that at the end of that mortgage lies someone’s home.

It’s said that Wall Street is now moving onto life insurance, because that’s going to take even longer to fail.  They’re doing the exact same thing. The word is that they are rolling up CDOs made out of crap life insurance policies, same way they rolled them up with crap mortgages a few years ago.

And those will probably take, I don’t know, 15 or 20 years to unwrap and unravel.  But what you see in the meantime, is that they are looking for things that are increasingly abstract, intangible, removed as far as possible from the experience of everyday life.

So maybe this is good. Maybe that’s financial services rising up. Lifting off. I think best case scenario now is that they actually leave humans alone altogether.  That, someday, they are just trading, effectively, completely arbitrary goods, the stocks could be anything at all, maybe for crops that no longer exist, and I’m just saying that then these bots would no longer affect what we do and what we are, it would just be a robot casino, an invisible paradise in the air.

People are the platform: How Games Can Be Engines of Innovation in Our Lives

See the video of Games that Know Where We Live here (screen shot above)

Kati London, Senior Producer, Area/Code, in her keynote showed how games that know where we live can shift players perspectives – from device aided augmented reality that can shift visual experiences of situated geolocal experiences to a kind of augmented reality that is aimed at shifting or changing a person’s social reality, e.g. the mayor badges in Four Square that change my relationship to the people and the place I am in, and augment engagement and reputation through socially driven consumer tie ins.

Area/Code has recently developed two games for the Knight Foundation that take people as the platform.  Macon Money, uses very simple games dynamics (for more see the video of Kati’s keynote) in a game designed to help “Knight’s continuing efforts to support revitalizing Macon and creating a vibrant college town.”

The other game that Area/Code has designed with the support of the Knight Foundation  is for the Biloxi and Gulf Coast community, a game called Battlestorm.  “The game’s purpose is to increase awareness about natural disasters and change the way people prepare for them.”

3rd Cylinder of Innovation: Build products, business models and entire industries.

Glympse – real-time, private location tracking

Julianne Pepitone, Yahoo! Finance, nailed the essence of Web 2.0 Expo, NYC, this year in her post, Web 2.0 Expo startups are big on neighborhoods, storytelling.  She writes:

“At the Web 2.0 Expo in New York City this week, executives from big sites like Facebook, Twitter and Pandora all spoke about industry trends. But the showcase of 27 startup tech companies stole the show.”

Listen carefully to Tim O’Reilly and Fred Wilson, Union Square Ventures, question their picks from the startup showcase at Web 2.0 Expo.  Also see this video of Fred and Tim discussing their conversations with all the start ups.  This  is one of the clearest public windows onto both how to present your company to VC, and how to figure out what are the most important questions for you as an entrepreneur  building a business in a world of data.

Glympse successfully pitches their “jet pony” strategy for a location based business, and is Fred’s pick.  They hold up well under pressure and answer Tim and Fred’s hard questions about how their start up will not get overtaken by an encumbent player with resources and market share before they can gain traction.   food52 responds to Tim’s probing about their strategy for business data analytics that he points out are vital if they want to survive with the small margins of ecommerce.  There is a list of all the participants in the start up showcase in Brady’s post here. hour.ly was the audience pick.

Shazam for Faces!

My favorite start up was a biometric service doing face, iris, and finger print matching, Tactical Information Systems.

Tim and Fred also liked them, and they have an interesting discussion about the merits or not of approaching your platform through a narrow first application as Tactical Information Systems are with WanderID -  an application to help identifying lost Alzheimer patients.  As Fred pointed out, they are potentially the Shazam for faces, so why start so small?

I  had asked TIS the same question when I met them in the “speed dating” session.  This is just their first toe in the water as they are a two person company at the moment. Their vision for their platform is big.  Mary Haskett and Dr Alex Kilpatrick, the founders of this quintessential jet pony for the algorithmic economies in the sky, are not only a partnership with the credentials to do a  Shazam for faces – see their bios here, they are the people I would want to be running a Shazam for faces!  They really get the consequences of living in a world of data – check out Dr Kilpatrick’s absolute killer Ignite talk, “Defeating Big Brother.” (screenshot below)

How Can Augmented Reality Add Value to the Real Time Internet/Data Operating System?

planefinder.net – an augmented reality app that lets you find information about planes by pointing your phone at the sky, “including flight number, aircraft registration, speed, altitude and how far away it is” (via MacLife).

The new opportunities in the algorithmic economies in the sky were center stage at Web 2.0 Expo and there are some interesting AR apps for the real time internet/data operating system emerging, like planefinder.net.  But Augmented Reality was still pretty low profile at Web 2.0 Expo (except that NVidia augmented reality demo attracted a lot of attention at the sponsors expo).  However, everyone working in the emerging industry of AR should recognize that apps big on “neighborhoods and story telling” are heading right up the AR street, and that platforms like Four Square and Pachube present enormous opportunity to explore the possibilities of AR.  And if augmented reality enthusiasts are not already paying attention to real time data analytics, and Hadoop, they should be (see this post for an excellent round up on Hadoop World).

At Hadoop World, Tim O’Reilly referenced the great tagline from the  IBM commercial:

“Would you be willing to cross the street — blindfolded — on data that was five minutes old? Five hours? Five days?”

As I have noted in several earlier posts – see here and here for starters,  we are just seeing the tools  for developing near field, vision based, mobile, social AR become widely available to developers, so there should be a new level of AR apps emerging through 2011.  There is a wonderful discussion in the comments of this post by Mac Slocum, “How Augmented Reality Apps Can Catch On,” between Mac, Raimo one of the founders of Layar, and Chris Arkenberg on what constitutes a platform for growth for augmented reality.

Mac’s post, the comments and Chris Arkenberg’s post on the latest edition of the Gartner Hype Cycle, that rather curiously placed Augmented reality almost at the peak of inflated expectations. really got me excited about exploring an idea I have been thinking about for a while, which is to get the AR community to discuss the Points of Control map.    See my discussion with Chris Arkenberg here, Platforms for Growth and Points of Control for Augmented Reality. The recording of  John Battelle’s and Tim O’Reilly’s webcast on Points of Control is posted here.

The interactive Points of Control map is an amazing tool to think with! Check it out in movements, territory and movements, acquisition mode.  There is a competition for the most interesting comment and most interesting acquisition suggestion.  The prize is a ticket to Web 2.0 Summit!

What is the Future of Social?

The recent “defection” from Google to Facebook – see Lars Rasmussen, Father Of Google Maps And Google Wave, Heads To Facebook,  is as MG Siegler of TechCrunch points out, “the biggest one since Chrome OS lead Matthew Papakipos made the same jump in June” (TechCrunch also notes “current Facebook CTO Bret Taylor was heavily involved in the launch of Google Maps”).

These moves have drawn my particular attention as did Bret Taylor’s response in his conversation with Brady Forrest at Web 2.0 Expo to Brady’s question, “How soon until we get the Facebook firehose?”

If you have been reading Ugotrade you already know how important I think an open, distributed, standard for real-time communications such as the very innovative Wave Federation Protocol could be for AR development  -  see ARWave and my presentation at MoMo13, Amsterdam last year, The Next Wave of AR: Mobile Social Interaction Right Here, Right Now!

The anticipated release of  Wave in a Box, has raised hopes in the developer community that  WFP will soon become easier to work with, and hopefully more widely adopted.  Like many others, I wonder what will happen to Wave in a Box now?

But the innovation of Wave is deep and broad (and as many have pointed out hugely ambitious).  Perhaps the boldest attempt yet to innovate both at the low level of architecture (where Google is so powerful) and at the high level of the Mark Zuckerberg, “big idea,” which as Tim O’Reilly notes is, “What is the future of social?” MG Siegler noted Facebook Groups Is Sort Of Like Google Wave For Human Beings.

But I deeply hope that the open, distributed standard part of the Wave big idea is not lost in the mix here.

Fourth Cylinder of Innovation: Keep the Ecosystem Going, Create More Value than You Capture

The Points of Control map is interactive, so please click here or on the image above for the full experience.

Tim O’Reilly points out that there is a worrisome dark side to the Points of Control Map – see Tim’s keynote here.  To paraphrase some or his points:

There are companies on the map that are forgetting to think about creating a sustainable ecosystem.  Rather than growing the pie, they are trying to divide up the pie and that threatens to cause the fourth cylinder of innovation to misfire.  This fourth cylinder is essential to the ecosystem.

Tim O’Reilly looks back to the lessons of the personal computing industry which was incredibly vital and creative, and lots of people made money until a couple of big players “sucked all the air out of the ecosystem” and innovation had to go elsewhere.

The Power of Platforms is to create value not just for your company but for other people.   Create value for yourself by creating value for other people.  Tim O’Reilly used the wonderful example of  Henry Ford inventing the weekend so that there would be enough people with time and money to buy his mass produced cars.  Think about building the ecosystem that will support the future your are going to build.  Grow the pie rather than cut up the pie.  This will be the vital fourth cylinder of innovation in a Web Squared world.

Tim O’Reilly has long proposed that  Web 2.0 is all about harnessing collective intelligence,  But as Gartner predicts, “By year end 2012, physical sensors will create 20 percent of non-video internet traffic.” Yet another previously unevenly distributed future is going mainstream, and if you haven’t read it already, now is the time to read this paper by Tim O’Reilly and John Batelle, Web Squared: Web 2.0 Five Years On.

The Consequences of Living in a World of Data

To bring this very long post to a close!  Here are just a few of the key questions re The Consequences of Living in a World of Data that Tim O’Reilly raised during his keynote for Hadoop World:

“How would we solve the problem of digital identity in the age of sensors? (Our smart phones are able to know their users by the way they walk – their gait!)

“How will we input data when our devices are smart enough to listen on their own?”

“How should we think about privacy in a world where data can be triangulated?”

“We are moving to a world in which every device generates useful data, in which every action creates information shadows on the net.”

“Shouldn’t we regulate the misuse of data rather than the possession of it?”

“How do we avoid a data arms race?”

“Create more value than you capture.”

Adam Greenfield’s new book, The City Is Here For You To Use, is coming soon (photo above by Pepe Makkonen is from Adam Greenfield’s Flickr stream). Adam told me:

“I’m aiming at a free v1.0 PDF release on 05 June 2009, with the book shipping as quickly thereafter as humanly possible. There will be a version zero or public alpha in about six weeks.”

I am not good at waiting for books I really want to read to arrive. But, on the upside, it brings out my already pretty highly developed investigative instinct. So when Adam very generously agreed to do an interview, impatience turned into delight in tasting what is to come. And Adam is encouraging this kind of engaged anticipation. He writes (see post) that The City Is Here For You To Use, is shaping up:

“as something of an unbook avant la lettre. It’s why we’ve [Nurri Kim and Adam Greenfield] always insisted on keeping you in the loop as to the book’s fitful progress, it’s why I take every opportunity to test its ideas here, it’s why I make explicit the fact that your response to those ideas is crucial to their evolution and expression. And it’s why, even though the process is inevitably going to result in a static, physical document as one of its manifestations – and hopefully a very nice one indeed – we’ve committed to offering a free and freely-downloadable Creative Commons-licensed PDF of every numbered version of The City, from zero onward.

You buy the book if you want the object. The ideas are free.”

I found the opportunity to ask Adam questions about some of his subtle renderings of technology, culture, and being in urban environments challenging and very illuminating.  Although I definitely get the feeling I am asleep at the wheel on some of the critical areas he is thinking and writing on.

Knowing the depth and range of Adam’s thought in his seminal book, Everyware, and his blog, Speedbird, before I began the conversation I asked Adam to point me to some of his posts that reflect key ideas he is working on at the moment (Adam has recently posted The City Is Here: Table of contents).  Adam directed me to these three posts.

Antisocial networking

More songs about context and mood

Messenger, space, messenger body, messenger mesh

I may ramble and diverge, as is my nature, but these posts inspired many of the questions I ask.

Adam is currently head of design direction for service and user-interface design at Nokia and living in Helsinki, so I did not have the opportunity to do the interview in person. But I have glimpsed Adam’s world through his Flickr stream and some of these images have found their way into this post. But I suggest you browse Adam’s photography for yourself. I cannot do justice to the thousands of nuanced perceptions of cities, networks and publics you will find there. In the meantime, here are three glyphs of Adam Greenfield that I liked a lot.

“My favorite shoes,” “My favorite chair” and photo by Adam Greenfield, “Favoriteplace”

A Conversation (in gdoc) with Adam Greenfield

Tish Shute: Could you explain a little about the evolution of your thoughts on urban environments, ubicomp and interaction design? What shifts in your thinking have taken place over the last few years re the dawning of the age of ubiquitous computing? It is a couple of years now since Everyware, what aspects of the uptake of Everyware have most surprised, disappointed or inspired you? Which of the many thesis you discuss in Everyware have become the most crucial for The City Is Here For You To Use?

Adam Greenfield: You know, there’s a little passage in the liner notes to the second Throbbing Gristle album that I always think of when I’m asked questions along these lines. As part of their stance, they’d adopted the dry tone of a corporate annual report, and the preamble began by saying, “Since our last report to you, many things have changed. Indeed, it would be foolish to assume that it could be otherwise.” And I think that’s just exactly right: the world keeps moving, and the positions we’d staked ourselves to not so long ago may no longer be correct, or even relevant, to the one we find ourselves inhabiting now.

So, first, I think it’s important to cop to all the places in Everyware where I just outright got things wrong. There’s a passage in Thesis 50, for example, where I unaccountably mock the idea that “the mobile phone…will do splendidly as a mediating artifact for the delivery of [ubiquitous] services.” OK, this was admittedly written in a pre-iPhone world – and was correct for that world – but you can really see my parochialism showing here. It took the iPhone to make the proposition as blazingly self-evident to me in North America as it had been for quite some time to folks in Europe and Asia.

Having said that, though, I think I’m justified in taking a little pride in what the book got right. The broader trends the book set out to discuss – the colonization of everyday life by information processing – well, take a good look around you. And so one of the points of departure for the new book is taking everything posited in Everyware as a given: the urban environment, and most everything in it as well, has been provisioned with the kind of abilities you mention. So what now?

How do you go about designing informatic systems so they don’t undermine the wonderful things about cities? How do you design cities so they can incorporate networked informatics to greatest advantage? How, especially, do you accomplish these things when the disciplinary communities involved barely speak the same language? And how do you keep everyone’s eyes on the prize, which is the ordinary human being asked to make sense of these new propositions? These are the questions The City Is Here For You To Use sets out to address.

Adam talking about the “Le Long Ici” in Paris (also see Adam’s post, “The long here and the big now”)

TS: You mention that the hardest parts of  producing The City Is Here For You To Use wasn’t “keeping on top of all the emergent manifestations of urban informatics, or even developing a satisfying spinal argument about their significance” but getting the voice right.  It seems that now is the perfect time for a book that would really speak to a wide audience.  But also it seems that the city that is here for you to use is manifesting quite differently in different parts of the world?  You seem to be somewhat of a nomad, Japan to NYC to Helsinki.  Can putting together different views of urban informatics give us more depth perception on the emergence of ubiquitous computing?

AG: There’s no question in my mind that the long-term experience of everyday life in Tokyo, New York, and now Helsinki has been an invaluable asset to me, as I imagine it would be to anybody interested in thinking or writing about the networked city. It’s given me a certain amount of parallax, you know? And that, in turn, throws a really interesting light onto how the selfsame technology can appear in substantially different guises in different social contexts.

But explaining those things – those complicated, delicate negotiations – getting them right, doing them justice, doing so in a way that doesn’t dumb anything down, and still remaining accessible? It’s a challenge, let me tell you. You want to remain approachable and humane, but you also want to explain things like different jurisprudential takes on property, or how advocates of RESTful architectures think that REST is the reason why Internet adoption spread as rapidly as it did. If you want to enjoy even one chance in a hundred of getting your message across, you’ve got to start with an understanding that those subjects are MEGO territory for most people – whether they hail from Shibuya, Shoreditch or San Pedro.

Everyware icons: Information processing dissolving into behavior (Icons inspired by Timo Arnall; design by Adam Greenfield and Nurri Kim).  [Adam notes on his Flickr page that he tweaked these icons as section headers for Everyware]

TS: Could you explain more about what you term “onto” and “ontome” and how this differs from spimes and spime wrangling?

AG: You know, I never did get to develop that idea as much as I would have liked. In my mind, at least, “ontome” referred to the totality – the global environment of addressable, queryable, scriptable objects. (An “onto,” then, would be any given such object.) I guess I was looking for words that would do two things: allow us to distinguish between the instantiation and the class, and leave us with a better word than “spime.”

TS: When you say better word than spime this is this because….

AG: Euphony, primarily. : . )

TS: When I first used the Android app,  Wikitude, on Broadway, NYC – a street I have traveled thousands and thousands of times, and it offered up new information about itself, it was definitely an “OMG this is big!” moment for me. Like the first time I clicked on a screen and Amazon sent out a book in the early nineties (something so ordinary now it seems impossible that it was exciting but I remember it was to me!). But if I understand your post here correctly, isn’t Android with compass the first easy-to-use context-aware mediator for wrangling onto, ontome and spimes?

AG: Wikitude sure looks pretty impressive, and maybe even useful. But I would never, ever call it “context-aware.”

To my mind, at least two more things would need to happen before we could comfortably think of it a “context-aware spime wrangler.” First, the buildings and other public objects around you would actually have to be spimy – they’d have to report something of their past and current state to the network. And then, some application running on your phone would somehow have to cross-reference that state information with some fact about your current state of being, and deliver you relevant information.

So, let’s take your Wikitude example. You’re walking down Broadway and you pass an unfamiliar building, and for whatever reason you want to know more about it. Your phone pings the building’s dynamic self-description, and it replies to the effect that Andy Warhol had his Factory there between 1973 and 1984. If Wikitude chooses to share this particular piece of information with you, and not some other potentially germane factoid from the building’s history, on the strength of the fact that “The Velvet Underground and Nico” was in your last.fm playlist? That would constitute some small measure of context-awareness.

But you see how hard we had to try just to come up with an example, how forced it is, how so-what. And I have to say that – short of some infinitely supple system that really could model your innermost desires ahead of real time, and present appropriate responses to them – most so-called “context-aware” applications and services are like this. They’re either trivial, or wildly overambitious.

Maybe we don’t need for things to be context-aware for them to be useful, anyway. Certainly a great many objects in the world are starting to report their own status, and many more will do so in the fullness of time. And for the most part, all you’ll need to avail yourself of them is a Web browser running on a device that knows where it is in the world. An iPhone or an Android device will work splendidly – I called the iPhone “the first real everyware device” the day it came out and I was able to play with it for the first time – and in that way, the answer to your question is “yes.” Not to be longwinded or anything. ; . )

This Object has imperceptible properties. [Adam notes on his Flickr page: “This is a custom RFID-enabled transit pass that Timo Arnall had made up for me here in Seoul. I’ve (clumsily) tagged it with the icon that Nurri and I developed to represent just such emergent situations as this in the everyware milieu – that there’s no way for anyone to understand that this object has puissance beyond the obvious simply by examining it.”]

TS: It seems that  we are just at the beginning of understanding how to create networks of spimes (e.g. Pachube). Gavin Starks of AMEE (”the world’s energy meter”) once suggested to me that AMEE could be described as a facilitator of networked spimes (everything will have an energy identity). I think you may be familiar with AMEE because you keynoted next to Gavin at Xtech 2007.

I would be interested to hear your thoughts on AMEE?

When you discussed onto and ontome in this post, you noted:

“The greater part of the places and things we find in the world will be provided with the ability to speak and account for themselves. That they’ll constitute a coherent environment, an ontome of self-describing networked objects, and that we’ll find having some means of handling the information flowing off of them very useful indeed.”

Is the idea of “energy identity” that AMEE proposes an ontome? 

AG: See below for a précis of my feelings regarding environmental/sustainability initiatives, AMEE included. Uh…is AMEE an ontome? No. There’s just one ontome, and it’s coextensive with what folks now call the Internet of Things. It sounds like individual AMEE sensors would be “ontos.”

But I think the difficulty we’re having is a pretty good indicator that the terminology is more trouble than it’s worth. Sometimes a coinage, as satisfying as it may be lexically, just doesn’t work for people. These days I’m trying to get out of the neologism trade.

TS: I know when Usman Haque talks about Pachube he talks about spimes and spime wrangling. I asked Usman for his thoughts on spimes and onto/ontome and he gave me some comments.

Usman Haque: I think I had somehow missed the conversation about onto and ontome but backtracked through blog posts to piece it together (unfortunately some posts at v-2 and Studies & Observations no longer exist!). There are a couple of things that have made me uncomfortable about the word ’spime’: (a) the fact that it might be too easy to confuse with an “object”. A ’spime’ should also encompass relationships between things, and not just the “thingness” itself. (b) the sound of it (as Adam noted above). But then I am reminded of that horrible gooey interface used to plug into people in eXistenZ – it somehow seems appropriate that it should be a horrible gooey word, and not something that can disappear politely… So I like onto/ontome because it speaks to my first concern about ’spime’; but my second concern, it turns out, is not the problem I thought it was, and so onto/ontome might be… ahem… too euphonic! On the question of this thing people are calling the “Internet of Things”, I’ve tried in lectures to reframe it as the “Ecosystem of Environments”. Further, Vlad Trifa makes a delicious point that just as ‘web’ is different from ‘internet’, so too should we consider the “Web of Things” rather than the “Internet of Things”, something I agree with.

TS: It seems like this point about the difference between “the web of things” and the “internet of things” is pretty important?

AG: The parallel distinction between Web and Internet sure is! They’re two completely different things, right? And http is far from the only protocol that runs over the Internet. Now, as to what Vlad means by extending this particular distinction to the domain of networked objects, I don’t yet know, I haven’t had time to check it out. But sure, in principle I’d totally be willing to go along with the idea that there’s a meaningful distinction between two environments named that way.

No information is collected here; network dead zone

TS: I was just going over Greenfield’s principles of ubiquitous computing.  I am not sure that I see any current manifestations of ubicomp that hold to these priniciples yet?

AG: Oh, sure there are. Look at the work Tom Coates has done on Yahoo!’s Fire Eagle; look at Dopplr. And look at some of the steps other, less compassionate developers (e.g. Facebook) have been forced to take by their own users.

Look, those principles are just codifications of common sense and basic neighborly virtues, expressed in language appropriate to the domain of application. The best, smartest and most ethical developers have never needed guidelines to do the right thing. But especially inside companies and other complex organizations, people who want to implement compassion in their design of a technical system may occasionally find it useful to have some color of authority to invoke in their struggles. That’s all those five principles are there for, and I’m well satisfied that people have been able to use them that way.

Boffi’s take on the smart home- photo by Adam Greenfield

TS: In your post, More Songs About Context And Mood, you suggest a direction for interaction design that you point out is not far from Yvonne Rogers’ ideas in “Moving on from Weiser” about a switch in goal of ubicomp from Weiser’s vision of calm living (”computers appearing when needed and disappearing when not”) to engaged living – ubicomp technologies not designed to to do things for people but to help people engage more actively in things that they do (ensembles, ecologies of resources).

You also suggest interaction designers should be:

“parsimonious about the interaction design challenges our organizations do take on, with an eye toward reducing the complications of context (and the attendant opportunities for default, misunderstanding, misfire, time-wasting, and humiliation) to some manageable minimum.”

As you have pointed out, “we don’t do “smart” very well yet.” But paradoxically smart grids, smart homes, smart products etc. etc. are ubiquitously coming to market right now.

Yvonne Rogers suggests interaction designers should be:

moving from a mindset that wants to make the environment smart and proactive to one that enables people, themselves, to be smarter and proactive in their everyday and working practices

What areas might interaction designers most productively direct their attention towards?

AG: You note that things called “smart homes” and “smart products” are coming onto the market, and that sure would seem to be the case. But as to whether or not these things are genuinely smart, we don’t have anything more to go on than the marketing department’s word. I think you can already see that I tend to take language very seriously, and I really don’t uses like the “smart” here, or the “aware” in “context-aware.” They overpromise, they cannot help to set us up for failure and disappointment.

You know what I’d really like to see interaction design wrestle with? I would love to see a rigorous, no-holds-barred examination of the complexities of the self and its performance in everyday life, and how these condition our use of public space (and personal media in public space). I would love to see the development of ostensibly “social” platforms informed by some kind of reckoning with issues like vulnerability, dishonesty, the fact of power dynamics. In other words, before we deign to go about “helping” people, wouldn’t it be lovely if we understood what they perceived themselves as needing help with, and why?

I’d also pay good money to see talented interaction designers turn their efforts toward tools for the support of deliberative democracy, for the navigation of complex multivariate decision spaces, and for conflict resolution.

Locative is a mood – photo by Adam Greenfield

TS: I know you said this would take too long to explain but I couldn’t help noticing that you seem to be, perhaps, skeptical about the role of everyware can play in sustainable living and yet, it seems at the moment, in the hacker and business communities at least, the role of everyware in reducing carbon footprint/energy management etc, is the great green hope?

Will everyware enable or hinder fundamental changes at the level of culture and identity necessary to support the urgent global need – “to consume less and redefine prosperity?”

AG: I’m not skeptical about the potential of ubiquitous systems to meter energy use, and maybe even incentivize some reduction in that use – not at all. I’m simply not convinced that anything we do will make any difference.

Look, I think we really, seriously screwed the pooch on this. We have fouled the nest so thoroughly and in so many ways that I would be absolutely shocked if humanity comes out the other end of this century with any level of organization above that of clans and villages. It’s not just carbon emissions and global warming, it’s depleted soil fertility, it’s synthetic estrogens bioaccumulating in the aquatic food chain, it’s our inability to stop using antibiotics in a way that gives rise to multi-drug-resistance in microbes.

Any one of these threats in isolation would pose a challenge to our ability to collectively identify and respond to it, as it’s clear anthropogenic global warming already does. Put all of these things together, assess the total threat they pose in the light of our societies’ willingness and/or capacity to reckon with them, and I think any moderately knowledgeable and intellectually honest person has to conclude that it’s more or less “game over, man” – that sometime in the next sixty years or so a convergence of Extremely Bad Circumstances is going to put an effective end to our ability to conduct highly ordered and highly energy-intensive civilization on this planet, for something on the order of thousands of years to come.

So (sorry again, Bruce) I just don’t buy the idea that we’re going to consume our way to Ecotopia. Nor is any symbolic act of abjection on my part going to postpone the inevitable by so much as a second, nor would such a sacrifice do anything meaningful to improve anybody else’s outcomes. I’d rather live comfortably – hopefully not obscenely so – in the years we have remaining to us, use my skills as they are most valuable to people, and cherish each moment for what it uniquely offers.

Maybe some people would find that prospect morbid, or nihilistic, but I find it kind of inspiring. It becomes even more crucial that we not waste the little time we do have on broken systems, broken ways of doing things. The primary question for the designers of urban informatics under such circumstances is to design systems that underwrite autonomy, that allow people to make the best and wisest and most resonant use of whatever time they have left on the planet. And who knows? That effort may bear fruit in ways we have no way of anticipating at the moment. As it says in the Qu’ran, gorgeously: “At the end of the world, plant a tree.”

Biowall! - photo by Adam Greenfield

TS: In your post “Antisocial Networking,” you make some telling comments on the sorry state of social networking systems.

But you do also state:

Which of your concerns is social networking more central to than you might wish and why?

AG: Well, you know I’m interested in social interaction, interpersonal behavior, and in how these things play out in networked environments. There’s virtually no way for me to avoid dealing with Facebook, as wretched as I think it is.

Facebook is pretty hegemonic, in that its reach and influence extend further than the universe of people who use it. I bump up against it constantly, in a few different ways. People send me links I can’t access, because I’m not on Facebook. People spend time and energy trying to convince me that I’m really missing out, because I’m not on Facebook. The last few months, there’s even been a few people who feel justified in expressing some kind of exasperation, that they’re really pissed off…because they can’t find me on Facebook. It’s become the sovereign interface to any kind of life in public, and as a result a great many people don’t question its modes, tropes and metaphors.

So when it comes time to build some kind of situated interpersonal mediation framework, some kind of intervention in the fabric of the city, those are the tropes they reach for: accounts, profiles, friend counts, friendings and unfriendings, nudges and pokes. And as a member of a team tasked with the design of such systems, as a potential user of them, and certainly as someone exposed to the social rhetoric flowing downstream from their use, you bet these tropes become central to my concerns.

But what if we admitted that Facebook and the whole paradigm it’s built on are broken? What would things look like if we started from a more sensitive understanding of the interaction between self and others? Say, the understanding Erving Goffman was offering us as far back as the late 1950s? Then you’d understand the need for provisions like a “backstage,” a place to swap out one mask for another, the ability to present oneself differently to different communities and networks. That’s what I’m interested in exploring.

TS: Social networking systems in their current form are crude and express a very narrow bandwidth of human relationship. But already people are connecting everyware’s networked social acts to existing social networking systems. At the ITP winter show there was kickbee – networked fetal communication (and tweetmobile which used twitter as an acctuator for an ambient display) and green everyware (energy monitoring) is showing up in a number forms on existing social networks. But rather than just hooking up everyware to these existing flawed social network systems, does everyware require a reimagining of networked social interactions and social networking systems?

AG: That’s a great question, and I think the answer is clearly “yes.” It’s one thing to confine the consequences of that brokenness to the Web, and entirely another to let it bleed out into the world.

Does that mean any such reimagining is going to happen, that people will somehow refrain from plugging real-world outputs into these terribly flawed frameworks? Not a chance in hell. It’s too late to put a fence on that particular cliff. But maybe there’s still time to park an ambulance in the valley below.

“A graphic representation of a portion of the Earth’s surface, as seen from above” – photo by Adam Greenfield

TS: I saw you tweet that you met Usman Haque from Pachube recently. What do you find most interesting about Pachube and EEML? Will you design a project for Pachube to push the conversation further?  Did Usman ask you to take a role in the future of Pachube. How does Pachube enable the vision of The City Is Here For You To Use? I could go on for ever with questions,  so please do tell!

AG: OK, I should probably reiterate that my fundamental interest is in people, and in what they choose to make and do with technology, not the technology itself. For the last few years, I’ve particularly been trying to understand how people interact with each other and with the urban environments around them when those environments have been provisioned with the ability to gather, process and take action on data. And this is how I come about my interest in what Usman is up to with Pachube, because those “gather,” “process” and “take action upon” functions are generally accomplished by different systems, designed by different groups of people, at different times and to different ends. What Pachube aims to do is make the difficult and not-particularly-glamorous work of connecting these pieces a whole lot easier.

Think of it as a step toward enabling the ontome, this so-called Internet of Things we’ve been talking about, the same way basic protocols like HTTP and HTML enabled the wildfire spread of the Internet we’re familiar with. What Pachube offers is a way – a relatively straightforward and self-explanatory way – to plug any given compatible input into a similarly compatible output. So if you’ve got an air-quality sensor or a soil-pH sensor or a personal biometric monitor, you can plug it into Pachube, and someone else can grab the data those things generate and use it to drive a visualization, or the state of a physical system like a window, or whatever else they can imagine. It’s as close as anyone’s yet come to providing a plug-and-play backbone for the creation of responsive environments.

And I think it’s absolutely brilliant that it’s designed to work with Arduino and Processing, two lightweight, open-source frameworks that hobbyists and researchers (and even one or two more serious developers) around the world are already using to build things. (Arduino’s a kit of parts for doing basic physical computing – using data to drive lights, motors, and other actuators that have effect out here in the world – while Processing is a very accessible language to do dynamic and interactive graphics for screen-based media). Given both its openness and modularity, and its willingness to build on top of the very popular frameworks that already exist, I’m very excited to see what people make of and with Pachube.

I have to be honest and admit that personally, I couldn’t really care less about the environmental angle, for reasons that I went into at embarrassing length above. What I’m engaged by in Usman’s work is the idea that Pachube is helping to create an open platform for people to share data more readily. And while, no, he hasn’t explicitly asked me to take any particular stake in things, I’m always happy to lend a hand in whatever way would be most useful. I think it’s a project worth supporting.

As to how Pachube enables some of the ideas in The City Is Here, the answer has to do with the book’s call for every “public object” – every lamppost, bus shelter, commercial façade, and so forth – to support an open API. Something’s got to string all those objects together, present them to people as resources to be taken up and used, and Usman’s offered us a critical first step in that direction.

TS: Usman suggested, it might be interesting to ask you about “the tension between ‘could’ and ’should.’

Usman Haque: There are a whole bunch of things that we “can” do, technologically speaking; how do we decide what we ’should’ do, as we find ourselves in an age where we can build almost anything we can imagine…? particularly with reference to technology/privacy/security triumvirate. e.g., leaving aside that the majority of the world is *not* in the technology ‘paradise’ that we’re in, here in the west, only a small fraction of people are currently producing the technology that the rest of us use; one aim is to get people more engaged in the productive process, but, in a sense that will also mean the whole wide ecosystem of technology will be even bigger, both “good” stuff and “bad” (that qualification firmly placed on how it’s used), as opposed to now when we can focus on quite specific things that government & industry are doing and saying “that shouldn’t be happening….”. part of this relates to something adam said on his blog  in the comments (see here).” 

AG: I think the first part of answering that question has to involve figuring out who “we” are in any given situation. A “we” composed of seven Helsinki-based Linux developers would most likely arrive at very different answers than the United States Air Force Materiel Command or Samsung’s board of directors, right? So clearly, a first challenge is getting to some kind of pragmatically useful alignment between those local and occasionally even painfully parochial perspectives with what’s best for the Big We. And this challenge is only going to become more vexing as the ability to imagine, design, build and deploy informatic componentry gets more and more widely distributed. In this respect the spread of simple, modular, low-barrier-to-entry tools only makes things worse!

The primary issue that I can see here is that the inherent clock speed of technical development is so very much faster than that of any meaningful deliberative process “we” might bring to bear on it. A concomitant concern is that the sources of technical innovation and production are now so widely distributed that you can be reasonably certain that somebody, somewhere will implement any given technically feasible idea, no matter how offensive, poorly thought-out, socially disruptive or frankly stupid. A public toilet you have to SMS to unlock and use? A “Friend Finder” visualization with high locational precision and no privacy features whatsoever? A first-person rape-simulation “game”? A clunky brown iPod knockoff? Somebody thought each one of these things was worth the time, expense and effort to actually go about making it. They exist.

But I’m pretty old-fashioned in some ways, in that I think the good old Habermasian idea of the public sphere still has some life left in it. And I think it should be self-evident by now that there’s no necessary contradiction between even the newest (cough) “social media” and the formation of such a sphere. So you’ve provided a forum, and in it I get to express my belief that these things are stupid and pointless and probably should not have been built. And if somebody gets all het up about that, they can argue right back at me in comments. And eventually one or another of these positions begins to tell, in terms of regulation, legislation, and other tools of the juridical order, in terms of protest campaigns or organized boycotts or litigation…in terms of nonexistent sales!

There’s nothing new in any of this, of course, though indubitably some of the dynamics are amplified or accelerated by e-mail, Twitter and YouTube. My main contention is that informatic technology now has such deeply pervasive implications, and for things like presentation of self that previous waves of technical development barely touched, that “we” as societies need to be very much more conscious of the consequences before committing to any one course of action.

I should also point out that I do not, at all, believe that we’re “in an age where we can build almost anything we can imagine,” though I might buy “…two or three of almost anything we can imagine.” On the contrary, as I implied above, I think the global constraints on our ability to operate freely are already becoming quite evident, and will continue to grow teeth over the next few decades.

TS: Also Usman  added …

Usman Haque: ..where Adam said: in this regard, I very much *do* have a problem with “just showing up.” — something I feel that as well. but i always wonder: What happens when one appears to be mandating participation…?

AG: Look, I happen to have a strong – maybe some would say obnoxious or hyperactive or overdeveloped – sense of personal responsibility and accountability. I think one is basically committed to some measure of responsibility for the commonweal simply by surviving to the age of majority. The choice of how, particularly, to discharge that responsibility can only be yours and yours alone, but it can’t be ducked or gotten around without severe and entirely predictable consequences. So to Usman I’d respectfully suggest that I’m not the one mandating participation. Life is.

TS: It seems we have grown accustomed to striking a Faustian bargain on the internet today -  in order to share and distribute parts of our identity we are expected to give up key information to one site to store and disperse our data. I took part in a discussion with David Levine, IBM and Eben Moglen on privacy last year.  And Eben Moglen gave a succinct description of the elements of privacy and how they have been treated in the American Constitution that is, I think, relevant to unpacking some of the challenges of ubiquitious computing. Here are some extracts from that conversation where, Eben notes:

there are three elements that are mixed up in privacy and we tend not to notice which one we are talking about at any given moment.

There is secrecy – that is the data should not be readable by or understandable by anybody except me or people I designate. There is anonymity which is the data can be seen by anybody but about whom it is should be knowable only by me or people that I designate. And there is autonomy which isn’t about either secrecy or anonymity but which is about my right to live under circumstances which reinforce my sense that I am in control of my own fate. And this form of privacy is actually the one we talk about in the constitutional structure when we talk about the right to get an abortion or use birth control.

“Anonymity” is a condition that is a deep structuring characteristic of the internet as you, Lessig and others have commented on.  And frequently we are promised (questionably) “secrecy” or anonymity as privacy protection by services handling our data on the internet.  But Eben (one of the US’s great constitutional lawyers) points out that “autonomy” is a key form of privacy in the  US constitutional structure that is often compromised in situations where our digital selves may constrain our non-digital selves.

The real issue here is about the forcing of choices on us…digital aspects of identity can quickly acquire an inflexibilty that constrains our non-digital selves.

I see again and again the ways in which people now find themselves unable to make certain life choices easily because there digital self has acquired an inflexibility that constrains their non-digital self.

As we go beyond the end to end internet and we lose the structuring characteristic that has privileged anonymity: How do you see these three elements of privacy, anonymity, secrecy and most importantly autonomy, being worked out in a networked world beyond the end to end internet?

Are there any new structuring characteristics that could privilege autonomy? (which Eben indicates is linked to having a flexible identity).

AG: If we accept for the moment a definition of autonomy as a feeling of being master of one’s own fate, then absolutely yes. One thing I talk about a good deal is using ambient situational awareness to lower decision costs – that is, to lower the information costs associated with arriving at a choice presented to you, and at the same time mitigate the opportunity costs of having committed yourself to a course of action. When given some kind of real-time overview of all of the options available to you in a given time, place and context – and especially if that comes wrapped up in some kind of visualization that makes anomaly detection and edge-case analysis instantaneous gestalts, to be grasped in a single glance – your personal autonomy is tremendously enhanced. Tremendously enhanced.

But as to how this local autonomy could be deployed in Moglen’s more general terms, I don’t know, and I’m not sure anyone does. Because he’s absolutely right: Bernard Stiegler reminds us that the network constitutes a global mnemotechnics, a persistent memory store for planet Earth, and yet we’ve structured our systems of jurisprudence and our life practices and even our psyches around the idea that information about us eventually expires and leaves the world. Its failure to do so in the context of Facebook and Flickr and Twitter is clearly one of the ways in which the elaboration of our digital selves constrains our real-world behavior. Let just one picture of you grabbing a cardboard cutout’s breast or taking a bong hit leak onto the network, and see how the career options available to you shift in response.

This is what’s behind Anne Galloway’s calls for a “forgetting machine.” An everyware that did that – that massively spoofed our traces in the world, that threw up enormous clouds of winnow and chaff to give us plausible deniability about our whereabouts and so on – might give us a fighting chance.

TS: The concept of autonomy is signaled clearly in the title you have chosen for your next book, The City Is Here For You To Use, and is a theme of all your writing!  While you talk about many of the possible constraints to presentation of self and potential threats to a flexible identity that ubicomp poses, your next book signals optimism. What are your key grounds for optimism?

AG: It’s not optimism so much as hope. Whether it’s well-founded or not is not for me to decide. I guess I just trust people to make reasonably good choices, when they’re both aware of the stakes and have been presented with sound, accurate decision-support material.

Putting a fine point on it: I believe that most people don’t actually want to be dicks. We may have differing conceptions of the good, our choices may impinge on one another’s autonomy. But I think most of us, if confronted with the humanity of the Other and offered the ability to do so, would want to find some arrangement that lets everyone find some satisfaction in the world. And in its ability to assist us in signalling our needs and desires, in its potential to mediate the mutual fulfillment of same, in its promise to reduce the fear people face when confronted with the immediate necessity to make a decision on radically imperfect information, a properly-designed networked informatics could underwrite the most transformative expansions of people’s ability to determine the circumstances of their own lives.

Now that’s epochal. If that isn’t cause for hope, then I don’t know what is.

Newson Obamanook – photo by Adam Greenfield, “The fact that it was one of the happiest days of my adult life may have colored my appreciation of this space. A bit, anyway.”

TS: In your writing you seem to imply that we will not find answers to our new relationship with Everyware by transposing the internet onto things for convenience’s sake but rather like the bike messengers -  we must explore the rich and complex terrain of the city that is ours to use in a give an take relationship.  Through our own exertions we find- how “anything reasonably smooth and approximately horizontal can become a thoroughfare,”  rather than be served up the city as something for us to consume.

You seem to be suggesting our city becomes ours to use because of the way we use it in our personal journeys -like “the messenger subconsciously maps the contours of an economic geography – known sources and sinks of courier assignments, or “tags” – and a threat landscape, this latter comprised of blind corners, cable-car and metro tracks, and traffic lanes.

But bike messengers are the lone ranger of our big cities. Others surf the city in tribes that ride the roiling tides of highly networked information together. How are the “natural” gestures of these tribes, e.g. day traders, who yoked to the tracings of a hive mind, part of the city that is here for us to use?  I thought the comment @ginsudo made shortly after joining Twitter and setting up TweetDeck particularly poignant:

“watching Tweetdeck is like watching stock market of your personality ebb and flow. needs analytics to maximize inherent self-involvement.”

But, for many of us our work has more in common with the day trader than the bike messenger, and are we pretty hooked on the ever growing possibilities for “contact” and identity sharing/construction, social media has produced  (with all the”Here Comes Everybody,” C. Shirky, benefits and risks).  Early theorizing of a “calm,” invisible” ubicomp seems out of synch with the excitable, active, engaged, contact driven, “users” that are watching stock market of their personality (or personal brand) ebb and flow.

How will these excitable/exciting processes of contact and identity sharing that have captured of a pretty large segment of popular imagination (not confined to the West -services like Gupshup does much of the same curating, linking and distributing of identity that web based social media does in SMS) be/ or not be part of The City Is Here For You To Use?

AG: Let’s remember that ubicomp itself, as a discipline, has largely moved on from the Weiserian discourse of “calm technology”; Yvonne Rogers, for example, now speaks of “proactive systems for proactive people.” You can look at this as a necessary accommodation with the reality principle, which it is, or as kind of a shame – which it also happens to be, at least in my opinion. Either way, though, I don’t think anybody can credibly argue any longer that just because informatic systems pervade our lives, designers will be compelled to craft encalming interfaces to them. That notion of Mark Weiser’s was never particularly convincing, and as far as I’m concerned it’s been thoroughly refuted by the unfolding actuality of post-PC informatics.

All the available evidence, on the contrary, supports the idea that we will have to actively fight for moments of calm and reflection, as individuals and as collectivities. And not only that, as it happens, but for spaces in which we’re able to engage with the Other on neutral turf, as it were, since the logic of “social media” seems to be producing Big Sort-like effects and echo chambers. We already “maximize inherent self-involvement,” analytics or no, and the result is that the tools allowing us to become involved with anything but the self, or selves that strongly resemble it, are atrophying.

So when people complain about K-Mart and Starbucks and American Eagle Outfitters coming to Manhattan, and how it means the suburbanization of the city, I have to laugh. Because the real suburbanization is the smoothening-out of our social interaction until it only encompasses the congenial. A gated community where everyone looks and acts the same? That’s the suburbs, wherever and however it instantiates, and I don’t care how precious and edgy your tastes may be. Richard Sennett argued that what makes urbanity is precisely the quality of necessary, daily, cheek-by-jowl confrontation with a panoply of the different, and as far as I can tell he’s spot on.

We have to devise platforms that accommodate and yet buffer that confrontation. We have to create the safe(r) spaces that allow us to negotiate that difference. The alternative to doing so is creating a world of ten million autistic, utterly atomic and mutually incomprehensible tribelets, each reinforced in the illusion of its own impeccable correctness: duller than dull, except at the flashpoints between. And those become murderous. Nope. Unacceptable outcome.

Uncanny Valleys - Adam comments,”Our apartment in NYC as rendered in Google Earth, with realtime traffic, weather, daylight and shadow as well as geodetic, street grid and service overlays. Camera view is South; that’s First Avenue just left of center-screen.”

TS: Smart phone’s are now drawing everyware data into the system and the net is reaching into who YOU are, WHERE you are, WHAT you are doing, WHAT is around you, etc..

Nathan Freitas says Android: “seems to be the platform most likely to socialize the idea that sensor data could be a piece of every application.” (Android APIs for a wide range of sensor data.)

What in your view will be the most likely platform, Android or what?, to socialize the idea that sensor data could be a piece of every application?

AG: An open platform. A platform with lots of hooks and ways to plug things into it, a strong developer community, a shallow learning curve and/or an easy-to-use, high-level development environment.

I don’t have a dog in this race, mind you. I couldn’t care less who gets there first.

TS: New location based services, e.g., Xtify and ViaPlace, are offering us ways to share location data across lots of different applications (eg Xtify and a dating application like MeetMoi ). In return for services that allow us to share information, we must give up key information up to one site to store and disperse (although there are many differences in approach to our data, from the Twitter stance “show but don’t own” as opposed to Facebook’s stance – “in order to show we must have rights to it”). But the basic model of Twitter – to provide a white noise platform for people to build service on top off seems to be being transposed to location based services. Obvious questions arise like what happens to our data in a start up like MeetMoi if they go belly up?  Apparently in the dot.com bust data was the first thing to go on the auction block in bankrupcy cases.

Also, I suppose it is hardly surprising (if disappointing to me) that some of the early location based services are trying to get mindshare by picking up on the glue celebrities give to mass culture. At the last New York Tech Meetup, OMGICU demoed a rather terrifying new pre-launch location based “participatory celebrity gossip application” which seems to combine all the worst features of social media with celebrity stalking, plus a narrative to change the notion of celebrity itself by “turning D listers into A listers.”

Hopefully location based applications  will not get stuck on “stalker, stalker, stalker” apps like OMGICU .

David Oliver, Oliver Coady gave me a good question: “How does timeliness and location-independence change our ideas of social media?

And how can we design new architectures that can reinforce the sense that I am in control of my own fate?

AG: But we’ve already come so far in terms of turning D-listers into A-listers! On a daily basis, I’m exposed to almost as many cues insisting I attend to nonentities and dullards like Robert Scoble as those insisting I attend to nonentities like Madonna or Thomas Friedman. It’s gotten ridiculous.

Now, how does timeliness and location change our ideas of social media? It makes them dangerous!

Look, even a proud Z-lister like myself – I’m a public person only in the most debased and degraded meaning of that word – I’ve had experiences that shook me up, like having someone approach me while I was quietly hanging out in the back of St. Mark’s Books, and wanting to strike up a conversation based on some talk they’d seen me give a year or so previously. Now part of learning to deal with this kind of thing is shrugging it off, being grateful and flattered that someone thinks you’re interesting enough to single out for that kind of attention, or chalking this up to Sennett’s observation about the constitution of urbanity. Or doing all three at once.

But let’s remember that at the end of the day, a “social network” is nothing but a group of arbitrarily distributed human beings joined by a communications channel, and those people have eyes and ears. The degree to which they recognize some shared interest gives them significance filters. If social capital accrues to those in the network who are able to claim some connection with a “celebrity,” no matter how fleeting, then such connections are going to be mobilized, made explicit. And now say the network has been provided with the tools allowing it to plot the appearances of those putative celebrities in space and time, and what do you get? You get a circumstance in which it is very, very difficult to maintain any membrane between the private self and the world, for anyone who’s even remotely a public figure, whether they particularly want to be a public figure or not. You get network effects that amplify those locational traces, and further undermine any possibility of anonymity, even anonymity-by-suspension-of-interrogative-awareness (which is a clumsy way of referring to that blasé matter-of-factness around famous people that most big-city folks eventually develop).

Am I letting myself off the hook? Not in the slightest. I passed Terence Stamp on the street not so long ago, and you bet I Twittered it. My only excuse was that I Twittered it to a closed loop of no more than a few dozen people. But then, who knows what those few dozen people will turn around and do with that fact, on the open networks to which they in turn belong? And that, too, is my responsibility.

I’m not sure there’s anything to be done about any of this but cultivate our own urbanity, learn to say “so what” when we happen to find ourselves next to Philip Seymour Hoffman in the line at Whole Foods.

TS: Zittrain in The Future of the Internet: And How To Stop It, foregrounds “generativity” and a generative devices (as opposed to appliances) as the most fortuitous starting point for: “tools to bring about social systems to match the power of the technical one.”

Are appliances a threat to the city that is here for you to use? How can generativity ensure The City Is Here For You To Use as Zittrain argues it has ensured, even if imperfectly, that the internet has been here for us to use?

AG: You know, I haven’t read the book, I’ve only heard him give the talk, so it’s certainly possible there’s a subtlety to the argument that I’m missing. But I’m not sure Jonathan isn’t simply wrong about this notion of generativity. Not that the concern is misplaced, but that he’s insufficiently trustful in human agency. Is a car “generative,” by his definition? Certainly not. And yet look at all the cultural production that goes on around “the car,” look at all the assemblages people make with cars, from Beach Boys songs to ghost riding the whip, from J.G. Ballard novels and Herbie the Love Bug to Tokyo Drift.

Or probably more to his point: look at the Japanese mobile-phone market – seemingly one of the most locked-down and unpropitious circumstances imaginable for the production of culture, in technical terms and Zittrain’s both. And yet fully 50% of the bestselling books in Japan last year were written on mobile phones. Not read, which would already be impressive enough (if “impressive” is indeed the word): written. What does that imply for his argument?

So, yes, I think there are grounds for concern in that we don’t allow technologies and frameworks to appear that unduly limit the scope of human creativity. Code is still law. But I also think people are quite amply able to reach into what would appear to be the least propitious technologies and tell their own stories with same.

TS: One aspect of Everyware that seems in need of some visionary yoga is the how we will relate to pixels anywhere.

In Urban Computing and its Discontents you mention how our technological trajectories often make it seem as if we seem to get fixated on particular scenes in movies, e.g., Minority Report. You point out that so many ambient informatics projects seem simply “to expand the reach of signage and advertising in dense urban spaces….as if we’ve become transfixed by the scene from Minority Report where heterosexual cop John Anderton is on the run from his colleagues.”

Ideas from the Minority Report continue to hold sway in designs as we saw in the recent MIT demo of SixthSense at TED.

But visions of augmented reality were pretty high profile in this years Super Bowl commercials this year (including a highly anthropomorphic imagining of ubicomp that was a kind of WoW mashup with a Pixar movie).

What recent movies/commercials have produced scenes mostly likely to be are new fixation fodder for ubicomp and why?

AG: I don’t think I’m qualified to answer that, actually. We don’t have a TV, so I don’t see much in the way of commercials, and most of the films I wind up seeing are the kind that play at Anthology Film Archives. What I can say is that science fiction is currently suffering in toto from an inability or disinclination to posit future scenarios that are any weirder or more visionary than those emerging from other sectors of the culture. And that would be fine, except sf has traditionally been the place where we wrestled with the imaginary.

We need that set of tools, badly. If for no other reason than something I glean from personal experience: essentially my entire professional career has simply been the leveraging of ideas and concepts I originally wrestled with in the encounter with William Gibson and Bruce Sterling when I was 16. Today’s visionary sf means tomorrow’s halfway-competent generalist.

Nurri Kim in the waiting zone – photo by Adam Greenfield

TS: My AR friend, Robert Rice, who is working on a markerless AR platform, notes that data visualization is one of the critical elements of AR in terms of “make or break.” Robert says, “even with the ultimate in ubiquitious data from everything, without good data vis it will all be useless”

Also something Cory Doctorow said to me last year has really stuck in my mind. When I asked him what happens when Cyberspace everts, he talked about a reverse surveillance society:

Cory: Well this is like Spook Country the new Gibson novel – What happens when cyber space everts – hmmm? I’m not sure I have anything very pithy to say on that EXCEPT………
Apart from all the traditional kind of overlay reality stuff, if there is one thing I am actually interested seeing from a virtual world migrating to the real world its instrumentation.
I think lot of things that are characteristic of very successful internet based business is that they are extremely finally instrumented so like Amazon knows in aggregate on a second by second basis how their site is being used by people and they can twiddle the dials in real time.

As users of the world we have very little access to that kind of instrumentation. We don’t even know how the tube is running. The tube knows how the tube is running and we kinda of don’t. I would be really interested in seeing that. You’ve seen Joi Ito’s WoW interface right. Have you seen it …

Joi Ito’s WoW interface seems a long way from the calm, invisible imaginings for ubicomp by early ubicomp visionaries?

AG: Well, he’s got a particular kind of neural wiring. And there’s not a thing that’s wrong with that, except that I’d never, ever want to assert that what’s appropriate for Joi Ito necessarily is or should be understood to be appropriate for anybody else. The point of calling for open systems and frameworks is to allow us maximum scope of diversity in the ways we choose to interface with the world’s richness and complexity.

TS: What new imaginings/possibilites do you see when pixels anywhere are linked to everyware?

AG: Product placement. Commercial insertions and injections, mostly.

Beyond that: one of the places where Mark Weiser logic breaks down is in thinking that the platforms we use now disappear from the world just because ubiquitous computing’s arrived. We’ve still got radio, for example – OK, now it’s satellite radio and streaming Internet feeds, but the interaction metaphor isn’t any different. By the same token, we’re still going to be using reasonably conventional-looking laptops and desktop keyboard/display combos for awhile yet. The form factor is pretty well optimized for the delivery of a certain class of services, it’s a convenient and well-assimilated interaction vocabulary, none of that’s going away just yet. And the same goes for billboards and “TV” screens.

But all of those things become entirely different propositions in everyware world: more open, more modular, ever more conceived of as network resources with particular input and output affordances. We already see some signs of this with Microsoft’s recent “Social Desktop” prototype – which, mind you, is a very bad idea as it currently stands, especially as implemented on something with the kind of security record that Windows enjoys – and we’ll be seeing many more.

If every display in the world has an IP address and a self-descriptor indicating what kind of protocols it’s capable of handling, then you begin to get into some really interesting and thorny territory. The first things to go away, off the top of my head, are screens for a certain class of mobile device – why power a screen off your battery when you can push the data to a nearby display that’s much bigger, much brighter, much more social? – and conventional projectors.

Then we get into some very interesting issues around large, public interactive displays – who “drives” the display, and so forth. But here again, we’ll have to fight to keep these things sane. It’s past time for a public debate around these issues, because they’re unquestionably going to condition the everyday experience of walking down the street in most of our cities. And that’s difficult to do when times are hard and people have more pressing concerns on their mind.

Citywarecrash – photo by Adam Greenfield, “An occupational hazard for urban screens.”

TS: I know in Everyware you mentioned that architects have play an important visionary role in imagining ubicomp and I know you work closely with your wife, artist Nurri Kim.  Robert Rice asked me the following question – which I will in turn ask you: “In terms of augmented reality do you think virtual worlds and virtual reality experts / leaders / are good pioneers for thought and guidance on AR? Or, should we look for new leaders, or where are new leaders emerging? Is the tech similar enough for the old crowd to be useful or is it different enough to be a disadvantage coming from the old models?.”

AG: I should make it clear that I have absolutely no interest in virtual worlds or virtual reality. The so-called virtual worlds I’ve experienced seem sad and really rather tatty – eversions of the most predictable adolescent fantasies of unlimited power, reinscriptions of all the usual politics – and completely lacking in just about everything that makes life resonant, meaningful and awe-inspiring. And anyway, to paraphrase J.G. Ballard, ordinary, everyday life is now far more vividly and fantastically weird than anything you’ll see in Second Life. I mean, Garry Kasparov was heckled by a radio-control dildocopter, Joe the Plumber’s off to Gaza as a war correspondent, a sea of dust-covered BMWs waits in the long-term parking lot at Dubai International for owners who are never, ever coming back.

Look to virtual worlds for insight into the hard work of negotiating the actual, with its physics, its entropy, its suffering, with all its constraints? Oh my goodness gracious, no.
And look to leaders? Never. Leaders are for followers, and who wants to be that? I don’t mean you can’t take inspiration and insight from the work of others – not at all – but use your own imagination, take some personal risk, do your own damn work.

Now, having said that. This opposition of virtual and physical worlds strikes me as increasingly a false one, as it does many people. The hard-and-fast distinction between “the real world” and virtual environments make less and less sense, as righteously satisfying as making it can sometimes seem. There may be attributes of this physical environment that are impossible to see or make use of without access to the networked overlay, and those attributes may in time come to constitute the primary wellsprings of a given place’s meaning. And if you’re offering me some insight that I think could be of utility in resolving the challenge of making this overlay accessible to all, equally, I’ll gladly accept it, no matter what domain or disciplinary background you claim as your own.

Am I aware of any such insight coming out of virtual worlds? No. As Bryan Boyer notes, “If you want to start talking about some serious cross-disciplinary pollination then you better take both sides of that disciplinary divide seriously. When your ubi- runs into my building with its boring HVAC, mundane load paths, typical finished floors, plain old foundations, etc., the transformative powers of comp are bracketed pretty seriously by the realities of the physical world.”

The Cloud Gate has landed – photo by Adam Greenfield, “Tell me this doesn’t look *just* like the descriptions of “stasis fields” in 70s SF. In fact, the picture looks practically CGId to me.”

TS: Some people thought the whole world would have been plastered with RFID by now.  But before that has happened markerless AR seems to be in our sights.

If I understand it correctly marker versus markerless AR has quite different implications for how the cyberspace of ubicomp evolves?  I asked Robert Rice (he is developing a markerless AR platform) to explain some of the differences.  He said:

Robert also pointed out to me that markerless AR may prove even more problematic for privacy:

Do you have any thoughts about the different roles that markerless versus marker techinologies will play in AR and Ubicomp?

AG: I need to admit that I’ve never until this moment heard the phrase “markerless AR,” although I’d think it’s more or less self-explanatory to anyone who’s been following this stuff. Let me make the distinction explicit, shall I, for anyone who hasn’t been? And you or Robert can correct me if I’ve gotten it wrong.

Augmented reality means that I have some mediating artifact that provides me with a visual overlay on the world. This could be a phone, it could be a windshield, it could be a pair of glasses or contact lenses, doesn’t matter. And you’re going to use that overlay to superimpose some order of information about the world and the objects in it onto the things that enter my field of vision – onto what I see. So far, so good: that’s AR 101.

Now where does that information come from?

What you’re calling marker-based AR implies that there’s some reasonably strong relationship between the information superimposed over a given object, and the object itself. That object is an onto, a spime, it’s been provided with a passive RFID tag or an active transmitter. And it’s radiating information about itself that I’m grabbing, perhaps cross-referencing against other sources of information, and superimposing over the field of vision. Fine and dandy.

But there’s another way of achieving the same end, right? Instead of looking at a suit jacket on a rack and having its onboard tag tell you directly that it’s a Helmut Lang, style number such-and-such from men’s Spring/Summer collection 2011, Size 42 Regular in Color Gunmetal, produced at Joint Venture Factory #4 in Cholon City, Vietnam, and packed for shipment on September 3, 2010, you’re going to run some kind of pattern-matching query on it. And without the necessity of that object being tagged physically in any way, you’re going to have access to information about it. But this set of information isn’t, necessarily, what the object itself, or its creators or merchandisers, want you to know about it; it could be derived from online discussion fora or review sites, or blog posts, or whatever. All there needs to be is a lookup table, essentially, that tells you where to find information about any object in the field of vision whose identity can be established.

Do I have that right? And if I do, then as I understand it, the distinction is primarily a pragmatic one: it’s just easier to get to an augmented world, by far, if we don’t actually have to go to all the trouble of tagging everything in the world with its own dedicated RF transponder. Easier, and cheaper, and quicker, and more environmentally sound besides, because the relevant traffic is in bits not atoms.

Unless I’ve missed something, you don’t, then, get the distinction between classes of objects and instances of same. Sometimes, when there’s a 1:1 correlation between the two, that’s not going to matter: I’m walking down the street in Madrid, and my glasses or whatever can easily recognize that this building is the Caixa Forum. There’s only one of it, and I can get a positive ID via pattern recognition. But for some edge cases – twins and lookalikes, mostly – the same thing is generally true of people.

But other times it will matter. Is this specific watch a real, $10,000 Panerai or a $50 Kowloon fakery? How has this black 1998 Honda Civic over here differ from this other one in terms of its use and maintenance history? Does this O-ring gasket need to be replaced? I don’t see how you extract data from specific instances of things without the necessary sensor instrumentation, transmitter, etc., being coextensive with the object in question or very closely colocated with it over time – in the terminology you’re using, a “marker.”

So using these terms, I’d say that “markerless” AR comes first, is relatively easy to deploy, and generates not-insignificant value. But – again, unless I’m missing something – there are some things that it won’t ever be able to do, and for those things you need some provision for self-identification and self-location.

Ultimately I think it’s a distinction without a difference, from the user’s point of view. People will care much more about the source of whatever information shows up on their overlay than the precise technical means used to get it there.

The surrender to cynicism – photo by Adam Greenfield

TS: Much early thinking around ubicomp seems to have come from visionary architects and engineers but recently I was at the O’Reilly Tools of Change for Publishing Conference (publishing in the Digital Age) and I met several book futurists.  It struck me how ubicomp from the perspective of the book created some interesting questions for how particular material cultures will shape and be shaped by Ubicomp differently.

I noted, Google seemed well down the path to holy grail “converting images to original intent XML.” And Peter Brantley talked about machine parsed books.

At TOC there were many suggestions about how books might manifest as everyware. (Although it did not seem that many people felt books had a special relationship to time and history and would not vanish as one of the great metaphors of calm and solitary enjoyment in our culture soon).  Books as everyware will, it seems, include, amongst other things:

books that read books

books that read context

context that reads books

books that read me

books linked to mobility – timeliness and location independence

books that are not books

books becoming babble

books bubbling up from the babble

There is an Institute of the Future of the Book. Will all former material cultures require their own institutes of the future to guide their cultures into everyware?  Do you think books transition into everyware is especially significant and why?

AG: But all objects have a relationship to time and history, no?

TS: Yes! What I meant to convey really was the idea that many people expressed at TOC that books had a privileged relationship to knowledge in our culture that was valuable and related to some aspects of their current form, and that books as everyware, e.g. machine parsed books, and more sociallly generated forms would not replace that entirely.

AG: Gotcha. Well, I certainly agree that books constitute an interesting category unto themselves – I’ve held onto my physical books, and in fact still spend a fortune buying new ones, where I stopped buying music on discs a long, long time ago. But I don’t think this state of affairs can or should obtain forever.

Lately there’s been a good amount of thought around the notion of “unbooks,” which I regard as a container for long-form ideas appropriate to an internetworked age. By building on some of the tropes of software development, mostly having to do with version control, open-endedness and an explicit role for the “user” community, unbooks can usefully harness the dynamic and responsive nature of discourse on the Web. At the same time, you preserve the things books are really good at: coherence, authorial voice and intent.

The important part is in acknowledging two points which have usually been understood as contradictory, but which are actually nothing of the sort: firstly, that the expression of ideas in written form has something to learn from the practices that have evolved around the collaborative creation of dynamic, digital documents over the half-century-long history of software; and secondly, that certain ideas require elaboration in the reasonably strongly-bounded form we know as a “book,” and cannot meaningfully be shared otherwise. A third point, concomitant to the second, is that despite recent technical advances, screen-based media still cannot, and may not ever fully be able to, deliver the extratextual cues and phenomenological traces that support, inform and extend the meaning of written documents.

The unbook lets you have your cake and eat it too. So, for example, when we publish The City Is Here, one of its manifestations will be a static, physical document – and hopefully, if we do our jobs well, a very nice one indeed. But even before that, you’ll be able to download a Creative Commons-licensed PDF of every numbered version of the manuscript, from zero onward. Bottom line: you buy the book if, and only if, you want the object. The ideas are free.

TS: David Brin sees two futures:1) the government watches everybody, and 2) everybody watches everybody (the latter he calls “sousveillance”).  My friend Ben Goertzel says “hooking AI up to a massive datastore fed by ubicomp is the first step toward sousveillance?” What do you think the role of AI in ubicomp will be?  Is it worth thinking about what is the first important “AI meets AR” app is?

AG: I don’t believe that artificial intelligence as the term is generally understood – which is to say, a self-aware, general-purpose intelligence of human capacity or greater – is likely to appear within my lifetime, or for a comfortably long time thereafter.

Having said that, your friend Ben seems to be making the titanic (and enormously difficult to justify) assumption that a self-aware artificial intelligence would share any perspectives, goals, priorities or values whatsoever with the human species, let alone with that fraction of the human species that could use a little help in countering watchfulness from above. “Hooking [an] AI up to a massive datastore fed by ubicomp” sounds to me more like the first step toward enslavement…if not outright digestion.

Sousveillance - the term is Steve Mann’s, originally – doesn’t imply “everybody watching everybody” to me, anyway, so much as a consciously political act of turning infrastructures of observation and control back on those specific institutions most used to employing same toward their own prerogatives. Think Rodney King, think Oscar Grant.

TS: I have one last question from Usman Haque.

Usman Haque: insofar as a lot of what adam describes as desirable could be said to constitute pretty radical socio-political change (or perhaps… “adjustment”) i would be really interested to know how his current work @ nokia is or isn’t able to gel with the themes of his writing. in some senses there’s quite an undercurrent strongly challenging corporate practices, in other senses it could be seen as gentle nudges. how does adam see it? and how about the nokia behemoth? does he have success nudging nokia towards the kind of world he would like to see (i imagine the answer is ‘yes’ otherwise he wouldn’t be doing it…) but i’d love to know more about the limits/challenges.

AG: I am told that Henry Kissinger, on his first trip to China in 1971, asked Zhou Enlai whether he thought the French Revolution had or had not advanced the cause of human freedom.
Zhou thought for a moment, pursed his lips, and replied, “It is too soon to tell.”