We are in the age of unearthing and uncovering data, and only just at the beginning of the age of processing data and dealing with it (see my interview with Anselm Hook, Part 2 upcoming).  O’Reilly’s Strata Confernence 2011, will explore, “the change brought to technology and business by data science, pervasive computing, and new interfaces.” It is, perhaps, one of the most important events of 2011.

Data is driving a revolution much as coal, oil, and steel powered the industrial revolution.  And the world changing insight from Karl Marx that “the industrial revolution polarized the world into two groups: those who own the means of production and those who work on them,” is taking on on new life, as  Alistair Croll, co-chair of Strata 2011, points out in his post,  “Who Owns Your Data?”

“The important question isn’t who owns the data. Ultimately, we all do. A better question is, who owns the means of analysis? Because that’s how, as Brand suggests, you get the right information in the right place. The digital divide isn’t about who owns data — it’s about who can put that data to work.”

Strata is where a vanguard will be meet, not only to discuss this revolution’s futures, but to define how to create, handle, and build the platforms and experiences that will harness the data. My flight is booked!  (Also check out BigDataCamp which takes place the night before Strata.)

The picture opening this post is from Michael EdgeCumbe’s  Fall 2010: ITP Winter Show Project.  A project exploring ways to intuitively get the feel of what it going on with big data sets using “the gestural manipulation and stereoscopic visualization of complex data to create a meditative state for data analysis.”  Michael project will be part of the Science Fair at Strata.  For more on Michael’s work see Noise Derived. I also have a number of the  interesting new interface sessions at Strata in my schedule.

The daily Strata Gems on O’Reilly Radar are great place to get a gestalt of some of the Strata themes, and this post by Edd Dumbill, program chair for Strata, Three key data trends for 2011, looks at the year ahead.  This week, I got the chance to ask Edd a few of the questions that I will have on mind at Strata – see his responses below.

If you have been reading Ugotrade, you will know I am interested in our mobile social augmented futures and there is no question in my mind that these will be unleashed by our new capacities to work with data (see my post here).

Data is the how.

The pic above is from “Secrets of BackType’s Data Engineers.” This post on ReadWriteHack by Pete Warden, an ex-Apple engineer, and founder of OpenHeatMap, really lives up to its title.  Check it out if you want to know how  “three guys (the BackType team ) with only seed funding process a hundred million messages a day?”

I asked on Quora, “What would be the most important developments for Augmented Reality in 2011,” Michal Avny, Strategist & Real Time search expert, wrote:

“AR strongly relies on localized personalized real time information.

Having a stream of tweets based on keyword search, location or circle of friends doesn’t really make the AR experience; it is the processed real time relevant information that will make AR useful and intensify the experience.”

In 2011 Real Time search and Social Search will drastically change to provide the infrastructure required.”

I followed up on Michal’s Quora answer with some more questions – see below in this post.

Also note the response from David Molnar, here is an excerpt:

“2. A wave of actionable, important data APIs opened up, enabling useful non-gimmicky AR apps for the first time. Think geoloqi.com , or the work Max Ogden has done with Portland civic data. Plus of course face.com , email providers and calendar providers, etc.”

Amber Case, one of the founders of Geoloqi, is on the programming committee of Strata and will be speaking. Be sure to catch her session! Posthumans, Big Data and New Interfaces, and if you haven’t already seen it, Amber’s TED talk is a must see.

Geographic proximity is a powerful filter, as is route, and time. But clearly social proximity, social relevance, and shared tastes are also key dimensions for location based experiences, (see my convo with Schuyler of Simple Geo, upcoming).

While the whole business of location based search and curation of augmented mobile social experiences is still, for the most part, uncharted terrain, the danger of key points of control being only really accessible to elite players looms large. I asked Sophia Parafina, a pioneer in the open geo space for some thoughts on real-time local /geosearch and geomessaging, and the future of openess & big data (see Sophia’s response below).

Is the market ready yet for P2P cloud computing?

This is another question I’m following,  Is the market ready yet for P2P cloud computing? It is one of those questions that we seem to have been asking in various forms for a very long while now, but without a  major shift in sight.  The pic above is from, The Cloud Made Open Source “Invisible” This Year.  But, perhaps, we are at the point when open p2p clouds will find a place in the market because of their potential importance in real time social search and discovery. Borislav Agapiev, Search Entrepreneur and founder of Vast.com, writes on Quora:

“I believe a P2P cloud is ideally suited for social & real-time search and discovery.

Consider MapReduce, a very interesting and popular paradigm for distributed computing. MapReduce is very much about bringing computation to data i.e. doing computation at nodes (map) and then aggregating results through network (reduce).

It is very clear now that user attention data (what they click on) is very valuable for search and discovery, yet a centralized model relies upon uploading all that to a single location and then doing a supposed local MapReduce. Clearly, MapReduce could be done across the network, without any centralized uploads.

In addition to the efficiency argument raised here, it is even more important to consider privacy issues. Uploading massive amounts of user attention data to a centralized location is not something that is going to make users warm and fuzzy as we are increasingly seeing.

In a P2P cloud, there is no big brother watching over anyone, all computation and data storage is done in the cloud, fragmented in many, many small encrypted pieces ala BitTorrent.”

Picture above from Brynn Marie Evans, “It takes two to tango: review of my social search panel“

The Delta of Now – Transforming Search into a Social Democratic Act

Picture of Maneki Neko “beckoning” cats from Journeyetc

New ecologies of human and machine intelligence are beginning to change basic social structures – see the Future of Work (Biewald and Chirayath Janah 2010). And projects like Swift River, using search and machine mining to filter out streams on topics of interest that can then be subsequently curated by human beings. This may be extended to the curation of real-time data streams and employment of machine learning algorithms based upon the explicit relationships.

Augmented mobile social experiences are a new frontier in which ideas and practices from a number of fields collide, including: ambient findability (Morville 2005), urban psychogeography, narrative structures, ambient games and devices, 4d (time-space), explorations of place and memory, enchanted objects and people (Kuniavsky 2010), and designed animism (Laurel 2010), to mention just a few.

Mobile local interaction presents an opportunity to invert the search pyramid and to transform search into a social, democratic act (see my interview with Anselm Hook upcoming).  Up until now search has been predicated around a very narrow revenue model. Google has an implicit model of a B2C – business to consumer brokerage. We are only just beginning to get a glimpse of the disruptive potential of C2C – consumer to consumer brokerages. Mobile local C2C brokerages that allow us to transact in a trustworthy way over our local geography in close to real time (Hook 2010) have the potential to enable new forms of social organization. Bruce Sterling’s short story about a networked gift economy, Maneko Neki, is a brilliant glimpse at the disruptive potential of such re-imaginings.

Augmented experiences that shift or change a person’s situated geolocal experience of social reality, and change our relationship to the people and the place by augmenting engagement in, and reputation through, socially driven consumer tie ins and game dynamics, like Four Square, & Gowalla are beginning to emerge, as Kati London pointed out in her excellent keynote at Web 2.0 Expo. And, while the integration of mobile local interaction and an augmented view that shifts our geolocal experience visually will involve creative solutions to some well churned mobile, tracking, mapping and registration challenges, the exploration and development of new dimensions through which we can filter and create trusted and meaningful augmented mobile social experiences is vital, whether you are considering a mobile screen, map, camera view, or futuristic HUDs and gestural interfaces.

Talking with Edd Dumbill

Picture from O’Reilly Community.

Tish Shute: First congratulations on Strata! On the Strata homepage there is a quote from Jason Hoffman:

“My gut feeling is that we’re going to look back at the upcoming Strata Conference like we do at the Web 2.0 Conference in 2004/2005.”
—Jason Hoffman, CTO/Founder, Joyent, Inc.

Why do you think Jason’s comparison might be prescient?

Edd Dumbill: Web 2.0 is a development that ran through every brand that has a web presence and radically changed the way business is done for many companies and brands.

Strata will have a similar impact: every business has data, every business collects an increasing amount of data. This data is the new oil – a valuable raw material that when refined or combined creates value and opportunity.

Tish Shute: The rise of real time was one of your three key data trends for 2011. Hadoop is bringing the capacity to work with big data to more than just a few elite players. But the challenge is still real time. You mention we will be seeing a hybrid approach to real time and batch MapReduce processing. Will we hear more about these approaches to real time at Strata? And, what do you see as the most important conversations on real time data analytics emerging at Strata?

You point out “open source projects and cloud infrastructure means developers can evaluate and learn to love technologies without requiring support or approval from above.” What are the most exciting developments on the horizon for open source tools?

Edd Dumbill: Here are some projects worth watching, in the key areas of real time, cluster management and Hadoop.

* Cassandra and MongoDB — NoSQL databases that will prove vital for anybody with real time big data needs

* Mesos — a compute cluster management tool, modeled after that which powers Google

* Hadoop ecosystem’s continuing maturation, especially HBase and Hive.

Tish Shute: Do you think the market is ready for p2p cloud computing?

Edd Dumbill: The market is emerging for decentralized and distributed cloud computing, and P2P technologies are one way of achieving that. They key trends will be moving computation nearer the data sets or nearer the point of user consumption of the result.

P2P is a difficult model for anybody wanting to commercialize a service, so I think it will tend to form part of a hybrid solution.

Tish Shute: We have seen enormous strides in our ability to work with giant unstructured databases recently. Do you think, perhaps, that the dream of a web of linked data – “a web of data that can be processed directly and indirectly by machines,” will be attained through brute force – i.e. through our ability to harness the power of massively parallel processing, as much as by Semantic Web approaches focused on machine readable metadata? [Also see my question on Quora, “Is this a good approach (www.dist-systems.bbn.com/people/…) to use Hadoop to build a scalable, distributed triple store?”]

Edd Dumbill: I’ve been an observer of the SW for over a decade and I tend to believe that on the web, data means to you whatever meaning you give it as the consumer. With that model, the links are made by the consumer rather than sitting out there explicitly. Some links become de facto standards, and some very few become web standards.

I think the actuality will be a mix of both explicitly stated metadata and that which is inferred. The Semantic Web is a great framework for certain operations, especially interoperable exchange of metadata. A great many more private meanings, never intended to be shared, will be created by consuming software.

There’s no question that machines will learn how to process most of the Web. Furthermore, machines will learn how to process most of the physical world we’re in. And that by the end of this decade.

Talking with Sophia Parafina

Picture of Sophia at Where 2.0

Tish Shute: Sophia you have worked in the trenches for a long time now to support the growth of open geo data. What do you hope to see emerge in 2011 in the field of geo-data?

Sophia Parafina: Better support for displaying and handling location data across multiple apps. Fred Wilson recently blogged about content-shifting, he talks about overcoming content silos across devices. We’ve worked very hard to reduce data silos via formats, but devices are creating their own silos. I would like to see a standard method for sending geo data and geo information to mobile devices.

Producing content for mobile is different from producing content for a computer browser. Web 2.0 produced a lot of infrastructure for browser based interfaces, but in mobile devices that gap has been filled with apps which is fragmenting how data is handled by various devices. What is even more interesting in the mobile space is that devices can push data back that contains location, user updates, photos and even sensor data.  If mobile data standardizes, it could lead to browser based applications and stem the continued fragmentation of the mobile application market.

Tish Shute: Simple Geo and Factual are startups emerging in the geodata space. What do you see on the horizon in terms of both the growth of business opportunities and an open geo data community?

Sophia Parafina: In the near future think we’ll see startups providing curated data + API and in response we will also see companies that provide a single interface across multiple data providers. We saw this when everyone released a mapping API and companies such as Mapufacture provided a single interface across multiple APIs.

We will see a resurgence in data providers repackaging the the 2010 US Census data in different ways to respond to market segments, some of this will be open data but all of it will be provided through an API instead of file. Additionally, we’ll see more data from outside the US.

Tish Shute: What are the biggest obstacles to having the open geodata sets available that we need to enable mobile local interactions and social augmented experiences?

Sophia Parafina: Licensing for both crowd sourced data and private curated open data will become an issue. We recently seen VLC, the open source video player, pulled from the Apple app store because of licensing issues. Also, licensing of content by geography will be problematic, limiting searches by geographical location. In addition, how will licensing of data that is updated by crowd sourcing work?

Multiple APIs for accessing data sources. The current trend for each provider to create an API for their data sets will result in data silos – there needs to be a single sign-on equivalent for requesting data.

Size of data on the wire, the current models for delivering data is based on broadband connections. However, as mobiles increasingly become the way people use the web, the data needs to be sized accordingly. This also goes for mobile interfaces. Have you tried to shop on a mobile device, or buy a train or plane ticket? It’s frustrating and error prone. There is a large untapped market of people who only use the Internet on mobile devices.

Tish Shute: You pointed me to this link in Strata Gems re “an interesting and pertinent (also a competitor to GeoLoqi),” – the Android Tasker app. What do these emerging services bring to the table in terms of the next generation of location based services?

Sophia Parafina: This app let’s your device interact with the environment. I think that this is a great way of using the sensors on existing platforms to increase interaction and to implement ambient findability. The basic premise of Tasker is that some action happens in response to an event in an application, time, date, location, event, or gesture. Tasker has defined 180 actions that can occur based and number or combination of events. This can provide a basic vocabulary for interaction between the user and the device and more importantly between users. Tasker also can use Android script plugins, which lowers the bar to creating your own ambient application.

Programs such as Tasker can provide a way for people to interact with social networks beyond sending messages. People can use their mobile devices to interact with their surroundings with out having to interact with the device.

Tish Shute: We have had many conversations about emerging ideas of geo-search, geo-messaging and geo-fencing. What are the most interesting developments in these areas and what do you see on the horizon for 2011?

Sophia Parafina: The map will fade into the background and become less important. Display of information will be context aware, that includes location. For example, let’s say I make a grocery list, when I’m at the grocery story, the list will just pop-up without the need for me to find the app that has the list. Or reminders or offers pop-up when you are near a place at a certain time, let’s say you need to buy a present for a birthday party for a child, you could send out a request that you are looking for an item and retailers could offer “on the spot” discounts if you are in the area.

Geo-search, geo-messaging, and geo-fencing are geared to towards mobile devices, so I expect to see them soon as part of apps. Building generic applications that implement geo* will fail because that sort of information is useful only within a context. Geo* apps are solutions looking for an problem. The killer mobile app will use these functions transparently to reduce the cognitive load of the user who is busy moving around in the world.

User data gathered from multiple web applications will become consolidated profiles that will used for context aware applications. For example, there could be a service which matches prices of items that you have shopped for on the web, so for example the service would have access to your cookies, know your favorite retailers, things you have shopped for, your location and activity patters (when you are at home, work, restaurant). When you are in the vicinity of a brick and mortar retailer with the same or similar items, the service can send you alert to match the price of the item you found on line. So your digital life will become more closely linked with your day to day activities.

Talking with Michal Avny

Tish Shute: At Web 2.0 Summit, one of the highlights for me was the, Q&A:The New Search Insurgents lunch where Charlie Cheever of Quora, IMO, stole the show. I tweeted:

“One of my takeaways from #w2s is that #quora points to future of augmented mobile social experiences – a search filter for experience! #AR”

In your view what are the biggest challenges for location Q&A to emerge as a search filter for location based experiences?

Michal Avny: The biggest location Q&A challenges yet to be conquered are immediacy (real time dynamic data), relevancy (strong personalized filters) and user experience (simplified interface).

Location Q&A enables different use cases. The most prominent are Follow (follow places, topics and friends to learn about a location), Interact (meet new people based on common interests), Plan ahead (plan a trip, night out or a shopping day by asking and searching for local information) and On-site (check for recommendations, friends, deals, events and traffic nearby).

Unlike Follow, Interact and Plan ahead that can be added to existing Q&A platforms (such as Quora) by attending location specifics as they share similar characteristics, the on-site mode introduces a completely different experience, first and foremost it requires immediate attention. It is real time based and the nature of the data is dynamic. Traffic updates, current events, nearby friends, all that changes constantly. Posting a location question on-site implies the response should be in real time (e.g. best kid friendly restaurant), the normal Q&A response latency wouldn’t work.

Strong relevancy filters are required to accommodate for the overwhelming flood of information. Moreover, some of the data should be filtered by user behavior and preferences, check in notifications (type of relation), restaurant recommendations (type of food, price level, etc), shopping deals (commercial categories) and more.

Mobile experience requires ease of use and simplicity. A new Q&A interface and query language that allows for posting questions should be defined as well as coherent summarized response interface. User on the go should not have to post lengthy questions, browse through tens of results or search for the right service, but instead use a simple intuitive tool.

Tish Shute: Real- time location based search is in its infancy. Real time questions can be answered using different services such as Yelp, TripAdvisor, Waze, Foursquare, IMDb and more. But what are the challenges to moving forward with aggregating these sources and then into “locals” that are able to process and deal with vast amounts of information?

Michal Avny: Using some of the leading location services to answer question is sufficient to start with.

In order to provide broad coverage (worldwide) and reliable information, aggregation of the different services is required for instance to normalize product and service rank, aggregate classified, and more. This is quite challenging as there is no one standard available.

When location Q&A user base is big enough, I foresee a tendency to rely more on ‘locals’ input as the base of information. As the platform grows, communities will be formed with different cultures, relationships and trust levels, making the information more valuable and customizable. Some of the challenges I already mentioned are implementing filters, query language and interfaces to enable using the vast amounts of real time data in a mobile environment. More of the challenges lying ahead are integrating the ‘locals’ data with location based services as they are integral components of the Q&A ecosystem. Merging trust levels and relationships while adhering to different privacy guidelines is a challenge yet to be explored. (This should be discussed in more detail under the protocols topic).

It is quite evident that Quora is now facing growing pains and is struggling to maintain its character. Same as with Quora, it will also be a challenge to support and maintain the ecosystem while allowing for massive scale-up.

Tish Shute: I have been very interested in exploring protocols that will be enablers to micro local interaction and mobile social interaction for AR – particularly the XMPP extensions and operational transform work of Google Wave (now Apache Wave), and PubSub protocols like PubHubSubbub and Erlang based RabbitMQ. We are beginning to see protocols emerging that could enable new real time local services. What do you think are some of the most valuable use cases for “locals” that this new generation of real time protocols can enable?

Michal Avny: AR is about interacting with digital information; the AR ecosystem is composed of layers and components such as devices, platforms, browsers, applications and content. For the different components to interact new protocols, security guidelines, and privacy policies must be in place. A standard will enable local vendors and service providers to publish specials, deals, updates and events for any application to broadcast, identify people and places by proximity (without having to use the same application or device), local recommendations will be shared by services, devices will be able to interact, location based platforms, such as Q&A, will have access to vast breadth of information, geo aware devices will provide consistent experience globally, and much more.

Tish Shute: What do you think are the biggest challenges to going mainstream for this emerging field of real time social discovery?

Michal Avny: The biggest challenge is building towards real time, geo-aware, localized, personalized ambient data. Discovery is in its infancy, location social based Best, Top, and Trending lists with some basic filtering options are available, and this is great as people are getting accustomed to information surrounding them. To some degree it can intensify the AR experience, for instance suggest the most popular dish in a restaurant, or map the best coffee shops nearby, but it is customized at best by friend recommendations and depends on the coverage and broadness of the specific discovery service.

There is a need for the next generation of discovery, customized geo social aware discovery that filters the vast amount of real time data by learning user preferences and behavior (built on top of the much needed local social real time open protocol)

Tish Shute: Who are your favorite startups/upstarts in the the field of real time search and why?

Micha Avny: My6Sense - My6sense provides a sharper and better way to experience your information from feeds you subscribe to (Social Networks, News, RSS feeds, etc.). It’s personal – Content is ranked based on what’s relevant to you. It learns what’s valuable to you by translating your consumption behavior into a personalized ranking function.
My6Sense – because it is a personalized prediction filter, a critical foundation for AR

Topsy – Topsy is realtime search powered by the social web that finds the most relevant conversations happening online. The site’s underlying technology examines popular links as well as the influence of each person citing a link. Topsy augments traditional search engines by finding information that people are talking about.
Topsy – because its ranking is based on retweets and influencers, a great social experience

Collecta – Collecta is a real-time search engine for the social web. It monitors the update streams of popular realtime blogs and sites like Twitter, WordPress, and Flickr, and shows results as they happen. Results can be filtered by status updates, comments, stories, or photos. The entire engine is built around the XMPP standard, which pushes out data on a continual basis, so that for every search you end up watching a stream that keeps updating itself.
Collecta – because it is built around XMPP, a real time experience

The biggest news at Where 2.0, 2009 came from the Yahoo! Geo Technologies Group. Tyler Bell, announced Yahoo! Placemaker and the opening up of the GeoPlanet data set, “all of the WOEIDs [Where On Earth (WOE) IDs] available as a free download under Creative Commons in June” (see Brady Forrest’s post for more details).

WhereCamp 2009 was held immediately after Where 2.0 and was a great place to chew on the events and ideas of Where 2.0.  In the picture above Anselm Hook addresses the WhereCamp morning circle in the courtyard outside the Social Text offices in Palo Alto. Anselm pointed out to me:

“there are interesting implications of placemaker in combination with other yahoo assets – in particular YQL – placemaker by itself is neat – but placemaker combined with everything else is a natural missing piece that is a big enabler.  Yahoo has been impressive.”

With all the Geo platform power available to us now, also (also see New Geo for Devs from Google I/O), there isn’t a shadow of a doubt in my mind Brady is right when he said, just before the Where 2009 conference: “Location is no longer a differentiator it’s going to become oxygen” (quote from WebMonkey).

The Yahoo! GeoPlanet team at WhereCamp – Tyler Bell, (talking to Brady Forrest in picture on the left) is sporting his spatial junkies T-Shirt. Photo on right, Aaron Cope, Tyler Bell, Martin Barnes, Gary Gale.

WhereCamp was alive with key figures from the social geography movement who knew the power of these new tools (see some of my photos of WhereCamp on Flickr here).

The importance of the Yahoo! announcement really became clear to me at WhereCamp where I attended sessions all day Saturday including the Curating Big Data Session led by Tom Carden, Stamen Design and Aaron Straup Cope, Flickr, (see Aaron’s slides from his Where 2.0 presentation on “The Shape of Alpha” here and video here).

Anselm Hook, a prime mover for WhereCamp, is a leading philosopher of place making and veteran software developer who led Platial engineering and is now at web consultancy http://makerlab.com. If you missed Anselm at WhereCamp he will be presenting on, Ubiquitous Angels at The OpenSource Bridge, Portland, Oregon, June 17th -19th, 2009.

Anselm describes where he thinks the challenges are:

“We should be mapping information that in some ways has been historically unmappable because it is 1) not valued or is 2) actively seen as threatening or is 3) simply too hard to map using traditional tools.”

The WhereCamp Schedule

The Shape of Alpha

Screen capture from Aaron’s Where 2.0 presentation on “The Shape of Alpha. Original photo from Flickr user “NickIsConfused”.

Aaron Straup Copes’s work on “The Shape of Alpha” puts key questions about curating big data center stage.

Firstly, the exploration of what it means to curate/collaborate over meaning from “the abundance of data produced in the precise but distant language of machines” (also see The Interpretation of Bias (and the bias of interpretation). The Shape of Alpha uses a process of reverse-geocoding to translate machine-generated geographic data into place names that people can understand and relate to.

The shapefiles are built with nothing but geotagged photos and some code called clustr (written by the brilliant  Schuyler Erie – co-author of Mapping Hacks). Anyone can make these shapefiles. You can get the shapefiles out of the  Flickr API. Aaron has been keying off WOEIDs (Where On Earth (WOE) IDs) but as Aaron noted you can key off anything you like – tags are an obvious choice.

Wow! You can reinvent mapping with this stuff.

Very importantly, “The Shape of Alpha,” tells us something about how we relate to place versus location. The emotions, disputes and behavior related to place also emerge through crowd sourced corrections.  For more see this very evocative post by Aaron about corrections and treating airports as cities.  There is a glorious thread/riff and ode to the genius of  J. G. Ballard pursued by Aaron and Dan Catt in their posts (also see Dan Catt’s, J.G. Ballard, Flickr, naked singularities and 3-letter airport codes, and Aaron pointed me to this brilliant “geo-detective work” on Concrete Island, by Mike Bonsall .

Dan Catt created geobloggers and “seeded the geotagging community around the Web.” I met Reverend Dan Catt (Twitter @revdancatt ) at Where 2.0 when he was kind enough to share part of his seat so I could join a very interesting discussion with Aaron on The Shape of Alpha.

As Aaron points out they decided to treat “the airport itself as the town…”  not (only) because they admired the work of J.G. Ballard,  “but because it is the right thing to do.”

Dan Catt has excellent blog posts “describing the nuts and bolts of how ‘corrections’ works.”  Aaron points out,  “in the nerdier of the two Dan sums it up nicely by saying”:

“On a slightly more philosophical level, it’s a never ending process. We’ll never reach a point where we can say “Right that’s in, all borders between places have been decided.” But what we should end up with are boundaries as defined by Flickr users.

…

For us, it’s a first small step into an experiment, and actually a pretty big experiment as we’re potentially accepting “corrections” from our millions and millions of users. We’re not quite sure how it’ll all turn out, but we’re armed with Maths, Algorithms and kitten photos.”

Psychosynthography – “Wearing Geography as a Perfume”

Psychosynthography screen capture from Aaron Straup Cope’s Where 2.0 presentation . Original photo from Flickr user,  “NiteLynx.”

As I mentioned before, many of the ideas raised at Where 2.0 were unpacked and worked through at WhereCamp. For example, Aaron introduced a word psychosynthography in the last 24 seconds of his talk at Where 2.0.

So I spent as much time as I could listening to Aaron at WhereCamp, and asking him about psychosynthography and more (post of this interview upcoming).

Aaron urged the Where 2.0 audience to pay attention to the Psychogeography movement seeded by Guy Debord, and “to wear geography like a perfume.”

Joseph Hart writes in a “New Way of Walking” psychogeography is:

“a whole toy box full of playful, inventive strategies for exploring cities…just about anything that takes pedestrians off their predictable paths and jolts them into a new awareness of the urban landscape.”

Curating Big Data

Tom Carden, Stamen, (picture above) paired with Aaron for the Curating Big Data session. Tom noted:

“The Curating Big Data session for me was an attempt to learn from other attendees (as opposed to teach/lead, as with the Stamen session, “Real Time Web-Based Visualization and Mapping”).  Also, it was an excuse to get Aaron to recap parts of the Flickr Shapefile story for WhereCamp folks, and to get *input* on how to do more things like it. I was a bit disappointed that nobody had really good examples for us, but I was happy with Brad Stenger’s suggestion to look into the upcoming census data as a relevant area.”

Aaron’s work on the The Shape of Alpha and The Corrections project shows, as Tom noted:

“what you can do once you have 150 million geotagged photos, and millions of users who are willing to say I took this thing here and my name for that place is …..”

And part of the significance of opening up the GeoPlanet data set is that now:

“we can try and start talking about the same places, as far as, [for example], these shape files go. So if you are interested in what comes out of the Flickr shape files project and but you also have your own opinion about what shape those places are so the IDs have be open you have to be sure that you are talking about the same thing in the first place.”

And, as Tom pointed out, collaborating over geo data informs us about curating any big dataset:

“it should lead to an overarching discussion about any kind of dataset geo or otherwise and ways in which we can talk about it, and think about patterns for improving that data, for collaborating, even on things like cleanup.”

Warp speed geo-genius Andrew Turner, Fortius One, took these excellent notes for the “Real Time Web-Based Visualization and Mapping” (on left) and “Curating Big Data” (on the right).

On my way to Where 2.0 I took the train from SFO to San Jose which was a delight but a little slower than I imagined. So, unfortunately, I arrived on Tuesday just after Michal Migurski (Stamen Design), Shawn Allen (Stamen Design) presented  Maps from Scratch: Online Maps from the Ground Up. This was on my MUST attend list and it was a wonderful opportunity to get into, “Real Time Web-Based Visualization and Mapping.”   I did get a chance to talk to Michal and Shawn a bit later in the conference but I will try to catch up with them soon for an in depth story.  Below is  Shawn Allen’s map of overlapping data sets from, “Trees, cabs and crime in San Francisco:”

Another follow up I am really looking forward to making is with Liz Barry and her work on S+EM, “an environmental mapping and social networking design project that links New York City trees with the people who care for them” (also see, Creating a Greener San Francisco Tree by Tree).  Also I got a chance to talk to another fellow New Yorker (we have to travel to the West Coast to find time to chat!), John Geraci of DIY City who presented DIY City:  An Operating System for Cities.

Machine Intelligence and Human Intelligence

Aaron Cope, Flickr, on the left is talking to Andrew Turner on the right the CTO of FortiusOne (see Andrew’s presentation at Where 2.0, “Your Own Private Geo Cloud”)

Many of the most interesting conversations happened in between sessions at WhereCamp and Where 2.0.

I caught this one in which Aaron Cope and Andrew Turner where discussing some of ideas Aaron raised in his presentation, “Capacity planning for meaning in the age of personal informatics” (see Aaron’s blog post, Tree planting and tree hugging in the age of personal informatics). The core question they were discussing was what happens when you wire the world at the scale people are talking about and it breaks… Aaron argues that you already have a whole class of people in systems operations that can tell us a lot about how to answer this question.

Ryan and Anselm shared the pulpit for the morning circle pulpit with Ross Mayfield of Social Text who was the generous host to WhereCamp.

Social Reality Mining

“As it stands today, we have no idea what terms and limits of a cloud based citizenship of the Google Caliphate will entail and curtail. Some amalgam of post-secular cosmopolitanism, agonistic radical democracy, and post-rational actor microecomics, largely driven by intersecting petabyte at-hand datasets and mutant strains of Abrahamaic monotheism. But specifically, what is governance (let alone government) within this?” from Benjamin Bratton’s talk at ETech 2009 (picture above), Undesigning the Emergency: Against Prophylactic Urban Membranes.

The other big take away from WhereWeek – Where 2.0 and WhereCamp, was not so much news, but a confirmation of something that has been pretty clear for a while now. (Check out Brady’s posts on reality mining at Where 2.0 last year).

We are moving headlong into the era of reality mining with all its myriad possibilities from: “hedonistic optimization” (this term came from Stig Hackvan when I asked him about some of the ideas central to the HeadMap Manifesto -more about HeadMap later in this post); to new forms of marketing (social reality mining the inside to predict if someone is going to trade business cards in the next 120 seconds – Alex “Sandy” Pentland, MIT, Where 2.0);  to stuff that matters to save us from mass extinction like distributed sustainability – greening production and consumption and our cities; to open government;  empowering indigenous communities (also see Rebecca Moore’s Indigenous Mapping: Emerging Cultures on the Geoweb Presentation); and not to be forgotten, the troubling possibility of new forms of social control.

Smart phones are powerful networked sensor devices in the palm of our hand

As Sandy Pentland MIT pointed out in his Where 2.0 keynote, “Reality Mining for Companies, or, How Social Networks Network Best,” mobile phones have created an ubiquitous instrumented reality that goes way deeper than location awareness. Smart phones are powerful networked sensor devices in the palm of our hand that know a lot more about us than location. With proximity, motion, (accelerometers), voice, images, call logs, email – what is enabled is not just knowing where people are but knowing more about them.

Many of the issues raised by Adam Greenfield in Everyware and in my interview with Adam were on my mind during WhereWeek, also questions that were distilled and explored in this presentation by Matt Jones last year, Polite, Pertinent, and… Pretty: Designing for the New-wave of Personal Informatics and Timo Arnall’s presentation, The Web in the World.

Google Wave, Pachube  Feeds, Sensor Networks and Microsyntax!

Visualizing 24 hours of Pachube logs, feeds all around the world -  built with Processing.

I found myself really wishing Pachube founder Usman Haque had been able to come to Where 2.0 this year – Usman was originally on the Where 2.0 schedule but had to drop out. My small contribution to WhereCamp was to discuss Pachube, Natural Fuse and OpenShaspa in the, Urban Eco-Managment session (see my interview with Pachube Founder, Usman Haque here).

Pachube announced – mapping mobile feeds in realtime, with 3d datastream value time & location based graphing just before Where 2.0.

And, as I was writing up this post, I was delighted to see this post by Bruce Sterling on Pachube Feeds and his challenge, offering:

“(((Extra credit for eager ubicomp hackers: combine this [pachube feeds] with Googlewave, then describe it in microsyntax. Hello, 2015!)))”

Also Anselm Hook, who has an extensive background in video game development, made an interesting point about Google Wave to me:

“btw – there is a preexisting metaphor for the wave – the wave is notable in that it is making the web like a videogame – its bringing real time many participant shared interaction to the web”

“Jens, Lars, and team re-imagined email and instant-messaging in a connected world, a world in which messages no longer need to be sent from one place to another, but could become a conversation in the cloud. Effectively, a message (a wave) is a shared communications space with elements drawn from email, instant messaging, social networking, and even wikis.”

For more on microsyntax see microsyntax.org

Aaron pointed out to me re microsyntax:

“This is ultimately the “magic word” problem, which is essentially the semweb vs. google-is-smarter-than-you problem.”

I will have some more questions for Aaron on the the “magic word” problem in my upcoming interview post.  At the moment I am busy studying some of the thoughts in these links.

http://delicious.com/straup/magicwords

http://www.slideshare.net/straup/the-papernet/22

http://www.xml.com/pub/a/2005/02/16/edfg.html

http://xtech06.usefulinc.com/schedule/paper/135

Privacy: Towards a Win Win and Community Sensing

While a key element of  Yahoo! Geo Technologies portfolio of platforms, FireEagle, not only gives an important set of tools to allow people to “share their location with sites and services through the Web or a mobile device” but also offers up some vital privacy tools, the community sensing work of Eric Horvitz takes privacy and data sharing into new terrain.

Eric didn’t have time to discuss his privacy work in his Where 2.0 presentation, Where, When, Why, and How: Directions in Machine Learning and Reasoning about Location, – it came up in his very last slide. But I ran up after his talk with my trusty old ipod recorder in hand, and got the part we missed! Fascinating stuff that will be the subject of an upcoming interview post. Here’s a little taste of what is to come. Eric describes one of the directions his team will be exploring.

“One thing I want to do, on our research team, I’d like to develop something very simple for people to use. A challenging problem with privacy is usability and controls. Aunt Polly and Uncle Herbie just don’t get all these authentication controls and sliders, nor do they want to invest in figuring them out. They also don’t get why they’re being asked with pop up windows to yes or no to various questions and so on. One Idea is having a useable privacy lens, that you can hold up anywhere and it tells you what you’re showing anybody or any organization, what does the world know about you. And you would like to have buttons to turn sharing off for some items. You’d also like to have a way to go back in time and view prior sharing and logging over periods of time, and to have buttons to push to say erase that segment of your logs.”

Understanding the social implications of what it means to live in an instrumented world is a topic that we cannot afford not think about. But luckily there are lot of people who have been thinking pretty deeply about this for a while now.

And I did my best at both Where 2.0 and WhereCamp to seek out as many of geothinkers as I could, and do interviews wherever possible (I have not had time to mention everyone I talked to in this post but hopefully all the interviews will get on Ugotrade soon!)

HeadMap Manifesto

In the bar of The Fairmont on the last night of Where 2.0, I heard some of the history of Where 2.0, GeoWanking, and The HeadMap Manifesto from Sophia Parafina, Director of Operations for OpenGeo and Rich Gibson, programmer, GeoWanker,  Gigapanner and co-author of Mapping Hacks with Schuyler Erie and Jo Walsh (Jo did a lot of key early work on bottom up urban informatics but unfortunately couldn’t make it to WhereWeek this year).

Check Gigapan.org out! “The GigaPanSM process allows users to upload, share, and explore brilliant gigapixel+ panoramas from around the globe.”

Also I interviewed Paul Ramsey, Senior Consultant, OpenGeo, so more on OpenGeo is upcoming (see Paul’s Where ReCap). Justin Deoliveira (OpenGeo) and  Sophia Parafina did a session, GeoServer, GeoWebCache + OpenLayers: The OpenGeo Stack, which unfortunately I missed as it was before I arrived Tuesday.

I met Rich Gibson at Etech 2009 playing Werewolf and Rich introduced me to his co-author on Mapping Hacks and alpha geek supreme, Schuyler Erie, who also wrote the clustr code that The Shape of Alpha uses.

Joshua Schachter founder of Delicious and the GeoWanking mailing list, [and GEOURL - and MemePool!] now at Google came to WhereCamp and was mobbed by a small crowd eager to get their hands on one of the developer G Phones he was handing out from a large box.

GeoWanking, which is now run by O’Reilly Media, has been the incubator for all things location aware and “neogeography” discussions since 2003 – check out ‘sproke for a Paleogeography vs Neogeography (which, as Sophia notes, was a common topic of discussion at Where 2.0) smack down in which geowanking rules in the form of a list traffic comparison.

Sophia and Rich shared some of their perspective on the early days of GeoWanking and the creation of the HeadMap Manifesto with me and pointed me to many other people to talk to. The prime mover of the Headmap manifesto, Ben Russell, has retired from the scene – perhaps bored by seeing a radical vision gone thoroughly mainstream, or exhausted by the rigors of carrying an idea through the early blue sky years, or just s simply doing something else? I don’t know.

The HeadMap Manifesto is still vibrant today even as much of what it envisaged has already been realized. HeadMap assembled the future in a poetry of fragments:

“you can search for sadness in new york people within a mile of each other who have never met stop what they are doing and organize spontaneously to help with some task or other.”

Anselm explained to me what powered all this social cartography revolution, from his POV, was actually IRC.

“We had a channel on IRC called “#geo”.  And many of us met there.  I met Ben Russell at MathEngine in the UK. Ben and I were fascinated by the future of maps.  Ben, Jo and I met Schuyler, Dav, Dan Brickley (who worked for Tim Berner’s Lee who invented the Web), Rich Gibson, Joshua Schachter (who was just a geek at Morgan Stanley at the time ) … and the snowball took off….  many others.

We stormed ETECH ( Schuyler met Jo there).  We got invited to FooCamp. Schuyler was married to Jo by Marc Powell (Food Genome) and lived at his house.  We pushed so hard on the social cartography revolution.

I did a spinny globe for geourl – a project by some hacker named Joshua Schachter…  we were all friends for years and we had never even met.”

“Can AR researchers harness these new approaches to index reality?”

Radiohead’s laser (as opposed to video) clip made using Lidar

If you have read my interview with Ori Inbar, you will know how excited I was to attend The Mobile Reality panel.  The video is up and it is really awesome to hear Raven Zachary (on twitter @ravenme) get into the fray with augmented reality.

The main take away for me from the Mobile Reality panel was that we shouldn’t get too hung up on the difficulties of achieving fully immersive visual augmented reality and twiddle our thumbs waiting for the long anticipated sexy lightweight eyeware – which is still in a coming soon phase (for more on immersive augmented reality see my upcoming interview with Blair MacIntyre). Because, in the meantime, there are plenty of delightful and useful ways to augment our experience of the world – and not all of these augmented realities rely soley on smart phones as John S. Zeleck showed in his presentation on “Wearable Sensory Substitution Device for Navigation.” Also I had an interesting discussion at lunch with Ori Inbar about the use of audio for augmented reality projects.

Where 2.0 clearly demonstrated that we have an unprecedented amount of information from mapping our world, Ori Inbar noted in his conference roundup. Ori writes:

“My point is not a shocker: all we need is to tap into this information and bring it, in context, into people’s field of view.”

As Ori noted Earthmine and Velodyne’s Lidar showed off two new approaches to mapping the world that have potential to create new opportunities for augmented reality:

“Earthmine uses its own camera-based device to index reality, at the street level, one pixel at a time. They have just announced Wild Style City an application that allows anyone to create virtual graffitis on top of designated public spaces. However, at this point, you can only experience it on a pc!”

Lidar, Ori notes, has also embarked on a mission to map the outdoors. But, the question Ori highlights is:

“Can AR researchers harness these new approaches to index reality?”

Brady Forrest inspects John S. Zelek’s “Wearable Sensory Substitution Device for Navigation” at Where Fair before putting it on and being guided by sensory nudges at the cardinal points in the belt.

Coolest Mobile Locative Media App. at Where Fair

Atsushi Shionozaki of Place Engine – “a core technology that enables a device equipped with Wi-Fi such as a laptop PC or smart phone to determine its current location,” demoed the coolest location aware mobile app in Where Fair – Oedo Yokai. Working with ethnologist, Dr. Hiro Kubota and artist Atsushi Morioka, “Oedo Yokai” is Koozyt’s “first attempt to cross IT (Location Information) and Folkloristics.”

“The Japanese “Yokai” are known to dwell and appear at specific locations. They can frequently be seen within the grounds of shrines and temples, believed to be the border between this world and the afterlife, or in more common places like on a hill or at a crossroads. If the “Yokai” symbolize the mystery, legend, and lore associated with places, as our interests fade from actual locations, the rol, es they play in modern day society will diminish, and the “Yokai” might then cease to appear at all.”

I love this idea of bringing the ancient spirits of place back into our lives with our new tools of location awareness.

Odeo Yokai also reminds me of Aaron Straup Cope’s work on “the idea of every spot being a “history box” which he explained is “one of the threads behind the ‘nearby’ project at Flickr.”

The Food Genome

I cannot end this roundup of WhereWeek without a mention of The Food Genome.

“Food Genome is a big hungry brain that scours the internet, trying to learn everything there is to know about food.”

Watch out for the upcoming launch of this project, it stole the show with an exciting presentation at WhereCamp. You can follow @foodgenome on Twitter now.

To get one of the gorgeous Food Genome brochures you had to ask Mark Powell a good question. Notice an eager hand reaching out in the picture below. I asked, “how would the basic building blocks of the food genome be licensed?” I got my brochure and a rain check on an answer to my question.

The Ubiquitous Media Studio

Another highlight of WhereCamp was hearing from Gene Becker about his new project, Ubiquitous Media Studio which will be located in Palo Alto. The project is still in the early stages of devlopment but it sounds really exciting. I am looking forward to being involved from East Coast.  If you’re curious where this is going, follow @ubistudio on Twitter to stay updated.