We are in the age of unearthing and uncovering data, and only just at the beginning of the age of processing data and dealing with it (see my interview with Anselm Hook, Part 2 upcoming).  O’Reilly’s Strata Confernence 2011, will explore, “the change brought to technology and business by data science, pervasive computing, and new interfaces.” It is, perhaps, one of the most important events of 2011.

Data is driving a revolution much as coal, oil, and steel powered the industrial revolution.  And the world changing insight from Karl Marx that “the industrial revolution polarized the world into two groups: those who own the means of production and those who work on them,” is taking on on new life, as  Alistair Croll, co-chair of Strata 2011, points out in his post,  “Who Owns Your Data?”

“The important question isn’t who owns the data. Ultimately, we all do. A better question is, who owns the means of analysis? Because that’s how, as Brand suggests, you get the right information in the right place. The digital divide isn’t about who owns data — it’s about who can put that data to work.”

Strata is where a vanguard will be meet, not only to discuss this revolution’s futures, but to define how to create, handle, and build the platforms and experiences that will harness the data. My flight is booked!  (Also check out BigDataCamp which takes place the night before Strata.)

The picture opening this post is from Michael EdgeCumbe’s  Fall 2010: ITP Winter Show Project.  A project exploring ways to intuitively get the feel of what it going on with big data sets using “the gestural manipulation and stereoscopic visualization of complex data to create a meditative state for data analysis.”  Michael project will be part of the Science Fair at Strata.  For more on Michael’s work see Noise Derived. I also have a number of the  interesting new interface sessions at Strata in my schedule.

The daily Strata Gems on O’Reilly Radar are great place to get a gestalt of some of the Strata themes, and this post by Edd Dumbill, program chair for Strata, Three key data trends for 2011, looks at the year ahead.  This week, I got the chance to ask Edd a few of the questions that I will have on mind at Strata – see his responses below.

If you have been reading Ugotrade, you will know I am interested in our mobile social augmented futures and there is no question in my mind that these will be unleashed by our new capacities to work with data (see my post here).

Data is the how.

The pic above is from “Secrets of BackType’s Data Engineers.” This post on ReadWriteHack by Pete Warden, an ex-Apple engineer, and founder of OpenHeatMap, really lives up to its title.  Check it out if you want to know how  “three guys (the BackType team ) with only seed funding process a hundred million messages a day?”

I asked on Quora, “What would be the most important developments for Augmented Reality in 2011,” Michal Avny, Strategist & Real Time search expert, wrote:

“AR strongly relies on localized personalized real time information.

Having a stream of tweets based on keyword search, location or circle of friends doesn’t really make the AR experience; it is the processed real time relevant information that will make AR useful and intensify the experience.”

In 2011 Real Time search and Social Search will drastically change to provide the infrastructure required.”

I followed up on Michal’s Quora answer with some more questions – see below in this post.

Also note the response from David Molnar, here is an excerpt:

“2. A wave of actionable, important data APIs opened up, enabling useful non-gimmicky AR apps for the first time. Think geoloqi.com , or the work Max Ogden has done with Portland civic data. Plus of course face.com , email providers and calendar providers, etc.”

Amber Case, one of the founders of Geoloqi, is on the programming committee of Strata and will be speaking. Be sure to catch her session! Posthumans, Big Data and New Interfaces, and if you haven’t already seen it, Amber’s TED talk is a must see.

Geographic proximity is a powerful filter, as is route, and time. But clearly social proximity, social relevance, and shared tastes are also key dimensions for location based experiences, (see my convo with Schuyler of Simple Geo, upcoming).

While the whole business of location based search and curation of augmented mobile social experiences is still, for the most part, uncharted terrain, the danger of key points of control being only really accessible to elite players looms large. I asked Sophia Parafina, a pioneer in the open geo space for some thoughts on real-time local /geosearch and geomessaging, and the future of openess & big data (see Sophia’s response below).

Is the market ready yet for P2P cloud computing?

This is another question I’m following,  Is the market ready yet for P2P cloud computing? It is one of those questions that we seem to have been asking in various forms for a very long while now, but without a  major shift in sight.  The pic above is from, The Cloud Made Open Source “Invisible” This Year.  But, perhaps, we are at the point when open p2p clouds will find a place in the market because of their potential importance in real time social search and discovery. Borislav Agapiev, Search Entrepreneur and founder of Vast.com, writes on Quora:

“I believe a P2P cloud is ideally suited for social & real-time search and discovery.

Consider MapReduce, a very interesting and popular paradigm for distributed computing. MapReduce is very much about bringing computation to data i.e. doing computation at nodes (map) and then aggregating results through network (reduce).

It is very clear now that user attention data (what they click on) is very valuable for search and discovery, yet a centralized model relies upon uploading all that to a single location and then doing a supposed local MapReduce. Clearly, MapReduce could be done across the network, without any centralized uploads.

In addition to the efficiency argument raised here, it is even more important to consider privacy issues. Uploading massive amounts of user attention data to a centralized location is not something that is going to make users warm and fuzzy as we are increasingly seeing.

In a P2P cloud, there is no big brother watching over anyone, all computation and data storage is done in the cloud, fragmented in many, many small encrypted pieces ala BitTorrent.”

Picture above from Brynn Marie Evans, “It takes two to tango: review of my social search panel“

The Delta of Now – Transforming Search into a Social Democratic Act

Picture of Maneki Neko “beckoning” cats from Journeyetc

New ecologies of human and machine intelligence are beginning to change basic social structures – see the Future of Work (Biewald and Chirayath Janah 2010). And projects like Swift River, using search and machine mining to filter out streams on topics of interest that can then be subsequently curated by human beings. This may be extended to the curation of real-time data streams and employment of machine learning algorithms based upon the explicit relationships.

Augmented mobile social experiences are a new frontier in which ideas and practices from a number of fields collide, including: ambient findability (Morville 2005), urban psychogeography, narrative structures, ambient games and devices, 4d (time-space), explorations of place and memory, enchanted objects and people (Kuniavsky 2010), and designed animism (Laurel 2010), to mention just a few.

Mobile local interaction presents an opportunity to invert the search pyramid and to transform search into a social, democratic act (see my interview with Anselm Hook upcoming).  Up until now search has been predicated around a very narrow revenue model. Google has an implicit model of a B2C – business to consumer brokerage. We are only just beginning to get a glimpse of the disruptive potential of C2C – consumer to consumer brokerages. Mobile local C2C brokerages that allow us to transact in a trustworthy way over our local geography in close to real time (Hook 2010) have the potential to enable new forms of social organization. Bruce Sterling’s short story about a networked gift economy, Maneko Neki, is a brilliant glimpse at the disruptive potential of such re-imaginings.

Augmented experiences that shift or change a person’s situated geolocal experience of social reality, and change our relationship to the people and the place by augmenting engagement in, and reputation through, socially driven consumer tie ins and game dynamics, like Four Square, & Gowalla are beginning to emerge, as Kati London pointed out in her excellent keynote at Web 2.0 Expo. And, while the integration of mobile local interaction and an augmented view that shifts our geolocal experience visually will involve creative solutions to some well churned mobile, tracking, mapping and registration challenges, the exploration and development of new dimensions through which we can filter and create trusted and meaningful augmented mobile social experiences is vital, whether you are considering a mobile screen, map, camera view, or futuristic HUDs and gestural interfaces.

Talking with Edd Dumbill

Picture from O’Reilly Community.

Tish Shute: First congratulations on Strata! On the Strata homepage there is a quote from Jason Hoffman:

“My gut feeling is that we’re going to look back at the upcoming Strata Conference like we do at the Web 2.0 Conference in 2004/2005.”
—Jason Hoffman, CTO/Founder, Joyent, Inc.

Why do you think Jason’s comparison might be prescient?

Edd Dumbill: Web 2.0 is a development that ran through every brand that has a web presence and radically changed the way business is done for many companies and brands.

Strata will have a similar impact: every business has data, every business collects an increasing amount of data. This data is the new oil – a valuable raw material that when refined or combined creates value and opportunity.

Tish Shute: The rise of real time was one of your three key data trends for 2011. Hadoop is bringing the capacity to work with big data to more than just a few elite players. But the challenge is still real time. You mention we will be seeing a hybrid approach to real time and batch MapReduce processing. Will we hear more about these approaches to real time at Strata? And, what do you see as the most important conversations on real time data analytics emerging at Strata?

You point out “open source projects and cloud infrastructure means developers can evaluate and learn to love technologies without requiring support or approval from above.” What are the most exciting developments on the horizon for open source tools?

Edd Dumbill: Here are some projects worth watching, in the key areas of real time, cluster management and Hadoop.

* Cassandra and MongoDB — NoSQL databases that will prove vital for anybody with real time big data needs

* Mesos — a compute cluster management tool, modeled after that which powers Google

* Hadoop ecosystem’s continuing maturation, especially HBase and Hive.

Tish Shute: Do you think the market is ready for p2p cloud computing?

Edd Dumbill: The market is emerging for decentralized and distributed cloud computing, and P2P technologies are one way of achieving that. They key trends will be moving computation nearer the data sets or nearer the point of user consumption of the result.

P2P is a difficult model for anybody wanting to commercialize a service, so I think it will tend to form part of a hybrid solution.

Tish Shute: We have seen enormous strides in our ability to work with giant unstructured databases recently. Do you think, perhaps, that the dream of a web of linked data – “a web of data that can be processed directly and indirectly by machines,” will be attained through brute force – i.e. through our ability to harness the power of massively parallel processing, as much as by Semantic Web approaches focused on machine readable metadata? [Also see my question on Quora, “Is this a good approach (www.dist-systems.bbn.com/people/…) to use Hadoop to build a scalable, distributed triple store?”]

Edd Dumbill: I’ve been an observer of the SW for over a decade and I tend to believe that on the web, data means to you whatever meaning you give it as the consumer. With that model, the links are made by the consumer rather than sitting out there explicitly. Some links become de facto standards, and some very few become web standards.

I think the actuality will be a mix of both explicitly stated metadata and that which is inferred. The Semantic Web is a great framework for certain operations, especially interoperable exchange of metadata. A great many more private meanings, never intended to be shared, will be created by consuming software.

There’s no question that machines will learn how to process most of the Web. Furthermore, machines will learn how to process most of the physical world we’re in. And that by the end of this decade.

Talking with Sophia Parafina

Picture of Sophia at Where 2.0

Tish Shute: Sophia you have worked in the trenches for a long time now to support the growth of open geo data. What do you hope to see emerge in 2011 in the field of geo-data?

Sophia Parafina: Better support for displaying and handling location data across multiple apps. Fred Wilson recently blogged about content-shifting, he talks about overcoming content silos across devices. We’ve worked very hard to reduce data silos via formats, but devices are creating their own silos. I would like to see a standard method for sending geo data and geo information to mobile devices.

Producing content for mobile is different from producing content for a computer browser. Web 2.0 produced a lot of infrastructure for browser based interfaces, but in mobile devices that gap has been filled with apps which is fragmenting how data is handled by various devices. What is even more interesting in the mobile space is that devices can push data back that contains location, user updates, photos and even sensor data.  If mobile data standardizes, it could lead to browser based applications and stem the continued fragmentation of the mobile application market.

Tish Shute: Simple Geo and Factual are startups emerging in the geodata space. What do you see on the horizon in terms of both the growth of business opportunities and an open geo data community?

Sophia Parafina: In the near future think we’ll see startups providing curated data + API and in response we will also see companies that provide a single interface across multiple data providers. We saw this when everyone released a mapping API and companies such as Mapufacture provided a single interface across multiple APIs.

We will see a resurgence in data providers repackaging the the 2010 US Census data in different ways to respond to market segments, some of this will be open data but all of it will be provided through an API instead of file. Additionally, we’ll see more data from outside the US.

Tish Shute: What are the biggest obstacles to having the open geodata sets available that we need to enable mobile local interactions and social augmented experiences?

Sophia Parafina: Licensing for both crowd sourced data and private curated open data will become an issue. We recently seen VLC, the open source video player, pulled from the Apple app store because of licensing issues. Also, licensing of content by geography will be problematic, limiting searches by geographical location. In addition, how will licensing of data that is updated by crowd sourcing work?

Multiple APIs for accessing data sources. The current trend for each provider to create an API for their data sets will result in data silos – there needs to be a single sign-on equivalent for requesting data.

Size of data on the wire, the current models for delivering data is based on broadband connections. However, as mobiles increasingly become the way people use the web, the data needs to be sized accordingly. This also goes for mobile interfaces. Have you tried to shop on a mobile device, or buy a train or plane ticket? It’s frustrating and error prone. There is a large untapped market of people who only use the Internet on mobile devices.

Tish Shute: You pointed me to this link in Strata Gems re “an interesting and pertinent (also a competitor to GeoLoqi),” – the Android Tasker app. What do these emerging services bring to the table in terms of the next generation of location based services?

Sophia Parafina: This app let’s your device interact with the environment. I think that this is a great way of using the sensors on existing platforms to increase interaction and to implement ambient findability. The basic premise of Tasker is that some action happens in response to an event in an application, time, date, location, event, or gesture. Tasker has defined 180 actions that can occur based and number or combination of events. This can provide a basic vocabulary for interaction between the user and the device and more importantly between users. Tasker also can use Android script plugins, which lowers the bar to creating your own ambient application.

Programs such as Tasker can provide a way for people to interact with social networks beyond sending messages. People can use their mobile devices to interact with their surroundings with out having to interact with the device.

Tish Shute: We have had many conversations about emerging ideas of geo-search, geo-messaging and geo-fencing. What are the most interesting developments in these areas and what do you see on the horizon for 2011?

Sophia Parafina: The map will fade into the background and become less important. Display of information will be context aware, that includes location. For example, let’s say I make a grocery list, when I’m at the grocery story, the list will just pop-up without the need for me to find the app that has the list. Or reminders or offers pop-up when you are near a place at a certain time, let’s say you need to buy a present for a birthday party for a child, you could send out a request that you are looking for an item and retailers could offer “on the spot” discounts if you are in the area.

Geo-search, geo-messaging, and geo-fencing are geared to towards mobile devices, so I expect to see them soon as part of apps. Building generic applications that implement geo* will fail because that sort of information is useful only within a context. Geo* apps are solutions looking for an problem. The killer mobile app will use these functions transparently to reduce the cognitive load of the user who is busy moving around in the world.

User data gathered from multiple web applications will become consolidated profiles that will used for context aware applications. For example, there could be a service which matches prices of items that you have shopped for on the web, so for example the service would have access to your cookies, know your favorite retailers, things you have shopped for, your location and activity patters (when you are at home, work, restaurant). When you are in the vicinity of a brick and mortar retailer with the same or similar items, the service can send you alert to match the price of the item you found on line. So your digital life will become more closely linked with your day to day activities.

Talking with Michal Avny

Tish Shute: At Web 2.0 Summit, one of the highlights for me was the, Q&A:The New Search Insurgents lunch where Charlie Cheever of Quora, IMO, stole the show. I tweeted:

“One of my takeaways from #w2s is that #quora points to future of augmented mobile social experiences – a search filter for experience! #AR”

In your view what are the biggest challenges for location Q&A to emerge as a search filter for location based experiences?

Michal Avny: The biggest location Q&A challenges yet to be conquered are immediacy (real time dynamic data), relevancy (strong personalized filters) and user experience (simplified interface).

Location Q&A enables different use cases. The most prominent are Follow (follow places, topics and friends to learn about a location), Interact (meet new people based on common interests), Plan ahead (plan a trip, night out or a shopping day by asking and searching for local information) and On-site (check for recommendations, friends, deals, events and traffic nearby).

Unlike Follow, Interact and Plan ahead that can be added to existing Q&A platforms (such as Quora) by attending location specifics as they share similar characteristics, the on-site mode introduces a completely different experience, first and foremost it requires immediate attention. It is real time based and the nature of the data is dynamic. Traffic updates, current events, nearby friends, all that changes constantly. Posting a location question on-site implies the response should be in real time (e.g. best kid friendly restaurant), the normal Q&A response latency wouldn’t work.

Strong relevancy filters are required to accommodate for the overwhelming flood of information. Moreover, some of the data should be filtered by user behavior and preferences, check in notifications (type of relation), restaurant recommendations (type of food, price level, etc), shopping deals (commercial categories) and more.

Mobile experience requires ease of use and simplicity. A new Q&A interface and query language that allows for posting questions should be defined as well as coherent summarized response interface. User on the go should not have to post lengthy questions, browse through tens of results or search for the right service, but instead use a simple intuitive tool.

Tish Shute: Real- time location based search is in its infancy. Real time questions can be answered using different services such as Yelp, TripAdvisor, Waze, Foursquare, IMDb and more. But what are the challenges to moving forward with aggregating these sources and then into “locals” that are able to process and deal with vast amounts of information?

Michal Avny: Using some of the leading location services to answer question is sufficient to start with.

In order to provide broad coverage (worldwide) and reliable information, aggregation of the different services is required for instance to normalize product and service rank, aggregate classified, and more. This is quite challenging as there is no one standard available.

When location Q&A user base is big enough, I foresee a tendency to rely more on ‘locals’ input as the base of information. As the platform grows, communities will be formed with different cultures, relationships and trust levels, making the information more valuable and customizable. Some of the challenges I already mentioned are implementing filters, query language and interfaces to enable using the vast amounts of real time data in a mobile environment. More of the challenges lying ahead are integrating the ‘locals’ data with location based services as they are integral components of the Q&A ecosystem. Merging trust levels and relationships while adhering to different privacy guidelines is a challenge yet to be explored. (This should be discussed in more detail under the protocols topic).

It is quite evident that Quora is now facing growing pains and is struggling to maintain its character. Same as with Quora, it will also be a challenge to support and maintain the ecosystem while allowing for massive scale-up.

Tish Shute: I have been very interested in exploring protocols that will be enablers to micro local interaction and mobile social interaction for AR – particularly the XMPP extensions and operational transform work of Google Wave (now Apache Wave), and PubSub protocols like PubHubSubbub and Erlang based RabbitMQ. We are beginning to see protocols emerging that could enable new real time local services. What do you think are some of the most valuable use cases for “locals” that this new generation of real time protocols can enable?

Michal Avny: AR is about interacting with digital information; the AR ecosystem is composed of layers and components such as devices, platforms, browsers, applications and content. For the different components to interact new protocols, security guidelines, and privacy policies must be in place. A standard will enable local vendors and service providers to publish specials, deals, updates and events for any application to broadcast, identify people and places by proximity (without having to use the same application or device), local recommendations will be shared by services, devices will be able to interact, location based platforms, such as Q&A, will have access to vast breadth of information, geo aware devices will provide consistent experience globally, and much more.

Tish Shute: What do you think are the biggest challenges to going mainstream for this emerging field of real time social discovery?

Michal Avny: The biggest challenge is building towards real time, geo-aware, localized, personalized ambient data. Discovery is in its infancy, location social based Best, Top, and Trending lists with some basic filtering options are available, and this is great as people are getting accustomed to information surrounding them. To some degree it can intensify the AR experience, for instance suggest the most popular dish in a restaurant, or map the best coffee shops nearby, but it is customized at best by friend recommendations and depends on the coverage and broadness of the specific discovery service.

There is a need for the next generation of discovery, customized geo social aware discovery that filters the vast amount of real time data by learning user preferences and behavior (built on top of the much needed local social real time open protocol)

Tish Shute: Who are your favorite startups/upstarts in the the field of real time search and why?

Micha Avny: My6Sense - My6sense provides a sharper and better way to experience your information from feeds you subscribe to (Social Networks, News, RSS feeds, etc.). It’s personal – Content is ranked based on what’s relevant to you. It learns what’s valuable to you by translating your consumption behavior into a personalized ranking function.
My6Sense – because it is a personalized prediction filter, a critical foundation for AR

Topsy – Topsy is realtime search powered by the social web that finds the most relevant conversations happening online. The site’s underlying technology examines popular links as well as the influence of each person citing a link. Topsy augments traditional search engines by finding information that people are talking about.
Topsy – because its ranking is based on retweets and influencers, a great social experience

Collecta – Collecta is a real-time search engine for the social web. It monitors the update streams of popular realtime blogs and sites like Twitter, WordPress, and Flickr, and shows results as they happen. Results can be filtered by status updates, comments, stories, or photos. The entire engine is built around the XMPP standard, which pushes out data on a continual basis, so that for every search you end up watching a stream that keeps updating itself.
Collecta – because it is built around XMPP, a real time experience

Only a two weeks after debuting their first Hands Free 3D video showing the possibilities for navigating Second Life “hands free” without a mouse or keyboard, Mitch Kapor (MitchK Linden in Second Life) and Philippe Bossut have a new demo out – Hands Free Object Editing in Second Life.

Philippe points out on the Hands Free 3D blog that they have already seen a lot of interest in their “hands free” project even from the main press (see this article from the NYT). Hands Free 3D, a project of Kapor Enterprises, is creating a prototypical interface using the 3D Camera designed by 3DV Systems to control virtual worlds like Second Life.

Mitch Kapor told me, they are now working “so that avatars can directly mirror body language and facial expression.”

Mitch very generously gave me an interview in which he not only describes his project to explore how:

the camera could be a central device to a whole new kind of interface the way the mouse became the central piece of hardware that enabled the whole graphical user interface and it enabled the transition from character based computing DOS to the GUI.

But also, Mitch shares some of his thoughts on the future of Second Life. A full transcription follows in this post.

“Moving From Science Fiction to Science”

Mitch explained to me he began to get excited with the idea of Hands Free 3D when he realized:

we had a shot at moving from science fiction to science as it were actually making some of this stuff work that people have been talking about for a long time

As Gwyneth Llewelyn points out much of the so called virtual worlds industry has backed off the bigger vision of a unified metaverse and is retreating into a more limited vision of a multitude of closed and controlled virtual worlds (see Digado’s post Raising Kids in Virtual Worlds and this video from fastcompany.tv to see how this controlled/controlling vision for virtual worlds plays from Disney’s point of view).

But while a bigger vision for virtual environments with a revolutionary role in adult life may not not be interesting to marketeers at the moment, it has a momentum that cannot be stopped. Mitch Kapor made a prediction during the interview that I wholeheartedly agree with:

the big vision of 3D is in the process of happening. It will be very transformative and anybody who is not counting on that happening, is likely to be run over by it.

I got very excited when I heard about the Hands Free 3D project because developing a natural interaction between people and virtual environments to me is one of the “it” projects for immersive 3D.

The dialogue between science fiction and science is of course the ongoing story of the metaverse. And seeing Iron Man which is alive with new possibilities for “seamless interfaces between people bits and atoms” made me think of how very exciting this new chapter in metaverse development is.

The Tangible Media Group, MIT, founded by Hiroshi Ishii has pioneered new couplings of the physical and the virtual. And, alumni John Underkoffler’s vision is definitely in play in Iron Man. Underkoffler’s exact credit flew by me too quickly – but he was clearly a futurist for Iron Man. Matt McGann points out that there is a very cool article about his work on Minority Report here.

Oh I cannot mention Iron Man without noting Iron Man in Second Life (see Massively) and Annie Ok’s latest great machinima!

And, Click on the screen shot below or here to watch the “Hands Free 3D: Second Life Object Editing Demo”

Interview with Mitch Kapor

Tish Shute: How did you get the idea to focus on Hands Free 3D out of all the possible areas you could have begun R&D in?

Mitch Kapor: You were asking me where did the idea come from? It originated in the fact that this kind of difficulty – of creating a natural, easier user interface – that we’ve had is characteristic of virtual world interactions.

There are things to be done about that at every conceivable level. From fixing all the little bugs to a bigger initiative. I was doing a thought experiment about what would really make a virtual world fundamentally easier to use.

I didn’t have an answer, but somebody had mentioned to me – one of the other investors in Second Life – that there are two Israeli companies working on 3D cameras. I had read about and heard about lots of things but this caught my attention. And I started to ask some questions about it. I had seen the video that Johnny Lee shot with the Wii on YouTube.

That had begun to prepare my mind to think about how you could use new types of input devices to control virtual worlds. So when I heard about the cameras I said this is really interesting and I started to make some phone calls and inquire. The Idea came to me that you could use the camera … the camera could be a central device to a whole new kind of interface the way the mouse became the central piece of hardware that enabled the whole graphical user interface and it enabled the transition from character based computing DOS to the GUI.

One of the other things is that I’ve now been around long enough, 30 years – active and professional – that I’ve seen many things come and go and I have a feeling for patterns. So I was fortunate in actually being able to get hold of a prototype of one of the cameras to do some experiments with it.

Tish Shute:
They’re not yet released generally are they, later this Summer, right?

Mitch Kapor:
Well .. it’s unclear. Sometime in 2008 or 2009. There will be multiple manufacturers. They have somewhat different approaches as to how they’re going to go to market. I’d say it’s all being sorted out soon. Everybody I’ve talked to is quite certain that by Christmas season of 2009 at the latest, they’ll be available in high volume at low cost.

Tish Shute:
I just got so excited when I saw you doing this because I think, basically, in terms of free form 3D programmable space which is how I’ve come to see Second Life now, it’s the future. Everyone’s been complaining that the problem with free form 3d programmable space for a mass audience is the difficulty of the interface. So there seems to have been this big retreat back into 2.5D, 3D chat rooms – plugins to Facebook etc. It seems like a step backward to me.

Mitch Kapor:
I think it’s inevitable that we’re going to get fully interactive 3D. It’s all a question of how we’re going to get there and how long it takes. It’s understandable why, for commercial reasons, people do more incremental things, but those are only going to get you so far.

Tish Shute:
Well it seems to me ideas about the evolution of 3D are to some degree being driven by marketing on the web forces at the minute. I suppose the thinking is that you can get these 3D chatrooms up easily and they are more amenable to marketing than a freeform 3D space like Second Life.

But my question is why you didn’t decide to go to game controllers? I suppose this is where a lot of thinking goes because all the kids have already a high level of skill with these?

Mitch Kapor:
Well, I’m not a gamer. It seemed to me that the possibilities with a camera to do the imaging and to be able in real time, to extract out a 3D model of the scene and the objects in it, is fundamentally just incredibly powerful. It feels like the right direction if you can develop it. What I was pleasantly surprised by was actually creating the first demo was pretty straightforward.

Tish Shute:
How did you prevent every random motion being sucked into the program?

Mitch Kapor:
It turns out that the cameras are pretty sensitive. They can detect relatively small motions like the resolution at a distance of 5 to 10 feet is a half a centimeter. That would be one part in several hundreds. maybe one part in a thousand. So it can detect slight motions. I don’t know the details of the software that the camera came with and that Philippe wrote. One of the other advantages is that Philippe, who is the engineer that did the work, has a PhD in computer graphics. And, he has been around the block quite a few times, and had a whole bag of tricks. I know that he spent some of the time writing filtering code to filter out noise in the signal and so on.

Tish Shute:
Do you have to be particular about where you stand at the minute? Can you smoothly go back and forth between when you have to type and things like that?

Mitch Kapor:
No, I’m not anticipating problems. We have another video coming up very shortly where we show object editing. The object editing isn’t as sexy as we would like it because it has to use the existing interface. They’re having to emulate keyboard and mouse. The point is that we have the concept of a control plane, a vertical plane, in front of you, that if you put your hand out so it crosses that imaginary plane, then it interprets what you do as controlling the mouse.

If you push through to the far side than pull it back it doesn’t. That actually works quite well as a gesture. And you get visual feedback when you’re in the control plane, it lights something up, so you can see – OK. It’s sort of like when you’re using the mouse to target an object you can tell tell when a mouse is inside a clickable button. Similarly there’ll be some kind of control zones. When your hand or other body part is in that you’ll get some feedback in the same way that a button highlights to indicate I’m clickable, or you’re over me. It’ll be a similar kind of thing.

Tish Shute:
But you have to avoid ending up with a mapping that’s more difficult to learn than the original one, don’t you?

Mitch Kapor:
I agree with you, but on the navigation and flying, we’ve had people learn to use this in less than 30 seconds. We just stand them up and say lean forward, lean back, stand up, lean to the side, raise your arms, and they’re moving, they’re flying, they’re walking.

Tish Shute:
And you don’t get a problem with the casual motion?

Mitch Kapor:
No. And this was just our first shot at this.

Tish Shute:
I know! I was really impressed that you could actually have done that in 3 weeks.

Mitch Kapor:
I think the start to finish time was a couple of months including the fact that Philippe had never seen the Second Life viewer code. So, he started like any other developer, just downloading and building the Second Life client. And, we never had a camera before! Ha!

Tish Shute:
But this is the great beauty of Second Life – the power that people have to do so many amazing things so rapidly.

Mitch Kapor:
He’s already re-written the code once. We’re totally prepared to give the code to Linden. It’s a little premature because the cameras’ aren’t available, but if the cameras’ were available, we would just donate the code. The nice thing is it’s actually pretty clean. It interfaces to the client at just a couple of points. We’ve isolated the dependencies.

Tish Shute:
But that’s my other question. If you donate the code will it be open source so that other developers could get involved? I know lots of people …

Mitch Kapor:
This stuff, the demonstration stuff, absolutely. That’s the intent. The purpose of this whole phase was just to test what we could do and to promote or evangelize the use of the camera. Get people excited. We’re thinking about what we might do with it.

I’m actually incredibly excited about the thing Philippe is working on now which is to use the camera so that avatars can directly mirror body language and facial expression. So that if I’m sitting in my chair and I have my arms crossed, my avatar will cross it’s arms. If I tilt my head to the side or smile or frown, the avatar will do the same thing. We’re quite optimistic that we can do something compelling in pretty short order, like less than a month.

Tish Shute:
Wow! That is really, really exciting. I think that has just been something people have been talking about a lot recently – to have gesturing and expressions transmitted to the avatar ..

Mitch Kapor:
The reason I get so excited is cause when I started believing we had a shot at moving from science fiction to science as it were actually making some of this stuff work that people have been talking about for a long time.

Tish Shute:
So the plan is to make your work part of the open source community and …

Mitch Kapor:
I don’t have a plan yet. I would say anything we’re doing in this phase we’re happy to give away. At some point I think things are going to become clearer as to the availability of the cameras, what Linden is going to build in, and then businesses that might be built off of what we’re doing.

But I’m very confident that the kinds of things we’re doing now and in the short term are just going to become part of the standard repertoire of things you can do in Second Life in code that’s available to developers.

I don’t have the exact road map.

Tish Shute:
I heard your recent talks in Second Life and how you were very interested in seeing how Second Life could become more of a business tool. I’ve talked about what Second Life and its “cousins” offers in comparison to other open source platforms like SUN’s Project Wonderland and the Croquet platform Quaq. For example, Second Life is a free form 3D programmable space that’s really accessible and easy to develop in.

But in Qwaq you can drag and drop documents in from 2D applications easily, and Wonderland has some great telephony/audio development. I’m totally psyched by what you’re doing because it has the potential to make the free form programmable space of Second Life more widely useful, and it could be bring much innovation to business communications.

I see a future in interactive data visualization, for example, the idea that Ben Lindquist of Green Phosphor has been developing, i.e., that you can actually model business processes dynamically in a collaborative environment. What are your thoughts on Second Life’s potential in business applications?

Mitch Kapor:
One thought is that a more general platform, more general purpose, more open, in the long run, all other things being equal, will be superior to more limited, less capable, more closed platforms, for building any kind of application.

And at the moment, Second Life is the most general and most open platform. So all other things being equal, which usually they’re not, Second Life should be viewed as superior by people who are building a variety of applications.

But there are clearly some things that need to happen. Well let me put it this way some of the other platforms have temporarily at least moved further ahead in enterprise related applications by developing collaboration capabilities.

So the imperative is for Second Life to provide comparable capabilities. It has to do that, in terms of fundamental stability, reliability, in all respects. If it does that then it’s actually going to win on it’s own merits.

Tish Shute:
I absolutely agree with you because in terms of ease of use, it’s the only dynamic networked general simulation platform around. There’s no one else close.

Mitch Kapor:
It’s also I think highly scalable in ways that some other things aren’t. Even though it doesn’t have as many 9′s in uptime as it needs to have, there have been recent signs of more progress. I guess the HTML on the prim stuff is rolling out finally or at least the first version of it.

I think it’s ended in beta now.

It’s not the full thing. But it’s a huge step. That’s going to help a lot.

Tish Shute:
Plus the fact it seems Linden Labs moving towards a more heterogeneous idea of a grid where there’ll be the potential to connect behind the firewall worlds with the main grid .

Mitch Kapor:
I also know that there are some third parties that have done that. They’ve sworn me temporarily to confidentiality. But they have done some very impressive stuff with integrating the web with Second Life in ways that you can for instance in a web interface just go and grab a PowerPoint. In your Second Life window. The power point will just show up. So there is a kind of work around to using the familiar web to get your intercollaboration stuff working. There’s progress. It’s going to be some time before it all sorts itself out.

But to come back to the camera as a more natural interface, I think for personal interaction, is important. It’s going to be a breakthrough.

Tish Shute:
It’s a huge breakthrough also to have the avatar related to your real life gestures. It’s a huge leap forward. When you introduced it at metaverse meetup that really got people’s attention. I have a question. Have you thought about going even further with the thought driven game controllers?

Mitch Kapor:
At some point I intend to take another look at that. I have the feeling that your not doing anything highly profound. Kind of a cute hack.

Tish Shute:
Again they’re not available, I would guess they would give some to you though.

Mitch Kapor:
From looking at earlier incarnations of this stuff I think what they can pick up on is very superficial. So I’m not sure that they’re going to be that interesting cause we really don’t know how to do, without some invasive type of surgery,

Tish Shute:
You can do it with very very complicated brain scanning you can do a lot more, but I agree. Although I did see the Japanese University was using them for severely disabled people. Looked like they were doing some interesting things.

My question is, this is something you mentioned in one of your talks in Second Life, you thought some of the steps forward to make Second Life truly a player in the business world, would be changes on the server level. Were you thinking more about the moves that are going on towards open source and making a heterogeneous grid?

Mitch Kapor:
Yes. I was thinking about letting people run it behind the firewall, and also it’s not just putting it behind the firewall, anytime you’re talking about an enterprise application, the enterprises want to integrate all of their existing IT systems. They already have these very sophisticated systems for managing say identity, and having easy integration of those thing with Second Life identity management is not glamorous but very important.

Tish Shute:
This brings to mind another question. I know I have some ideas about what Second Life really brings to the table for business. No one else has taken on working with dynamic melded states on the internet in 3D to the degree Second Life has. That’s sort of, to me, the essence of it – having groups of people working around 3D objects that can be updated on the fly and modeled on the fly.

Mitch Kapor:

If we do things well there will be a good level of interoperability and all of the open source work and the reverse engineered clones will actually be a good thing.

Second Life is, and I’ve probably used this line, faced with insurmountable opportunities on all sides.

Let me ask you a question. I’ve read your blog, or some of it, but what do you actually do?

Tish Shute:
I spend a lot of time on my blog at the minute!!! You can tell I have kids and dogs driving me crazy [dog is barking in the background], which is exactly why I took this up a year ago. I worked in film and special effects for the early part of my career.

When I had my kids and dogs and all of that it got to be just too much to do 24/7 film production. My son’s nearly 9 now. I tried academia for a while, then I just said forget it..too hard to be in a medieval guild as a second career!

And I actually a year ago when I started looking at this (Second Life) I thought my goodness this is what we sat around and talked about every night when we were doing multiple pass motion control photography in the eighties. And so I started writing about it and that took a life of it’s own. And now it’s become a little ridiculous because it’s an excessively time consuming hobby!

Mitch Kapor:
Are you in New York or the UK?

Tish Shute:
Yes. I’m in Manhattan.

Mitch Kapor:
The reason Second Life has gotten as far as it’s gotten is because of people like you who have become inspired and become obsessed and feel the possibilities and feel them to be so utterly compelling to cause some rearrangement of life priorities.

Tish Shute:
It’s interesting cause it’s like every week I say “Oh I really can’t spend all this time writing!” Then I see something, like this week I saw all the new wave of 3D chat rooms coming out. And it just got me going again! I just can’t bear to not to have a voice because when you see the big picture you want the really innovative stuff to move forward. That’s why when I saw your work on hands free 3D, I said: “Oh my goodness, someone’s taking it the next step. And as you say there isn’t a path that’s clear. There’s no guarantees. But its a path worth traveling, in my view!

Mitch Kapor:
I firmly believe, I have complete conviction, that all of the 3D, the big vision of 3D is in the process of happening. It will be very transformative and anybody who is not counting on that happening, is likely to be run over by it.

Tish Shute:
Right. Of course you’re much more knowledgeable of this aspect of it but in terms of business applications, has anything interesting happened in a long long while?

Mitch Kapor:
There are some interesting things that are happening, I just learned this by accident, that are being kept under very close wraps. There’s at least one consultancy that is doing extremely well with very large prestigious global corporations. They have done a lot of development of this integration of web with Second Life. Their clients are shy. They do not want public exposure at the moment because of the backlash against the overhyping of Second Life that happened last year. I was very heartened to hear about this. I think it’s going to start coming out in the next few months what some of these companies are doing.

Tish Shute:
I agree. Many of the interesting things I know about I can’t write about either because there’s no interest for people developing business applications to have a lot of web publicity about it in the early stages.

Mitch Kapor:
Right. I think we’ll be in this phase for a while. But then we’ll get out of it.

Tish Shute:
In terms of specifics about business application, do you have any dreams for Second Life?

Mitch Kapor:
I would like to just personally have a really good meeting application. Just simple like when you and I want to get together and meet in world, I would like that to be easy, bullet proof, convenient, natural. I’m imagining that we both have cameras, so that we can see each other and you get body language and you get a sense something like what you would get in a face to face meeting. And I want people to have the ability to easily get more realistic avatars, if that’s what they want. And actually there’s a lot of good technology around that now. Where you can just basically take a picture or two with an ordinary digital camera, upload it and get back something that pretty much looks like you.

Tish Shute:
What do you think are the biggest obstacles to this kind of free form 3D programmable space?

Mitch Kapor:
There’s a lot of software that has to be written to bring out its full potential. And not just by Linden or any one company. It’s really a collective effort that is the work of a whole generation.

It’s comparable to all of the work that went into making the ecosystem of the personal computer. Or for that matter the ecosystem of the internet. It requires having the right architecture, it has to stay open. If that can happen I think it’s mostly just a matter of time and some patience.

It is going to happen. There are lots of individual challenges. Tons of problems to solve. I’m not a technological determinist, but at this point I don’t think anything can hold it back.

In a way though having lived through the onset of the internet, while it has changed things a lot, and in certain ways it would be very difficult to imagine life without it, it also has left things the same. I mean people bring all of themselves and their issues into every technological medium. The drama gets played out in a different ways, but it’s neither going to be a good thing or a bad thing. It’s going to be some of both. And so the question is, to me, how people of good will who want to make the world a better place are going to use whatever new things get created in a positive way.

Tish Shute:
I know Mark (Zero Linden) heads up a lot of interoperability work in his office hours and other meetings. But I got a couple of emails this week saying that all these groups that are working off of either clones or reverse engineered, and there are so many of them, and some are under wraps too, need to actually meet on an even more regular basis?

Mitch Kapor:
That’s true. I guess it’s much more desirable for people to meet and talk, and if they don’t for awhile, you get more noise in the system. It just will take longer to put things back together.

Tish Shute:
That’s what I was thinking, that it’s become pretty clear to me that cooperation, if it is going to happen, has to happen around the clones and the reverse engineered versions of Second Life because other platforms are not prioritizing interoperability at the moment, that I know of.

Mitch Kapor:
People will call – this and that should be happening but my view is that the ecosystem is still sufficiently underdeveloped that there is a risk of attempted premature standardization.

If you look at the history of things, It’s very important for there to be working instances before anybody attempts to standardize anything.

There’s a lot to be learned in the early history of the internet. pre-history, from the 60′s up through the 80′s — when the basic protocols were being developed. There’s some very smart people working on that and a certain amount of looseness is actually quite important now.

There’ll be people who want to prematurely standardize and get everybody together and all you’ll wind up with is a massive crud.

I thinks that the power of the open systems is so much greater than the walled gardens Also the open source ethic is so deeply established in large parts of the development community, even in enterprises, that overall I’m not too worried about it.

When the functionality of whatever it is, is that well known and well understood, that’s the period in which the open source alternatives can really flourish. When there’s still a lot of evolution in functionality, and design in the user experience, open source techniques can become too slow.

So it’s going to be somewhat chaotic. I think we have to embrace or at least make peace with a certain amount of chaos right now and the understanding that it’s likely to settle down. The chaos is not the last word.