Tim O’Reilly has outlined some of the world’s big problems in his talks, and urged technologists to “work on stuff that matters.” I was one of O’Reilly’s listeners at the Web 2.0 Expo in NYC (see my post here).

But, I found out at the Head Conference in London, recently, that O’Reilly is doing more than just talking about solving the world’s problems. The O’Reilly VC company is investing in technologies that tackle these big problems, for example, a very interesting startup, AMEE.

AMEE is a new company with a VERY big, world changing idea – “to create  a neutral technology platform to aggregate all the energy consumption data in world ” – “to be the world’s energy meter.”

I was fortunate, when I was in London, to get an opportunity to chat with Tim O’Reilly about AMEE, Web 2.0, and the role of virtual worlds in positive global development. Also, I met the CEO of AMEE, Gavin Starks.

We are still, just, in the pre “Internet of Things” era.  But, soon, as Bruce Sterling puts it, we will be able to “Google our shoes” or find out which super market shelves are out of sandwiches at any particular point in time! But for now, it can still be very hard to find a sandwich, even in central London. So, I had plenty of time to talk to Tim O’Reilly whilst searching for a hand held bite to eat.

We journeyed past several sandwichless restaurants (Tim picked up the Financial Times under his arm in the picture above in one of them), and super markets with shelves stripped bare except for some end of the day sushi (it looked scary so we passed on that).

Finally, McDonalds came through for us with the sandwich in the top left corner of the photo above.  The full interview is later in this post.

First, more about AMEE.

Tim O’Reilly says he doesn’t like predicting the future. But the future comes to Tim O’Reilly in very powerful ways.  And AMEE asks us to play a new proactive role in our own future. AMEE’s call to action is:

“If all the energy data in the world were accessible, what would you build?”

AMEE, to me, is a quintessential example of an effort to harness the key paradigm shifts of Web 2.0 (see O’Reilly, “What is Web 2.0?“) to tackle some of the world’s most pressing problems.

AMEE’s mission is to be a neutral technology platform, using open source and standards, and an architecture of participation, to address the need to standardize measurement, encourage collaborative development, and create a market place for energy data.

AMEE’s goal is to enable us to understand energy consumption from the level of the individual to the scale of whole countries.

This would address the need O’Reilly notes here his son-in-law Saul Griffith argued at Emerging Technology Conference earlier this year:  “to pick a target CO2 concentration and work backwards to get to an energy policy, rather than guessing at an energy policy with fingers crossed, hoping for a climate outcome that is tolerable.” AMEE is also involved in Saul’s Wattzon initiative.

Gavin Starks, CEO, AMEE, (pictured above standing under the Head Conference banner – a recording of his talk is here), explained:

AMEE’s vision is to aggregate all the energy data on Earth. By energy I include electricity, gas and all types of fuel, water, waste, you name it: everything we do is energy consumption, which means really building towards our sustainability footprint rather than just our carbon footprint. The initial thing we’ve focused on is Carbon and CO2, because that’s the most pressing issue we have to face: but it’s quite a thin layer on top of the whole sustainability question.

AMEE is not building the front-end applications to harness this energy data. Gavin noted:

We’re aggregating all the standards. This is a massively complex area, so we’ve got a science team whose job is to harvest all the scientific research and methodologies. That’s not something developers tend to want to go anywhere near: it unpacks itself into enormous amounts of complexity very quickly (e.g. building methodologies that have 700 data points).  Our Chief Scientific Officer, Dr Andrew Conway, is ex-NASA and has worked on massive scientific data analysis.

We aggregate government standards and other international standards, so that you know when you’re integrating with us, that you’re working to those standards. But much more than that we actually publish those standards on an open Wiki (the Wiki and the API actually talk to each other). As these standards emerge and evolve, we’ll be tracking them, ensuring that we are up to date, and granularity is added as needed.

We were very fortunate that one of our first clients was Defra which UK’s Department for Environment, Food and Rural Affairs. Defra is currently restructuring into DEC – the Department for Energy and Climate Change. So in the UK they’re actually fusing Energy and Climate Change into a single government department. I think this is quite remarkable: a giant step forward.

We were contracted by them because they were building a national campaign to raise the awareness of every citizen’s personal and household footprint, and they were looking for an Open Source solution – which AMEE is. Defra/DEC now use AMEE for two purposes: they make their standards available through the AMEE platform, so that everybody else can use them, and as the back-end solution to their national campaign, “Act on CO2″.

AMEE is SaaS – a web-service API – we’re not trying to compete with anyone on the front-end development and delivery. Because we aggregate two moving targets: standards and consumption, we enable those integrated with us to be current at all times.

Who Owns the Data?

But, if AMEE hopes to harness global network effects as a neutral aggregator of energy consumption data from individuals, businesses, and governments, one of the key questions that AMEE must answer (also a key question for Web 2.0 in general) is: Who owns the data?  I have delved into this question before on Ugotrade. See David Levine’s conversation with Eben Moglen on privacy here.

Gavin told me that this is a question AMEE has given a lot of thought to.

How AMEE answers this question, Who owns the data?, will probably determine the success of their mission as an ethical endeavor,  and their ability to scale and leverage the network effects of the internet as a platform while still allowing ” a very granular level of energy activity to be tracked.”

Gavin explained where AMEE is going re their approach to this issue. And, how this relates to AMEE’s business model – software as a service (SaaS).

We’ve spent a lot of time thinking about the best way to approach this, from both a commercial perspective and an ethical perspective.

We’re in a position where we’re aggregating vast amounts of personal and business information, on an ongoing basis. If we were to integrate with your credit cards, your smart meter, your supermarket, and your vehicles: we actually know everything about your life. Aside from the Data Protection issues we felt, “How could we protect everybody around ‘excessive’ exploitation of that data and ensure the user has long-term control?”.

As the saying goes, “we don’t know what we don’t know”: when it comes to predicting what our privacy issues will be, and as the data around our physical lives becomes digitally available, we wanted to err on the side of caution. Danny O’Brien, the EFF and MySociety have certainly helped to shape our thinking in this respect.

So if you’re a Google user and you go into the Google and you kick off a calculation, all your answers to the questions are stored in AMEE – but we don’t know who you are. We’ve got an anonymous key, Google’s got the anonymous key. Google will have your user name and so on and so forth. In AMEE we’ve got the aggregate of the responses to the questions. It’s up to Google’s Privacy Policy to determine what they and their user’s can do.

However, this doesn’t preclude us from enabling data-portability on behalf of the user. The anonymous key is not dissimilar to an OpenID, but applied to a specific data set. We are heading towards allowing you to control your data’s portability, as an individual.

It’s a fragile space. We want to give you the opportunity to glue together your AMEE IDs with your OpenID, or whatever login you choose, so that you’re in control. We aim to enable this to be both cloud and edge-based, which while fragile, is in the interest of the user. The more value we can provide, we believe, the stronger the value of our proposition.

But, because were dealing with a range of different massive organizations … imagine credit card companies sharing data with energy companies sharing data with petrol companies, sharing data with airlines etc. That’s a massive challenge from a business and political perspective – almost impossible to navigate.

We can enable people to collaborate, by making it opt-in all the way through the chain. We don’t undermine peoples existing databases: we can actually add value to them, or we aim to add value to them. We think this has got a huge amount of potential to stimulate new business for our clients.

It’s very “web” in its execution: we are part of an ecosystem. Part of our imperative is to be commercially enabling to everybody else. If we’re not being commercially enabling to other people, we’re not going to get the kind of scale of change that we need.

This was another design feature. We felt, “how could we create something which other people could build businesses or platforms on top of”?  And, how could that scale incredibly quickly? If we’d gone beyond our boundaries as an API, we would have been competing with people we want to work with.

While many services have taken similar strategic approaches, most seem to start with a form of lock-in, or evolve quickly to the point of lock-in, which satisfies a current trend in their valuations. We believe this trend will change and adapt to a more “privacy-based” intelligence, which has substantial value.

Tim O’Reilly in The Magic Circle

The motto of The Magic Circle where the Head Conference was held, “Indocilis Privata Loqui,” can be roughly translated as “not apt to disclose secrets.”  One of the wonderful displays of memorabila there was Robert Houdin’s Mystery Clock (picture below).  Luckily, for me, I was was treated to a full explanation of the “Mystery Clock” by another attendee during the Head Conference cocktail party.

As Tim O’Reilly pointed out, in his interview with Aral Balkan, he felt it was a privilege to be talking in this theater and center of magic.  Capturing the magic, spreading the magic, and sharing the magic is at the heart of what he has spent his career doing.  He explained:

I guess this is kind of a root idea for O’Reilly…… When I look back on my career….What did we really do?  Find cool people who were doing cool shit. They didn’t really need any help from us. But then there were a bunch of people who were saying, “How did they do that”? Those are the people we help.

We find the people who are doing what appears to be magic. The Arthur C. Clarke kind of magic you know…. We document – we teach people how to do it.

It is such a great  privilege to be here in a theater devoted to magic – The Magic Circle. This is really what we try to do.  We try to capture the magic, spread the magic, share it with other people.

Interview With Tim O’Reilly

Tish Shute: I was interested in your comment on Chris Brogan’s blog post the other day.

Tim O’Reilly: Actually it wasn’t Chris Brogan’s post. Dennis Howlett was a guest blogger. I was reacting to him saying there is nothing new in Web 2.0. My reaction was: “Well gosh it’s very easy to make a straw man out of Web 2.0 and say, ‘What’s new?’”

Howlett was specifically reacting to the Web 2.0 start ups that are superficial and not really what the trend is all about.  For me, Web 2.0 is about the internet becoming a platform. Does he think that is over?

And, it is about understanding that the rules of business change when the internet is a platform. I think a lot of people do that with Web 2.0 [make it a straw man].

They don’t like the term Web 2.0 and they attach ideas to it that reflect the most superficial elements. And then, they say these aren’t interesting. And, what he was saying was that there is a lot of superficial social media stuff – consumer apps, and what really matters is what will bring ROI to business.

I just said that he is totally missing the point because learning how to use the network as a platform matters very much to business. The same rules that apply to everyone else apply to businesses.

So, for example, I have made the point in my talk in New York, just a few months ago, that in many ways you can think of Walmart as a Web 2.0 company. They are infused with IP – they are taking the data that the users give them by buying things and making themselves a more responsive organization by using that data.

That is the heart of Web 2.0 in the enterprise. Not, do they use social media or not, or the social media buzz words.

Tish Shute: I am going to do an interview with Gavin Starks, CEO of AMEE. Could you tell me about your role in this project?

Tim O’Reilly: My role is as an investor. Our venture firm, O’Reilly AlphaTech Ventures, has just finished closing on investment in AMEE. We think global warming is a really important issue to come to grips with. And, a big part of it is actually keeping track of all the carbon we’re emitting. AMEE helps with that problem.

Tish Shute: What is the potential, now we are beginning to break virtual world technologies down into basic open source building blocks, to create useful mashups with sensor technology, Web 2.0 and Virtual Worlds? Could Virtual Worlds play a key role in this work of instrumenting the world?

Tim O’Reilly: First of all, I don’t think that Virtual Worlds in the Second Life style will have this role …while I like the concept of Second Life, in that we have a Second Life in a virtual world, I am not sure that 3D avatars are [the way to go]…. at least they are certainly not my Second Life.  My Second Life is in other types of media.

But, when I look at this idea of instrumenting the world, one of the things that is very, very clear is that we are turning all the millions of consumer cameras into sensors.

For example, Microsoft’s Photosynth demonstrates how these consumer sensors can be used to build 3D models. We are starting to build a 3D representation of the real world, not a separate virtual world. And, we are all going to be part of that world. So I think that the real Second Life will be ………..well I think the first layer is going to be….to get the 3D models of the world as it is, and then we will have doorways into additional rooms and additional spaces.

That’s when its going to take off because people are going to get used to it through navigating the real physical world with maps – with 3D imagery of buildings and spaces.

And, another piece of this… I talked recently with Carl Bass, CEO of Autodesk……and he was talking about how much even the Autodesk workflow is shifting to scanning things first.

He was describing how they have built this new demo center in an old building.  The first thing they did was photographs and measure it exactly. Then they go and make stuff that fits in the exact space.

Also, he was talking about how in his own hobbyist work, he found this shark’s jaw and he loved the curve of it.  So he scanned it and made it into the arms of a chair.

Tish Shute: I think the two important pieces that might come out of virtual worlds technology are the real time interactions where people can view the same application or context as and when you do, and the rapid prototyping.

Tim O’Reilly: Yes rapid prototyping, no question. But, as I said, the idea of moving an avatar around isn’t very compelling to be quite honest.

Tish Shute: But the web doesn’t do real time interaction very well does it?

Tim O’Reilly: Well Twitter is doing a pretty good job!

Tish Shute: Yes I love Twitter. But, in terms of if you want to get a 10,000 foot view and gather around and interact with data with other people…..what about that?

Tim O’Reilly: Well that’s true. I am using that example of the Squeak based virtual world that Fidelity is using. And, that is exactly what they are using it for – business interaction.

Tish Shute: What about the role Virtual Worlds might play, for example, in instrumenting the world through facilities management?

Tim O’Reilly: Yes, SAP did a very interesting project on property management – and that is interesting.

Tish Shute: Yes, on my way back to New York City tomorrow, I am going to stop off in Zurich and visit Oliver Goh who worked with Denis Browne, SAP, on that project. In fact you showed a picture of Oliver’s Goh’s avatar demoing the virtual counterpart to his instrumented Playmobile house in Second Life in your post, “SAP as a Web 2.0 Company?” (see the picture below).

Tim O’Reilly: I make no attempt at predicting the future.  So you could well be right that Virtual Worlds will be a very powerful tool.  But, I think with the pace that other technologies are progressing, we will get there with photorealism and video, etc.  I think the fundamental problem in most virtual world stuff is the idea of the avatar.

Tish Shute: Why do you say that?

Tim O’Reilly: Well just imagine if when interacting with people in the real world you had to look at yourself interacting with someone else.  First person point of view is our fundamental experience.  And, you are being forced to see yourself in the third person.

Tish Shute: I have always been more interested in avataring the world than in my avatar identity.

Tim O’Reilly: There is something very interesting in having an avatar that you don’t ever see.

[At this point there were less than ten minutes before Tim's interview for the Head Conference, so it was time to concentrate on eating!]

Virtual Worlds: Where Web Meets World

As Ian Hughes, IBM, notes in a recent post on Eightbar:

The last few days have seen a plethora of virtual world pitches, reports, articles and blog posts around certain types of virtual world platform. The first was over at @monkchips a.k.a James Governor analyst blog around a visit to Microsoft to see about the ESP platform. This appears to be a high fidelity simulation platform and toolkit. The second was widely reported. But Wagner’s New World Notes is the one most of the metarati will have read on the matter. This centres around some statments by Craig Mundie that avatar based interaction was of limited interest and really it was photosynth that was the way forward, modelling the real world from photos…….

I agree with Ian when he says:

the interesting thing here is that all the discussion is not about why would anyone want a virtual world, but instead what sort is best.

Tim O’Reilly was not questioning, in my view, that we are moving towards new understandings of virtual spaces or virtual worlds but whether avatar based virtual worlds will be the most useful model.

I began my questions to Tim O’Reilly by bringing up his comment on Dennis Howlett’s post not only because he succinctly states there what is really important about Web 2.0, i.e., “internet as platform, and the rise of applications that harness network effects to get better the more people use them.”

But also, this comment caught my attention because Tim used virtual worlds as one of the examples of the value of Web 2.0 to enterprise.  He wrote:

Guess what : they [SAP] understand that harnessing users is good for their business. At O’Reilly, our InPractice division is working with them to actually turn their documentation into an open source, crowdsourced project. They are doing fascinating experiments at SAP Labs with how to integrate virtual worlds into property management. They have built a great internal social network for employees that has already affected their HR practices.

It is Oliver Goh’s instrumented virtual house in Second Life that Tim mentions in his post “SAP as a Web 2.0 Company?” to make the point that:

This prototype is also very on trend with one of the big ideas we have about where Web 2.0 is going, towards Web 2.0 applications that are fed directly by sensors, so that “participation” no longer just means typing on a keyboard, but the accidental information we create “merely in living as and where we live.”

I have been blogging Oliver’s work prototyping various use cases for virtual worlds in facilities management and energy optimization, e.g., virtual operations centers, in Second Life and OpenSim, since its inception (see here and here).

Now, Oliver has developed a complete solution for sustainabililty in the real estate industry that optimizes energy consumption through the entire life cycle of properties - see here for more.

Also, look out for some interesting intersections between AMEE’s mission – ” to create the world’s energy meter,” and Oliver’s mission to “optimize the world’s energy usage” in the future!

I visited Oliver in Zurich on my way back to NYC from the Head Conference.  In the picture below, Oliver is standing by the Playmobile house that is RL counterpart to the virtual control center house pictured above!

The ability of virtual worlds to play a role in solving the world’s pressing problems is, in my view, linked both to their ability to fully integrate in Web 2.0 and “real” world  data.

I have been blogging a lot on these issues!  Rob Smart, IBM, (see my recent interviews with Rob here and here – “Web 2.0 to OpenSim Made Easy”) has been doing some very interesting work recently integrating JSON support to OpenSim. This is one of the recent important steps forward in virtual world to real world communication.  See this cool video, “OpenSim Meets MQTT jedi mind numbers.”

The powerful value add that virtual worlds, even in a basic form, have shown in the realm of social media that “the people with you can view the changing states of that application or context as and when you do” can also play an important role in the front end applications for projects like AMEE and Oliver’s work.

This is not to discount the role of social media virtual worlds in the participatory work of instrumenting our planet. There is already  a nice integration of AMEE  with Second Life. See Jim Purbrick’s Carbon Goggle’s for Second Life here.

Carbon Goggles from Jim Purbrick on Vimeo

But, if virtual world technology is going be part of the evolving power of the internet to help us solve the big problems facing humanity, there must be an evolving vision for virtual worlds and their relationship with the “real” world.

Most likely, many of the dichotomies, e.g., the notion of avatar based or non avatar based, or simulation versus augmentation, and mirror worlds versus virtual worlds, will increasingly dissolve as all these aspects of virtual reality are woven together into the fabric of everday computing to form new digital/physical realities. And, while I’m not trying to predict the future, perhaps, this will happen sooner than we think!

Here is the video of our last Metaverse Meetup: OpenSim & Virtual Worlds Interoperability 7.23.08 (from Vimeo). The video of this landmark event was produced thanks to the awesome Annie Ok, Artist, Creative Director, Curator, Video Director, Metaverse Evangelist/Consultant, Co-Organizer of Metaverse Meetup.

While Annie’s first love is art, she has been involved in an extraordinary number of projects (see her bio here). Notably, Annie Ok, with Jeff Crouse, & Stephanie Rothenberg made the documentary and helped with the amazing Invisible Threads project which shows how excellent Second Life is for such innovative mixed reality installations. The documentary premiered at Synthetic Times. Annie also created the interactive, collaborative Timeline of Virtual Worlds that the whole community can help with.

Photos of the meetup are now posted here on Flickr and some nice portraits here on Facebook.

With the video Annie sent out a great write up about the meetup.

Annie noted:

“Adam Frisby and David Levine gave us incredible insight into OpenSim and shared compelling details that really expanded on what has previously been known about its amazing potential and revolutionary role in the future of the metaverse.”

And she was very kind about my really minor supporting role!

“Tish Shute was great as the guest moderator, asking key questions and adding salient commentary.”

And I really agree with Annie’s synopsis about what is at the heart of our metaverse meetups!

“It was so nice to see all the familiar regulars as well as meet the new ones. In true Metaverse Meetup style, we migrated en mass to a local bar where we continued the conversation about all things metaversal and had fun hanging out with fellow avatars until the late hours.”

Thanks to everyone and especially Global Kids for making the meetup possible.

Please be sure to check out the list of Metaverse Meetup links on the new About page. There are now Metaverse Meetup group on LinkedIn, Flickr and FriendFeed, as well as a list of Metaverse Meetup chapters in other cities for those of you who are not based in NYC.

Looking forward to seeing you at the next meetup!

Screen shot from realXtend’s wickedly cool avatar tech demo (see video here).

Some people may have walked away from Virtual Worlds 2008, NYC, thinking the vision of the metaverse has boiled down to two notions: 1) every toy should have its own own virtual world and 2) may a thousand walled gardens flourish. But, if you did come away thinking that, you missed out on another important current at the conference – the rapid growth of the open metaverse and the excitement of developers, architects and visionaries who are exploring its potential.

The discussion at the Open Source Virtual Worlds Round Table included so many of the key players, including Philip Rosedale, and covered such a big chunk of issues that that I have transcribed it and published it at the end of this post – the audio is here. The audio quality is poor (except for the round table facilitators from OpenSim, Sun’s Project Wonderland, Qwaq and myself as we were sitting right on top of my ipod!) So, I hope the transcription of the discussion will be useful to all those involved in pioneering the open source metaverse.

The dichotomy of visions – an open metaverse or a thousand walled gardens – present at VW 2008 did not escape the very savvy virtual world writer Wagner James Au (Hamlet Au in Second Life) who narrates this tale of two conferences on GigaOm, here and here. Hamlet, author of The Making of Second Life, and part of metaversal thinking from the early days is in unique position to understand the accomplishments and vagaries of its prodigal children.

The inadequacies of the short term constrained visions that held the main stage at Virtual Worlds 2008 were also commented on by Cory Ondrejka, one of the founders and former CTO of Linden Lab who wrote on his blog:

Is this really the Metaverse? Is this even the 3D internet? Isn’t this the same week that we saw Congressional testimony on virtual worlds, on their potential impact on education, community, business, and communication? Technology is just enabling us to take incredibly bold steps, to connect people in entirely new ways. From 3D camera technology to spatialized voice to novel interfaces to mobile to augmented reality, we should be ready to embark on the next exponential curve, building on everything learned from Second Life over the last 8 years.

Not game over by a long shot – the party has just started!

The young guns are working with the open source and reverse engineered derivatives of Second Life to explore the full potential of avatar presence in a 3D, interactive, dynamic, networked environment. And this is just the very beginning.

On 3rd of April the OpenSim platform was load tested with the amazing Antigone (top image), who sang live in OpenSim in an event sponsored by the Sine Wave Company (boardwalk leading to the stage in OpenSim above).

And, if you were thinking that Philip Rosedale stepping down as CEO of Linden Lab was a sign that Philip was giving up a leadership role in the future of the open metaverse, think again. Philip’s continuing deep engagement with the technical and business challenges of the Open Metaverse was quite clear when he showed up and sparked off an intense discussion at the Open Source Virtual Worlds Round Table.

In this picture, Philip Rosedale, Linden Lab, Zafka Zhang of HiPiHi, Wagner James Au (author of The Making of Second Life), Tess Linden, Eilif Trondsen of SRI Consulting Business Intelligence are just some of the metarati at the round table.

Also very visible at Virtual Worlds 2008 was Cory Ondrejka. And while Cory is now consulting on a wide range of entrepreneurial, technology, and innovation projects, he has a tremendous amount of domain knowledge about the design, architecture, and scaling challenges of virtual worlds. And, as I saw Cory chatting with the new kids on the block, I found myself thinking, how interesting it was that his experience was actually on the open market at this critical juncture for open source virtual worlds. (But Cory did hint to me that he may not be as free to consult in the near future.)

Cory noted in a brief chat after the conference that there are a lot of potential stumbling blocks for Second Life competitors and the aspirant architects of the Open Metaverse face challenges linked to design (repeating failures from the late ’90s), architecture (given target market and use, are you picking the correct technologies?), and scaling (do any aspects of your design require vertical scaling? what are the choke points?). Cory will be writing up more of his thoughts about some of this on his blog, I think.

What is the architecture of the Open Metaverse?

Picture from Nicole Yankelovich of Sun Microsystems Wonderland blog post – from left to right, Remy Malan, Qwaq, Nicole, me, Jani Pirkola, realXtend, Adam Frisby OpenSim, Adam Johnson, Genkii.

The power of virtual worlds for business collaboration was the emphasis of Sun and Qwaq’s presentation during the Open Source Virtual Worlds round table. Nicole Yankelovich demoed Project Wonderland’s multiple group voice chat that cleverly simulates “watercooler chitchat” that real-world office spaces provide and impressive telephony that allows users to communicate in or out of the virtual world space by phone (See Nicole’s blog and Hamlet’s write up on GigaOm here for more). But the discussion centered on the open metaverse as something akin to the next generation internet where business, consumers, communities and the individuals and organizations of public life have the possibility to interconnect and interact as well as stay behind firewalls. And the voices for this vision came from the open source initiatives with their roots in the Linden Lab Second Life technology.

Topics discussed were:

What is the business model for Linden Lab in the open metaverse? (Philip gave the most clear and convincing explanation of this I have heard.)

How will forking not become an issue and break up the open metaverse before it has begun?

Will the open metaverse have a virtual currency?

How can truly wicked avatars using blended animation and inverse kinematics be deployed without choking performance?

How will IP be protected and will obfustication be employed?

How will asset/content development flourish in the open metaverse?

The latter question included a discussion about different models of content production and content monetization in virtual worlds including new ideas like the open source content project of Clever Zebra. For info on their upcoming vBusiness expo see here.

I relayed a couple of questions from Peter Quirk, EMC, who unfortunately couldn’t attend the conference. Peter’s questions produced some excellent discussion and responses.

1) Is the lack of useful assets to populate a world, whether it’s OpenSim, Croquet or Wonderland the number one business issue?

2) Instead of driving to a complete implementation of LSL, has OpenSim gone off in open source fragmentation land inventing their own scripting extensions which are guaranteed to cause problems going in the other direction?

If you are interested in any of these questions you may want to study this transcript that includes lengthy comments from Philip Rosedale (Linden Lab), Adam Frisby (OpenSim), David Levine (IBM) – Zha Ewry in Second Life, Jani Pirkola (realXtend), Christian Westbrook of WelloHorld, and several other key architects of the open metaverse.

What’s New?
Enterprise Applications in Open Source Virtual Worlds?

I moderated two enterprise round tables at Virtual Worlds 2008, one on Open Source Virtual Worlds and one on Enterprise Applications and the discussion at both was driven by key innovators in these areas.

The Virtual Worlds Conference and Expo in Fall 2008 will have a full on enterprise track Chris Sherman says. But the “knights of the enterprise round table” gave us taste last week of what is to come.

It was fascinating to hear Michael Osias from IBM and Oliver Goh from Eolus and who are pioneering enterprise command and control centers for building automation, green data centers, energy and facility management debate with Mark Phillips from the Simulation Business Unit of MASA Group Inc.

“What’s new?” about these enterprise applications on OpenSim, asked Mark Philips who works at the very highest end of business simulation. It is true, from the perspective of the lofty budgets that high end business simulation is accustomed to, command and control centers in 3D environments are nothing new. But Michael and Mark who have worked together in the past did come to agree that never before has this kind of software been accessible for cheap and rapid protoyping/development/and deployment in this way and with the potential to be used both inside and outside of firewalls in both in secure and massively networked environments.

Virtual worlds for children maybe a marketers utopia/cornucopia but the open metaverse is still the most exciting social and technical paradigm shift since the mass adoption of the internet.

A New Era of Business Tools and Business Process Modeling

Melanie Swan from MS Futures, one of the facilitators of the Enterprise Applications round table described how open source data visualization tools will open a new era for business tools that have given us little that is new in recent years.

And Ben Lindquist of Green Phosphor described how virtual worlds will be more than collaborative spaces they will become where business processes are modeled on an ongoing basis within the enterprise.

What I see happening is knowledge workers, analysts, middle management, spending time in a virtual space modeling the actual business that they do and doing that on a continual basis.

Imagine a network of pipes and other objects that actually represents your business processes, your organizational model, your supply chain; and you can see your people working on it in the virtual world. They’ll be able to perform “what if” scenarios – answering questions such as “what if we combine these two offices – what does it do to responsiveness”, and then when a change works well in the model, it can be implemented in the real world through integration with the ERP system.

IBM’s big news at the conference was that they would be working with Second Life behind their firewall. But with 6000 plus IBMers in Second Life and a working interest in interoperability issues, it is common knowledge that IBM gets the open metaverse and its potential. Perhaps what is more surprising than the news of Second Life being experimented with on IBM blade servers is that this collaboration hadn’t happened sooner. For more insights on what the IBM behind the firewall project is about read David Levine’s (Zha Ewry in Second Life) post here.

Transcript of Discussion at the Open Source Virtual Worlds Round Table

Philip Rosedale (LL): Blended animation and IK (Inverse Kinematics) is a really cool thing it’s also a really hard problem, I would love to see progress on that. Its got to be one of thing to make the world really ??. We wanted to do that from the very beginning. Its a daunting problem of course. You’re simultaneously having to use the animation in-world as a kind of a mechanical guide to move what is supposed to be a mechanical hand, and the problem is there’s a lot of corner cases where trying to do that with an animation kind of won’t work. In the same way that say break your arm you can’t put it anywhere, you run into this interesting problem. But I have to say that I think that is a great piece of work. It is one of the things that in my personal opinion it’s one of the key elements of believability that the avatar lacks today that we essentially have this odd situation where we have a little bit of physics going on the avatar bumping into things and getting up on a table and then all the animations are happening without any respect for the kinetics of the environment so its a very hard problem and I’d love to see some work being done on it.

We love to work on it! But it is a question of having the people..

(for more click on “read entire post” for the rest of this transcript)

Ben Goertzel (Novamente): One of the things we’re doing is trying to use AI to control avatars in a broad sense. We run up constantly against the lameness of having just having a fixed set of animations… you know what I mean? We’d rather for AI take a more robotics type approach, so you can learn to grab something with your fingers in different ways depending on what the shape of the object is rather than having a grab animation or even a grab-1, grab-2, and grab-3 animation.

Jani Pirkola (realXtend): Right. Actually we could do it in a way that …?? knows the animation that should be played.

Philip Rosedale: Of course the other challenge you have to face and it’s true of a number of the other features discussed here today is that they may run into sever complex computational problems without really sophisticated solutions around level of detail. The problem with blended animation that I raised is that the computational cost generally is doing that for even a single avatar historically is noticeable a significant amount of ….. So if you had 30 avatars break dancing and bumping into each other it’s not computationally attractive……

David Levine (IBM): When you bring the physics simulation onto the client you are asking the question how much of that world am I transferring down the ?? boxes…..

Philip Rosedale: Even if you have a finite model well established in the client and then you try to bump the avatars into each other you’re done.

David Levine: Even worse cos you can’t just do that you also have to have regular physics model ……..

Ben Goertzel: We’ve used the IK library of the University of Pennsylvania which is Inverse Kinematic IK it’s just for an arm, right?And just solving all those non-linear differential equations just for the arm is significant and when you get a bunch of guys with a bunch of arms you’re assuming that there is a powerful CPU on their client machine.

Philip Rosedale: It’ll be done though, it’ll be done. I mean it’s solvable by breaking it down into a finite method or a lattice method or something. There’s a bunch of smart thinking about it, I’m just saying it is a great thing to work on.

Jani Pirkola: Yes, It’s a big challenge for us and I’m not saying that we will solve it once and for all but we’ll take the first step here.

Ben Goertzel: We’re an AI company (Novamente) doing AI for virtual worlds. We’re starting out with virtual pets in Multiverse.

Yesha Sivan (Metaverse Labs): My name’s Yesha Sivan, I’m working on a long term more official standardization project out of Europe a … question for the OpenSim guys what are we doing with money of any kind?

Adam Frisby (Open Sim): Well this question keeps coming up. And this is, perhaps, a good question to come up because effectively a lot of these virtual worlds you want to make money out of them you want to be the Amazon where you can sell this this and this. The problem is the money shouldn’t be part of the protocol itself, if you look at it on a fundamental level on the web today, you do not have something in there that handles just Visa transactions so we could put visa transactions in the http specification and then suddenly everyone’s going to use visa for every transaction. And the same thing applies to virtual worlds, there’s no reason to imbed a payment processing method into the protocol itself. You can make sure the protocol’s extensible so you can do things on top of it, but I think it’s fundamental that you allow people like PayPal to spring up and satisfy the niche and allow competition to occur because if you standardize these things too strictly then you’re going to run into edge cases where you won’t be able to do this or this without running up to walls. We are not too keen on developing a money standard for virtual worlds, what we’re keen to do is develop a virtual world server that handles a 3D simulation and has a very finite goal. I think a lot of problems with virtual worlds is they try to do everything. You want to do this this this and this. The fact is you don’t need to do everything. You can solve just what need to be done to allow other people to solve every other problem. So my answer is we’re not going to touch money, We’ve got support in the server software for adding a currency module if you want to. So you can write a module to extend and do that. But we won’t do it ourselves.

Yesha Sivan: Let us move it a bit further, it’s not just the money order that pay pal is receiving it has to do with permissions, it has to do with obfustication of the object so that you cannot copy it…..

Adam Frisby: I’m going to get onto this one and say that any obfustication of an object is rubbish because at some point you’ve got to have displayed it on the viewer. All the viewers that we work with at the moment are open source. And if you’re going to have obfustication someone can just look at the viewer code and say hey this is how it’s being obfusticated or they could grab it off of the video card where it has to be decoded, there is no way you can display an encrypted or obfusticated object on a video card. There’s no way you can do it.

David Levine: On the video card it’s acceptable and if the rendering is done in open source client it’s acceptable. However, that said, marking intent, making the creators intent, and supporting that is of the essence. So from a mutual operability point of view, we’ve got to be able to say this object was created by so and so, they’ve attached these creative comments or permanent licensing terms to it, and if you violate them you are on notice for stealing their item. We can’t technically stop them, there are limits there.

Philip Rosedale: You know, a couple of high level points about this topic – money, permissions, inventory, interchange. One of the reasons why we, I think it’s somewhat mystifying sometimes why we at Linden been so proactive. People who understand that acknowledge that people are indiscriminate. People who understand less about how virtual worlds work are such an empty example of how .. why would we at Linden so aggressively tried to open-source all our code and participate so actively with open-source projects when it means that if we succeed we would lose the ability to make money in business at all in the presence of a lot of open-source products. Well that’s the answer actually. Because there’s going to be enough people, not everybody, not every simulator wants to have these capabilities nor normally need it. But yeah if you’re going to sell a virtual pet in a virtual world, and you want to take it somewhere or you want to have it be able to pay for it at all, you’re going to have to use some single global mechanism for that because otherwise it’ll just become .. you don’t have to use one but it’ll become inconvenient for you to say move money around different worlds. So I agree that money should be separate from the system, and it’s an interesting point cause this is one of the ways that we as a company have been smart in thinking about the strategy here and realizing that there will be reasonable ways for us to charge fees for things like money systems, and inventory permissions, and of course it’s not just obfustication, it’s simply the marking that this was made by Amee Weber. You’ll only trust that the cloths you’re wearing were made by Amee Weber, if you care, if there’s some sort of common naming scheme that says Amee Weber. And, what our hope as a company is is to provide those types of global services and make money on them or at least some money in the market place.

And we’re so sure having thought about it that we can reasonably do that it has allowed us to take the mission and principal position which we’re very happy with that we as a company can do that which will move virtual worlds forward for everyone fastest which is to allow basically protocols, code and everything else to be out in the open (inaudible) ???. And we’ll still make money. Hopefully a lot of it! We’re making a lot of money today as a company. We’ve done well thus far, admittedly it’s been by hosting these services.

David Levine: I think this is of the essence part of that success comes from having the content created by users, and respecting their desires to do that in an economically satisfying way. And one of the things that I try to take in these interoperability discussions is how do we do interoperability in a way so that at the end of the day the people who are creating the content that makes this a special world in some ways thats one of the things that Second Life is visually unique by are feeling safe and whole and protected. That they can market their content and they have some reason to believe it’s not all being leaked out into the world instantaneously because they’re technically an open source work. That’s a really challenging work. The fact is as Adam pointed out there’s nothing we can do to prevent the client from seeing the textures. Once you give the texture to somebody they put it on your shirt, your shirt’s in world, it’s going on to somebody’s client and they’re (inaudible)???(33:03.375).

Philip Rosedale: But one point though that’s also important to make about this is Amazon.com has a really big website. Can somebody here just copy that Amazon thing and put it on your laptop and inaudible ???(33:13.750). No! You know why not? Because that Amazon stuff is connected to a bunch of back-end code that you can’t just ???(33:22.906). There’s always protection. This is the way the world works. The stack of intellectual property protection moves upward, hopefully at the speed at which horizontal ubiquity make it necessary to have uniqueness there. And people start building property at higher levels. Animals, Ben’s working on these animals in Second Life do you think you can just rip one of those off by pulling it out of the frame buffer?

Ben Goertzel: That case would be an example of behaviour the ability to learn and yet to steal a certain number of units of server time for pet learning and intelligence (inaudible) ???(34:04.844) you have to go to …

Philip Rosedale: You guys don’t have to, it’s your choice. You don’t have to expose the code that is behind these pets, either as scripts or as back-end technology. It’s elective as to whether you expose that code.

Ben Goertzel: We’re not quite sure to what extent we will. And even that (exposing the code) I don’t believe is fatal to our business model whatsoever. Most of the value is in the knowledge. It’s taught to the virtual pets and AI rather than in the source code itself anyway. The knowledge of what people have taught to the AI which is in the knowledge base on the AI server rather than the AI source code.

Adam Frisby: I dare to make a point here on this though. Some things will be ripable. Take for instance looking at the web today. If you upload a script to your web host, your web host has a copy of that script. They can rip it. Likewise if you have sell something like Vbulletin thats a forum software that’s used on world web sites. If you sell that software to someone, then you’re giving them the code and the moment you give them the code then it’s out of bounds. The solution to that to protect something exclusively, that relies on a server back-end component is to host it yourself and provide it as software as a service model. If you build the object and you allow it to be transferred from one server to another then at that point the people who it’s being transferred to can copy it. And this is just a fundamental thing. If someone is going to copy something they can make another copy.

Tish Shute: Also another question from Peter Quirk of EMC, his question is also part of the discussion we just had here about assets, and working with this wonderful artistic situation..and storhouse that’s developed in Second Life that hopefully is going to be interoperable with OpenSim. Peter’s question to OpenSim is what is your plan to make sure you stay interoperable and that you remain compatible, and are you not forking? He actually is quite specific, he felt OpenSim LSL is already forking from the Second Life LSL.

Adam Frisby: That’s actually probably a good point. I mean we have extended LSL. We’ve got something and we’ve actually renamed it from LSL to OSSL. And the reason for that was we started adding new things. We’ve added some ??? commands. Then we started changing the syntax adding things like switch statements etc. that people have been requesting for years. We still are compatible with LSL though. If you take any LSL script on the grid, it should compile under OpenSim. Some of the backend functions, LSL’s got about 327 functions, I think we’ve implemented about 118 of those. So as long as you use those 118 then they’ll run just fine. Once we’ve finished the rest of them then yes, we’re compatible with LSL 2.0. On the other hand if you use OSSL then that’s not backwards compatible with LSL you can’t take an OSSL script and push it back into Second Life. In most cases I think some of them will be, cause OSSL is a superset of LSL.

Philip Rosedale: Are you going to continue to use Mono?

Adam Frisby: We will.

David Levine: Absolutely.

David Levine: I know Babbage is already having some fairly rich discussions with OpenSim developers on things like should we share some of the mono libraries.

Yani Pirkola: OK. I just wanted to say something about content and realXtend. We at real realXtend see in the future or we hope and see that there are going to be many realXtend servers around the world, like today there are Apache servers. And each of those worlds needs content on them so that’s a problem. Where are we going to get initial content to populate the world to make it look like something. We have started donating all our content that we do in the project for public use, in this CC license, attribution only, corresponds to the BSD license on the code side. So you can take and use it. I haven’t yet seen any, at least I don’t know any good open source project around open content.

Adam Frisby: It’s an interesting point. There is the Google warehouse. That would be a fantastic thing to integrate so you can grab drag and drop items from there and they’re all in collada. There’s also commercial services. Turbo squid is a fantastic one and I know there’s some guys from them hanging around this conference. But I think there are big libraries available. The key is using standard format, using Collada, or using some common mesh format ???

David Levine: There are groups now in Second Life like Clever Zebra doing open content intentionally for open use.

Philip Rosedale: We made a statistical assessment of ratio of free to not free content. It was very interesting, I can’t remember the number. But there is a double digit percentage of items in Second Life that appear to be fully promiscuous in effect of permissions. People were OK with them being copied and so it’s be interesting to note what mass of content that is within a given sort of ???(43:28.781) of users.

You’re going to have an equilibrium point according to how much stuff people are fine giving away and how much stuff they want to take and monetize. You can do both things in Second Life. So it is interesting to take a look at how that number meshed out. You know the total volume I think of content is a significant driver of experiences even behind the firewall experience. You know the stuff we’re doing with IBM is because we strongly believe that you really still want to be to get content from the main grid must be somewhere quite easily even if you’re just use do private business conferencing on a sim???(43:58.889)

There’s no question… I mean no business right now is going to want to start from …. and build up a nice business conf. in Second Life. They’re going to have to get content from somewhere so whether it’s from a system or not it’s .. the sheer size of the content base is interesting for comparatives. About a billion assets at this point about a hundred terabytes of data. So it’s a considerable amount of data probably makes .. … the google warehouse probably wouldn’t be measurable at this point so it’s so .. this is a big issue. A lot of the experiences in Second Life, the reason they’re cool is because they have all the content that you probably haven’t seen before. And that’s the way it’s going to be and its not just visual content it’s got to be interactive content. It’s everything the most important content is actually living.

David Levine: Actually the point that you’re on there about that even though it’s behind the firewall experiences is exactly spot on. Nobody wants A) to have 500 avatars interact with 499 other companies. and that’s not going to do it the MIM gets sheared at about 5. and B) nobody wants to have to go recreate the 500 conference rooms that people seem to insist on creating though why thay don’t want to sit by a lake and enjoy the view I don’t know. Not always the same place but a waterfall and trees .. the point being that you don’t want to create these little corporate walled gardens which are empty, you want to create corporate walled gardens which are lively, and be able to say to people of this part of the corporate walled garden not only is lively but is quasi public. And those kinds of discussions are why it’s not a simple standard like http. It is not we’d be throwing content over the walls and forgetting about it.

Philip Rosedale: About a third of those billion objects by the way, as far as we can tell, are scripted.

Round table participant:
Going back to the original question about things remaining compatible. Because OpenSim is an open source project there is always someone around who can come along and say revision have gone to far and it would be valuable to bring any forks back together.

Adam Frisby: There’s a lot of commercial interest in making sure things are compatible. I’m sure that when there’s lots of installations around say if it’s a version that 3Di does which is Ginsei or it’s the realXtend server, the point is people are going to want to go from one to the other and the one that drifts too far apart and you can’t do that anymore, is going to be shunned a little bit until that’s fixed. And, there’s a commercial incentive for people to have a compatible standard here. I think that’s probably the reason or driver behind why there are so many organizations now looking to standardize virtual world technologies.

Question (inaudible) Christian Westbrook I think!

Adam Frisby: That depends entirely on the content creators whether they’re willing to license their things to go outside the walled garden or not. If they are then fantastic, that’s great.

David Levine: It depends also on getting to a model of the protocol where we can mark whether or not the end point is an end point we trust. One of the challenges is going to be the trust model. Because we want to be able to say to somebody you can mark your content always stay on Linden’s grid because that’s the agreement you want, or on any trusted grid where there’s a legal relationship that the DMCA requested and will be honored. or I don’t care, I think it’s a cool object, anyone can have it. we want to be able to get all those gradations. and again we can’t enforce them in the sense that yes some of this stuff is going to bleed. but we want to be able to know the intent was conveyed all the way to the end point ..

Adam Frisby: Interesting point here however that I think there’s a good commercial incentive for things to be more permissible. Take say someone’s avatar, someone’s got a nice furry avatar that they go from one sim to another. Now if you teleport from one region to another server that isn’t trusted your avatar’s going to disappear and you’re going to be pissed off at the creator of that. And say Hey, why can’t you let me take my avatar with me, that’s part of me. So I think that we’ll see that there will be an incentive for people to start marking their items as transferrable because the customers are going to demand it. It gives value to your item, and to your end user so that they can use it.

Rountabler?: Wouldn’t it be an incentive for the landowners to have a certificate allowing that content on his land?

Adam Frisby: There will be models all over the place. I fully expect people to show up and say -

David Levine: Practically there will be an open ecosystem where we explore all these relationships within the essential, not just interesting, the essential. find out this model really drives people to your concept, this model drives people away. And I don’t predict which one it’ll be, I’ve got some guesses, but let the market tell us. Let people explore what happens when I create a wide open sim, do I get a ton of user created content and it’s cool? Or does it turn into a real wasteland? And I think predicting that is not only sort of -

Prokofy Neva: I want to ask a philosophical question about the relationship of the average consumer to the developer in OpenSim. From here it sounds like you you are making virtuality but not virtual worlds – a world. (very hard to hear here) They may all be interoperable with an open source that they share but they are all (inaudible) … and what that sets the developer up to do is that he has to endlessly create professional content that the consumer burns through.

Adam Frisby: Sounds like the internet today doesn’t it?

Prokofy Neva: No sounds like World of Warcraft…

Adam Frisby: There isn’t going to be a single..

Prokofy Neva: ….(inaudible) how do people get the motivation….I mean three people have three things to do either play house, play store, or play war…..

Adam Frisby: Each single one of those is a use case in this thing. There’s also the corporate ones where you’ve got people who want to work the case is that you have people conglomerate their regions into some sort of federation of sites. So you’ll have web site equivelants where you’ll have big portals set up that link website A to website B to website C. And that happens on the internet today people conglomerate in certain locations so if for instance the digg.com virtual world would potentially be huge it would be giant it would be a mass collection of subsites which formed something. Also you would have things like Second Life. Second Life is just one such use case for this environment. Second life is a big virtual world where you’ve got people involved and they all exist. The key here is that you can not just go to Second Life though, you can go from Second Life to digg.com and back again. Yes some of the experience will change, some environments will decide to be consistent in their visual scheme or other types of simularities and yet other regions will group together based on simularities I expect that to happen. But like the web today, there’s nothing linking say one site to another. If they choose to make a link then great but there’s nothing linking every single site on the net to every single other site on the net.

Prokofy Neva: Sound inaudible

Adam Frisby: One of the reasons it doesn’t work is because I would say the browsers at the moment aren’t really set up to allow you to browse from one site to another. In Second Life the browser is extremely limiting, you can’t type in a URL and teleport from one area to another. You have to load up a SLURL which is a pain in the ass. Or you need to open up the world map and find where you’re going and teleport to it. Or you need to fly to it. There’s nothing that gives you a convenient A to B link.

Discussion – sound inaudible…….

David Levine: I think one of the things we talked of as a concern about like a thousand simulators sort of flower is that we’ll get a thousand simulators sort of half empty and not interesting. I think the reality is that’s not going to happen because simulators are going to end being either run or not run based on whether or not people actually come there and play. People are going to come and play because we create compelling spaces. You look at virtual worlds that die and they’re plenty of them out there which are half dead and they’re half dead because the users don’t have a reason to hang around. And yeh there are some people who draw up a simulator in their garage because they think it’s cool. And they’ll play with it and a dozen people wander in and out and it’ll be like that web page you put up because you were bored one Tuesday and you wanted to put a picture of your nephew up. And it’ll die. Eventually somebody forgets to pay their domain service or registration and they go away. I think what’s going to happen is there’ll be self grouping. We’re going to have large clusters of people who have a similar philosophy on content, and they’re going to find ways to interoperate.And a lot of them at least in the early days are going to hang off Second Life as an appropriate place to socialize and a place to get some large shopping. Situations happen. A lot of things are going to gradually coalesce and we have no good prediction what that’s going to look like. Just like we didn’t know which of the ISP’s which were going to look promising in 1985, whether it’s the AOL or Compuserve, or Prodigy extensions you go to media if you want date yourself thoroughly. Bits of those models are still here with us today. you can get an @AOL.com email address but, lots of those went by the wayside when their content models stopped being relevant. And we’ll figure that out over time.

Inaudible question……

Adam Frisby: I think that you’ll find ther’ll be a very tiered relationship. For instance say one webhost allows you to use scripts, and allows your scripts to consume 100% of the processor time. That’s going to be an expensive web host. Likewise you’re going to have environments which are very cheap. They’re not going to let you run scripts. They’re going to let you have this many objects. They’re going to let you use this many resources on things. You’ll pay for more resources. Say you’ve got one world where every single blade of grass in the environment is physically simulated, that’s going to happen on the server and you’re going to have to pay an expense for that. You’re going to find that all these web hosts are going to show up, and they’re going to offer different competing feature sets. They’re going to say well I’ll give you a really rich experience for end users but you’re going to pay out the nose for it. Or you’ll have people who do the discount web host. and to a degree this happens on the internet today. you pay for the extra database access, you pay for scripting access, you pay (YOU PAY FOR RELIABILITY) yeah you pay for reliability of services.

Roundtabler?: And there are low prim sims.

David Levine: And the other piece that I think you’re going to see is you’re going to find sims that say everybody on this sim is private, we’re not going to identify you. You’re going to find other ones where there’s going to be a OpenID requirement to login, and every range of that. And that evolution I think it began, it’s essential like the internet, we will toss out thousands of models over the next ten years in virtual worlds and just like we suddenly discovered point cast didn’t work. and why it didn’t work, (to pick on another old technology). Some of it’ll fall by the wayside and some of it’ll take off and the exciting thing is having an ecosystem where we can afford that.

END…

The social consequences of the architectural decisions that will take us into a future of openness in immersive software are potentially vast. Open immersive software is poised to begin to play a disruptive role in the next generation of the internet, and decisions about its design may turn out to be very important ones for all of us.

EbenMoglen Euler the Second Life avatar of Eben Moglen of the Software Freedom Law Center), and Zero Linden (Mark Lentczner, Linden Lab), Neas Bade (Sean Dague, IBM, Linux Technology Center), and Zha Ewry (David Levine, IBM Research) met in Second Life last Sunday to discuss crucial issues of open architecture for immersive software in a discussion on Intellectual Property and Privacy/Identity in Open Virtual Worlds facilitated by John Jainschigg and I that kicked off Life 2.0 Summit Spring ‘08 (more details on the panelists here).

It was, I think, a landmark conversation. And, with the permission of United Business Media, here is an exclusive first chance to hear it, if you missed the live event Sunday.

Audio here, © United Business Media.

Picture below of the panel members from left to right, Tara5 Oh (moderator), Zero Linden, EbenMoglen Euler, Neas bade, Zha Ewry, and John Zhaoying.

Open Virtual World platforms are just beginning to get on the radar. But open software is clearly the path to the future and as Philip Rosedale (founder of Linden Lab) has said several times re the complete opening of Second Life software (the client is already open), “Only open will win!”

In two weeks, I will be part of a round table at Virtual Worlds 2008 in the new enterprise track. This round table aims to give people an opportunity to see a variety of implementations of open source virtual world platforms and to learn a bit more about the individual platforms presented, and what they are trying to achieve. Adam Frisby of OpenSim, Nicole Yankelovich of Sun’s Project Wonderland, Jani Pirkola of RealXtend, Remy Malan of Qwaq will be the co-facilitators

Avatar Rights!

At the heart of the discussion with Eben Moglen about freedom and openness in immersive software were some propositions about avatar rights. And, as Zero Linden explained, the new open architecture of the next generation of the Linden Lab grid crucially separates avatar identity from what constitutes their environment. Separating the production of identity from the material substrate is, Eben Moglen explained, at the core of avatar rights. (For a technical view of the next generation of architecture for Second Life see the first draft of Second Life Grid Open Grid Protocol a.k.a. SLGOGP, and for more on these protocols see Tao Takashi’s (Christian Scholz in RL) blog. where the stream for yesterday’s Architectural Working Group 2 meeting held in Second Life is posted.)

Pictures below of Zero Linden (left), Second Life avatar of Mark Lentczner (right), Linden Lab.

Eben Moglen put the fundemental issue of rights in immersive software very eloquently at the start of the discussion. I have transcribed the beginning of this discussion but to see how the thoughts developed through an in depth probing of the issues, you will need to listen to the audio here.

Eben:

I think we have an interesting and powerful problem to put our minds to with respect to want it means to think about freedom and openness in immersive software. The free software movement which I spent a lot of time trying to understand and in trying to assist took for granted that the person who needed to have rights in software was the person who got a copy of a binary, and that his rights in the binary should include the right to understand, and to study, which implied access to the source code, to modify, improve and share.

Now the reason for getting into OpenSim and open virtual worlds is to achieve some of the same kinds of social consequences that the free software movement was trying to achieve including innovation that can be shared by everybody that innovates and the value of technology in commons. But, because this software is immersive software we have another set of values to take into account which are closer to rights for users who don’t have a copy of the binary, or at any rate, whether they have a copy of the program on the client side and may posses some of the code derived from the server is sort of less important than the fact they are inside the software and if they are not free inside the software, they are substantially unfree.

That led me to ask some questions about what it means for an avatar to exist as a beneficiary of the freedom that the user is supposed to get in relation to software. It seemed to me that from there one could begin to try to deduce some rules about how the open virtual worlds have to operate. The most important one seemed to me to be that there is a right to continuity. To have the avatars existence and accumulated experience trapped inside one Terms of Services contract raises the the possibility of what lawyers call unconscionability. That after a while you have so much accumulated value in the avatar that the Terms of Service can be changed on you in a way that you can’t very well resist.

That implied, it seems to me, that if the spaces in the virtual worlds are to be regarded as open they have to be contractually open. It has to be possible to move between them without being artificially constrained by Terms of Service Agreement. That also seemed to me to imply something about the question of what it was one carried from place to place. And it seemed apparent to me that in the process of traveling from place to place an avatar has to carry some rule set as was true in many parts of the world before the modern era of the Nation State. The law that you obeyed traveled along with you. And it seemed to me that we were talking about a situation very much like that.

If you move an avatar in open virtual space from one part of the grid to another or from one grid to another governed by different servers one is not in a position to be asked to surrender ones’ sense of fairness or ones’ understanding about what can be done as a consequence of standing in a particular place. And so I reasoned my way to the conclusion that we had to provide an infrastructure for both declaring persistent preferences and expectations with respect to treatment.

Those were the lines of thought which led me to the propositions that Tara5 explained. And they seemed to me merely propositions in search of simplification. I feel as though I’m looking for some axioms, like Richard Stallman’s four freedoms in the free software movement’s genesis to explain what it is that we need to do as we open the space up.

Zha Ewry:

Reacting on that just a little bit, one of the things that came to mind when I heard Eben talk back in December is something Zero actually said when we did the kick-off, back in I guess September now, for the Architecture Working Group and some of the inter-op work which was, he used a phrase I liked a lot which is “an avatar bill of rights.” It’s what are the expectations an avatar should have in a virtual world. I thought that was a very compelling way of expressing it.

Zero Linden:

I believe at the initial Architecture Working Group which is Linden Lab’s sort of open forum for developing an open protocol set we used the term – looking for “an avatar’s bill of rights.”

I think I would respond to those two thoughts that are quite good. In one case I can give you a different approach for why we came there, and the other I can give you a simplification, which I think you’re looking for. In the first case, when I looked at viewing what would have to exist in an open virtual space, and the value of immersion. Strictly looking at it from the point of view of what made this space work. Granted here I’m a technologist not a lawyer so I’ve perhaps a different view, but surprisingly the same conclusion, is in fact the ability for you to identify and for you to to associate so closely with your avatar that led me to posit as a fundamental aspect of how we build this future open protocol that you must think of it as avatar portability.

It must be a fundamental right that users are in control of their identity not the services which help provide the existence and the immersion. That’s pretty radical thought actually, at least in terms of technology, because it is so much easier to architect, and so much more quote unquote natural to build systems which work another way. Witness every single web-site where you create your own account on every single web-site, it’s much easier for each web-site to do that than it is for web-sites to understand that they somehow agree to opt into a protocol in which you can control your identity.

In virtual world services, we have to have a world in which your avatar and your identity are in control of the user not matter what the service provide is. So what is going on in the AWG and the structure of the future protocol is the surprising separation at the server side, at the internet side, of those services that provide aspects of you identity. And by separating those out, and by building the entire protocol based on a mutual understanding between those two sets of servers, we enable users to choose servers to represent their identity that meet with their needs, trusts and ideals and to still interact with other servers that provide land with other things.

Right now when we are standing on this piece of land, the server we are standing on is both providing our identities as well as providing the land. And if you come to visit this piece land you basically have to trust what this server decides your avatar can do. In the future open that we are deciding we separate those notions. There is a server which represents your identity that you have chosen, and there is a server that represents the land that the landowner has chosen. And, we embed in those systems the negotiation between them.

Eben:

That is beautifully elegant, I have to say. That does indeed make an enormous difference………(to continue please go to the audio here).

Also see Sean Dague’s (IBM and OpenSim) interesting post on his blog that highlights one of the key freedoms Eben discussed – the freedom to leave.

“freedom to leave”, an open-standards based assurance that users can move their data easily between interoperable platforms and services.

Sean notes:

Today, if you decide to leave any virtual world platform (even OpenSim), you pretty much have to leave you data behind. I think that one of the features people will be looking for in the virtual worlds of tomorrow is the same freedom to leave that they get from any standard web or mail infrastructure provider today. Part of what has made Google successful in the application hosting space is by ensuring it’s easy to leave the platform.

One of the biggest reasons I left LiveJournal was that it was hard to leave, and the longer I built up content in that environment, the harder it was going to be for me to get it out.

Also see the coverage of the panel at SLNN.

Mitch Kapor – Second Life 2.0

In his keynote at Life 2.0 Summit Spring ‘08, Mitch Kapor predicted that virtual worlds will become a very important part of corporate strategies for adapting to a world defined by global warming. Kapor the recent board chair of Linden Lab (now a board member), chair and founder of the Open Source Applications Foundation; co-founder of the Electronic Frontier and Mozilla Foundations; and the creator of Lotus 1-2-3, the spreadsheet that revolutionized enterprise computing in the 80s spoke about his current work to enhance the user interface for virtual worlds, which he likened to being in the DOS stage now. His vision to make virtual worlds useful and accessible to all is backed by work on in his own lab. He noted that videos of some of these experiments will be available on You Tube in a couple of weeks. See Hiro’s epic post for more. Kapor also made some comments re the open sourcing of virtual worlds.

I am personally very encouraged at the progress being made with OpenSim and open source components of a virtual world eco-system- some people inside the company at Linden may feel threatened by this. But my personal view all along is the most important thing that can happen is to have the largest most vibrant innovative ecosystem for virtual worlds as possible. And that means something that is open and interoperable. One wants to have the biggest pie not a little slice of a small pie. And similarly I know that there are a lot of people interested in avatar portability or open avatar in the company.

This last remark I think is clearly supported by the number of Lindens that came to the annual open Architectural Working Group meeting. I attended and there were indeed a bunch of Lindens present. The picture below is from Tao Takashi’s (Christian Scholz in RL) blog.

Eben Moglen – Open Source, IPPI panel in Second Life

Life 2.0 Summit Spring ’08 will kick off with the Open Source, IPPI (IP and Privacy/Identity) in Virtual Worlds On Sunday, March 16, at 1 PM PST, with special guest Eben Moglen (his avatar pictured above).

The event will be held in the CMP Amphitheater at CMP 1, 2, 3, 4 ( SLURL ). To attend the Second Life events or watch video you must register for Life 2.0 here.

Eben Moglen, is Director, Chair and Chief Counsel of the Software Freedom Law Center. Moglen, professor of law and legal history at Columbia, is a pioneer of the opensource movement, former general counsel for the Free Software Foundation, and one of the architects of version 3 of the GNU GPL.

Zero Linden

Eben Moglen’s co-panelists on Sunday will include Zero Linden, a.k.a Mark Lentczner, Linden Lab. Zero is one of main the architects of Second Life’s evolving infrastructure. Zero recently published the first draft of Second Life Grid Open Grid Protocol a.k.a. SLGOGP a important step forward on the path to opening up the Second Life grid (see Tateru Nino’s post on Massively, Tao Takashi’s post at mrtopf.de, and mindblizzard).

The brilliant and very elegant Zha Ewry (a.k.a David Levine, IBM Research) will be joining the panel from JFK airport while he waits for his flight to San Francisco. David Levine and Eben Moglen had an interesting conversation back in December that you can find on Ugotrade here. They explore some of the problems of defining digital public space and issues of privacy on the internet, offering many suggestions on how to implement online privacy enhancing technologies and insights as to how we could design the next generation of these technologies in responsible ways.

Also Zha was a interviewed recently on Metanomics with Beyers Sellers (a.k.a Robert Bloomfield). This interview is highly recommended as some of the key issues facing Second Life’s Architecture Working Group (AWG) and a future Open Grid “that will ultimately allow the cohesive operation of both Linden-operated and non-Linden-operated Second-Life style simulators and grids.” (see Massively for more) are unpacked.

Download the video (Quicktime)
Download the audio (MP3)
Read the transcript
Metaversed video archive at SLCN

Also, OpenSim and Linux guru Sean Dague (IBM) will be a panelist. Sean Dague has been a member of IBM’s Linux Technology Center since it’s inception in 2001. He has worked on numerous Open Source technologies over the years including: Cluster Management (SystemImage and OSCAR projects), Hardware Control (OpenHPI), Virtualization (Xen), and now Virtual Worlds with OpenSim. Sean has been an active member of the OpenSim project since July 2007.

Sean’s avatar (picture below) is Neas Bade.

Tara5 Oh (below – that’s me!) will moderate with CMP’s John Jainschigg (John Zhaoying).

See you there! (SLURL)

We have built the technology for monitoring almost everything, almost everywhere and we are making 99% of it open-source. Thanx to Bruce Sterling who inspired us, we called our technology “OpenSpime.” This is a concrete opportunity to monitor the earth, and everyone on this planet will be able to contribute to this.

OpenSpime have prepared a concept video of CO2 monitoring (still from the video above) and Google maps mashup via their OpenSpime infrastructure. For more see, co-founder, David Orban’s post OpenSpime: What do you know about your planet?

Instrumentation versus Surveillance

Surveillance is all about when people in authority know a lot about you. Instrumentation is when you know a lot about the world. And it allows you have more agency. When people know a lot about you it takes away your agency.
Cory Doctorow, Craphound.com, boingboing.net

OpenSpime and SL Data Viz project (a project in Second Life “where people can contribute, review and copy open-source data viz tools” which OpenSpime will participate in) are taking up the challenge that Bruce Sterling made at end of his visionary book, Shaping Things. Sterling gave an imperative to humankind to start to make “decent technology” – social software entities that can answer questions. Questions about our world. Questions about objects. Not the profit-centric questions – serious questions. (Shaping Things).

The protagonists of this new era of social software – the narrators of the instrumented world to come – are Spimes.

Sterling invented the term by compressing ‘space’ and ‘time’. Spimes are aware of their environment, they know where they are, and when they are, and keep track of some parameter around them. Sensing, memory, and ubiquitous communication enable spimes to accurately map the physical world around them. The progressive saturation of the world with spimes is creating what is called the Internet of Things.

The closing words of Sterling’s great visionary book on “spimes” and “the internet of things” are:

Its not enough to think about that, or even write about. If it is to be any use to humankind, it will have to get done.

OpenSpime is doing it! SL Data Viz project is doing it!

If you are not yet in tune with “the internet of things” here is a music video Royksopp’s “Remind Me” that Sterling suggests is some kind of “spime” theme song. This is the third time I have posted this link but Spimes deserve all the air play they can get!

Our technology enables individuals and corporations to better understand their environment, through the use of a series of GPS-enabled sensors. We provide a set of open APIs and communication protocols to manage the data collected.

See an interview with the founders of OpenSpime here.

With free hardware, free software, open APIs and communication protocols, OpenSpime’s business model is about the provisioning of the SpimeID identification numbers for the trusted communication of validated data streams between spimes and the OpenSpime servers.

Here are David’s responses to questions about the revenue model:

The revenue model is based on the sale (in large quantities to the hardware makers) of the certified SpimeIDs. Anybody can build spimes that conform to our open specifications, but if you want us to validate the data that your spime collects, and aggregate it together with other trusted sources, than you have to pay €1 per device to get the ID. Ah, and we already have LOIs for several hundred thousand IDs to be built in devices. [smile]

The estimates are that there will be ten or more spimes per person in the developed countries within ten years. That is approximately 25 billion spimes, and counting, since by 2020 the number of people who live in countries we define developed will also increase. That’s a
market for you! And we want to define the way the data generated is managed.

What comes after humans twittering to each other? Spimetalk. And spimetalk is going to be several orders of magnitude more intense than any human2human or human2machine communication before that.

From Spimes to Mirror Worlds

Agents, avatars and spimes will eventually hang out together in Virtual Worlds, interacting in real time in networked virtual environments built of live data and 3D info machines.

Melanie Swan points out:

Virtual Worlds have been used for architectural builds and interaction and the next obvious step is making them alive with data, streaming in data and representing it visually. Data visualization in Second Life is starting to take off with an open-source movement to make open-source building-block tools available to the community and developers and end users creating specific-purpose enterprise and science applications. There is a community wiki and a “Data Visualization” group in Second Life.

The picture above is of a Bashiba Panorama – “a commercially available ambient data panorama that dynamically reflects the overall “mood” of a rich information environment. The so-called “data atmosphere” reacts in real-time multivariable changes (e.g. stock market data), that are then mapped to visual counterparts (e.g. ocean waves, sun strength, wind speed, rain). the resulting information display can be interpreted ‘without cognitive effort’.”

“once the stock market opens, our 3D simulation comes to life, & people start ‘breathing’ business information.”

The streaming Bashiba Panorama in Second Life is an experiment in collaboration with Melanie Swan (MS Futures Group) and powered by the Ohio University VITAL Laboratory streaming server (many thanks to Dr. Chang Liu + Stephen Carroll). Look for Bashiba to come to Second Life in full immersive 3D!

For videos of Bashiba in SL see here and here.

3D Command Centers the killer app of the 3D Internet

A Mobile Tactical Data Comm Unit built by Illuminous Beltran in OpenSim

Data visualization will develop increasingly into and become inextricable from operation/command centers – the 3D information machines that will reinvent the relationship between humans and the up to now invisible but most crucial layer of modern society – software. For more on Illuminous Beltran’s, (a.k.a Michael Osias, IBM) virtual operation centers/command centers in Second Life/OpenSim see here.

These 3D command centers are being used for energy management, virtual network operations centers, and for building automation (see the work of Eolus). But, as long as our society is wedded to war, the “killer app” of the 3D internet will also be a killer app in more ways than one. The instrumentation of society has grown up hand in hand with surveillance and military technology. But it is up to us, as a society, to grow the peaceful and people centered aspects of this technology.

The nascent spiming technology of RFID had its roots in military innovation.

In 1946 Léon Theremin invented an espionage tool for the Soviet Union which retransmitted incident radio waves with audio information. Sound waves vibrated a diaphragm which slightly altered the shape of the resonator, which modulated the reflected radio frequency. Even though this device was a passive covert listening device, not an identification tag, it has been attributed as a predecessor to RFID technology. The technology used in RFID has been around since the early 1920s according to one source (although the same source states that RFID systems have been around just since the late 1960s).^[2][3][4][5]

Similar technology, such as the IFF transponder invented by the United Kingdom in 1939, was routinely used by the allies in World War II to identify aircraft as friend or foe. Transponders are still used by military and commercial aircraft to this day. (for more see Wikipedia on RFID)

But, Gelertner’s vision in, Mirror Worlds 1991, is the transformation of computers into seeing machines that will empower people to understand and work with the machinery of their society. And with projects like OpenSpime everyone can contribute to the task of asking important questions about our world. And as these questions are increasingly incorporated in virtual operation centers, we will be able to interact with the previously invisible machinery of our modern world – tinker with it, hang out in it with other avatars, and agents, and command it in new ways.

Life2.0 Summit Spring ’08

Life 2.0 is the leading event on virtual world application and business development. Produced three times yearly by CMP in Second Life, the six-day virtual event draws a fully-registered, global audience of over 1000 software architects, creatives, CMOs and key executives seeking to harness the power of virtual reality for marketing, application creation, commerce, education, and to connect with the Net’s fastest-growing, smartest, most-engaged communities.

Also check out CMP’s John Jainschigg’s (John Zhaoying in Second life) post, “Life 2.0 – As green as Five Brazilian Households,” which demonstrates “that importing certain kinds of real-world activities into virtual reality saves a bagload of carbon. Or put another way, that it enables us to enjoy the benefits of global human community at small cost to the planet.”

Eben Moglen to join IPPI panel at Life 2.0 in SecondLife

The Kickoff Symposium will Discuss Opensource, IP and Privacy/Identity in Virtual Worlds On Sunday, March 16, at 1 PM PST.

Life 2.0′s IPPI (Intellectual Property, Privacy and Identity) symposium will kick off with a rousing panel discussion on Opensource, IP and Privacy in Virtual Worlds — including special guest Eben Moglen, Director, Chair and Chief Counsel of the Software Freedom Law Center. Moglen, professor of law and legal history at Columbia, is a pioneer of the opensource movement, former general counsel for the Free Software Foundation, and one of the architects of version 3 of the GNU GPL.

The panel (still growing!) will be moderated by Tish Shute (Tara5 Oh), who blogs on virtual worlds (hey that’s me!), and CMP’s John Jainschigg (John Zhaoying).

Also see, A Conversation with Eben Moglen on Second Life.

There will be a Data Viz panel at Life 2.0, Monday, March 17th, 2pm to 3pm PST, organized by Melanie Swan (more on this soon). See the impressive list of data visualization tools that SL Data Viz project has already gathered together below.

Directory of Data Visualization Tools in SL:

(excerpted from the sldataviz wiki)

Interactive Data Exhibits:

• Embodiment Island Data Visualization Exhibit Hall, SLurl, contact Mark Dubin/ThreeDee Shepherd to exhibit material
• NOAA real-time U.S. weather sim, SLurl
• Daden Prime real-time U.K. weather sim, SLurl
• Daden Prime real-time LAX Air Traffic Data, SLurl

Tools – scientific:

• CAIA – Cheminformatic Assisted Image Array visualization laboratory. Status: Available to view (Gus Rosania/Caia Alter), SLurl
• Hiro’s Molecule Rezzer – Rezzes molecules from a notecard. Status: OpenSLedware. (Andrew Lang/Hiro Sheridan) ONREZ SLurl
• Histogram (20 period, real-time, on-demand stock market data), SLurl. Status: Open-source download available (Melanie Swan/Xantha Oe and Eloise Pasteur)
• Orac – Takes smi, InChI, or inchikey as input, queries three web services and rezzes the returned minimized structure in SL. Status: Avaliable upon request. (Andrew Lang/Hiro Sheridan)
• StoryMachine – Generic visualization tool for dynamic interactions. Status: In development (Peter Miller/Graham Mills), SLurl
• StoryMachine – PubMed Search. Status: In development (Peter Miller/Graham Mills), SLurl
• Protein Rezzer Toolkit – In-world rezzing of 3D protein backbone structures based on specially parsed PDB files. Status: In development (Peter Miller/Graham Mills; based on scripts by Troy McLuhan).

Tools – enterprise:

• Airline flight booking – connecting Second Life to a SAP NetWeaver system to create a 3-D Business Process (Bjoernaeon). Also see Craig Cmehil’s blog.
  
• Green Phosphor, New Jersey, USA – enterprise data visualization applications (Ben Lindquist/Arkowitz Jonson, David Cooper/Coop Upshaw)

Tony Manninen, the CEO of LudoCraft games studio (the client side development division of realXtend) who has being doing all this amazing recent development on OpenSim, has a vision for Avatar 2.0 that he is bringing to OpenSim. The possibilities for the future integration of realXtend features (that include meshes and the ability to import proper 3D models) with Second Life is currently under discussion – more on this soon.

We have tried to keep the rexviewer as compatible as possible. We totally appreciate what Linden Lab has done and we are trying to do our best to co-exist with their beautiful social innovation.

When I interviewed Tony (see interview later in this post), I was very excited not only by the scope of his vision and his devotion to enhancing the user experience and possibilities of virtual worlds, but also by his absolute determination to manifest this vision in code with the the utmost speed

The next release of realXtend server and client will be published on 29th of February, 2008 at 4pm (Helsinki time). The features that will be in this release and a roadmap for the future are here.

Virtual Worlds that make our “real” and “virtual” lives better

What do I mean by an exciting vision for virtual worlds? There are many visions for virtual worlds floating around and some of them are not so appealing to me, like the notion of a virtual world as a walled garden/company town put forward by the CEO of There.com, Michael Wilson.

When Michael Wilson was interviewed in Second Life by Beyers Sellers (Robert Bloomfield, Cornell University) for Metanomics this week, he gave the impression that creating a controlled environment for marketing is the number one priority of There.com.

In my view, which seemed to be shared by many of the Second Life audience participating in the group chat, this notion of designing virtual worlds as “big brother houses” is the epitomy of “the metaverse gone wrong.” You can see the full interview here on SLCN. Also see here for Christian Scholz’s (Tao Takashi in Second Life) excellent summary and analysis of a virtual world according to There.

I hope there are some more positive aspects to There.com than this interview conveyed.

Despite ups and downs and growing pains, Philip Rosedale’s vision for Second Life has always, in my view, had at its heart the motivation to make people’s “real” and “virtual” lives better. See The Making of Second Life by Wagner James Au for a fascinating look at the early days.

While Linden Lab is still working hard to break free of the “walled garden” model, with OpenSim and realXtend, the expansive vision that they have pioneered has been open sourced even before they have finished their own open sourcing project. And now, an open source development community of genius and depth (including Linden Lab) is rising to the challenge of taking this vision of making our “real” and “virtual” lives better to the next step.

There are several Open Source virtual world platforms available now, including Ogoglio, Croquet and Sun’s Wonderland. In the news today is the Open Virtual Worlds Project from The New Media Consortium (NMC) and Sun Microsystems which as VWN points out is similar to efforts announced by the Immersive Education Initiative to create an Education Grid across Second Life, Wonderland, and Croquet.

But OpenSim stands out from the pack because it inherits from Second Life an awesome set of tools to facilitate user generated content. This emphasis on user generated content is, of course, key to how Second Life has become the largest and most highly developed 3D immersive world to date. The fact that OpenSim is a platform built for people to build in was of one of the reasons that realXtend chose to work with OpenSim after they investigated all the available open source options. Also important, Tony noted, was the depth of the development community in OpenSim and Second Life.

Philip Rosedale speaking in Second Life at the MacArthur Foundation event on The Future of Philanthropy in Virtual Worlds

The Future of Ubiquitous Game Design

There are many inspiring alternative visions for virtual worlds emanating from Game Developers Conference 2008 which have been percolating through the blogosphere. Particularly interesting, to me, are the new possibilities for avatar expressiveness and some interesting ideas on the future of ubiquitous game design.

I was thrilled to discover that Tony Manninen has been doing research on avatar expressiveness since 2000 (see his papers here). Rich Interaction in networked virtual environments, avatar expressiveness, and the future of ubiquitous game design are his forte. How very cool!

Tony’s vision is to take social gaming to the next level and to produce games that are so heavily collaborative that they reach deep into our experience of the pleasures doing things together, our enthusiasm for team sports, and childhood memories of the enjoyment of playing together. He has been exploring these ideas of team play and community in earlier LudoCraft games like AirBuccaneers.

Ray Kerzweil’s keynote GDC 2008 laid it on the line: “Games are the harbinger of everything.”

But some of my favorite GDC 2008 quotes are from Jane McGonigal (see her blog post “Reality is Broken” – My GDC Rant) and in Liz Lawley’s copious notes on “Pouring Fuel on the Fire: Game Designers’ Rant.”

Jane McGonigal: I’m not mad at game designers. Compared to the rest of the world, we have it all figured out. Our medium kicks all other media’s ass. We make more people happy than any other platform or content in the world. (If you don’t believe that, you’re not paying attention to what’s happening.) We’ve won. Games have won. As an industry we’ve spent the last 30 years learning how to optimize experience. Brains, bodies (recently), and hearts are all engaged. That’s the good news.

The bad news is we rule the virtual world only. Reality is broken, and we’re not fixing it, we’re offering alternatives to it. We offer better experiences, better socialization, in virtual experiences. That needs to start changing. If reality is broken, why aren’t game designers trying to fix it? It’s our responsibility to design systems that make us happy and successful and powerful in real life? We have the power and the responsibility.

Jane shows an image of her favorite “I’m not good at life.” graffiti (from Liz Lawley’s notes).

Tony Manninen is an innovator committed to bringing the power of games into arenas we are yet to dream of.

Jane McGonigal has some nice examples to stretch your mind in this direction. For example, the Sniflabs collar remembers other dogs you’ve encountered. “We can play games with our dogs. What if I could play an MMO with my dog?”

Why will it make our “real” and “virtual” lives better to take play into every nook and cranny of our experience? Because, as Kurzweil points out, “Play is how we principally learn and principally create.” Jane McGonigal goes further:

I have spent the last year doing research on happiness. Instead of trying to figure out what’s broken, these people are trying to figure out what makes us happy. Every positive psychologist has found the same thing. Happiness is 1) having satisfying work to do, 2) the experience of being good at something, 3) time spent with people we like, and 4) a chance to be part of something bigger than ourselves.

“If you are a game designer you are in the happiness business.”

Jani Pirkola, realXtend project manager, works on innovation in the offices of LudoCraft.

LudoCraft = The Art of Designing Games and Play

Ludo – Theories of game and play

Craft – Art of game design and development

Reinventing the technologies of expression and experience.

At the Metaverse Roadmap meetup, Mitch Kapor pointed out “3D cameras would make virtual worlds easier to use.” And Bob Moore (on left above, me on right) asks in this excellent post, Is “free gesticulation” for avatars here yet?

When I asked Tony this question he replied:

“Our avatar will have enough “bones” for full facial expressions, etc. When the actual base architecture of the avatar is fully functional, there’s a possibility to use webcam, voice, or other input devices to control your avatars facial expressions. It can be true 1:1 mapping, but of course it can be something else as well. You can be yourself, or, you can change your “output” to something else.”

Tony, through realXtend, is reinventing the technologies of expression and experience.

Man Upside Down from The Book Of Urizen, William Blake

Interview with Tony Manninen, LudoCraft

Me: Second Life as a platform has been pretty much ignored for game development up to this point. Do you imagine transforming OpenSim and Second Life into platforms suitable for MMOG?

Tony: I am running the company and also making sure the realXtend development reaches the required quality and performance standards you would expect from MMOGs.

We’d definitely love to make games for Second Life, but at the moment the end-user experience is not exactly what you would expect from a game system. Therefore, we’ll pay heavy attention to things like responsivity, graphics quality, frame rate, etc. If we manage to keep up the momentum of realXtend development, then I’m sure there will be some interesting games spawning up in the near future…

Developing a sophisticated game engine is not an easy – or cheap – task, so there’s a loads of challenges ahead. But I truly believe that is the only way forward. With game-like interfaces and features you’ll be able to get much more heightened experiences.

Me: Have you worked out anything with LL yet re keeping the realXtend viewer compatible with SL?

Tony: We have tried to keep the rexviewer as compatible as possible. We totally appreciate what Linden Lab has done and we are trying to do our best to co-exist with their beautiful social innovation.

Me: But will you be able to work out a licensing deal under the GPL so that they can integrate your code into their browser?

Tony: The whole licensing scheme is still undergoing some serious thinking processes. We will try to find the best possible option in order to satisfy the needs of hungry virtual world adopters. GPL has its challenges. But, on the other hand, everything invented by a man can be re-invented by another. I am sure there will be a fruitful solution to the licensing issue. At the moment we are concentrating on releasing all the code to the general public, so that all the enthusiastic developers out there can join the forces and increase the momentum.

I’m pretty sure the pieces will eventually find their right places. I am really happy about the response we’ve got from Linden Labs – it’s great to think we might be able to give something back for them.

Me: Is it safe to say the licensing issues are on the table and being worked out?

Tony: There’s definitely some serious working-out being done, so I suppose it is safe to say that.

Me: I know Will Wright creator of Sims online and Spore has spoken a lot on the spiraling costs of content production and that diminishing returns for content development at these high costs. He has gone to procedural programming with Spore to take gaming on another path. But, it seems to me that you are taking another approach by trying to bring SL up to gaming standards – is this correct, or are you doing something different?

Tony: I guess that’s quite correct assumption – at least in relation to LudoCraft. This is not necessarily a conscious decision. It’s more like the costs of licensing a decent game engine are generally so high that they more often than not fall out of reach of start-up companies and small developers – let alone universities, communities, etc.

We have tried to find a suitable platform for our collaborative games, but since there were no perfect solutions, we decided to try and make one. Not alone, but by joining forces with our partner company Admino and several other keen developers.

The OpenSim merger will increase the development base even further, so there’s a great chance we’ll actually pull this off.

Like I said, there’s loads of challenges, huge amount of work and some design issues involved. But the Open Source communities have proven themselves earlier, so why not now.

Me: What do you think are the chief design issues to be addressed?

The realXtend project includes LudoCraft and Admino, plus we have several sub-contracting developers doing work with us.

The main issue is the divide between social 3D worlds (like SL for example) and MMOG. The gamers tend to avoid these social virtual worlds for obvious reasons [the quality of the experience from a gaming POV]. However, if we manage to develop a platform that can serve both purposes, then I’m sure things will change.

The main design issues, therefore, are the performance, audiovisual quality, rendering, frame rate stability, responsivity, interaction, etc.

Numerous issues that are not necessarily critical in purely social virtual world, but are absolutely essential in any multiplayer game environment.

Plus, if well taken care of, these issues will boost the whole end user experience in the non-gaming situations as well.

Me: You have gone a long way with the rendering and meshes what will be the next most important features and when do they arrive?

Tony: Inverse Kinematics and procedural animation are essential features if we want to have truly expressive and adaptive animations. We see avatars as main tools for self-expression. We’ve researched the issues since 2000 and we believe we are on the right track. With flexible and powerful expression potential and accurate controllability, the users will be able to communicate with their avatars so much more than is possible nowadays. The concept of Rich Interaction is something we will utilize here and it will be interesting to see the results when the system is actually usable by the general public.

Vehicles and projectiles are really important so they sit heavily on our roadmap.

Me: Are you going to do 3D face mapping etc – I know that this is getting close to doable now?

Tony: Oh, the required list of features is endless. There are several key features in terms of game development, but also some interesting stuff for more serious applications.

Our avatar will have enough “bones” for full facial expressions, etc. When the actual base architecture of the avatar is fully functional, there’s a possibility to use webcam, voice, or other input devices to control your avatars facial expressions. It can be true 1:1 mapping, but of course it can be something else as well. You can be yourself, or, you can change your “output” to something else.

Me: OpenSim is very attractive for a number of vertical applications – building automation is one you mention. How focused are you on getting the OpenSIm platform ready for the huge mirror world, 3d info machine/command center market?

Tony: In terms of the mirror world, we already have some interesting pilot projects that target those features. It may be less juicy than making cool games, but it provides some highly interesting scenarios.

Me: What are your pilot projects?

Tony: I don’t think I should go into details at the moment. We will include a list of initial projects and collaborations for our next announcement, which is due 3rd of March (latest). We want the third parties to have their saying first before making these public.

Me: The realXtend avatar server is a new way of connecting sims – could you tell be more about your vision there?

Tony: The main vision is: there should be only one. Only one avatar, one user account, one login, etc.

It is ridiculous that I need to have an avatar in SL, character in WOW, nickname in IRC, and whatnot. I mean, it’s ok if I want all these different identities, but what if I would be, eh, me in all these worlds, perhaps doing business, shopping, hobbying, and so on. There should be an option for me to jump from world to world without the hassle of logging in and logging out

Me: And how will the work you have done on the avatar server alleviate this problem unless SL, WoW and other cooperate on interoperability?

Tony: Think of it more like the 3d web. realXtend/OpenSim is like the Apache of virtual worlds, rexViewer is the Mozilla or Firefox of whatever. When “surfing” the web, you are not constantly required to prove and change your identity when loading different pages.

Me: But because there is no equivalent of http for virtual worlds yet, how will this work re the other worlds?

Tony: Ah, that’s the critical question: it won’t. There is some interoperability work going on in terms of creating the standards. The problem is, this work tends to be slow. We are not willing to wait that long. We want to see some action. That’s why we thought that we’ll start the wheel rolling and then see what happens.

Me: So at least now OpenSim worlds and hopefully Second Life will be able to connect this way?

Tony: The openess is the key to the lack of unified protocols. When the interfaces and APIs are open and transparent, people are free to develop applications, converters and bridges between different systems.

Right, at least OpenSim is now reachable, plus we want to get the teleport between SL and OpenSim/realXtend up and running, at least in the quick-and-dirty way so that people will have a chance to experience the feeling of world-to-world hopping.

Me: So in terms of this experience of avatars hopping between OpenSim worlds how do you think this will change the need /or not for large grids like SL or any of the new ones like OpenLife that are springing up?

Tony: I think there’s still need for various technical solutions like there is need for lighter and heavier websites. There are different applications, different scenarios and different user requirements. It is still really enjoyable feeling to be able to actually walk – ok, virtually at least, from one server to another. With hopping there may be some loading involved (like when surfing the web). So I guess the big grids won’t go anywhere. We’ll just have more modalities in terms of servers/worlds.

Me:So what are your expectations for concurrencies on OpenSim in the near future?

Tony: Hmmm… I’m not sure yet. OpenSim is an excellent initiative and there’s already a substantial user base. There are a lot of work to be done in terms of integrating realXtend codebase to the OpenSim, but since the visions and missions of the people involved are aligned, the hard work is a minor detail in the process.

Astrophysicist Piet Hut (Pema Pera in Second Life) has come into virtual worlds to explore questions about reality. Piet’s interest in reality is the question of what reality really is. His exploration starts with scientific insights and moves into the reaches of contemplative traditions and other ways of knowing to ask: “What else is true?”

Simulating stars and galaxies and putting them in the palm of your hand computationally speaking is interestingly complementary to much of the ancient imagery of India, Tibet, etc.

Tibetans/Indians used “waking up” as a metaphor for seeing more directly into reality, but we now have many more examples we can use, besides waking up from a dream: realizing that a movie is a movie, getting out of and into virtual realities, so many metaphors that all can serve to help us talk about radically different ways of seeing in a new light what what is already right here.

Piets experiments in virtual worlds began with Videoranch (founded by Michael Nesmith, one of the original Monkees, and his wife), and now Qwaq Forums and Second Life. Piet is planning to write a book with the tentative title “Exploring Reality in Virtual Worlds.” He is a professor at the Institute for Advanced Study in Princeton, but is currently logging in from Kyoto Japan where he is involved in joint research with Japanese astrophysicists.

In the picture opening this post, by Noelani Lightfoot the proprietor of Quixotic Photography in Second Life (see more of her great work here), Piet and I are at a Go table in Second Life (see the newsblog and  the website for a schedule of events). Piet is a Go player (shodan rank in Japan) and at the time of the interview he had just discovered this place, to his delight.

Piet explained to me how the trajectory of his research into the notion of “the lab” and his journey as an astrophysicist in virtual worlds relates to the history of astrophysics:

Science goes back to astronomy, studying the regularity of the heavens, and moderns science started with Galileo. Yet, though astrophysics is the oldest science, it is the youngest laboratory science. Until half a century ago it had no lab, while all other sciences did. You couldn’t put a star in a lab, or a galaxy. But with the advent of computers, you can, virtually! So experimental astrophysics equals virtual worlds, in fact a virtual cosmos.

Piet takes the notion of “the lab” into new terrain.

Piet began with two experiments in Qwaq. The first is a Qwaq forum called MICA (Meta-Institute for Computational Astrophysics). Aimed at his astrophysics colleagues, MICA explores what happens when astrophysicists enter their virtual labs themselves, in virtual form as avatars. The second Qwaq organization, WoK (Ways of Knowing), is aimed at a widely interdisciplinary group of scholars interested in thinking deeply about reality and asking the question, “what do you find if you really delve into contemplation, doing it and reflecting on it, both, i.e. using your life as a lab.”

In astro I am putting galaxies and stars in a lab and in WoK we are treating our own life as a lab and virtual words are great labs!

Piet’s recent article, “Virtual Laboratories and Virtual Worlds,” reports on his experiences with these Qwaq forums and offers many fascinating and intriguing insights. Piet was initially surprised at which group made the transition first, as he put it from tourists to neighbors and then to collaborators. Apparently the task of getting astrophysicists into virtual worlds was a lot like herding cats. An interesting point Piet makes is how:

The main attraction for coming into Qwaq Forums was presence. Presence in a persistent space, a watering hole that quickly became a familiar meeting ground, this is what was felt to be the single most important aspect of the whole enterprise. Everything else was clearly secondary.”

But this week, as Piet explained:

We will start putting our N-body simulations in Qwaq and we will also start playing with interfaces to their in-world applications. Now from here on, Qwaq will prove to be useful for integrating simulations and avatars, i.e. no more screen dividing scientists and their science.

This may be a considerable draw for astrophysicists who will now be able to enter virtual laboratories running real time simulations, produced by a supercomputer, of galaxies “or molecules interacting for that matter” to inspect the results of the simulation, and “even change the way the simulation are run in real time” together with their colleagues in a shared virtual space.

Another potential of virtual worlds that Piet is interested in exploring is “for researchers who are geographically remote to start writing code together in a virtual space.”

The biggest problem in computational science, which is quickly becoming all of science, is to find ways to let scientists write software together.

He notes in “Virtual Laboratories and Virtual Worlds,” that keeping full digital records of coding sessions can be part of a process in which we move “from open source to open knowledge.”

The idea is that if someone uses a legacy code, twenty years from now, and wants to modify a particular function, she can click on a pointer next to the old lines of code and see a replay of the session that produced that code. This will give full disclosure of the original motivation for writing the code, the way in which is was debugged, what considerations came up and were tried out, anything you could possibly have wanted to ask the original writers (except that they by then will have forgotten most of it!).

Breaking out of walled gardens (intellectual disciplines and proprietary virtual worlds)

Piet’s commitment to exploring different ways of knowing extends now beyond breaking down the boundaries between science and contemplative traditions. He is also forming groups that bridge virtual worlds. Piet has invited me to join two groups: one is the Qwaq Second Life Liason group, QSL, (more about this later in this post), and the other The WoK Qwaq forum .

I will have some first hand accounts of my own experiences bridging worlds soon. My first WoK forum featured a presentation by Dr. Karen Sobel Lojeski, founder of Virtual Distance International, and a discussion of the New York Times article on virtual worlds for children – what the implications of the widespread adoption of virtual worlds by children may be for mind and society, and whether these burgeoning children’s worlds were benefiting children or merely opportunist business ventures.

Karen’s book with Dr. Richard Reilly, “Uniting the Virtual Workforce: Transforming Leadership and Innovation in the Globally Integrated Enterprise,” is scheduled for publication in April, 2008, and you can order now on bn.com, amazon.com, or wiley.com.

I am interested in the endeavor of working across the disciplines of science and contemplative traditions. But I wonder how the emphasis of many contemplative traditions to go beyond conceptual reality meshes with science.

For example, in Buddhism there is a distinction between certainty about the nature of reality and the experience of what reality really is – which is beyond conceptual fabrication. To go beyond conceptual fabrication is to go beyond the language of science, indeed beyond any language. If science made its endeavor the experience of the true nature of reality beyond concept would it still be science? Piet told me that while current science cannot play such a role, he fully expects a future science to be able to do so (see his one-page summary, Beyond Methods and Goals).

I am equally interested to see how Second Lifer’s and Qwaqers will find new ways to relate. As Piet put it:

Let’s build bridges. The human part is always the most difficult, so why not start there scaling the walls. And Qwaq and SL communities have a lot to offer each other. They are totally complementary.

Exploring Qwaq

Here are some pictures of my first day in Qwaq. I had a bit of culture shock when I checked in and found myself not the rather lovely Tara5 Oh, my Second Life avatar, but a block person. Luckily being able to drag Powerpoint presentations from my desktop onto a wall in a virtual room in one second with one mouse motion saved the day, so I was a happy blockhead.

Then I learned how to become a sphere (see picture below) which for me actually felt a whole lot better. There is no rich culture to support avatar identity construction in Qwaq, as there is in Second Life, and without this I preferred to retreat into a non figurative abstraction.

In Qwaq people frequently use a real life still image of their face to give more of an identity to their blockheads. Piet explained how they had experimented with using streaming video. But, he noted, the effect was a bit disappointing because the disjuncture between the moving image, which had no connection to what the avatar was actually saying or doing, was worse than when a still picture was used. Yet the richness of facial expression added to the communication, so the jury is still out, he said, as to what might work best: “time and further testing and tinkering will tell.”

Piet came into Qwaq and showed me this nice mirror in the space of Dr. Robert Magrisso, Doctor of Internal Medicine, an avid Qwaqer. In the picture below the mirror with images of our qwaq avatars in it has Dr Magrisso’s paintings to the right and left. There is a reflection of my happy blue sphere. And, Piet and his alt are the orange spheres (you can run multiple Qwaq sessions on the same computer, making it easy to log in with two avatars, or to be present in two Qwaq organizations simultaneously).

The mirror is very nice and I have not seen one in Second Life. Apparently there are a few according to my friend Zha Ewry. But they are not easy to make because of rendering issues, “You need a primitive that ‘acts like a mirror’ in the graphics system.” Apparently there has been some discussion about adding “mirror” to prim surfaces in Second Life but it’s graphically expensive and would use too much rendering resources as it would need to be calculated every frame. But Zha notes, “They are pretty. And, there are some tricks you can do on higher end graphics cards to make them less costly for clients.”

Second Life the great virtual lab of fairy tale, dream, technology, community and more….

In the last few weeks Piet has expanded his explorations of reality in virtual worlds into Second Life. Though as Pema Pera he was born ten months ago, he was so busy with Qwaq that he had to wait for the Xmas holidays to get acculturated into SL, which has a learning curve steeper than for Qwaq by two orders of magnitude, he said. But now that he’s in, he shows all the signs of being hooked.

Second Life as the largest and most popular virtual world, whose rich social structure of networks built around user generated content, is the supreme realm or “lab” to explore, “fairy tale, dream, technology, community, etc., etc, …” and the infinite possibilities of networked human intelligence.

Piet has started a Qwaq Second Life Liason group with Kat Lemieux, a Director and Founder of The International Space Museum in Second Life. “Kat has been a wonderful mentor for me here in SL,” Piet told me. Henrik Bennetsen (formerly Henrik Linden), from Stanford, was instrumental in getting Piet involved with Second Life. Piet explained, “I met Henrik in RL at a conference in Stanford about using virtual worlds as collaboration tools in August. Henrik pointed me to Kat and the rest is history.”

Because Kat’s last name is in French, I came up with the name liaison, Qwaq-SL Liaision. Then, with an incredible stroke of good luck, it turned out that QSL, the abbreviation of the group , had a deep meaning in early radio broadcasting, which I did not even know.

So then QSL grew very fast. Emileigh Starbrook showed her simulation of an asteroid impact on Mars, an amazing performance. And I got astronaut Ed Lu in at the space station. Kat showed some Qwaqers how to build things in SL. And I got Troy Mcluhan and Kat into Qwaq, in my MICA organization, and now you too, in my WoK organization.

Adventures with The Qwaq Second Life Liason Group

The Qwaq Second Life Liason Group, QSL, has grown rapidly and is now holding three meetings a week, each next one roughly six hours later than the previous one, since members span Europe, Asia and North America. I joined them on Wednesday, January 9th for a tour of a replica of Chernobyl in Second Life given by Maxwolf Goodliffe. This Second Life build is part of an effort to raise money for Chernobyl charities. For more about this effort see this YouTube video.

Zazen Manbi, who introduced me to Piet, was there for the tour in his yellow protective suit. Piet is in an earlier incarnation of his avatar here – the bearded avatar in a maroon shirt (just before he learned how to shed his noobe image ;>). Zazen, who is Jeffrey Corbin in RL, recently obtained funding for teaching an environmental impact course in Second Life. Zazen is building a virtual Nuclear Power Station in Second Life to help explain and demystify nuclear power and its vital role in controlling global warming.

A couple of days earlier, Piet brought his friend, astronaut Ed Lu, into Second Life to visit the replica of the International Space Station that he lived in for a half a year in 2003. Ed Lu and Piet were joined by a number of members of the Qwaq Second Life Liason group. The picture above is from Troy McLuhan’s Snapzilla. Ed Lu is the avatar in the white shirt. For more on Ed Lu’s visit to Second Life see Kat Lemieux’s post on the International Space Museum blog.

Piet and Ed Lu, together with Rusty Schweickart and Clark Chapman, founded The B612 Foundation named after the asteroid that was the home of Le Petit Prince. The goal of B612 is to significantly alter the orbit in a controlled manner by 2015. After the tour of Chernobyl in Second Life, Piet and I talked for a while on the solar panels of the SL International Space Station.

Piet described to me how Ed Lu came into Second Life.

He had called me on my cell phone from the space station, when he lived there for half a year in 2003 and two days ago I called him on his cell phone from the SL space station. He was so intrigued, he was willing to come over the next day. We had quite a party! After ten minutes in SL he could talk, walk, fly, and have a text based conversation with his admirers while inspecting the space station. These astronauts are really incredible!

Ed Lu’s visit gave a very auspicious start to the year for QSL which has taken off at rocket speed – good to have some guidance from an RL astronaut! I am very excited to be part of it. Piet has launched some ground breaking communities in virtual worlds in the last year. I can’t wait to see what happens as they develop and expand in 2008.

I think QSL will be visiting the, “The Rhodospin Membrane Protein,” soon. This build on the IBM Research Shared Work Island resulted from a simulation done on the Blue Gene supercomputer. In the picture below, Zha Ewry, Pema Pera and I are sitting in the molecule discussing the meta implications of a visualization of a visual molecule in a virtual world (for more on this build see this post on eightbar, SLNN, and the molecule tour on Destroy TV’s Flickr photostream).

Recently I met with Eben Moglen, the founder, Director-Counsel and Chairman of the Software Freedom Law Center, and David W. Levine, a researcher at IBM’s Thomas J. Watson Research Center and IBM representative to the Architectural Working Group, for an informal conversation that looked at many of the fundamental social, technological and legal questions of building 3D immersive online spaces like Second Life.

I live only a couple of blocks from the Software Freedom Law Center. And, as the opening sourcing of the Second Life Architecture is pressing forward, I decided I must at least try to get the thoughts of my neighbor who is the great advocate for the role of free software as a fundamental requirement for a democratic and free society.

I was delighted when Eben Moglen said that I could stop by and ask some questions. But he made clear, from the outset, that he wasn’t the optimistic advocate for immersive virtual worlds that I am. But the stage was set for what I felt could be a very important debate, so at the next Architectural Working Group meeting in Second Life I asked David W. Levine (a.k.a Zha Ewry in Second Life) if he was willing to come along and take part in this discussion. The photo of Zha below is by Noelani Lightfoot the proprietor of Quixotic Photography in Second Life (see her great Flickr stream here).

David is one of the most experienced architects, not from Linden Lab, involved in the Architectural Working Group’s efforts to open source the server side architecture of Second Life and develop open standards for virtual worlds. What followed was a fascinating and wide ranging conversation in which Moglen and Levine discussed how the choices we make about the design of virtual spaces and avatar interfaces in general can affect the whole path of human society.

Moglen and Levine explore, in depth, the problem of defining digital public space and issues of privacy on the internet, offering many suggestions on how to implement online privacy enhancing technologies and insights as to how we could design the next generation of these technologies in responsible ways.

A Conversation Between Eben Moglen and David W. Levine. (instigated and transcribed by Tish Shute, Ugotrade)

In this interview David Levine is speaking personally, not representing IBM’s official position on any of the issues under discussion. Also, neither Eben Moglen nor I (Tish Shute) represent or have any affiliation with IBM. We are all speaking from our own perspectives. And I apologize in advance for any errors I may have inadvertently introduced through faulty transcription and/or editing to the speech of Eben Moglen and David Levine.

David: This is entirely Tish’s idea, I’m tagging along to ask interesting questions and hear your insights.

Tish: Yes I roped David in because in terms of the Architectural Working Group in Second Life and the open sourcing of the server side architecture as he is one of the most experienced architects not from LL.

David: Yes IBM has an interest in promoting virtual worlds as standards based environments.

Eben: On the theory that avatar based interfaces have some broader purpose?
David: Right. Something akin to Second Life or variations of it in five or ten years is going to be deeply disruptive. Exactly which piece of it, I’m not making predictions actually, or exactly how it becomes disruptive.

I think it turns into the flavor of things like right now you go into Amazon, you go to book reviews. And there are 50 or 60 people who very carefully putting their opinions down and there are maybe ten other people who are curious about the author — who currently are thinking about buying their book. But you can’t see them on the web at all. But they’re there side by side pulling the pages reading the reviews commenting on them. But you have no chance of interacting with them at all. If in fact we can say now that you’re interested in talking about this author and this book, there are ten people happen to be in the internet context, interested in doing that.

Can we bring you together and have a conversation. You can ask a question rather than simply read someone’s written report. Why do you think the character is interesting? Was it because they do interesting things? Was it because their thought processes as described by the author interesting? What makes this a compelling book to you personally? And we can have a dialog rather than a static web.

Things like that, I think, are going to happen. Do they look like giant wolves wandering around freely through the Internet – which is half game space and half social a space, half social interaction spacee. Partially because in fact because Linden Lab didn’t set out to create that thing, people did, so I figure some of that meets human needs.

But the disruptive change is this weird blend between Wiki, comment pages, chat, talk, and 3D helps along with it. I’m deeply opposed to calling it 3D internet. I think you focus on avatars and visualization and you miss the fact that it’s a blend between what My Space does and classic internet chat that’s really exiting -which a lot of social community building.

What followed at this point was a wide- ranging discussion on whether Virtual Worlds may effect fundamental aspects of human neurology -“Is this an interface that grows the mind or shrinks the mind?” – and underlying issues and problems with privacy on the internet. Two key questions Moglen raises are: “Is store it yourself fundamental to freedom in the 20th Century?” and, “Who gets the logs?” I touched on these questions in an earlier post, “If The Metaverse Goes Wrong….” And there is an excellent podcast of Eben Moglen speaking on these issues at the MySQL conference. Also see David F. Flanders blog.

To focus this post on some very practical suggestions re privacy and avatar rights that came up in this particular discussion, I begin the transcription where the discussion began to dig down, both conceptually and technically, into the architecture of digital public space.

Eben: When I teach about this with law students, I say there are three elements that are mixed up in privacy and we tend not to notice which one we are talking about at any given moment.

There is secrecy – that is the data should not be readable by or understandable by anybody except me or people I designate. There is anonymity which is the data can be seen by anybody but about whom it is should be knowable only by me or people that I designate. And there is autonomy which isn’t about either secrecy or anonymity but which is about my right to live under circumstances which reinforce my sense that I am in control of my own fate. And this form of privacy is actually the one we talk about in the constitutional structure when we talk about the right to get an abortion or use birth control.

The Supreme Court says this about the ability to make our own life choices about having control over the serious consequences of circumstances in our lives in which we make moral judgments. But in the original companion to Roe against Wade, Doe versus Bolton, which was the Georgia Law that was being challenged at the same time that the Texas Law was challenged, Mr. Justice Douglas wrote the opinion for the Supreme Court in which he talked about the freedom to walk, loaf, or stroll meaning that it was more than just a question of some particularly important life decisions.

There was a feeling of a constitutional liberty in making ones own decisions about all the matters in ones life, which I think is also the liberty interest that the court protects in a case like [[inaudible]?? The guy simply wanted to walk about LA in the night time and got tired of being asked to show his drivers license at every step because he was a six foot tall black man with dread locks. He sued over the right, which I thought we used to have in the US, to walk around without having to identify ourselves to people all the time which isn’t just an anonymity problem but becomes a problem because it forces certain kinds of choices on us.

I think that the real issue here is about the forcing on choices on us. I see again and again the ways in which people now find themselves unable to make certain life choices easily because there digital self has acquired an inflexibility that constrains their non-digital self.
Lets start with really simple stories of minor behavior.

A law student comes to me and she says: “I had this bad break up with my boyfriend. And I didn’t want to be on My Space all the time looking at all the photos of him with his new girlfriend. So I dropped out of My Space and I went away. But the problem was that I was the photo collector for my little circle of friends which wasn’t so little. And, I had eight thousand tagged photos that was everybody’s life in law school. And, I was the one who was the keeper of everybody’s photo album. And, when I dropped of My Space they couldn’t get to those photos anymore.

So this was a real cost to people and they were really pissed about it. And they needed to decide how to go ahead and persuade me to come back to My Space. But of course they had already graduated and spread out among law firms and cities in N. America. So they had to coordinate the campaign to get me to come back to My Space. So they made a make so and so come back to MY Space page, in My Space. And, they went and hashed out all this over their photos because they weren’t going to leave all their photos from law school in a place which they couldn’t get to them because of my break up with my boyfriend.

Now there is a point that a fundamental decision occurs that she feels pretty seriously about as an individual. But she is being subjected to a campaign of peer pressure to hold her in circumstances that she is not going to like in order to get the photo album back.

Oh we might say oh there are a million other ways to solve that problem you can upload them all to Flickr and get the hell out of there. But what is actually happening she wanted to leave town and she couldn’t.

We have got to understand that when she wanted to leave town and she couldn’t. The digital self was trapped by a fence that the physical self had no problem passing through and moving on from.

We don’t want that to happen to people. We understood when the Soviet Empire decayed that all over it were places where people felt trapped in webs of surveillance and betrayal and interaction that had a kind of sinister feeling even if there is no Gulag and there is no shooting. And many of us feel very uncomfortable with the changes in the society we live in the United States in the past several years where for us there is no Gulag, no shooting, no being swept away with out charges.

We don’t actually think that like some poor lawyer in Oregon that our finger print is going to be misidentified as that of a suicide bomber’s accomplice in Spain. But we are aware that these webs of knowledge about us are beginning to control us because our digital persona is subject to leverage and to being interfered with in ways that matter.

I think that the question is there a power in projecting yourself into new environments in which you can meet people in new terms and in rich ways has no second answer, and there is no downside to it. But we owe it to people to build safe spaces.

When you said its Anaheim and they own it. We are touching on one of the really important problems for me as a lawyer which is these concepts of property.

I have just got done with guys who think protocols are property. Now I am going to have guys who think music is property. And after that I am going to have guys who think carriage is property. And at the other end is this sort of “you are on my digital island and I can look through your clothes if I want to because it is my island and that’s the way I make the rules.”

Now we have got a very strange notion here of property and we have a very unusual structure because most of the property that they could have in Anaheim is subject to governing. And even if it is Celebration Florida they still have only as much Home Rule as the Florida legislature will give them. And if there march against Alabama or even Logan Valley shopping center there’s rules that say what you can do in a company town about controlling people’s speech. I feel absolutely certain that when Google does it there will not be a guaranteed right to carry a sign that can be read by everybody in town explaining why Google sucks. It’s just not going to happen that way.

David: This is the “I write the TOS for people coming onto this public space. And there is no constitutional guarantees in this space.” Argument.

Eben: And I get the benefit of calling it a public space by which I actually mean my private space, real estate I invented.

David: Now in fact if you wanted to do something interesting there. This is to toss something into the tools to play with. And to toss it to you, in particular, because you are good at playing with these kind of tools. Which is, if I was going to attack that, I would go and attack that under common carrier. If you want to be a public space, if you want the benefits in law of being a common carrier then you have to accept these constraints on you behavior.

Eben: Well this is why this network neutrality thing rolled up out of nowhere. This is where this came from. The problem with this it is that it already begins by saying what you really like is innkeepers and stagecoach guys. This is an attempt to throw your selves 800 years in the past in order to come up with an analogy.

I think what we really want to say is something like this. If you are talking about a public space your talking about a thing that has not just a TOS contract but a social contract.

It’s a thing which has to do with what you get and what you give up in order to be there.

There ought to be two rules about. One: Avatars ought to exist independent of any individual social contract put forward by any particular space. And two: Social contracts ought to be available in a machine readable form which allows the avatar projection intelligence to know exactly what the rules are and to allow you set effective guidelines about I don’t go to spaces where people don’t treat me in ways that I consider to be crucial in my treatment.

Its one thing to say that the code is open source – let’s even say free software – it is another thing to say that that code has to behave in certain ways it has to maintain certain rules of social integrity.

It has got to tell you what the rules are of the space where you are it has to give you an opportunity to make an informed consent about what is going to happen given those rules. It has got to give you an opportunity to know those things in an automatic sort of way so I can set up my avatar to say, you know what, I don’t go to places where I am on video camera all the time. Self, if you are about to walk into a room where there are video cameras on all the time just don’t walk through that door. So I don’t have to sign up and click yes on 27 agreements, I have got an avatar that doesn’t go into places that aren’t clean and well lit.

David: So if I am going to walk into a space there is an “Ehum – you about to go into a space you really don’t want to go into.” So I can make an informed decision whether the trade off is worth it or not.

Eben: But that’s fine but you are going to have to go even further by saying here is the reason why you don’t want to go in there. Here is what you said in the past about why you don’t like places like this. You got to have an angel sitting on your shoulder – a code angel.

David: But before we can even do that we have to mark the world in such a way that I can make – that I can detect that I am making that choice.

Eben: That’s right you have to force the existence of social contracts in terms that are explainable. It is like I say to my students: “Should there be informed consent for My Space or Facebook?”

If I were doing this as a University project – if I was setting this up inside Columbia University – and building one of these space, and if we were going to have people there, volunteers, I would have to go to the IRB. I would have to explain why this was permissible human subjects research. And I would have to show the IRB an informed consent disclosure that showed what the risks were and allowed people intelligently to decide whether they wanted to run them.

Now I have got libertarian colleagues around the university, in the Law School in the Philosophy Dept in other universities who think this IRB stuff is creeping totalitarianism because it is inhibiting the rights of researchers to find out about the world.

And I understand their point of view. Just as I also understand why we have decided that there is some unethical human subjects research conducted in the past of which we consider so abhorrent that we won’t allow researcher to use those numbers. We consider that data to be the fruit of the poison tree. Not even to save our own pilots shot down in cold water will we use the studies about what happens when you put people in freezing water because we think that evidence comes from experiments you shouldn’t be allowed to conduct.

Now I look at these immersive experiences for children that the Times was writing about last week where you have four and five yr olds buying virtual gear in immersive spaces. And I think that is unethical activity. I think that the rules about children television are weak and not very important. But they are way stronger than that would let you get away with that kind of stuff on TV.

I think that is really serious screwing around with children’s wiring to explain to them that they have to be consumers of stuff that doesn’t exist where the only reason that they can’t have what they want is because the software is programmed not to give it to them unless they pay.

That’s a fundamental education in control of life by unfree software so elegant that Stallman probably doesn’t even know it is there because it is so horrible. But that is another perfectly imaginable outcome of this stuff. We create these things we create beautiful 3D amusements parks. We sell children rides for actual money extracted from real parents because the ride won’t go unless you put another nickel in the slot. And we going say – well its ….Disney Land.

Tish: Yes the children’s worlds can be very extreme….

Eben: Well Disney Land is a little extreme. When you go to Disney Land they open a file on you and they follow you through life. Disney buys everything that is can buy about all those children who have ever been to Disney land and they have got a reason which is bringing them back.

The average number of visits to Disney Land by people brought to Disney Land as children is four in a lifetime. Disney’s goal over the next 20 years is to make that six. And in order to make that six all they have to do is keep buying everything they can buy about all those people they opened a file on as children looking for opportunities by data mining to find an opportunity, like the oncoming 65th birthday of a parent or an oncoming retirement or a this or a that. They just have to play the siblings off against one another – you’ve never been to Disney Land but Johnny has. … They are going to do what they can do to create feelings around that person that says remember what it felt like being in Disney land when you are child – do that again.

Tish: So what can we do about this – for example now we have an architecture working group – a community group – that is working with Linden Lab on creating an open architecture that aims to get away from the current model – the one that you are talking about right now that is currently the model for LL and everyone ….

David: And Linden Lab are better than most in that they are actually relatively explicit.

So, for instance one of the things that came up in their kick off for doing their next generation of architecture was what are the Avatars Bill of Rights what things do we prevent the technology from letting happen to avatars. They actually had that discussion.

And Lessig who was relatively enlightened about this had some comments about “Well we don’t want to force going to a public space that somebody else hosts other than Linden Labs to be a way of taking away rights from avatars explicitly.” Which is good. But none the less the model for 90% of the service providers is always going to be: “We have got you’re data. We have got your chat stream, we may have your voice stream, and we have your action stream. How much do you realize we data mine?”

Tish: Is the architecture group thinking about these issues?

Eben: One way of imagining that business …is that there is a “screen” and somebody is going to get it. Another way of looking at it is there may be some element in which the avatar projector is itself the guy who has a part of the screen.

If you want to be really aggressive about it lets assume we have infinite processing power and can slow things down to a crawl. We can take that screen and split it up in a shared secrets manner so that you could reconstruct the screen out of the data held by all of the people who were there. You could even reconstruct the screen if you had opt in co-operation from say 4/5ths or 5/6ths of the people who were there but no one person who was there could reconstruct the whole screen.

This is the sort of thing which cryptography was designed to make it possible to do. Its how escrowing systems in the high security world are supposed to work.

Lets just go ahead and finish this up. For every one of these that is built in the non secret world there is going to be one built in the secret world. So in the Government world we are going to have secure structures. One possible way of having secure structures is there is a trusted server and that will work for intranet kinds of use.

But lets just imagine for a moment what I think of for a conceivable reason for having this stuff is for the conduct of diplomacy, actual meetings. There is a whole lot of stuff that diplomats do because which is very expensive to do because it involves sitzplatz.You send guys to Vienna and they meet every week for thirty years for no good reason . You could do a lot of that in other ways – some of it by video conference – but lets us just for a moment assume that there are things you could do by means of certain kinds large meetings that occur in a neutral environment.

You could set it up so that no one party at the negotiating table actually has a record of what happened. Only by cooperation of multiple parties is the record of what happened there reconstructable. Because the way the space works is that it automatically divides all of the stream data up amongst the avatars and gives it to them in a shared secret structure. So without the cooperation of avatars you don’t have a reconstructable event.

At the moment, I am talking about technology that I can spec but it would be a little bit burdensome in performance terms to implement. But that doesn’t mean anything if we just keep doubling the speed of the chips.

David: Presumably if any of us get together and we have a conversation at my client end I see everything that is said, in some form. The question is how you prevent me from keeping that stream.

Eben: yes right we are going to be living inside an encrypted environment ..

David: Yes right and right down to the eyeballs of course because at some point the digital stream is there and I can capture it…

Eben: maybe although but remember the data mining isn’t just have having the chat stream…

David: No I agree, so the question is how much can you reconstruct – you can reconstruct a lot of it. I can pretty well capture the stream of everything my avatar saw. I can’t see what the other people had in the background conversation without their cooperation. But I can do everything that is said in public.

Eben: Yes, of course, and the most important thing is that, that is as much as the space manager can see too. And the space manager can see no more than what happened in public which is usually what we are concerned about.

David: So one set of reasonable desiderata here would be that stuff that we build enable a lot of that – which is to say that assets held by private individuals it should be possible to put them on private servers and pair wise or group wise discussions should be securable in and of themselves.

Eben: The protocols ought to be agnostic) But, it would be enough if the protocols were agnostic with respect to how that collection occurs. If the protocols of operation don’t necessitate that the data of operation wind up in any particular place then you got a protocol consistent with both what is now the familiar model and the less familiar model where more information stays at the end place.

David: Right now every single bit of data that’s in Second Life is owned by Linden Labs because you put it on their server and they fetch it back out.

One thing we are going to do for lots of good reasons is go to a web centric model where lots of the assets get pulled on to web servers traditional old fashioned web servers. At that point there’s absolutely no reason why I can’t say these are my assets I will let people come to me and ask for permission to see them. And many of them I will share freely, because the clothing I am wearing is the clothing I want them to see the clothing I’m wearing. But the contents of my pocket, my wallet, I’m not going to be so eager to give them. And it ought to be possible to have this office in a virtual world in which you say…

Eben: My filing cabinet is locked.

David: Right! The furniture, everything in here is public. But I’m going to pull out this document in which I’m going to show you some really cool ideas about this law suit we’re engaged in, and you can read it here, and you can walk away with what I gave your client to read. But you can’t find out the rest of my file cabinet at all because it’s safe in my machine.

And you may not even be able to take that document off the island as it were, out of the office. You may be able to take the pages you’ve read from it. The other thing that’s equally important is can I do the chat pier to pier rather than through the central server. Is there any reason why Instant Messaging as opposed to public chat gets dragged through public space. It certainly shouldn’t be.

Eben: Right. The protocol design ought to take as a feature that there is no requirement for intermediation of any communication among parties.

David: Right. Other than that which we need to do to actually get the consensual state. What we’re doing is taking my word, your words, Tish’s words, putting them into one spot and then sending them back out.

Eben: There are things we can agree to do together, but once we are together there are things we can agree to do apart.

David: Right! But there is some core tapping. If we were each sitting on a computer 500 miles apart, in order to have this discussion we do have to bring the space together and spread it back out. But we don’t have to let the guys over there see it unless we want to.

Eben: What we have is both point to point and many to many kinds of conversational modalities and that there shouldn’t be any intrinsic reason to block them off. Now there may be situations since that famous parental controls system. I don’t actually want people sideling up to my children and whispering in their ears in public places. I’m content to let my children be publicly addressed in public places because transparency is an opportunity to avoid certain kinds of misbehavior. And there may be locations which are transparent locations. And if you enter a transparent location, that should be something known to the avatar in a technical way as an entry point and it should be possible for avatars to either seek or avoid such spaces.

I think the goal here is to confine the conception ‘property’ to meaning something like traditional right to exclude and not let property in the form of the greater includes the less since it’s my property I may define everything that goes on here, and I may make whatever rules I please here. There is some community of communities which defines ‘states’ which it is OK for places to be in.

There was a story in the New York Times a day or two ago, about a public park in Oregon that had gone bad. It was written in a slightly frivolous but not entirely frivolous way. It’s an ex-urban park, meant originally as a kind of highway rest stop park, far enough from everybody else that bad things began to happen.

People began to have sex in public, there are child molesters who pitched tents there and lived there, and so over time it becomes more a locus of crime than a locus of recreation. And what the parks authority of Oregon does is they shut it down. They close it up. They chase everything away from it. And now they’re in what they think of as a sort of rebooting period for the park. And then it’ll come back and a different community will be attracted to it and it will be a different park.

This is an example of ‘space gone bad’ in meat space places. You can certainly imagine renegade space in a immersive world. Places where operators are not following the rules. Places where the operators are taking advantage of the free software nature of space building to build spaces which are deceptive about how they look.

They seem to be spaces of type X and they’re actually spaces of type Y. That should be at least a regulatory interaction if not a crime. That should constitute socially actionable misbehavior. Making spaces that don’t behave the way they seem to behave, that don’t give you what they seem to give you. And if we come out of all of this saying the FTC will occasionally make an order about this, I’m going to feel very dissatisfied.

David: There are a couple of interesting things lurking there again, one of which is, in the web today we don’t generally think of it as having two parts. There’s the very public part of the web, where you go to a page like CNN and you get the content and it’s assumed to conform to fairly common broad norms. And then there’s a different part of the web where people know that that’s not true. All pornography and large portions of adult community which are not necessarily pornographic but have some content of theirs that’s restricted and they tend to have click throughs often. So you get something that says you need to be informed at some level that something’s happening.

At the moment none of that interestingly enough is in the protocols, there’s nothing in http that says I have to mark my page as mature, or I have to mark my page as you might not want to bring your child here.

Eben: Right. And as you may recall Larry was himself a big believer in picks and the idea of privacy platform standards built in the web pages. In the end the theory was look there’s a big first amendment problem with making people grade their web pages.

Sandra Day O’Connor went for this theory of doors and locks and walls in cyberspace in the first child decency case. Larry realized he had started the United States Supreme Court down a very dangerous crooked highway and to climb back out again, it’s never happened. And Tim Burness Lee and the others weren’t enthusiastic to say the least, so it never occurred.

Now here it’s a little bit different and the reason it’s a bit different is your public accommodation law kind of aspect. You’re making a public space OK? We’re not telling you grade your web pages, we’re telling you if you’re going to have a public space you’ve got to obey the fire code. You have to post maximum occupancy. You’ve got to avoid chaining the fire doors shut.And you can’t put a surveillance camera in the ladies room. You can’t do it! And that’s sense. Not it’s my space I can do whatever and if you don’t want to take a piss in my monitored men’s room, you don’t have to. That’s not going to fly.

David: Right. There’s a series of layers here just like we say look there are certain things that you can never do in your public space or even in your private space. We don’t allow you to say this is my 5000 acre ranch therefore I can shoot people. We say there are limits to your right to use your space as you desire. Even if you have your Celebration – home rule ends with the norms of the state.

Eben: You can even own a city but you can’t own a city which is an island in space all by itself. It’s got to be part of the rule of law it has to acknowledge sovereign power

David: Except for Mogda Dishu perhaps.

Eben: Well — and that’s what we call a failed state. And I guess what I’m worried about is building a…

David: failed digital space.

Eben: That’s right.

David: That I think there is an argument here that is interesting and worth exploring which is what would be good ways of – you know – we don’t have good rules for digital space we have none in fact. They are a Wild West in fact. There isn’t any sense that the constitution will apply in digital spaces.

Eben: This is why the whole political evolution of free software turns out to be interesting. Stallman creates in effect a constitution of the project, that’s GPL. It functions very narrowly but within its very limited range, it’s supreme, as Judge Marshall would have said right.

Debian free software guidelines and the Debian social contract tried to imagine the constitution of a distribution. Tries to imagine a constitution of a state solely about software building. Privacy policy in the United States which his supposed to be market driven and grow out of positive interaction with the FTC gives you the Ebay privacy policy, which is basically an attempt to imagine the constitution of a flea market.

David: And it is a reasonable thing to do if you are running a digital flea market.

Eben: And what you just said is that Second Life didn’t have a constitution because it it overdrove its headlights. And now your questions is so as that explodes as it inpupalizes as it loses its org chart and begins to float free as technology what would it mean to imagine the constitution of digital spaces in a multiplicity of reproducible forms ? That would be when it franchises.

David: What would it mean if in fact we succeed at the desire to say now we have an open software specification that says anybody can host a chunk of content they can create a public space using this technology and they can invite people to come into it and can bring content into it and share it. What would be good social contracts for those space is a very cogent question.

Eben: So one aspect of this is the debate between those who believe in the so called Aferro GPL and those who don’t. Let’s go back to this as a question of Free Software specialism.

I am going to make a pitch that if you don’t copyleft the software you have got another whole problem. Because in effect we all then l go to work for the guy who wins the race to the bottom. If we imagine this a BSP software we all become developers for the guy who has the one that is most abusive and least free.

That is why, in situations like this where we are talking about software with profound but multiple social consequences, I feel strongly about the utility of copyleft. The second thing I would say that if you imagine within the world of copyleft the fundamental difficulty of the, “its my server I made private modification I want to run this code for people who are willing to have this program run for them,” you can sort of see why even if you’re not distributing the software, if you’re offering services with this code, you have got to release the source code so we can see the modifications and learn from them becomes an important property in licensing.

That’s why the young guys like Mako Hill and others around Stallman are partisans of the Aferro GPL, because they see this in the long run as a really important freedom related question.

Google of course has entirely the opposite point of view of resting on rights which they want to exercise. And Google increasingly says, “well we’re rich enough to re-implement anything we want to, If it’s got a license and we don’t like on it, we won’t use the free world version. We’ll just write our own.

And that raises questions about when do you have to regulate regardless. Now in the world of e-bay because the free market takes real money for real goods and is patently engaged in interstate commerce nobody wants to rumble with the federal government and so everybody has a chief privacy officer and everybody acknowledges that there’s a regulatory role to play.

I think you could get to a point where that would be less obvious than true with respect to other kinds of entrants to the field. And I’m not convinced that the way you can get this done is by licensing the code correctly. But I do think that licensing of code is probably a part of the solution. I think the question of who is the steward of the code also matters. That is to say I think it probably does matter who the legal personalities are who maintain the spaces.

That Central Park out there, that’s a very municipal park but it’s got a bunch of people called the Central Park Conservancy, who are rich people who sit on a board of directors, some of whom hold public offices, some of whom don’t. But they put up a lot of money to keep that park and they have some kind of mysterious rights in Central Park that I’m not entirely sure I understand very well.

Just as I’m not very sure I understand this business improvement district for Lincoln Square that has these little rent-a-cops who patrol my neighborhood. As a local small business, I’m supposed to be very happy about those guys. But I’m perfectly aware that that’s a layer of government in which I am complicit but which I don’t control. And if I see one of those rent-a-cops kicking a homeless person it’s not exactly clear to me whose telephone number I’m supposed to dial. Even though in theory I’m the owner and supporter of those guys.

So I think that there is an ethical advisory board kind of question. I think there’s a dialogue. I think it’s a structured dialogue. I think it interacts with architecture at technical levels. And I think it interacts with structures of licensing and IP control. And I think it interacts with the strategists who invest on the basis of expectations about business laws. Because I think among other things it has a public informative role to play.

I’m sure that we’re going to enter here at maturity a dialog which looks a lot like some American style discussions about regulation. The people who think that disclosure is enough, the people who think that rules are enough, the people who think that you need agencies with quasi-prosecutorial powers to unearth wrongdoing, the people who think you should reward whistle blowers and so on.

The part that most interests me is the engineering because I believe in the durability of technology which is path dependent more than I believe in the congressional will to get things done or to stay where you’re put. The most corrupt man in Abraham Lincoln’s cabinet Simon Cameron Said, “the definition of an honest politician is a man who when bought stays bought.” and I’m very dubious about the honesty of our politicians.

David: There are clearly some really nice social activism open source related questions and any of that is…

Eben: And I don’t want to press this decline of reading stuff either because we agreed that is whatever it is so now let’s try and talk about the stuff you came to talk about.

David: By the end of the day I think there are 2 questions here one of which on a personal basis is as technologists one should have a responsibility to build technology that’s socially responsible. If you don’t you should stop doing it frankly. There’s a fairly good question: “Are we in fact as stewards of the technology doing the right thing architecturally?”

Eben: Well your proposition about disruption is the right proposition in response. Something is going to happen.

David: Right. But as technologists we can’t prevent the disruption necessarily. But we can shape it, shape some of the consequences of the disruption and thoughtful voices like yours saying here are things to look for.

Eben: It’s what lawyers usually do. Having thought about this let’s now let me try and tell you about things you’ve written [I submitted some written questions from myself and other Second Lifers prior to meeting Eben in person].

Tish: Well when I mentioned that I was going to talk to you several people had some questions in Second Life. This one came from Gareth Nelson (a.k.a Gareth Ellison in Second Life):

LL get people to sign a contrib agreement allowing them to relicense their work commercially. They also use trademarks to make it very difficult to release a product based on the viewer without their consent in order to release a non-SL product based on the viewer you’d have to remove all mentions of SL in the code or LL have stated their
policy is to enforce the trademark. It would be interesting to get a legal opinion on that – i.e is it reasonable for them to deny forks with such means?

Eben: you can imagine somebody so embedding their trademark material in a GPL program that the difficulty of removing it is so severe that in practice they’ve essentially made it impossible. And, if they then behave in a way which attempted to inhibit distribution of the software through the exploitation of their trademarks they would in fact have in one sense or another made the GPL a nullity.

But here we have a complexity because it’s theirs to start with. They haven’t got any third party code in there. So until they have some third party code in there they could have not had GPL at all and that would have been perfectly acceptable. And they can GPL it but nonetheless make their proprietary commercial license business attractive to people by creating incentives to use the proprietary licensed version instead of the GPL’d version.

Since removing all the trademarks from the GPL’d version is a job that only has to be done once, because after that all you have to do is remove the trademarks from patches, and removing the trademarks from the gifs doesn’t mean going through the whole codebase again. The general likely outcome is I’m going to say no foul has occurred. It’s their code you can scrub it once.

It would be highly desirable if they took say the step that Red-Hat has taken of finding ways to segregate all their trademark graphics and other ancillary elements into one branch of the code tree so that you can come along and lop that branch off and replace it with other graphics or other materials that don’t bear trademarks and be done with it.

And if in the long run it’s the Linden Lab position that they want to live as easily as possible with the community that’s what they’ll do. But I don’t think the short way across is to say they can’t do what they’re doing now. The short way across is to say if they inevitably continue to do what they’re doing now and they mean it about aggressively protecting their trademarks, they will wind up defeating the community they seem to want to have.

Tish: This next is a question about virtual assets causes a lot of controversy when it comes up in discussions on Second Life:

How can the Architecture Working Group (a community alliance working on an open architecture for Second Life) think about assets in a broader world than SL? And how can AWG encourage and make fair content creation? What properties would be desirable, in such a scheme, and what legal issues show up?

David: Let me toss in a little background. This is a question I’m particularly interested in, which is people are busy creating digital assets and they want to share broadly and in particular they want to sell them. And, in some sense, one can argue whether encouraging content makers to do content for money is a good thing or a bad thing. But it is certainly what we do for a world these days. But, How do we deal with music? How do we deal media? How do we deal with digital assets that people want to share?

Eben: So the problem again is that we live in a world in which duplicating bit streams is easy and asset value consists of the artificial hardness that we put into the work of duplicating the bit stream.

We create a scarcity which we protect by technological means. The more that scarcity value increases, the more desire there is to subvert the mechanism that inhibits the doing of the actual trivial act of copying the bit stream. The consequence of which is that we get a whole lot of weight on the paratechnology of protecting bitstreams against doing what networks do to bitstreams which is copy them. That’s one of the things which bears down on this.

You have to have everything on one server, because then it’s the operating system code on the server that protects against the copying of the bit streams. So the Fort Knox of the asset system becomes the server which has little rules about who’s got what.

Unfortunately there is no actual difference in the world between Second Life assets and first life assets because money has also ceased to exist in the twenty-first century. Money in most of the long history of human beings was stuff with scarcity value in lumps which could be weighed and measured. After that money was the fiat currency of governments whose credit was either infinite or knowable which was hard to counterfeit.

In the 21st century money is information flows arbitrarily assigning to me the right to purchase so many goods in the world and to you the right to purchase so many goods in the world, and so on. And the banking system is essentially a system for securely keeping sets of information books which if hacked, or otherwise interfered with, can radically result in the transfer of immense amounts of money in no time because all that’s happened is some bits have changed.

So we’re moving very rapidly into a world in which the Fort Knox’s of first life and the Fort Knox’s of Second Life are the same. They’re numbers in protected files protected by access control levels of one sort or another.

The good news is again through shared secret kinds of technologies or other ones we can break those assets up and distribute them across a million servers so that it doesn’t have to be centralized in order to be as safe as it is when centralized.

I can’t spend the money, you can’t spend the money, and she can’t spend the money because no one person or even two out of the three of us can reassemble the check. Bring us all into one place, make her J.P.Morgan, you Morgan Stanley and me the borrower and suddenly we can put our pieces together and the lost dollar bill is resuscitated.

Again it’s not a perfect solution to the problem. The 21st century economy is the first time there wasn’t anything that prevented me from stealing all the gold in Fort Knox. Sooner or later all the gold in Fort Knox is going to get stolen.

We faced in the first round of crypto wars the presence of the thing called the briefing which is what the incoming president and vice president used to get from the NSA about how we saved the planet through surveillance and if you had unclean encryption there’s going to be nuclear destruction.

I can remember when Mike Nelson of IBM got ‘the briefing’ as an incoming Clinton administration figure and was then sent to Phil Zimmerman and to me, Oh if you only know what I know now you’d give up the fight. And they said well publish it Mike and maybe we will. Instead we constructed a thing called the alternate briefing which was a briefing to policy makers which said because you don’t universalize strong encryption in the world because everybody doesn’t know how it works and everybody doesn’t have access to it, a secret hole built up last night in the world banking system three trillion dollars had disappeared. Nobody exactly knows where it is this morning. Markets open up in three minutes what do you do now Mr. President?

The problem with the Second Life asset is really the problem of the [First Life] asset. It’s not different. And one of the scary things about the 21st century is that it isn’t different. Having said all that, I agree that this question of making money out of nothing, so that assets come into existence suddenly is scarier for many of the people who run the world’s monetary system than the sudden disappearance of money.

It’s really scary that somebody could be making something that would have value in the system that they built because they think they’re the only people with right. In theory it could get to be a really important problem, The good news is that most of the people on earth can’t eat virtual food. In the end the power of the real economy is going to keep this thing restricted to a certain kind of luxury production. And we are probably good at that. That may cushion some of the difficulties that arise in the world where Pay Pal does more business than all the Forex markets everyday.

The stupidest piece of economics I ever read in my life was the paper which when I got to law school was the most admired piece of economic reasoning of the past generation which was Milton Friedman’s classic article about why currency speculators contributed to currency stability Friedman wrote this paper and most of the economists who taught me when I was in college in the seventies thought that that paper which was then 16 years old attacking the Brenton Woods’ system on the ground that currency speculation is the primary means of maintaining currency stability was the most brilliant thing they had ever read.

Bernie Saffron, my teacher at Swarthmore, who had served on Jimmy Carter’s council of economic advisors gave me a B+ in micro economics in 1979 after the A+ I got in macro economics in the fall because I wrote on the exam that Milton Friedman’s paper was the stupidest thing I ever read. Now as George Soros can explain I was correct. He was the one who got rich knowing that Milton Friedman was an idiot.

Currency speculators don’t contribute to stability of the world currency reserves. And the reason that they don’t is because they are not the marginal arbitragers that Milton Friedman thought they were. Milton Friedman’s logic was revealed to be horseshit the minute that the Forex traders started trading more every year than the monetary reserves of the large countries because that made central bank intervention irrelevant. And that’s what shelved the Rubel.

And it is what produced the great problems of currency in the late 1990s. And it is what’s about to produce the enormous difficulty we are going to have with the United States dollar – which is governments don’t control currencies levels anymore. Friedman thought that these arbitragers, these beautiful capitalist business men operating at the edges of a system whose gravity is over here can do nothing but stabilize it. Soros said wait a minute the axis has been flipped governments can’t even stabilize what I can trade.

Now we about to live in a world where that is square. And there is effectively no way to get larger than Pay Pal or the other bit traders eventually get because they have every business from two kids in a bar sharing a bar tab. I watch these guys in my twenty something crowds in LA first then in New York. When they go out for a round of drinks they Pay Pal the money back and forth to themselves from their handhelds at the bar. That is how they square the drink stab at the end of the night.

They have everything from that micro handle to all the business on EBay to the point at which if I want to get money to somebody elsewhere in the world I am way better sending it through Pay Pal than almost any other mechanism. It is almost one thirtieth the cost of sending a wire transfer internationally, and my bank is going to give me all this bullshit, and the Federal Government, and the know your customer and all that stuff.

Now admittedly the United States Government crawled inside the ass of Pay Pal when [inaudible]? were running it in the first place. They had it totally wired. But that’s OK it is less paper work than I have with dealing with J.P. Morgan.

Now we come to the Second Life asset stream look at what happens. We have virtualized the entire world …this stuff is stuff over in the corner but you can’t eat it. In the end you can’t trade it in for the stuff you can eat at anything like the speed that would be necessary in order to build it as a hole in the world financial system. When it is a hole in the world financial system you will suddenly meet some very powerful people making a real fuss.

Is anybody from one of the major investment banking houses sitting in on the Architectural Working Group.

David: No they are not.

Tish: But they are hovering around Second Life.

Eben: Well I will make a prediction. They will show up. And they will show up very heavy, very hard. If you are lucky they will show up through the securities industries financial management association or one of their sort of combined IT shops. If you are not lucky they will show up with the storm troopers because this is their stuff you are fucking with.

David: It is only a million a day turnover today so that is not scary to anyone. But if it goes from 1 million, to 10 million, to 100 million over the next ten years, they need to understand what is going to happen.

Eben: And they can expect to either be looking at an unregulated wild west that they would approach with both vigor and prudence or a highly regulated environment in which they will look for a way to use the house edge. And what they each want will depend on forces I can’t foresee about what Merrill Lynch and Citicorp are each thinking.

David: If I personally had the decision about this I would say we have pretty good micro-payment systems – called Pay Pal – why on earth would you make a new one.

Eben: Well yes… except that that one has its roots dug deep in the actual financial system which means the minute you go there, there is no fire wall. The play money aspect of the thing makes it possible for a whole lot of stuff to happen.

Your guys who are all so interested in their assets values they haven’t met the tax authorities yet.

David: Some have.

Tish: Yes in Europe with the VAT.

Eben: Yes I have heard that.

David: But the vast majority of them haven’t worried about the tip they get for playing live music being taxed the way they should be.

Eben: What will happen if that thing gets its hooks into the financial system, if it did share a payment platform with the real world, the taxing authorities would be there immediately. Look at the United States government’s attitude about gambling.

The asset values built up in an Antiguan casino online are the asset values in second life. They’re paper profits and losses generated in a flow of offshore money which if it has an entree to the American Credit Card and bank account is the story. So you could model this with the online poker explosion.

My CTO Bradley Coon was there every step of the way going up and every step of the way going down. Bradley used to sit at home in the evenings and play 12 tables while his wife was doing the Live Journal. They had a beautiful little digital console. Bradley played blindfold poker 16 tables while working for me. And of course made lots of profits which as the United States government began to close down the online poker shops and arresting British business executives on their way through the United States on their way to Acapulco and stuff like that, slowly closed down. And figuring out how to get your money out of each of the online poker houses as they each surrendered to the government and they effectively quarantined peoples winnings became what all the poker players were doing.

If Second Life actually ran head on into the tax system there would be a sudden recalculation of a whole lot of trading positions. This whole problem is the problem with the internet sales tax moratorium. It’s the problem of the bit tax not just the VAT on the goods but the VAT on the telecommunications services which is the great unregulated question in European tax laws.

David: And Australia for that matter. They have the same regulatory scheme, pay for bandwidth, pay for each bit.

Eben: And that proposition leads to the clear sense that the telecoms provider ought to calculate cost going in and cost going out and pay the VAT on the difference which produces an enormous windfall to the public fisc and which has uncalculable consequences for the nature of the digital economy. So people back off.

In an European law faculty there will be a person with a tenured chair in VAT that’s a separate legal specialty OK? And I’ve heard the European VAT professors when they get together and start fighting about the bit tax. They use services. Sales taxes on services have a very powerful European history. There was an 18th century system for sales tax on services which were brilliantly effective. It was called stamp taxes. It meant putting little pieces of revenue stamp on the pieces of paper on which services were performed. And if you were a service businessman all your documents in and out had revenue stamps on them and stamped paper was what you did business on. British have been paying stamp taxes since the early 18th century in perfect peace.

When you attempt to introduce sales taxes on services in North America you get trouble. It was done by the British Empire in the late 1760s and there was trouble. It was done in Florida in the 1990s and there was trouble. Our state advertising agencies didn’t like being taxed by Florida sales taxes on services and they contributed a great deal of advertising to making it very hard for the government of Florida to continue until it withdrew the sales taxes on services.

We on the other hand paid poll taxes in the United States until the Supreme Court outlaws them with the greatest happiness in life, in part because it helped us to keep black people from voting. Poll taxes were imposed in England twice, once in 1381 resulting in the peasants rising, and once under Margaret Thatcher resulting in the end of Margaret Thatcher’s regime.

Which cultures will pick what taxes is a very hard thing to understand. Nobody knows what virtual communities will bear and how they will pay their taxes. But when their values rise to a certain point they will pay their taxes and the question of that imposition will be disruptive in another way.

So I think that people ought to go cautiously they ought to understand that the big players with the big power, the people who use the money system around the world with a great deal of muscle and the people who use the tax system around the world with a great deal of muscle have not arrived at this party yet.

If they start making their rule under the assumption that those guys are never going to show up, they’re going to be very negatively surprised by the outcomes.

What they really need to be doing is inconspicuously and thoughtfully planning for what their positions are going to be when the big guys show up. And they need to get ready for that more than they need to have pipe dreams of their own about how they’re going to use technical means to preserve their asset values in currencies that will never make it to market, and that will never be subject to income tax and VAT. It’s going to look very different when it reaches equilibrium.

But while Cascio pointed out many of the ways which the metaverse by “making visible the invisible” (Bruce Sterling) can enable us to make smart and hopefully wise choices about our world, his presentation had a cautionary note. He said he is a “clear eyed optimist.” He believes that the future is in our hands. And, we need to make the right choices and build the future we want.

We are living in an exciting moment as these technologies are emerging today.

But we live in a world of “asymetric transparency” where people who have economic, political, social and religious power have the ability to know what we are doing and we don’t have the reverse unless we go through a lot of trouble.

Questions about digital public space and control of you own persona are keys to whether the “metaverse goes wrong” or not.

Cascio showed the slide (opening this post) of the George Orwell Plaza in Barcelona which is surrounded by surveillance cameras. Brad Templeton has argued that widespread transparency technologies will inevitably be corrupted and co-opted by those in power. Templeton argues:

If you put in place the tools which would make a police state possible, they make the police state inevitable. Repressive states are real; near-panopticons are real; transparent utopias are hypothetical.

See here for Cascio’s look at the privacy debates from the Accelerating Change Conference 2004 with David Brin, author of The Transparent Society ,who argues “that surveillance technologies are here, and we should fight to make sure that they are two-way, and not just in the hands of elites” and, Brad Templeton, chairman of the board at the Electronic Frontier Foundation).

Jamais Cascio’s has put forward the notion of a participatory panopticon. This idea is exemplified in the Witness project which provides video cameras to human rights activists around the world in order to document violations and abuses. Cascio suggests a similar model might work for an “Earth Witness” project, “a second superpower” army of networked environmentalists:

imagine a web portal collecting recordings and evidence of ecological problems (human-caused or otherwise), environmental crimes, and significant sources of greenhouse gas emissions.

Cascio has suggested designs for an earth witness phone.

During his talk, Cascio gave several powerful examples of the potential for a participatory panopticon to benefit people and the planet. He mentioned the work of Dr. Larry Brilliant who is famous for his role in eradicating smallpox. Brilliant is now Executive Director of Google.org. One of Brilliant’s current projects is to skim info out of public databases and newspapers from all over the world and construct a map to aid in a prediction about where the next pandemic might come. But the question of privacy loomed large in Cascio’s talk.

“Privacy is what they take away when they want to torture you.” The Earl of Spencer

The above quote was side by side in Cascio’s slide of surveillance cameras in George Orwell Plaza, Barcelona. While privacy advocates are winning some battles, notably in the recent changes by Facebook to the Beacon advertising system (see New York Times, CNET and MSNBC for more):

[they] will allow users to “opt-in” to sharing information through the service, which broadcasts purchases made on outside websites to Facebook users’ friends.

MoveOn.org, the on-line advocacy group commented that this was a “big step in the right direction.” But, we are still a long way from a situation where surveillance technologies are not primarily in the hands of elites. Albumoftheday reveal serious “big brother” underpinnings to Facebook.

I visited Eben Moglen, founder, Director-Counsel and Chairman of Software Freedom Law Center, recently (the interview is upcoming in next post). Moglen is deeply concerned that:

An entire generation of people are now fundamentally mis-reasoning about what it’s like to behave in surveillance spaces, who are being led to the slaughter in a very unpleasant fashion, in which they have agreed to move large parts of their private life into instrumented environments, big brother houses, built by people who want to use that data for the purpose of influencing their behavior economically and in other ways. And, they don’t have informed consent. And, they don’t have structures of regulatory intervention.

I attended Cascio’s talk as my avatar Tara5 Oh (above) in Second Life. Henrik Bennetsen was fielding questions from Second Life, so I asked Jamais Cascio two questions that have stuck in my mind since talking with Eben Moglen. The whole of Cascio’s talk will be posted to the web here soon. But, here is a transcription of Cascio’s reply to the questions I asked:

“Is store it yourself fundamental to freedom in the 20th Century?” and, “Who gets the logs?”

The question of who gets the logs is actually does summarize very nicely the fundamental dilemma of this kind of environment. Because who gets the logs is really a proxy for who gets control. Who ultimately has the final say in what we do with the information we create about ourselves. I think that it would be wonderful if we could have complete individual control over that information. That is however unlikely. So rather than rail against the unfairness of it all, and rather than simply lying back and accepting it, the choice here is to really fight for as much as we can, to try to be active, be vocal, be responsive, and push for whatever you can get, as much as you can get, knowing that ultimately the compromises that will emerge. And it will be a compromise as something that will be the best you could get as opposed to something that was just imposed upon you.

Jamais in his post meetup blog post continues to examine the question: “If privacy is effectively unattainable, or the institutions to protect privacy are too weak to withstand the relentless expansion of Internet observation, what recourse would those wishing to maintain some control over their external visibility have available?”

He makes a provocative suggestion:

One possible alternative: intentional misinformation about oneself, reducing the “signal to noise” ratio of networked transparency.

A Fine Balance – privacy and identity portability

On November 21st a workshop was held in Second Life to look at “How can you protect your privacy in virtual worlds?” This workshop was part of a “real” life event: “A Fine Balance 2007 – Privacy enhancing technologies: How to create a trusted information society” held in London. The keynote speaker, Paul Ledak (IBM, USA) and chairman: Jan Camenisch (IBM, Zurich), pointed out that a key area of “fine balance” is “maintaining privacy while enabling identity portability.” Protecting privacy poses some challenges to the goals of interoperability because large parts of the metaverse might end up to be shared reality only to a community of trusted peers.

Organised by three of the UK’s Knowledge Transfer Networks (KTN), and supported by the European Commission, A Fine Balance 2007 is an independent forum for discussing privacy in relation to the development of new technology (see Knowledge Transfer Innovations).

Building on last year’s event of the same name, this year’s conference discusses the development and integration of technologies which can build privacy into new devices and services at the design stage.

Privacy Enhancing Technologies (PETs) will encourage industry to recognize that valuable emerging technologies can be designed with privacy and data security in-mind from the outset. Full details may be found here.

On May 2nd, the European Commission adopted a Communication “Promoting Data Protection by Privacy enhancing Technologies (PETs)” in which it calls for stepping up research in and development of PETs. In this context, the outcomes of this event will be taken under consideration by the European Commission in its formulation of upcoming work programmes for funding calls in this area of the FP7 – ICT programme and will influence the direction of future research in the fields of privacy and technology.

The original European Commission Communication can be found here:

There is an interesting argument by Kightlinger that the different approaches of the US and EU may not be as different as they appear. (The European Union (EU) adopted comprehensive privacy legislation in 1995 and special privacy legislation for electronic communications in 2002.) While the EU emphasis is more on the need for bureaucratic control to “protect” individual privacy the US emphasis more on the need to “protect” individual “freedom.” Kightlinger argues that debates about the relative merits of the two regimes, regardless of differences in detail, strengthen the hold of a the same post enlightenment paradigm where “we find ourselves living in the culture of bureaucratic individualism, and debates about public policy issues predictably become debates about which side of the dyad, individual or bureaucracy, to emphasize at any given time.” And the important point he makes is how very hard it is to see these issues from other viewpoints.