Blair MacIntyre is one of the original pioneers of  augmented reality and an extraordinary amount of creative work is coming out of his Augmented Environments Laboratory at Georgia Tech – see YouTube videos here.  The screenshot below is from, ARhrrrr, a very impressive augmented reality shooter game created at Georgia Tech Augmented Environments Lab and Savannah College of Art and Design, (SCAD- Atlanta), and produced on the NVidia Tegra devkits – watch the demo here.

Blair has spent much of his career working on immersive augmented reality and more recently the integration of augmented reality with mirror worlds. Blair explains:

“I am interested in the intersection of mobile devices – whether they are head mounts or handhelds – and parallel mirror worlds…I think that parallel mirror worlds are a direct manifestation of the intersection of the virtual world we now live in (the web) and geotagging.  As more and more information is tied to place, and as more of our searching become place-based, we will want to do those searches about places we are not at.  A 3D mirror world may provide one interface to that data.  Want to plan your trip to London;  go their virtually and look around, see what is there (both physically and virtually), teleport between areas you want to learn about, and so on.  More interestingly, talk to people who are there now, and retrieve your location-based notes when you are on your trip.”

But, at a time when many augmented reality developers are focusing on AR apps for smart phones, including Blair (the picture on left opening this post is Blair’s augmented reality iphone app ARf), I was interested in finding out from Blair what the state of play was for the real deal Rainbow’s End style AR, as well as the potential he sees in smart phones to mediate meaningful AR experiences.

There is enormous amount of  innovation in mapping our world, see my post, “Location Becomes Oxygen at Where 2.0 and WhereCamp,” and  Ori Inbar’s  Where 2.0. conference roundup. But as Ori notes, to move augmented reality forward:

My point is not a shocker: all we need is to tap into this information and bring it, in context, into people’s field of view.

And this is what Blair MacIntyre’s work is all about.

Talking With Blair MacIntyre

Tish Shute: There do seem to be broader implications to augmented reality today than when this term was first coined. I am interested to have your perspective on how augmented reality may go beyond some of our early definitions?

Blair MacIntyre: I still think the original definition of the term is useful:  media (typically graphics) tightly registered (aligned) with the physical world, in real time.  Many people talk about many things that relate virtual worlds to places, spaces, objects and people.  There is room for many of them, and they don’t all have to “be” augmented reality.  I like using Milgram’s definition of Mixed Reality as everything from the physical world (at one end) to the virtual world at the other;  it’s a spectrum, and augmented reality just sits at one point.

The reason I like the old definition is I believe there is something special about graphics that are tightly, rigidly aligned with the physical world.  When things appear to stick to the world, and an obviously identifiable location, people can start leveraging their natural perceptual, physical and social abilities and interact with the mixed world as they do the physical world.  We’ve found this with the two studies we’ve done of tabletop AR games (Art of Defense and Bragfish);  one key to those games is that the graphics were tightly aligned with identifiable landmarks in the physical world (gameboard).

Art of Defense (pic on left) Bragfish (pic on right)

Tish: I know that you are involved with ISMAR 2009 which is the key US augmented reality conference.  What do you think will be the hot themes, applications, innovations at this year’s conference? Do you think this will be the year that AR really breaks out of eye candy into truly useful and sustained experiences?

Blair: Unfortunately, I won’t be involved this year.  I was supposed to be helping run the technical program, as well as the art/media program, but sickness in my family prevented me from having the time, so I am not helping this year.

First, I would not agree with the implication of the last question — I don’t think AR has just been eye candy up to now.  I do agree that the “high profile” uses of it have largely been that, which is mostly because of the limits of the technology.  I don’t think we’ll see huge changes in that regard by ISMAR this year.  However, we will hopefully see a mixing of communities that hasn’t happened at ISMAR before, and I do believe that this year (independent of ISMAR) we will see more and more AR apps.  Whether they go beyond eye candy is still a question.  I’m hoping that some folks (including myself and other ISMAR folks!) will help push AR in new directions.  But I also expect many folks new to ISMAR and AR to play a big role, because it is this new blood, especially those folks with real problems to solve, new art and game ideas, and a fresh perspective, that will open new doors.

Tish: You have been working on integrating augmented reality with virtual worlds. You mentioned that the way you use Sun’s Wonderland is really about pulling the virtual world into the real world, i.e., Wonderland, “is just a place to put data.”  How is your use of the persistent virtual space different from what we have become accustomed to call virtual worlds?

Blair: The approach we are taking in our project at Georgia Tech is to use the virtual world as the central hub of the information space, and allow the virtual world to be the element that enables distributed workers to collaborate more smoothly.  This is work we are doing with Sun and Steelcase (and the NSF), and is an outgrowth of a project (the InSpace project) that’s been going on for a few years.

What we are trying to do is use mixed reality and ubicomp techniques to pull as much of the physical activity into the virtual world, and then reflect that activity back out to the different participants as best suits their situation.  So, folks in highly instrumented team rooms will collaborate in one way, and their activity will be reflected in the virtual world;  remote participants (e.g., those at home, or in a cafe or hotel) may control their virtual presence in different ways, but the presence of all participants will be reflected back out to the other sides in analogous ways.  We may see ghosts of participants at the interactive displays, or hear their voices in 3D space around us;  everyone will hopefully be able to manipulate content on all displays and tell who is making those changes.

A secondary benefit, I hope, is that by putting the data in the virtual world and making that the place that gives you more powerful and flexible access to the data (e.g., by leveraging space and giving access to history), distributed teams will begin to have the virtual space become a place they go to work, bump into each other and have those casual contacts co-located workers take for granted.

Creating the Information Landscape of the Future

Tish: At the end of my interview with Ori Inbar he said, in order to have a ubiquitous experience “you’ll need to 3d map the world. Google earth like apps are going to help but it is not going to be sufficient. So let’s leverage people. Google became successful in part by making people work with them.  Each time you create a link from your blog to my blog their search engines learn from it.  So let’s find ways to make people create information that can be used for AR.” What ways do you think people can create information that can be used for AR?

Blair: I think the big part of that is the creation of models and environments, the necessary “baseline” for specifying experiences.  Google and Microsoft are clearly working toward this;  recent videos from Microsoft show them starting to move the photosynth work toward Virtual Earth.  Similarly, I came across a page where people are finally starting to mine geotagged Flickr [see my post, “Location Becomes Oxygen,” and here for more on the “The Shape of Alpha” project from Flickr]  images to create models.  It’s that kind of thing that will be useful first;  using the data we all create to enable modeling and (eventually) vision-based tracking in the real world.

After that, it’s a matter of time till more of what we “create” (e.g., Tweets and blog posts and so on) are all geo-referenced;  these will become the information landscape of the future, the kinds of things people think about when they read “Rainbow’s End”.   The big problem will be filtering, searching and sorting.  And, of course, safety and security.

Tish: You are working with Unity3D to research the integration of mobile location based AR with persistent mirror world like spaces.  What has attracted you to Unity? What is the difference between this and your Wonderland project? I know you mentioned. you will be using head-mounted displays are part of this Unity project. What are your goals for this project?

Blair: We started to use Unity3D because it gave us what we wanted in a game engine.  Most importantly, it’s very open and let us trivially expose AR technologies into the editor.  Similarly, it can target the iPhone, so we can begin to work with it on that platform, too.  The biggest problem with creating compelling experiences is content;  and a show stopper for creating content is not getting it into your engine.  Unity has a nice content workflow.

Unity3D is a front end engine, for creating the game;  Wonderland is both a front end, and a backend.  We are actually looking into using the Wonderland backend with Unity as well.  Wonderland also has growing support for doing “real work” in a virtual world, which is key to our other projects.

Eventually, we’ll be using HMD’s.  The goal for the Unity3D project, initially, was to explore what you can do with an AR/VR mirror-world; this is a project are working on with Alcatel-Lucent, and demo’d at CTIA this year.  It’s continuing to grow, though, and now includes a number of our projects, including some work on mobile social AR and soon, some performance and experience design projects in the area of AR ARG’s.  It’s really quite interesting to imagine what you can do when you have an “MMO of the real world” (which we now have for part of campus) that supports both VR-style desktop access simultaneously with mobile AR access.

Tish: Have you taken another look at OpenSim as a possible backend for augmented reality?  Recently I talked to David Levine, IBM and he is thinking about some possibilities to optimize OpenSim to dynamically load a large amount of objects at once (i.e how fast OpenSim can bulk load into an existing sim) and make it better suited to augmented reality/mirror world type projects.

Blair: I haven’t looked at OpenSim recently.  We will probably look at it this summer.

Tish: Why did you select Unity as a good client for augmented reality?

Blair: Unity is a 3D game authoring environment so at some level it is no different from using Ogre, if all the associated stuff was just as well done. It has integrated physics, scripting, debugging, etc. – you can write code in javascript or C# or whatever.   It has a good content pipeline, as well, and supports a range of platforms.

It has simple networking built in, so multiple unity engines can talk to each other but it is not a virtual world platform out of the box – there is no back end …

Tish: Someone described Unity to me as a great client waiting for a great backend? So what are you going to use as a back end?

Blair: There is no real processing except in the client right now.  We will eventually have to create a back end.  We are thinking of using Dark Star because someone on the Sun Wonderland community forums has already built a set of scripts connecting Unity to Darkstar.

But for us, we are not proposing right now to build a real product.  This is research to demonstrate what you could do if you actually had the back end.

Tish: What are the most important aspects of the backend from your POV?

Blair: We want to simulate a variety of the interesting aspects of the back end.  So I very much care about notions of privacy and security and how these sorts of AR/VR Mirror Worlds would work in practice.  But I care about how those things as they impact user experience, not really about how we would really implement them.

Tish: So looking at some of the big problems from the perspective of user experience? Are we are going to go through the same growing pains that the web and VWs have seen, for example, will we have to type in passwords to get into everyone’s little worlds….

Blair: Well you know the SciFi background to this, you’ve mentioned it in other posts on your blog.  Because when you look at the Rainbow’s End model where you have security certificates flying around, that is in effect what cookies and so on are now.  You can authenticate yourself once and then have those certificates hang around. So you can easily imagine how it could be done.  But the big question is how does that change user experience.  There are all kinds of things that start coming into play – like what happens if nearby people see different things – it goes on and on!

Tish: Sounds Like this is very valuable research.  It seems to me that there will be a lot of investment soon in putting the pieces together to do location based markerless AR and it would be nice if we knew more about it from the user experience POV.

Isn’t it vital for a productive intersection between mobile AR and persistent mirror world spaces for us to have markerless AR?  Aren’t we right at the beginning of people really saying yeah markerless AR is doable now? But it seems to me not many people researching or working on fully immersive AR and its integration with mirror worlds?

Blair: I think some of the AR community is thinking about this. There’s probably people who are doing stuff in some other non technical communities. It wouldn’t surprise me to find out that people in the digital performance or ARS electronica world who are thinking a little bit about these sorts of things. Although not necessarily at the level of actually trying to build it, because they probably can’t right now.  But experimenting with the precursors.  My colleagues in digital media like to point out that this is often the purpose of digital art, to point out new directions and push the boundaries.

Obviously Science Fiction has explored the possibilities because that is what Rainbow’s End and the Matrix were all about.

Tish: and Denno Coil…

Blair: There has been some research – people like my adviser Steve Feiner up at Columbia, Mark Billinghurst in New Zealand, myself and people at Graz University in Austria .  But partly it has been so hard to do mobile AR up to now – so many people mock head worn displays and can’t get past current technology – you have had  to be willing to ignore the bulky back packs and cables and batteries and so on.  That is changing which is good.

My current response to the anti-head-mounted display people is if 5 years ago you told me you told me that fabulously dressed people who care about their looks and wear stylish clothes would have had big things hanging from their ears that blink bright blue light, so they could talk on the phone, many of us would have said you were crazy, because it would be ugly and so on.  But because there is an intersection of demonstrable need and benefit…Bluetooth headsets are really useful and the sort of early gestalt feeling that grew up around them – that people who use them are so important that they always have to be in touch, they wear these things – so people accept them.

It will likely be a similar thing with head mounted displays. And I don’t know if it will be that people wearing them so that they can read their mail while driving, god forbid. But it will be something.  And when we get the 2nd generation of the wrap glasses that look more like sun glasses and are not bulky and so on, we will have the potential for them catching on because you will look at them and you will think that the person is wearing because they are doing x…

X might be surfing a virtual world or reading their email or keeping in touch, or being aware. It will happen. But they have to get unbulky enough and there has to be more  than one important application, not just watching TV.

Picture above shows  an outside view of the KARMA AR system;  the knowledge based maintenance system Blair built in his first year of grad school (“first AR system Steve Feiner, Doree Seligmann, and I worked on”).  Blair noted, “The Communications of the ACM paper on it (from 1993) is a pretty widely cited AR paper.”

Tish: I think the need for  full on transparent, immersive, wraparound, Gucci stylish eyewear with a decent field of view are the elephant in the room in terms of realizing the full potential of augmented reality.  There are a few new players in the field Digilens,  Vuzix, others?  What is the progress in this area and what do you hope for in terms of near term solutions?

Blair: I agree with that sentiment.  I think that, in the near term, there is a lot we can do with handhelds, as we’ve been doing in the lab.  However, because it’s awkward and tiring to hold up a device, even a small one, for any length of time, handhelds will only be good for “focused” uses of AR.  Such as the table-top games we’ve been doing, or the constellation viewing app that I heard came our recently for the Android G1.  I don’t even see something like Wikitude as that compelling (beyond the “gee wiz” factor) for a handheld form factor.   Many proposed AR apps only really become compelling when users have constant awareness of them, and that requires a see-through head-worn display.

I’ve seen the mockups of the Vuzix ones;  they seem pretty interesting, and are getting to were early adopters could use them (they will be cheap enough, and will hopefully be good enough).  Microvision’s virtual retinal display is also promising;  the contact lens displays will be the most interesting, if anyone can ever make them work.   I don’t know of anything else out there.

“its not really a killer app you care about, it is the killer existence that all of the technology and small applications taken together facilitate”

Tish: While location based services are accepted now and people are understanding that it is something that opens up a new relationship to everything, we still haven’t found the experience that will get everyone holding up their mobile devices?

Blair: Well that is actually the killer problem.  Gregory Abowd is one of my colleagues who does ubiquitous computing research here at Tech.   Way back when we started the Aware Home project (Aware Home Research Institute at Georgia Tech) when I first got here about ten years ago, there was always this question of what is the killer app.  So Gregory comment in a meeting once that its not really a killer app you care about, it is the killer existence that all of the technology and small applications taken together facilitate. It is not that any one of these AR demos we see right, whether it is seeing your photos in the world or whatever, is important. Its that when taken together, there is enough of a benefit that you would use the whole environment.

In the original context we were talking about an instrumented home, but it is the same thing here with AR.

The problem with the mobile phone as a AR device is that problem of awareness. If I have a head mount on and I walk down the street and there is bunch of probably-not-useful-but-potentially-useful information floating by me, that’s a good thing, because I may see something that is useful or makes me think of something else.  But if I have to hold up my phone to see if something might be interesting nearby, I will never hold up my phone because at the time there is a high probability that there won’t be anything particularly important there.  You might imagine you can get around this by using alerts or something like that, but then you overload whatever alert channel you use.  For example, I forward maybe 5 or 6  people’s updates from Facebook to my phone – started with my wife, a few friends, my brother, and the net result of that is I never get SMSs’ anymore because when my phone buzzes, usually I ignore it because it is probably just somebody’s random Facebook update. So if we start overloading channels like that with “oh there might be something useful here in the real world, if you pick up the phone and look through it you will see it … and I will buzz you.” People  just start ignoring the buzzes.

So it is a very hard problem if you think about the kinds of applications that people always imagine with global AR — names over peoples heads and other random information floating in the world — until you have a head mount and all that information is around you all the time. That is when those sort of applications will actually happen.

Tish: Robert Rice notes: “AR is inherently about who YOU are, WHERE you are, WHAT you are doing, WHAT is around you, etc.” (see my interview with Robert, “Is it ‘OMG Finally’ for Augmented Reality?). And I think the iphone experience has laid the foundation for the increasing desire to experience the network wherever we are – and not be stuck behind a pc.  We cannot perhaps do all we want to do yet. But even in the range of things we can do know, we are not even sure exactly what it is we want to do where yet is it?

“imagine your iphone Facebook client supports AR and that all data on Facebook might be georeferenced – pictures, status updates etc…….”

Blair: Yes that is a huge problem. I have been lucky to be able to teach two fun classes this year that let the students and I start to explore some of the potential that handheld AR might bring.  Last fall I taught a handheld AR game design class — coordinated with a class at the Savanna College of Art and Design’s Atlanta campus — and we had the students build a sequence of prototype handheld AR games, which was a lot of fun.   This spring I taught a mixed reality/augmented reality design class with Jay Bolter (a professor in the School of Literature, Communication, and Culture here at GT).  Jay and I have been teaching this class off and on for about 9 years; this semester we decided to say to the students “imagine your iphone Facebook client supports AR and that all data on Facebook might be georeferenced – pictures, status updates etc…….” and have them do projects aimed at such an environment.

Tish: Not many of our favorite social media today have much sense of location do they? But Flickr  are  utilizing the geo-referenced pictures to create vernacular maps…..The Shape of Alpha

Blair:Yes that is because lots of cameras put geo location data into the exif data so they can extract it…

Some mobile Twitter clients like the one I use in my iphone will let you add your location.  But in general Facebook and other sites don’t have any notion of location. But if you look at all the things people do in Facebook, such as sending gifts and other games, its easy to imagine what these might look like with geo-reference data.  So, the high level project for the class is the groups have to design experiences people might have using mobile AR Facebook.  We told them to assume Facebook as it stands now, but add geolocation and AR to the client.  The class boiled down to “What would you imagine people doing?” So it has been kind of fun.

And we are using Unity for the class too – the same infrastructure I am working on in my research linking mobile AR to persistent immersive mirror world type spaces – and we having the students mock up what a mobile AR Facebook experience would be like.

Tish: Can you describe some of the ideas you class came up with that you think have potential? I know Ori mentioned that from the games class he liked Candy Wars.

Candy Wars

Blair: In the end, they had a nice range of projects in the Spring class.  One created tag clouds out of status messages over spaces, others looked at analogies to virtual pets and gift giving out in the world, one looked at leveraging geolocation to help with crowd-sourced cultural translation, and three groups did straight-up social games.

[See all of the projects from the handheld AR games class on YouTube here]

iphone, Android, or NVidia Tegra devkits or the Texas Instrument’s OMAP3 devkits?

Tish: Is anyone in the class working on Android?

Blair: Nobody is using Android because no-one in the class has the phones. We have ATT microcell infrastructure on campus.  Some ATT people joke that we are better off than them because we have a head office on campus so we can build in the network applications which people even at ATT research can’t do.  But because  we have this infrastructure on campus, and a great relationship with ATT and the other sponsors, we have the ability to provision our own phones without having to pay for long-term contracts, which is vital for research and teaching.

Tish: So does this lock you into the iphone?

Blair: Well the G1 is of course not AT&T but it is GSM so we could probably buy them unlocked and put them on our AT&T network. But the students I work with are much more interested in the iphone right now.

Tish: Is that because the iphone has the market?

Blair: For me the reason I am not interested in the G1 is because you can’t do AR on it – there is Wikitude and a few other apps, but it is all hideously slow.  Worse, because the Java code isn’t compiled like it would be on the desktop, you can’t do computer vision with it, so you can’t do anything particularly interesting on the current commercial G1s.  We could probably take the NVidia Tegra devkits or the Texas Instrument’s OMAP3 devkits (both are chipsets for next gen phones — high end graphics, fast processing),  and install Android on those and we may actually do that yet.  But, it seems like a lot of work right now, for not much benefit.

Augmented Reality shooter game ARrrrr from Georgia Tech and SCAD Atlanta on the NVidia Tegra devkits – watch the demo on YouTube here.

Tish: Everyone seems very excited about the iphone OS 3.0 and the addition of compass. Compass is pretty essential for AR right?

Blair: It is necessary if you can’t do other forms of outdoor tracking, but the problem is that the compass on the G1 isn’t very good, relatively speaking and the iPhone one probably won’t be much better. It does not have very high accuracy, nor is it very fast (compared to, say, the high end 3D orientation sensors we use, from Intersense and MotionNode). As far as I can tell, it doesn’t even give full 3D orientation. I don’t have a G1 (although I have pre-ordered an iPhone 3Gs), but people have told me it only has absolute 2D orientation, so you can only line things up if you are careful.  Your can’t look around arbitrarily…

Tish: You can’t sweep your phone?

Blair: You can look left and right, but if it doesn’t have full 3D orientation, you can’t go up and down. You can’t tilt it in weird directions. It is not fast in the form that you would want to look around quickly.  So it is nice demo.  And it is good for what the Android people use it for which is to let you do your Google street view by looking around, which is actually really useful.

I think there are lots of really useful things you can do with such a compass.

And, it is clear that compass is a necessary feature if we want to do AR.  It’s just not sufficient.

Outdoor Tracking and Markerless AR

Tish: Isn’t it essential for markerless AR? I guess not I just saw this post about SREngine on Augmented Times!

This wasn’t up when we spoke so perhaps you have some comments about what it brings to the table?

Blair: Maybe. The folks at Nokia are working on outdoor tracking, they demoed some stuff at ISMAR last year on the N95 handsets that is all image based.  We are trying to do some work with them, one of my students is working on it.  And probably Microsoft is going to do more on this as well, they had a video up showing that they are also working on vision based techniques.  If you give the phone the equivalent of those panoramic Google Street View images (assuming they are up-to-date) and you are standing at the right place, you don’t really need a compass, you can figure out which way you are looking by looking at the camera video. Ulrich Neumann (USC) did some work on tracking from panorama’s years ago, I don’t know what ever became of it.

Regarding SREngine, that project appears to be a pretty simple first step, but is probably just a demo at this point, and limitations like “only works on static scenes” and “doesn’t work for simple scenes” means it’s probably extracting some simple features out of the image and then matching those to some database.  The trick would be getting this to work on a large scale, where the world changes a lot.   It’s not obvious how to get there.

Tish: So forget RFID for AR…

Blair: RFID is not really useful.

Tish: not at all?

Blair: RFID is useful for telling you what things are near you.  The problem is it doesn’t give you any directional information – it just tells you you’re in range of the tag. So can use it to tell you when you are near a certain product for example.  So it is useful in terms of telling you what thing you are near, and then you can load up a vision system or something else that will recognize that thing.

In that way, it could be useful as a good starting point.

Similarly for computer vision, the compass and the gps are very useful for giving you an initial guess at what you may be looking at that can then speed up the rest of the process.  But, computer vision by itself will not be a complete solution because if I have my panoramic Google Street view (or whatever image database I use for tracking) and you are standing between me and the building -  I am not going to see what I expect to see, I am going to see you.

So I think it is all going to be part of one big package – you are going to see accelerometers, digital compasses, and gps and then combine that with computer vision and other sensors, and then maybe we are going to start getting the things that we have always dreamed about.  I like to show this video from the U. of Cambridge (work done by Gerhard Reitmayr and Tom Drummond) of an outdoor tracking demo because it gives a sense of what will be possible.  Techniques like this will be an ingredient in the future of things.  It becomes especially interesting when you have these highly detailed mirror worlds.  It is sort of one of those chicken and egg problems where if I have an highly detailed model of the world then techniques like they have can be used to track.  But that mirror world needs to be accurate or you can’t use it for tracking, and why would you create the mirror world if you couldn’t track?

Tish: I noticed in your comment to “my interview with Robert Rice” that you said you thought that is was important not to collapse AR into ubicomp – “forgetting what originally inspired us about AR” is, I think if I remember correctly, the suggestion you made. But aren’t ubiquitous computing and AR basically coextensive?

The vision of ubicomp Mike Kuniavsky describes – “sharing data through open APIs and the promise of embedded information processing and networking distributed through the environment” demonstrates how much can be done with very little processing power.” In its most immersive form augmented reality requires a lot of processing power. I think we have all become very conscious about trying minimize levels of consumption.  Can you explain why you think people shouldn’t see AR as the Hummer (energy squandering indulgence) of Ubiquitous Computing?

Blair:  I think there will be a hierarchy of interfaces. You are going to have the rich Rainbow’s End like experience – you are totally submerged in a mixed environment, if you have a head mount on (its not going to be Rainbow’s End for while) but if you don’t have the headmount on that information might be available to you other ways, whether it is a 3D overlay using your handheld or just a 2D mashup with Google maps.  But there will be some circumstances and people who will want to get the compelling experience you can only get with the headmount.

Tish:  Are you doing any research on how all these hierarchies of experiences will fit together – what aspects of this are you looking at?

Blair: The thing that really needs to happen is you need to have this backend architecture that allows you to collect your data from different sources and aggregate it much like the web. Right now Google Earth and Microsoft’s Virtual Earth are much like the old pre-web hyper-text systems that were all centralized. And what we really need is to have the web equivalent where Georgia tech can publish their building models and I.B.M. can publish their building models and their campus models, and your client can aggregate them, as opposed to Microsoft or I.B.M. puts their building models into Google Earth and then somehow you get them out with Google’s google earth browser. That’s just not going to fly.

Tish: so what does it take then to get us to this backend architecture, because I’m in total agreement?

Blair: The nice thing about augmented reality versus virtual reality is that you don’t need everything modeled. You can do interesting AR apps like Wikitude with absolutely no world model.

Tish: So that means we can start with what we have – utilize cloud services without a full blown backend architecture?

Blair: It may very well be that Google Earth and MS Virtual Earth act as a portal because people go and build models and link them with KML, and they can see them in google earth but they can also download the KML’s through some some other channel. So it may be that those things end up being something that feeds some of this along. Then people start seeing a benefit to having these highly accurate models so then you start integrating the Microsoft photosynth stuff and leveraging photographs to generate models.

It’s just keeping up with it and building it in real time is the challenge. A lot of folks think it will be tourist applications where there’s models of times square and models of central park and models of Notre Dame and the big square around that area in paris and along the river and so on, or the models of Italian and Greek history sites – the virtual Rome. As those things start happening and people start building onto the edges, and when Microsoft Photosynth and similar technologies become more pervasive you can start building the models of the world in a semi-automated way from photographs and more structured, intentional drive-by’s and so on. So I think it’ll just sort of happen. And as long there’s a way to have the equivalent of Mosaic for AR, the original open source web browser, that allows you to aggregate all these things. It’s not going to be a Wikitude. It’s not going to be this thing that lets you get a certain kind of data from a specific source, rather it’s the browser that allows you to link through into these data sources.

So it’s that end that interests me. It’s questions like “what is the user experience”, how do we create an interface that allows us to layer all these different kinds of information together such that I can use it for all my things. I imagine that I open up my future iphone and I look through it. The background of the iphone, my screen, is just the camera and it’s always AR.

I want the camera on my phone to always be on, so it’s not just that when I hold it a certain way it switches to camera mode, but literally it’s always in video mode so whenever there’s an AR thing it’s just there in the background.

When we can do that I can have little alerts so when I have my phone open I can look around and see it independent of the buttons and things that I’m tapping and pushing to use the phone. That’ll be a really a different kind of experience.

Of course it is not known yet if the next gen iphone will have an open video API.  And of course, the current camera is pretty low quality, so why would they give it an open API until they put in a better camera?  I am not expecting anything one way or the other until the 3Gs comes out and people start using it.

But there are many things about the iphone 3.0 OS that are hugely important, like the discovery API that allows people to play games with other people nearby, that don’t have much to do with AR.

Tish: You have an iphone AR virtual pet application ARf.

Macrumors wrote it up and suggested that the neg gen iphone will have compass and open video API.  What are your plans for ARf?

Blair: ARf is just a demo right now.  I know what we’d like to do with it, but it would require tons of work;  imagine what it would take to do a multiplayer, social version of Nintendogs?  It’s not clear what we’d really learn by doing that, but there are lots of other game ideas we have that we want to explore.

Tish: I think it was on Twitter where Tim O’Reilly said, “saying everything must have a RFID tag is like saying we can’t recognize each other unless we wear name tags. Look at what’s happening with speech recognition, image recognition et.al. and tell me you really think we need embedded metadata.” What would you say to that?

Blair: I think that whatever extra data is there will be used. So if we put machine readable labels on some objects then they’ll be used if they make the identification and tracking problem easier. But it’s pretty clear that people are already working on tracking and so on.

A lot of these mobile AR apps are clearly putting ideas in people’s minds things that won’t really be doable in the near future. Like being able to look down the aisle of the store and it recognize all of the products. Given the distances and complexity of the scene, the number of pixels devoted to each of those objects, and so on – you just can’t recognize things in that context. But if I’m standing in front of a small set of objects, or looking at one thing, or I’m standing in front of a building, or if I’m in the store and because of the location API — imagine an enhanced location API that can tell me within a few feet where I am, and then combine that with some use of the discovery API that allows the store to tell your device you’re in the toothpaste section. Now you only have to look for different brands of toothpaste. So now you can recognize the big letters “Crest” or whatever. It’s all about constraining the problem.

That’s why I like that particular piece of Drummond’s work, the tracking web site I mentioned above. The general tracking problem of looking around and recognizing objects and tracking is still impossible. But if I know roughly what direction I’m looking in and I have a good estimate of my position, and I have models of what I should be seeing when I look in that direction, then it becomes a tractable problem. And so it’s not that a compass and a GPS are 100% necessary. But if you have them it certainly makes things possible that you wouldn’t otherwise be able to do.

Imagine for example  if there’s a new version of GPS, I just noticed that some of the new satellites going up have this new L5 channel. There’s the L1 & L2 signals  that the military and civilian ones use and they added this civilian L5 signal, which should make GPS more accurate. I haven’t found anything online that says how much more accurate.

But someday, hopefully, all GPS will get to be the quality of survey-grade GPS. Right now, if you get an RTK GPS from one of these companies that make the survey grade GPS systems, they give you position estimates in the range of two centimeters, and update 10 to 20 times a second. When you have that kind of positional accuracy combined with the kind of orientational accuracy you get from the orientation sensors we use in the lab from Intersense and MotionNode, everything is easier because you’ve pretty much got absolute position. You put that into a phone and now when I look up, it’s still not perfectly aligned because there will still be errors (especially in orientation, since the compasses are affected by metal and other magnetic noise). But it does mean if you and I are standing 5 feet apart from each other and look at each other, I can pretty much put a little smiley face above your head. Whereas now, with GPS, if I look at you and we’re 5 feet apart our GPS’s might think we’re on the opposite side of each other because they’re only accurate to two to five meters.

And that depending on the time of day and weather!

Putting RFID tags everywhere is easy; the problem is the readers – they currently require lots of power and they have a limited range.  Sprinkling RFID tags everywhere is fine. But you have to be able to activate those tags and read back the signal.  In certain contexts it works.

Tish: And one final question!  What do you think can be done re beginning to think about standards for AR.  Is there a meaningful discussion going on yet? Thomas Wrobel left this comment on my blog rcently and I was wondering what your position was on some of the ideas he raises?

Wrobel wrote, “The AR has to come to the users, they cant keep needing to download unique bits of software for every bit of content! We need an AR Browsing standard that lets users log into an out of channels (like IRC) and toggle them as layers on their visual view (like Photoshop).Channels need to be public or private, hosted online (making them shared spaces) or offline (private spaces). They need to be able to be both open (chat channel) or closed (city map channel) as needed. Created by anyone anywhere. Really IRC itself provides a great starting point. Most data doesn’t need to be persistent, after all. I look forward too seeing the world though new eyes.I only hope I will be toggling layers rather then alt+tabbing and only seeing one “reality addition” at a time.”

Blair: I agree with him, in principle.  But, I’m not sure there’s a point yet.  It can’t hurt to try, of course, from a research perspective, and I’m interested in the experience such an infrastructure would enable (as we’ve talked about already).

HomeCamp is a home hacking, automation and green technology community that will be gathering in London tomorrow, Saturday 25th April 2009, 10am until 6pm BST (GMT + 1), and in an OpenSim event running alongside for virtual participation, to brainstorm new possibilities for distributed sustainability, creative smart meters, monitoring, graphing and visulaizing energy usage.

More details and videos on the blog. The wiki, which includes signup, is the main portal to all the online activity.

As James Governor notes here:

there has been a huge amount of code and applications released focused purely on using technology for home energy monitoring and automation.  We have an active google group and quite a few videos and content showcasing the various applications and hardware currently being used by geeks to save money and live greener.

Now the challenge is to see how this seedling home energy management movement can really grow into widely adopted distributed sustainability solutions that everyone can use, and participate in.

Both Chris Dalby (see here), Andy Piper, James Governor of Monkchips (see here),  and Tom Raftery of GreenMonk (see here), have posted on tomorrow’s HomeCamp event. So I am just going to add some quick notes, especially to highlight some of what will be going on virtually for those of you, like me, who can’t make it to London.

You can tune in either on the live video ustream, or sign up on ReactionGrid and join the OpenSim event. Also, you can keep up on what is happening on Twitter #homecamp. I highly recommend that you catch Tom Raftery’s talk which will be streamed from Spain live into the London meeting, the OpenSim event on ReactionGrid, and Ustream. Tom Raftery, a leading Green technology analyst at RedMonk (see also GreenMonk), will be picking up, in depth, on some themes raised in his brilliant ETech 2009 presentation, “Electricity 2.0: Applying the Lessons of the Web to Our Energy Networks.”

There will be homecampers dropping in to virtual homecamp in ReactionGrid throughout the day, including Chris Hart (the awesome “girl-geek”@dstrawberrygirl), MiketheBee, and Cminion, who has a number of cool projects to demo, including his energy turbines.  Dave Pentecost (pictured above with his Tweetawatt, Pachube Orb) and I (see our presentation for EarthWeek SL here) plan to be at Virtual Homecamp on ReactionGrid between 9am and 10.30am EST. Dave has done a number of cool energy monitoring hacks including a Pachube link to and from OpenSim.

Also keep your eye on Dave’s blog, The Daily Glyph, for what’s new in distributed sustainability. Dave just posted some great links on Sustainable Interaction, design and work by ITP researchers and others in sustainable use of technology.

Sustainable Interaction | Main / Papers

Sustainable interaction design | Sustainable Minds

Design For the Other 90% | Cooper-Hewitt, National Design Museum

If you are in London, look out for Oliver Goh of Shaspa as Oliver will be at Homecamp in London. As I mentioned in my previous post, Oliver will soon be launching both Shaspa commmunity and enterprise hardware and software packages for “Intelligent Energy Management.”

For a bit of homecamp history, James Governor (picture below from Chinposin), recaps  some of the successes of  the first HomeCamp here.

And last but not least, a big thanks to sponsors, CurrentCost, Greenmonk, Pachube, Onzo, and ReactionGrid,  and media partner theattick.tv who are making the London and virtual homecamp events possible.

]]> https://www.ugotrade.com/2009/04/24/homecamp-2-home-energy-management-and-distributed-sustainability/feed/ 2 https://www.ugotrade.com/2009/02/11/people-meet-people-meet-big-data-sciencesim-explores-collaborative-high-performance-computing/ https://www.ugotrade.com/2009/02/11/people-meet-people-meet-big-data-sciencesim-explores-collaborative-high-performance-computing/#comments Wed, 11 Feb 2009 22:40:02 +0000 http://www.ugotrade.com/?p=2855

Wilfred Pinfold, Director, Extreme Scale Programs for Intel, and the Supercomputing Conference general chair, is working with some Intel colleagues to make a project called ScienceSim the centerpiece of a special workshop event at the SC09 conference (see Supercomputing Conference, an ACM and IEEE Computer society sponsored event).

Recently, I interviewed Wilf Pinfold (see interview below), Mic Bowman (also see my previous interview here), and John A. Hengeveld (see interview below). I wanted to find out what are the underlying goals of this SC conference program?  Why are members of the SC community being encouraged to participate with the ScienceSim environment? What projects are beginning to emerge? And, what are Intel’s goals in giving infrastructure support to further the conversation between high performance computing and collaborative virtual worlds?

The vision of creating new ways to collaborate and interact with big data does seem to be one of the more significant steps we can take at a time when we find many of our most complex systems roiling and threatening total collapse. As Tim O’Reilly has pointed out – from financial markets to the climate, the complex systems we depend on for our survival seem to be reaching their limits.

But,  how can we get from the place we are now – see this example of an n-body simulation in OpenSim, to the point where we can collaboratively steer from our visualizations big data simulations of climate change, financial markets, or the depths of the universe.  The picture opening this post is a:

Frame from a 3D simulation of gravitational waves produced by merging black holes, representing the largest astrophysical calculation ever performed on a NASA supercomputer. The honeycomb structures are the contours of the strong gravitational field near the black holes. Credit: C. Henze, NASA

Wilf Pinfold explained to me part of the reason to begin a dialogue on collaborative visualization at SC ’09 is that super computing communities (that tend to be highly skilled and visionary) have played key roles in internet development in the past. Wilf pointed out,  key browser technology  developed out of these communities in the early days of the internet – see this wikipedia entry that gives  a background on the role of NCSA (National Center for Supercomputer Applications).

The hope is, while there are many obstacles to overcome, the super computing community has both the skills and motivation to find solutions to creating collaborative environments capable of the kind of rapid data movement that scientific/big data visualization needs. Solving the problems of realtime collaborative interaction with big data will  have many ramifications for the way we understand virtual reality, the metaverse, virtual worlds (all these terms are becoming increasingly inadequate for cyberspace in the age of ubiquitous computing, an argument I will make in another post!).

There have already been a number of blogs on ScienceSim (see Virtual World News, New World Notes, Vint Falken, and Daneel Ariantho). There have also been Intel blogs – see this post by John A. Hengeveld (a senior business strategist working with Intel planners and researchers to accelerate the adoption of Immersive Connected Experiences). And Intel CTO Justin Rattner’s post announcing the project this November.

But to blow my own horn a little, I think i was the first to blog the encounter between OpenSim and Supercomputing (an encounter I to some degree provoked by making the introductions) see this post.  So I have been following the ScienceSim initiative with great interest.

Very shortly after N-Body astrophysicicsts Piet Hut and Jun Makino, creators of  – GRAPE (an acronym for “gravity pipeline” and an intended pun on the Apple line of computers) – a super computer that will become one of the fastest super computers in the world (again), met Genkii – a Tokyo based strategic company working with OpenSim, the first N-body simulation appeared in OpenSim.  And in a matter of weeks  this video went up on YouTube – the result of a collaboration between MICA and Genkii.  But the nirvana of being able to create visualizations using real time data from super computers that can be steered from a collaborative environment is still a ways off.

Super computing communities tend to be geographically very dispersed and researchers often find themselves far from simulation facilities so there is both the motivation and skills to pioneer new tools for collaborative visualization. I know that astrophysicists certainly see their value (Piet Hut has some profound ideas on this). Astrophysicist Piet Hut and others  (see here for more) have been pioneering the use of VWs for collaboration.  There are two Virtual World organizations, both founded by Piet Hut and collaborators, that are currently exploring the use of OpenSim for scientific visualizations.  One is specifically aimed at astrophysics, MICA, the Meta Institute for Computational Astrophysics, and the other is aimed broadly at interdisciplinary collaborations in and beyond science, Kira, a 12-year old organization focused on `science in context’.  As of last week, there are two weekly workshops sponsored jointly by Kira and MICA that explore the use of OpenSim, ScienceSim, and other virtual worlds.  One of them is “Stellar Dynamics in a Virtual Universe Workshop” and the other is “ReLaM: Relocatable Laboratories in the Metaverse.”

MICA was founded two years ago by Piet Hut within the virtual world of Qwaq Forums (see the paper “Virtual Laboratories and Virtual Worlds”). The Kira Institute is much older: it was founded in 1997.  Later this month, on February 24, Kira will celebrate its 12th anniversary with a presentation of talks, a panel discussion, and a series of workshops.  See the Kira Calendar for the main event, and the Kira Japan branch for a special mixed RL/SL event in Tokyo.  During both events, Junichi Ushiba will give a talk about his research in which he let paralyzed patients steer avatars using only brain waves.

Other early adopters of ScienceSim include Tom Murphy, who teaches computer science at a Contra Costa College. Prior to teaching, Tom spent 35+ years working for supercomputer manufacturers. Tom said:

it is very natural for me to find significantly new ways to visualize and interact with scientific mathematical models via ScienceSim and the OpenSim software behind it. ScienceSim also allows us to interact with each other and teach students in new ways.

Also Charlie Peck, chair of the SC09 Education Program, (his day job is teaching computer science at Earlham College in Richmond, IN), is working with Wilf Pinfold, Tom Murphy and others “to explore how 3D Internet/metaverse technology can be used to support science education and outreach.”

Cristina Videira Lopes, University of Irvine, is doing very interesting work  on road and pedestrian traffic simulations. Crista is also the creator of hypergrid in OpenSim,

People Meet People Meet Data: A Conversation With Mic Bowman

Screenshot of ScienceSim from Daneel Ariantho

Tish: How does this work on ScienceSim  fit into a wider dialogue on linked data? Where people meet people meet data, and where data meets data?

Mic: Yeah… that’s hard by the way.  Open integration of data (and more interestingly the functions on data) is very hard if it comes from multiple, independent sources.

That’s the people part. For example, if Crista can build a model of the UCI campus somebody else builds an accurate model of several cars and another expert provides the simulation that computes the pollution generated by those cars in that environment…its bringing people together to solve real problems, no matter how far apart physically.

Tish: You mention three different simulations here. Could you explain why it is difficult to integrate data from multiple sources?

Mic: integrating data from multiple sources has always been one of understanding & interpreting both the syntax & semantics of the data. Even relatively simple things like multiple date formats require explicit translation. More complex formats, like the many formats data is represented for urban planning, are barely computable independently let alone in conjunction with data from other sources (each with its own representation for data). Its often the expertise & the collaboration of bringing people (and their bag of tools) together that solves these problems.

Tish: and in this case the bag of tools is high performance modeling..?

Mic: high performance modeling, rich visualizations and data. Its the three that matter… data, function, and interface.

Tish: Some people have a very hard time wrapping their head aropund the fact that anything that seems related to Second Life can do this.  Can you explain more about the difference between SL and OpenSim?

Mic: OpenSim potentially improves data & function because it can be extended through region modules. Region modules hook directly into the simulator to provide additional functionality. For example, a region module could be implemented to drive the behavior of objects in a virtual world according based on a protein folding model.

We need to work on additional viewer capabilities to address the user interface limitations.

Tish: Yes Rob Smart’s (IBM) recent data integrations with OpenSim (see here) are impressive. Re viewers one of the biggest objections to virtual worlds is the mouse pushing and pc tied interface.

Mic: There are great opportunities for improving the interface

Tish: Yes I really like where the Andy Piper’s experiments with Haptic Interfaces for OpenSim lead, see Haptic Fantastic! And I think that we will have cyberspace ubiquitous in our environment, not just stuck on a pc screen, sooner than we think.

Mic: Mic’s opinion (not Intel): until we get souped up sunglasses with HD screens embedded (or writing directly into the eye) there will always be a role for the PC/Console/TV).  But, it isn’t about the device… its about the services projected through the device… sometimes you’ll want a very rich experience… sometimes you’ll want an experience NOW wherever you are.

Tish: I think people are only just realizing that VWs will be a now and wherever you are experience very soon.

Mic: That’s the critical observation the virtual world is not an application you run… its a “place”… and you interact with it where you are or maybe interact through it. Speaking for Intel… it is the spectrum of experiences that are critical to support.

Interview with Wilfred Pinfold

Picture from National Science Foundation – “Climate Computer Modeling Heats Up.”

Tish Shute: I know your day job for Intel is in High Performance computing. Could you explain to me a little bit more about what you are working on in this regard – a mini state of play for high performance computing from your perspective?

Wilfred Pinfold: My title is Director, Extreme Scale Programs. This program drives a research agenda that will put in place the technologies required to make an Exa (10^18) scale systems by 2015. The current generation of high performance computers are Peta (10^15) scale so this is a 1000x increase in performance and this increase will require significant improvements in power efficiency, reliability, scalability and new techniques for dealing with locality and parallelism.

Tish: The nirvana in terms of linking supercomputers to the collaborative spaces of immersive virtual worlds is to be able to create visualizations using real time data from super computers in collaborative VW environments, and ultimately for researchers to be able to collaborate and steer their simulations from their visualizations.  Where are we at now in terms of scientific data visualization in VWs? And what are the current obstacles to using realtime data from super computers?

Wilf: Being able to steer a simulation from a visualization requires both a visualization interface that allows interaction and a simulation that operates at a speed that is responsive in interactive timeframes. For example a weather model that predicts the path of a hurricane would need to operate at something close to 1000x real time. This would run through a day in ~1.5 minutes allowing an operator to run the simulation over several days multiple times with different parameters in a single sitting to understand the likelyhood of certain outcomes?

Tish: Do you see a networked online collaborative virtual world being capable of being a visualization interface that allows meaningful interaction with the hurricane scenario you describe in the near future (next 6 to 18 months)?

Wilf: I was using the hurricane example to explain the usage model not an imminent capability. Hurricane Simulation: Accurate hurricane simulations require multiscale models able to resolve the global forces working on the storm as well as the microforces that define precipitation. We can build useful weather models today that run faster than real time (anything slower is not useful for prediction) but we are a long way from the ideal.
Visualization: There are excellent visualizations of weather systems but I have not yet seen a virtual world that can track a simulation and allow the scientist or team of scientists to see what is going on at both the macro scale and zoom in to see precipitation conditions. Today’s supercomputers are much better at this than they were a few years ago but they are a long way from ideal.

Tish: Open Source Virtual World technologies are pretty diverse in their approaches, Croquet, Sun’s Wonderland and OpenSim are quite different and have different strengths and weaknesses. As you have become more familiar with OpenSim, what have you found about the technology that particularly lends itself to this project – ScienceSim (Mic mentioned Crista’s hypergrid code for example, modularity is another feature often cited).

Wilf: We have found OpenSim’s client server model is well suited to the visualization model and the ability to put the server next to the supercomputer producing the visualization data is critical. We are however very interested in other environments and encourage papers, demonstrations and research on any of these platforms at the conference.

Interview with John A. Hengeveld

Tish Shute: OpenSim’s dependence on Second Life based viewers is sometimes cited as a limitation, and sometimes as a strength. What are your views on this?  What would a strong open viewer project directed at science applications bring to the picture?

John Hengeveld: There may be more than one strong open viewer project required for opensim compatible experiences.  The strength of the Hippo viewer, for example, is availability and its weakness is the size of the client.  We would love a ubiquitous, client.. that runs on all platforms, but each hardware platform brings tradeoff and restrictions of its own.  Today, probably all of the folks innovating in the space can deal with the size of a very fat rich client ap.. they have big computers anyway.  But as we get into more 3D entertainment and augmented reality applications.. virtual mall, collaboration apps.. etc… there is a great deal of room to optimize for the specific experience.  Balancing visual experience with bandwidth and compute performance available .. tying into standard browsers, etc… people have done some of this work.. and I think all of it adds to the usefulness of these worlds.

Tish: Integrating highend game engines and OpenSim opens up new possibilities. But licensing issues have been an obstacle. Could a project like ScienceSim get a non-commercial license on a high end game engine?  What would that bring to the picture?

John: Anything is possible. Game engines can give a great deal of design power for high value experiences, but the programming of these experiences must be simplified.  Mainstream adoption in enterprise can’t be premised on the programming model of studio games… that’s a big step to get over I think.  There are very interesting possibilities when we take that step tho.  Simulation, training, agents of various types (I just finished watching “The Matrix” for like the billionth time… I think agents are cool…)

Tish: Where does Larabee fit into the picture of ScienceSim and next generation virtual worlds?

John: We are all very excited about the Larrabee architecture and its application to work loads like next generation virtual worlds, both in the client.. delivering immersive reality.. and someday potentially in a distributed architecture simulating and producing these worlds.  For Intel CVC is an all play.  Atom will be used in strong mobile clients.  Core will be used in Enterprise PCs, Laptops and Desktops   Xeon will be simulating these environments and handling the data communication, and Whatever we brand Larrabee… will be enabling compelling visual experiences. Oh.. and our software products (Havoc, tools and others) will be building blocks in knitting all this together.  Larrabee is a part, but there are a lot of other pieces in our vision…

Tish: If the kind of rapid data movement that scientific visualization needs is achieved in virtual worlds, this will be quite a game changer for business applications of VWs too. Also it will blurr the boundaries between what we call virtual worlds and mirror worlds. It seems to me this kind of rapid data movement is a vital step towards what Mic described to me as Intel’s vision of CVC: “Connected Visual Computing is the union of three application domains: mmog, metaverse, and paraverse (or augmented reality).” It almost seems to me that if you achieve your goals for ScienceSim you will change how we think about virtual worlds in general? What do you think?

John: I certainly hope so..  Part of our goal is to stimulate innovation in the technology and usage models that will enable broad mainstream adoption of CVC based applications (what we categorize as immersive connected experiences).   By tackling the scientific visualization problem, we hope to find the key technology barriers and encourage the ecosystem to solve them.

Tish: To me virtual worlds and augmented reality should be complimentary and connected experiences. How do you see this connection evolving?

John: We certainly see them as related.  In the long term, there are many common building blocks.. but they aren’t united per se.  Its about the user experience, and in some usages these two are almost identical…  in some.. they don’t look or feel at all alike… the viewer is distinct by a lot.  Our approach is to enable building blocks that people can quickly build out usages that are robust.

Tish: What is Intel’s vision for ubiquitous mobile computing and an internet of objects?  How can high performance computing be an enabler for this vision?

John: Mobile computing is a central part of our life, culture and community in economically enabled economies.  It feeds the data of our decisions, it connects us to entertainment, it is the access point to our soapboxes, pulpits, economy and families.  This creates a massive increase in data, a massive increase in interactions, transactions and visualizations.  While many HPC applications will be behind the scenes (finance, health, energy, visual analytics and others), HPC will emerge as a part of a scale solution to serving some of this increase… particularly that part where interactions and visualizations are complex or compelling.. or where scale enables the usage per se .. I talked about my love of agents earlier.. and some of that comes in here.  Compute working behind the scenes to help managed the data complexity, manage some of the base interactions between ourselves and technology.  The other thing we talk internally about the “Hannah Montana usage” where millions of people use their mobile devices to access and participate (using the sensors in the device) with an interactive live concert.  When Mylie hears the applause of a virtual interactive audience… and can scream back at them.. we’re there.  Access to ubiquitous compute will be mobile, and interactive experiences will be complex.. and HPC can help make that real.  Watch out for the mental trap that HPC is always high end super compute clusters tho… the “mainstream HPC”.. smaller clusters… high threads, etc… will play a key part in all of this as well.

Interesting that John ended on this point as this just came in from Wired.

Google Street View Hacking

This Google Street View Hack (via @timoreilly) will get my nomination for a Hacking the World Award this year, if there is such an award.

A parade (the screenshot opening this post), a marathon, a mad-scientists laboratory, a sword fight, and more (see The Infonaut Blog) were staged all along the route of the Google Street View truck by artists Robin Hewlett and Ben Kinsley working in conjunction with the local community and Google Street View.

The Google Street View Hack suggests at a myriad of possibilities for anyone with their eye on the prize for a great world hack for 2009.  In my mind’s eye, I imagine the Google Street View truck’s trek across the planet triggering local environmental street action carnivals wherever it goes.

Local energy conservationists, “passive house” architects, retrofitters, could turn the arrival of  Google Street View into an occasion to create projects for a sustainable future – a traveling StreetCamp (see my post on HomeCamp ’08 here).  As Google Street View intends, surely, to go everywhere,  this would be a global hack for sustainable living that crossed the bounds of the physical and the virtual.  And the vast public record of Google Street View would became a generative engine and global resource for sustainable living.

Working together on the noble aim of sustainable living

- this is my (and many other people’s) big theme for 2009.

A Hacking the World award should also go to  Pachube – “patching the planet” – for demonstrating that instrumenting the world is not merely a Sci Fi  fantasy anymore.  By facilitating “interaction between remote environments, both physical and virtual,”  Pachube demonstrates (see diagram here) how we have only just begun to dip our toes into the many new opportunities we have to work together to save energy, rethink our culture of consumption, and to reboot our failing economy under a new sustainable operating system.

Energy awareness unlike the glut of information we have in entertainment and games suffers from a dearth of information. We really have very little idea about what we are consuming and the waste we are producing.  So more Hacking the World Awards should go to projects like AMEE – creating the world’s energy meter, and Wattzon – your personal energy meter, for giving us new ways to understand and work with energy data.

Many people and organizations, given the information, will change their behaviours. But the cultural changes necessary for sustainable living are deep and old habits die hard (see this disturbing report on the recent return to SUV buying in November as soon as gas prices fell!).

A  Small Community of Volunteers Can Bring Change on a Global Scale

Picture above by Benjamin Ellis, “HomeCamp – The Throng,” from his Flickr stream.

One of my favorite “instrumenting the world” projects to date and another top contender for a Hacking the World Award is HomeCamp ‘08 (see my previous post).  HomeCamp brings together a community of creators and enthusiasts of  “smart stuff,” creating a wikiculture for the noble cause of sustainable living.

The key to whether “instrumenting the world” empowers people and changes our lives for the better will be the capacity our systems of instrumentation have for what Jonathan Zittrain in “The Future of the Internet: And How To Stop It:,” defines as generativity, i.e.:  “the system’s capacity to produce unanticipated change through unfiltered contributions from broad and varied audiences” ( Zittrain, 2008).

Generativity is the “secret sauce” that makes the difference between, for example, Wikipedia and its all but forgotten predecessor – the “written by experts” Nupedia.

Jonathan Zittrain writes:

Wikipedia stands for more than the ability of people to craft their own knowledge and culture.  It stands for the idea that people of diverse backgrounds can work together on a common project with, whatever its other weaknesses, a noble aim - bringing such knowledge to the world. (p.147)

At Web 2.0 Summit, Jonathan Hochman (Known as Jehochman on Wikipedia), shared with me his insider perspective as a Wikipedia administrator. The full interview with Jonathan is later in this post.

Jonathan comments on the role of wikiculture in sustainable living:

“Sustainable Living requires everything to become more efficient. Incentives need to line up with conservation priorities. This requires a radical change to the way we govern ourselves. Command economies, whether commanded by politicians or capital, lead to huge inefficiencies.”

And surely, if we have learned anything in 2008, we have learned that very bad things happen when the complex systems of modern life are left in the hands of a few people motivated solely by the urge to make profit.

Hacking Design and Planning Processes for Real Estate and Transportation with Virtual Worlds

This great machinima by Azwaldo Vilotta shows the progress so far on the Wikitecture 4.0 project, ‘Re-Inventing the Virtual Classroom’ for the University of Alabama.

Though still a niche market Virtual Worlds are growing at a steady pace.  As I mentioned in my previous post, energy hungry avatars themselves will be a target for optimization in 2009.  But as my personal power usage breakdown from Wattzon shows, cutting down the amount of flying I do in 2009 would be far more effective in reducing my carbon footprint than deciding not to log into Virtual Worlds!

Note: Read Write Web’s recent post, “Report Enterprise Virtual Worlds More Effective Than Web Conferencing.  Also check out Web.Alive, and Immersive WorkSpaces and Dusan Writer’s post on “ThinkBalm,The Immersive Internet and Collaborative culture,”

My friend Melanie Swan points out in her Top Ten Computing Trends for 2009, that Virtual Worlds not only have the power of the 3 Cs (communication, collaboration and commerce) but they are fast expanding into rapid prototyping, simulation and data visualization.

My Hacking the World, 2008, Awards for Virtual World innovation would go to three potentially world changing projects for sustainable living:

1) Studio Wikitecture, (see “Reinventing the Virtual Classroom” for The University of Alabama).

2) Oliver Goh’s work on “The Path to Sustainable Real Estate.”

3) Encitra, a company recently co-founded by Crista Lopes and Christer Lindstrom focused on improving urban planning processes, starting with transportation, using virtual worlds (see my previous post here for more).

The latter two projects are being developed in OpenSim – the open source project that should also get a Hacking The World Award for creating an open modular architecture for virtual worlds that is unleashing all these new possibilites for integrating physical and virtual worlds.

The 2008 code contributions to OpenSim of special note re world hacking are Crista Lopes’ OpenSim Hypergrid – see Justin CC’s blog for full details on “What is the hypergrid?,” and David Levine’s work (IBM), in collaboration with Linden Lab (see Architecture Working Group), on interoperability (see my earlier post here).

Both these projects expand the frontiers of interoperability for virtual worlds although they “slice the problem from different ends,” as David Levine put it.  The emphasis in the LL/IBM approach is on security so assets are not moving yet.  In Crista’s solution you can have assets but the security issues are not addressed yet. But this work is vital to expanding the usefulness of virtual worlds and both projects should get Hacking the World Awards IMHO.

I asked Jon Brouchoud (full interview upcoming) what he thought were Studio Wikitecture’s most important successes to date:

“I think the greatest success has been in proving, on some level, that everyone has important knowledge that can inform and improve the design of a building, not just architects.  If we can continue building on that success, I hope we can eventually start to hack the traditional design process, and find ways to harness the wealth of knowledge held by the general public, instead of ignoring or avoiding it, as is most often the case.”

Harnessing the “Smart Stuff” to the Noble Cause of Sustainable Living

Robert Scoble’s, The Interview of the Year: Tim O’Reilly, is not to be missed. Tim O’Reilly discusses the key trends for 2009 that are bubbling up at O’Reilly Media.  And, Yes, Tim O’Reilly, as the guru of Hacking the World, gets the “Distinguished Thinker – Hacking The World Award of 2008!”

Tim O’Reilly’s trend list includes:

1) big data- vast peer produced data bases in the cloud accessible by mobile devices

2) “smart stuff” – sensors and robotics and hacking on stuff for fun and not for profit

3) Green Tech

4) Advances in Biological/Life Sciences.

And, in Robert Scoble’s interview, there is a nice titbit of history re his attendance of early Foo Camps.  Foo Camp is the wiki of O’Reilly conferences and a lineage holder to my favorite Hacking the World event of 2008, HomeCamp ‘08.

But what will be the “secret sauce” for these big ideas  – the generative engines that harness to the noble cause of sustainable living these vast peer produced data bases and all the creative “smart stuff” hackers across the globe are creating?  What will motivate the mass adoption of Green Tech and sustainable living?

What can Wikipedia teach us about how generative systems and bottom up approaches can change the world?

Jimmy Wales (interview coming soon!)  writes in his recent personal appeal for support for Wikipedia.

At its core, Wikipedia is driven by a global community of more than 150,000 volunteers – all dedicated to sharing knowledge freely. Over almost eight years, these volunteers have contributed more than 11 million articles in 265 languages. More than 275 million people come to our website every month to access information, free of charge and free of advertising.

To answer questions on a how to create a successful wikiculture for sustainable living, an insider’s view of Wikipedia may be a good place to start.

Interview With Jonathan Hochman on Wikipedia.

The picture on the left is from the Wikipedia article, Gamma-ray Burst, that Jonathan Hochman is currently working on.  It is a drawing of a massive star collapsing to form a black hole. Energy released as jets along the axis of rotation forms a gamma-ray burst. Credit: Nicolle Rager Fuller/NSF

The picture on the right, Jonathan at Web 2.0 Summit, is taken by me. Jonathan was part of the, Defending Web 2.0 from Virtual Blight, panel.

Known as Jehochman on Wikipedia, he serves as an administrator and as a leader in addressing online harassment, disruption and sock puppetry. He is also the founder of Hochman Consultants, an Internet marketing consultancy, and the director of Search Engine Marketing New England, a regional conference series.

Tish: Second Life and Wikipedia are the two great experiments in collaborative co-creation what do they have to teach us about the future of the internet?

Jonathan: Yes, Wikipedia and Second Life are key social spaces.  Some people have been seeing Second Life as the beginning of Web 3.0 – a wrap around environment where you can almost experience another life. Wikipedia is sort of another example of this.

All the problems that exist in the real world are mirrored right into that little universe.  For example, the Armenians and the Turks are at each others throats and the Japanese and the Koreans are going at it, the Palestinians and the Israelis, and the “Troubles”  … all the conflicts are imported into Wikipedia.  People are fighting over the content of these articles. They want to have it their way because these are first ranked in Google and they have a big impact in public opinion.

There was a huge fight on the waterboarding article a while back. Some guys from Little Green Footballs – they are a very conservative reactionary type of media. They are trying to change the article to say that water boarding might not be torture – change it to say it is probably not so bad.  Crazy stuff. They were trying to water it down.  And it is very clear, from every source out there, that waterboarding is torture.  We did a study and there are 115 sources that say waterboarding is torture. You simulate drowning – you simulate killing someone – that is a violation of the Geneva Convention and everything else. People were fighting, fighting, fighting!

One of the things I did was to try and clear people out who were being disruptive.  We actually had to go to arbitration over that article. It is like the supreme court of Wikipedia. There is a panel of 15 arbitrators.  They hear the case. There is evidence, arguments and decisions. It is really like a simulated law suit. You get all the experience of a simulated law suit with the real threat that you could be banned. If they don’t like what you are doing they can actually ban you or restrict you from topics.

So it is really fascinating how this social space Wikipedia becomes a very real platform though it is in a virtual world for real world disputes.  Most disputes are over the definition of things.  If you have a you suit most disputes are about how things are defined. And Wikipedia has become the defacto definition of things in the real world.  People want to know what are “The Troubles.”  If you go to Wikipedia you find out  The Troubles are a dispute over Northern Ireland.  What the article says has a profound impact on public opinion.

Tish: So who is on the court of Wikipedia?

Jonathan: They are volunteers. these people work two or three hours a day to run this court.  There are all kinds of projects.  There is a WikiProject Spam which has people who can write computer programs to statistically analyze Wikipedia projects – not only Wikipedia. But all of them are looking at the links and reporting them and banning those people who are abusing or gaming the system.

Tish: You were on the Stopping Virtual Blight Panel at Web 2.0 Summit – what are the most important things to think about on this topic?

Jonathan: Yes we were talking about how to defend the web against virtual blight. The thing I find interesting about Wikipedia is that because it is the eighth largest web site and possibly the second largest web site comprised of user generated content after YouTube. The problems that exist in Wikipedia are larger and more detailed than any other site.  For whatever problem someone has for their social media site or their Web 2.0 site these problems already exist in Wikipedia and the solutions are there and they are transparent. You can actually see the history of what’s been done.

If there is, for example, a problem on Digg – some problem with sock puppetry or vote stacking – it happens, it goes away.  You don’t get full disclosure.  With Wikipedia you can actually go in and look at a dispute and watch it unfold.  You can watch the arbitration cases that are filed, the arguments, the decisions, the logic, the rationale.  You can see the successes and the failures and the different things people have tried to control blight. For example, we tried to resolve this dispute one way but it was a disaster, so we have tried something else and that worked.

Wikipedia is a large laboratory for social media. Wikipedia and the large universe around it Wiki and WikiMedia projects that individuals, enterprises and put together like Commons.  Wikimedia Commons is a repository of publicly licensed images that anyone can take and reuse. They have sound and they have video, and all of this stuff is being stitched together now.

So if you go to the article on Obama  you can probably now hear his acceptance speech because that is public domain – its been stitched into the article.  If you go to the article on Richard Nixon – his resignation speech – you may even hear his conversation with the astronauts when they landed on the moon.  So this becomes a giant repository of all our culture and knowledge.  When I design a website, a lot of times I go to Commons to find images I use for free.  I don’t want to pay for an image I can get for free. 

Tish: And the Commons images get contextualized in Wikipedia too.

Jonathan: Some of these articles are fascinatingly detailed. If you want a quick summary of the Dr. Strangelove, the article is fantastic.  It is enjoyable, a pleasure to read.  I was reading about S.A. Andree’s North Pole balloon expedition of 1897. Some guys from Sweden decided to fly a balloon over the North pole.  They managed to get aloft then they flew over the icepack for 24 hrs then they crashed.

They unloaded their stuff and hiked back across the ice toward the island they had launched from. They ended up being on the ice pack for three months before they finally holed up in an ice cave and starved to death.  There weren’t found until thirty years later!  There was a camera with these guys and the frozen pictures taken 30 yrs earlier.  They developed the film and those pictures are now on Wikipedia.  It is just a fascinating thing!

Tish: Do you see real time collaboration beginning to play more of a role in Wikipedia – whether virtual worlds or just voice/IM — how could real time collaboration change the wikipedia editing process?

Jonathan:  The Presidential candidate articles were being edited very rapidly yesterday. There are certain real time problems.  Some of the more interesting problems are when you get two administrators who “get into it.” One administrator says I am blocking this user and the other one says I am unblocking him, and the other one “NO I am blocking him!” And so on…… And everyone says, “Stop fighting. You are not allowed to do that!” And they both get their powers stripped. People do get very heated over the silliest things. Wikipedia does have some mailing lists attached and there are some IRC channels. So there are some real time elements.

Tish: What is the role of avatars in Wikipedia?

Jonathan: In Wikipedia you have a user page and many users are anonymous.  They create an avatar and they personalize it and show themselves in ways they want to show themselves through an avatar. In many ways it is a lot like Second Life.

Some users have created second accounts – or a humerous second account. Bishzilla – a Swedish lady who is in tremendous command of the English language and has a razor sharp wit.  She has created this secondary account – almost like in a baby language.  Her avatar is a dinosaur that is not very bright that goes around frying people. Bizarre what people do! People may be editing a topic like an interest they have – e.g. Pokemon that they don’t want associated with their professional avatar. Or people may be editing a topic about hot political issues.  There have actually been some death threats issued to people over stuff they have been putting into the encyclopedia.

Tish: So avatars are important in Wikipedia.

Jonathan: Absolutely because people may be going in and editing articles that they may not want their friends and family to know they are editing.  One editor may say to another, “Stop putting stuff in or I will come and kill you!” Well then we have to ban them.  We have to call the police.

Tish: Can you build reputations on multiple avatars?

Jonathan: You are allowed to use multiple avatars as long as they don’t cross paths.  You can’t have two avatars editing in the same area beacuse you are going to be giving yourself double weight commenting on a discussion.


Tish: How do you know when this is happening?

Jonathan: You can watch the style of a users editing.  You have to watch behavior.  And if you have enough evidence through behavior that suggests accounts are controlled by one person you can go and request a technical check.

There are some uses who are called Checkusers who are able to access information desired from the server logs and check the technical characteristics of these accounts to see if they are using the same IP address.

Tish: So if you want to understand avatar interaction on the web it helps to understand Wikipedia.

Jonathan: Yes it is a fantastic way to understand how avatars work in some aspects, and also how to deal with community dynamics.  We have some very strong willed people – people in their 40s, 50s, and 60s – who are very successful in business.  They have plenty of money and spare time and they are doing this as a hobby. And some of these people can really butt heads.  You can have a problem when you have an editor who has been writing fantastic articles but also happens to be rude and chew other people out and tell them to f**k off if they are not behaving. What do you do?

Tish: Sounds a bit like Second Life!

Jonathan: The person is a great contributor to the community but they are telling noobies to f**k off, so you can’t allow that.

What do you do?  Vested contributors are a major problem to some of these sites. They are vested in the community but they start misbehaving. You can’t block them, because if you block them there is a huge upsroar from all their friends and it causes a cataclysm.  It requires very careful diplomacy to deal with some of these situations.

Tish: How many Wikipedia volunteers are there now?

Jonathan: Think of a Venn Diagram – a big circle. The total number of contributors are about one million different people that contribute.  But there are probably about 5,000 active editors  that are consistently and regularly contributing.  And within that kernel there are fifteen hundred people that have administrator access and probably only eight hundred of them are active.  People have a natural life span with the community.  People come an typically stay for 6 months to 3 years.  Usually after that they become bored, disillusioned or get into a conflict with someone.  There is a natural tendency for people to stay for a while and move on. Some people stay longer, a few, but the majority will move on at some point.  So it is a lot of fresh faces moving in.

Tish: What lessons of trust does Wkipedia have to teach us about new projects like AMEE that aims to aggregate the world’s energy data?

Jonathan: Well you have to know who is releasing the data. Who is creating the data? The beauty of Wikipedia is that you have an edit history so you can see exactly who has done what.  So you can judge whether this person is trustworthy or not.  That’s a huge problem on the web today.  We don’t have enough identification information.  When you see a web page you don’t necessarily know when that page was created and by whom, or how many revisions it has had.  Sometimes you can glean information by checking it.  If you see typos and errors you may decide that that page probably didn’t receive as much attention as it should have, and probably it is not that good.

Typos are an interesting thing.  People always try to figure out how Google ranks web pages. Matt Cutts was here from Google today.  And he was talking about spam.  But Matt also did a blog post about how he was in an airport once, and how he has a policy – when you are reading a document as soon as you come to the first error just stop because if the author hasn’t taken the care to make everything correct, you don’t need to read it. So he was in the airport, there was a sign, he came to a typo and stopped reading it. Somehow he got in trouble for not reading the sign and not having the information.  But it is interesting to think whether Goggle is looking for for typos, misspellings, broken links and using that as a signal of quality to rank pages.

Tish: Aaaagh typos might bring down your page rank!!!  That certainly is a scary thought for a blogger like me who likes to write impossibly long posts that are hard to check………

I have just got back from attending two conferences in the UK, the Head Conference, and Virtual Worlds London.  I was on a mission at both the events to ask questions about how Virtual World technology will answer the call Tim O’Reilly made at the Web 2.0 Expo in New York City to “create more value than you extract” and do something worthy and useful with the internet.

The Head Conference was an ambitious, timely, and much needed creative exploration of the potential for “green” conferencing using Adobe Connect Pro, Second Life and  local conference hubs in various cities. For more on the conference organization see this pre-conference interview with Aral Balkan.

Head will be the focus of my next post, so more on Head soon!  One of my main goals in attending the London Hub of Head was to interview the CEO and founder of AMEE, “Avoiding Mass Extinctions Engine,” Gavin Starks. AMEE aims to be “the energy meter of the world.”

AMEE is a neutral aggregation platform designed to measure and track all the energy data on Earth.”

AMEE is a project with the kind of big goals that O’Reilly talked about in his keynote at Web 2.0 Expo, NYC.  Tim O’Reilly is an investor in AMEE. He announced, at Head, that the O’Reilly VC company has just closed a deal with AMEE.

I had an extraordinary opportunity to spend time some time talking with Tim O’Reilly while looking for a sandwich in Euston Square.  More on this sandwich adventure and my interview with Tim O’Reilly, and my long talk with Gavin Starks about AMEE, in my next post!

Tim kept saying in London that he doesn’t like predicting the future. But the future comes to Tim O’Reilly!

And, after talking with Tim and Gavin, I felt I had a very exciting glimpse of what is emerging from the tech’s burning issues. George F. Colony, Forrester, summarized these issues nicely in his post, “Why This Tech recession Will Be Different.” Colony noted, “Virtualization, social computing, mobile computing, Green IT, SOA, extended Internet (connecting the physical world to the digital world) are front and center on the agendas of large companies.”

And, yes, this is supposed to be a little bit of a teaser for my next post on AMEE!

Virtual Worlds Road Map.

The final keynote at the Virtual Worlds London was what Ian Hughes in his post on the conference for Eightbar, aptly described as a call to arms for the Virtual Worlds Roadmap. As Ian pointed out: “This needs a post in its own right as we all need to get on board with this across the industry and help.” Ian Hughes’ (IBM) own presentation on “Business Process Management” was one of the best I attended in conference.  Yes, amazingly, he made this topic very interesting and fresh!

The pictures opening this post are the Virtual Worlds Road Map presenters. Victoria Coleman (Samsung) -seated at center, Sibley Verbeck (Electric Sheep Company) – in trademark hat, Jeffrey Pope 3Di – far left, and  Bruce Damer – close up in the picture on the right.

I am delighted to join Bruce Damer, later today, for a FastCompany.com Technology Group Call-in: “Next Generation Interaction: Are Virtual Worlds Waiting in the Wings?” with Donald Schwartz (October 28th at 4:00 PM EST).

I will also be in Second Life at Train 4 Success (SLURL) on Thursday, October 30 (starting at 9AM PST) with Peter Quirk, EMC, and Jani Pirkola, realXtend talking about OpenSim and realXtend for an event organized by Eilif Trondsen of the Stanford Research Institute and the Gronstedt Group.

John Hengeveld (Intel) – was off screen for this group picture (above). But, Intel is doing some very interesting work in Virtual Worlds see my earlier post here.  And, John is  “helping NASA work out how to deflect extinction level event asteriods from Earth!).”

As Ian noted, the main aim of Virtual Worlds Road Map, “is to gather together and cut through use cases to understand and help people come to terms with which applications need to be built for which case.”

For more great coverage of Virtual Worlds London check out Ian’s post on Eightbar. And, check out Roo Reynolds’, live blogging here and here. Also see Roo’s post on his panel on “ARGs [Alternative Reality Games] and Virtual Worlds.” which includes slides and audio. Picture below is Roo in action live blogging. Roo is Portfolio Executive for Social Media at BBC Vision.

Tribal Media: A Teacher Training Intranet For The Swedish Government on OpenSim

One of the more interesting developments I saw at Virtual Worlds London was a highly customized training intranet for 50,000 teachers being developed for the Swedish Government by Tribal Media. The flexibility of OpenSim to provide cost effective custom intranet solutions was nicely demoed by Darren Guard, Tribal Media R&D (pictured above). Darren is one of the more reclusive founders and phenom developers of OpenSim.

Virtual Worlds and Web 2.0

In my earlier interviews with Rob Smart here, and Teravus Ousley here, we discussed the work to integrate OpenSim with Web 2.0.

To meet the O’Reilly challenge – to do something useful with the internet and help solve some of the world’s big problems, in my view, Virtual World technologies must engage more fully with the power of the internet-as-a-platform – “a system without an owner, tied together by a set of protocols, open standards and agreements for cooperation.” (see O’Reilly, “What Is Web 2.0?” ).

Unfortunately the worst presentation at Virtual Worlds London was purportedly on standards for virtual worlds.  I do not want to waste energy rehashing the misinformed and misguided presentation on the MPEG-V’s archaic blunderbuss approach to standards in this post.  I completely concur with Jim Purbrick of Linden Lab’s characterization of this talk as “the worst talk I’ve heard in a long time.” (Also, see Jim’s post for an astute commentary on other aspects of Virtual Worlds London.)  Luckily, there is much productive work from quarters aimed at leading to standards for Virtual Worlds. And, some of these efforts I have blogged here on Ugotrade.

Because there is confusion, sometimes, in Virtual World discussions about how business models work on a “system without an owner,” here is the concluding quote from, “What is Web 2.0.”

This is not to say that there are not opportunities for lock-in and competitive advantage, but we believe they are not to be found via control over software APIs and protocols. There is a new game afoot. The companies that succeed in the Web 2.0 era will be those that understand the rules of that game, rather than trying to go back to the rules of the PC software era.

What is the Killer App. for Virtual Worlds?

“The killer is that any app you do create is automatically presence enabled.
The people with you can view the changing states of that application or context as and  when you do.” Rob Smart, IBM.

The picture above are the presenters for the “Platform Integration Considerations for Enterprise Virtual Worlds” panel. From left to right: Jean Miller, German Market Development Manager, Linden Lab, Matt Furman, Software Engineer, Northrop Grumman, Rob Smart, Emerging Technology Specialist, IBM Hursley,

Interview with Rob Smart, IBM: Part 2.

Tish Shute: Up to now, Virtual Worlds have been relatively isolated from Web 2.0, living somewhere between the gaming world and the Web 2.0 world. How are the curtains lifting and virtual worlds becoming the linking the space between social media, and online gaming?

Rob Smart: Virtual Worlds that allow user created content and the association of behaviour to that content via scripting put themselves forward as the ideal platform to combine realtime social interaction with existing Web 2.0 tools. The data and function out there currently on Web sites can serve to augment the real-time social interactions. For example enhancing/enabling cross cultural communication with chat translation (example my translation HUD from wayback in 2006).

Another example is augmenting personal spaces with flickr images, video etc. In many flash room based Virtual Worlds this level of integration exists. However without the ability of the users to create their own gadgets and gizmos the pressure is on the development team to innovate and give users what they want, tough to do in the long term. A blended approach is to open APIs and content creation to registered developers.

Rob Smart: When you’re integrating any system with another it becomes so much simpler if the creators have provided,  services and APIs for external systems to interact with. It becomes even easier if those system entry accept/give inputs and outputs in a common way e.g. xml/json. The same goes for both data and media.

By using common existing standards we shorten the development time taken, because if a standard is widely adopted there will be a multitude of programming language libraries for it. The existence of which means the developer can get straight onto the important task of creating the logic for their application/gadget rather than messing around trying to understand some weird data encoding method you’ve invented.

Having an Open Source platform spreads the work load around, as long as the method under which the OS software is licenced isnt too prohibitive then developers from all walks of life will contribute. Spreading that workload also leads to an increase of innovative features as people always bring their experience and interests to bear, the features they create can be shared back and others build on top of them.

If a company chooses to implement a feature they specialize in or integrate with their existing products they can sell this as an add-on, this creates a market where the base product can improve through contributions from companies making a living of the OS product, it also introduces some competition and financial incentive to the platforms well being.

People keep talking about killer apps within Virtual Worlds, the killer is that any app you do create is automatically presence enabled. The people with you can view the changing states of that application or context as and  when you do.

Rob Smart: I still think Virtual Worlds is a good term, though it is very fuzzy. If we’re talking about VWs that can be extended and integrated with web 2.0 then maybe we need to talk about Immersive Application Platforms. Yep not very catchy but probably something more people in the enterprise world would say out loud in front of their boss In addition another term that could be used is 3D Internet it conjures more of a picture of integration between the different parts of what is a vast networked system.

Tish Shute: The  original metaverse roadmap had four distinct segments Augmented Reality, and Life Logging at the pole of augmentation, and Mirror worlds and Virtual worlds at opposite corners of the pole of simulation. How are these areas coming together?

Rob Smart: There’s no reason these need to be separated, its all down to the use of the VW platform these four segments are just applications of a virtual world platform. A platform like OpenSim can merge several of these together if neccessary. For example the Publish Subscribe messaging module written about on eightbar that I created lets me do things like bring in Realtime Flight data and show planes positions etc. across a region I could at the same time call an API that gives me more details on that flight. I could even search for blogs that mention that flight number and bring them into the same space. I could add additional script functions to the plane objects so that when a visitor clicks on a plane it thereafter sends them messages about its position.

Tish Shute: Virtual worlds are being broken down to open source basics building blocks and modules that can be mixed and matched and mashed up with Web 2.0 to create a new ecosystem that enriches both what has been know as virtual worlds and traditional web environments. What kind of innovation do you see coming out of these new opportunities to mashup virtual worlds with Web 2.0?

Rob Smart: I’m hoping to see as a number one priority an increase of accessibility, despite a number of people saying that browser based virtual worlds aren’t worth the effort they certainly are. The ability to just send a friend a URL or Instant Message etc.. and pull them in with you is an important step to adoption. As are simplified interfaces that don’t scare off those unfamiliar with gaming. An example of this is the Lotus Sametime 3D work with OpenSim that lets you invite a friend or colleague in via an instant message.

Virtual Worlds For Enterprise: A Coming of Age Party?

As Ian mentioned I did think that the London Conference was a coming of age party for enterprise virtual worlds. In the picture below there are just some of the Lindens who were there, many to promote the Linden Lab collaboration with Rivers Run Red on “Immersive Work Spaces” which was written up in Wall Street Journal. Also see this post yesterday on Silicon.com, “Virtual Worlds Set For Second Coming.”

Someone please help me with the all the names of the Lindens in the picture below!  Matt Furman from Northrop Grumman is center and Joey Seiler from Virtual World News is on the right.

Justin Bovington said to me that this conference was in his view: “the enterprise virtual worlds coming out party – an acceptance that this is a tangible solution- about selling relevant tools and relevant ROI – rather than talk about virtual worlds it is about relevant tool sets.”

And, while the conference was small, I think the engagement level of the enterprise attendees did back up this assertion of Justin’s. Matt Furman, Software Engineer, Northrop Grumman was asked by more than one attendee how he was dealing with scaling up the behind the firewall virtual world he is developing for Northrup Grumman with Linden Lab to meet a big demand internally to start using virtual worlds for collaboration.  Apparently some attendees were seeing so much interest in virtual world solutions for internal collaboration in their own companies, they were concerned about meeting the needs of thousands of employees in short order.

Immersive Work Spaces

I asked Justin a few questions about Immersive Work Spaces while waiting for an elevator!

Tish Shute: And what are the relevant tool sets from your point of you?

Justin Bovington: Collaboration, sharing, integration of existing backend systems and applications.  For example, we have developed seamless ways to share powerpoint or share screens. And, also going back down to the ROI models as well,  tangible ROI based on subscription based system where basically in four or five usages it has paid for itself. We have never had that with Virtual Worlds. It has always been in the bounds of experimentation or the bounds of isn’t it cool technology. Now we are seeing this become a serious collaboration tool.

And as I have said before that argueably the twentieth century ended two weeks ago and the twenty first century is now with us.  And that is about companies rengineering their thinking particularly in the financial sector they have to restart again. And that is going to be about  using additional tools and additional guide lines to do that. This is the change over and I have said this in the panel as well. This show in particularly is enterprise virtual worlds coming out party.

And again we see a massive change between the last three shows – there is a level of interest we have never seen before and also an acceptance that this is a tangible solution not just something that is cool…

We have hundreds of users in out product and it will go  to thousands and tens of thousands in the next year.

And we know where it is going – data visualization is going to be the next big thing and getting this 10,000 ft view of your company. We are using this term called snow globing which lets you pick up a snow globe and shake it and let you see exactly what a company is about and this is exactly what virtual worlds are about.

It’s about having a ten thousand foot view of your company because that’s when it becomes powerful because then it becomes a broadcast medium. And I think it will change people’s perception of data. And it is also moving to beyond just having the avatar as the main presence. The environment itself becomes an essence or a kind of dynamic level that is inside there. We are working on stuff at the moment that allows you have direct influence on data or the environment you are in which on a massive collaboration scale could actually give you a huge amount of input and ideas around company. And there is a genuine need to have this kind of collective intelligence.

Sine Wave Dinner!

The grand finale for me was the excellent Indian meal very generously hosted by Rohan Freeman of Sine Wave Company. Standing on the left is Chris Collins, Linden Lab, seated left front is, Steve Spangaro, bigpipemedia, and on the right Ren Reynolds of the Virtual Policy Network. Many other metarati were there including Bruce Joy, Vast Park, Corey Bridges, Multiverse, Dave Taylor, Imperial College, Gia Rossini, Sloodle, Peter Haik, Metaversality, Adam Frisby, OpenSim, Mal Burns, and many more – please help me out with the name tagging!

The curtains are lifting between Virtual Worlds, Web 2.0, and online gaming. There are many indications of this in the news including, the rebranding of the Virtual Worlds Conf. and Expo in New York City as “Engage! Expo – 3D Web, Virtual Worlds, and Virtual Goods,” and Google’s Lively opening up an API for game development.

And, If you have been reading Ugotrade recently, you will know I have been up late several nights trying to keep up with the pace of the  OpenSim, Architecture Working Group and OpenGrid Protocol teams that are proceeding at a fast clip with their work on Web 2.0 integration for immersive Virtual Worlds (and there is still much more to write on this!).

Also, this week, there was the preview launch (the full launch is scheduled for November) of a new collaboration, “SHASPA,” between EOLUS^® One and The Serious Games Institute.  SHASPA was unveiled to a select audience of business decision makers at the Globe Theater, London on Wednesday.

SHASPA- making a “smart” world with “shared spaces.”

Oliver Goh said, “SHASPA” is a unique cooperation to bring together Virtual Worlds (OpenSim, Second Life^®), Web 2.0 Applications and the world of mobile applications….”

The pictures above show Oliver speaking (far left) and David Wortley, Director of the Serious Games Institute (SGI) at Coventry (far right), and some of the audience (center). I recognize several very important virtual world pioneers and innovators in the midst of the frey – Rohan Freeman (CEO of the Sine Wave Company), Prof Jeremy Watson (Arup), Dr Anthony Dennis (infoterra),  and Dave Taylor, Programme Lead, Virtual Worlds and Medical Media at Imperial College London.

EOLUS^® One initially started as an innovation project with the focus to develop new service oriented offerings for the real estate industry. Oliver explained to me:

The focus is on sustainable real estate, enhancing the structural and technical performance of properties which will be the first use case for SHASPA. SHASPA uses the combined power of the work done by the SGI and EOLUS One to create Smart Shared Spaces for various industries. The first use case will be in the Real Estate industry to revolutionize approaches to facilities/property performance optimization and energy management.

RealXtend harnesses OpenSim as engine for their server side development

Adam Frisby sent me the picture opening this post today.  Adam pointed out It shows:

…Realxtend just running under OpenSim rather than the forked version of OpenSim realXtend did. It’s been converted to a set of OpenSim plugins – we’re still at a semi-preliminary stage, however, we’ve got Rex avatars and a few other features now working.

Realxtend is now able to leverage the OpenSim core, and OpenSim developers can work with realXtend innovation as plugins. For more details on this modular integration see my previous post.  This modular architecture will create many new opportunities for mashups.  And the Web 2.0 integration and interoperability work that is central to the OpenSim vision will be aligned with the advanced 3D Internet layer realXtend has been building on top of it.

On their diverse and multifaceted team, RealXtend has a number of world class game developers  who have in a very short time progressed rapidly towards the goals Tony Manninen, Ludocraft, described to me back in February, “making sure the realXtend development reaches the required quality and performance standards you would expect from MMOGs.”

And as Jani Pirkola, Project Manager for RealXtend, points out:

For realXtend it means that we can have all the OpenSim development directly benefit realXtend, whether it is Web 2.0 or other features.

Web Friendly Standards for Virtual Worlds

I am off to London next week to the Virtual Worlds Conf an Expo.  But, while I am very excited to meet old and new friends there, it is disappointing to note that the open source developer communities and the interoperability and open protocol efforts of OpenSim and Linden Lab are sadly unrepresented in London.

Making virtual worlds part of the fabric of the internet and everyday computing will not happen because some arbitrary standards body pontificates on elaborate requirements and then tries to get the backing of big business to implement their standards from top down. There are many white papers on why this old fashioned way of developing standards is not applicable to the fast moving internet environment.   As David Levine, IBM, so nicely put it a while back, interoperability and standards for virtual worlds:

will emerge battered byte by battered byte from the hands of grubbie techies each with an agenda. Except on Second Life some of us are blonde, with a pert smile but yeah….

It is, in my view, unfortunate that Dr Yesha Y Sivian, Metaverse1, in his talk “Virtual Worlds State of Standards (SOS): MPEG-V, Metaverse1, Open-SIM and more” has put OpenSim in his title (and Architecture Working Group in the body of his abstract) when he does not seem to have (yet) invited anyone from OpenSim or Architecture Working Group or OGP to represent their own work.  Again, unfortunately, a panel including key industry leaders and representatives from OpenSim and Architecture Working Group did not get the opportunity to present in London because Dr Sivian’s proposal gave the conference organizers the impression there was already a “similar panel.”

MPEG-V and Metaverse 1 are Dr Yesha Sivian’s projects and they are at a very early stage of development (basically an effort to define a set of requirments and garner business support for the notion of creating so called MPEG-V standards). To have credibility, in my view, these projects need to engage with other groups that are working on standards and actually have working code, as  Architecture Working Group and OpenGrid Protocol (OGP) do.

There are some common misunderstandings about the approach of the Architecture Working Group that should be cleared up.

As key architects of OpenGrid Protocol (OGP), and the Architecture Working Group, frequently stress, OGP is a point of departure.  While its focused on the existing code of OpenSim and Second Life, the overall framework is as broad, or broader than the meteverse work.  The goal is to create a fully described set of web based protocols and formats which will do anything MPEG-V wants to do, but meshed far more fully into the web.

Metaverse1 needs to be in dialogue with the standards work that has already produced code, if they are serious about creating good standards.

Out of the Trough of Disillusionment onto the Slope of Enlightenment

It seems Virtual Worlds may have started onto the Slope of Enlightenment (see Gartner Hype Cycle).  Virtual Worlds, and immersive Virtual Worlds (in particular Second Life^® and OpenSim), continue to garner broad consumer interest. And, the ability of Virtual Worlds to deliver added value in key areas of collaboration and energy conservation is fueling a a lot of interest from education and enterprise.

While worries of depression and recession in the global economy abound, and the internet is abuzz with discussion of Joe The Plumber (as Mitch Kapor noted in Twitter: “Not since Nixon have we heard so much about plumbers”).

Nevertheless, there has been quite a steady flow of positive news from Virtual Worlds. See Caleb Booker’s roundup and  Virtual World News and check for yourself. And just in, Forbes.com post, “A ‘virtual’ Escape from Economic Pain,” After scanning my reader I checked my perception in Twitter and quickly got replies from Mitch Wagner of Information Week.

@MitchWagner – while you didn’t find any negative stories have you seen an increase in positive stories in mainstream media? about 13 hours ago from web in reply to MitchWagner

Recents News Events of Note

The coming of age of Open Source Virtual worlds is attracting some mainstream attention now.  One of the leading authorities on Open Source Software and Services,  OStactic has several recent posts on OpenSim and Open Source Virtual Worlds. And, of course, I was thrilled that Ugotrade got a mention in the most recent one, Open Source Virtual Reality Spreads Out.

This post, and my previous post about integration of OpenSim into Web 2.0, explore how immersive virtual worlds, through a full architectural integration into Web 2.0, will become part of the fabric of everyday computing.

The diagram above shows where OpenSim sits in Web 2.0 (click on the diagram to see a readable enlarged version!). The following interview with OpenSim developer, Teravus Ousley, describes some of the work being done to create documented protocols that will make OpenSim fit seamlessly into Web 2.0 architecture.

OpenSim is in the news a lot these days, explicitly as in the case of the announcement last week by 3Di of their “3Di OpenSim” Standard (for more see here and here), and implicitly with the launch of ChinaQ.  Adam Frisby, OpenSim, pointed out to me if you download the ChinaQ client that it is based on OpenSim, it connects nicely to OSGrid too. There is speculation the client is a rebranded version of the realXtend viewer (which is based on the open source Second Life viewer) as all the version numbers are the same.

So OpenSim is not only attracting the interest of business giants like IBM, Microsoft and Intel, it is becoming the architecture of choice for virtual world initiatives from Chinese and Japanese telecoms (see here and here for more). Also, see the press release about Nokia and the City of Oulu, Finland, joining as supporters of realXtend.

But, as Raph Koster in his post commenting on 3Di’s OpenSim announcement notes, the question how immersive virtual worlds can go from strong niche or enterprise markets to mass adoption in consumer markets must be answered.  As Raph points out, Lively, Whirled, SmallWorlds, Vivaty, and yes, Metaplace have a very different architecture that they hope will attract broad consumer markets.  (I did a long interview with Raph on this at The Virtual Worlds Conference and Expo in LA which I will post as soon as it is transcribed, so more on this soon!).

Architectural integration into the heart of Web 2.0, I would argue, is the key to mass adoption for immersive virtual worlds. While architecture alone will not guarantee the necessary breakthroughs in usability for widespread consumer adoption, it will create the ideal conditions for the innovation through which usability obstacles will be overcome, and the enormous potential for immersive, real time interaction over the internet will be realized.

Interview with Teravus Ousley

Tish: What has been  the most fundamental problem re virtual world architecture that has kept immersive virtual worlds isolated from web 2.0 to date? 

Teravus: a lack of standardization, licensing issues, and the difficulty of entry into the industry.

1) Standardization

Tish: In order of importance what in your view are the priorities for standardization?

Teravus: Probably the same order that OpenSimulator was tackled in, basic connect (current state of OGP – Open Grid Protocol).  Basic Service (interaction standards).  Advanced connect/mashup/aggregate extensions.   Preferably people will have working code in the various spaces there to use freely under various licenses..

Tish: Can you show me where OpenSim will fit in this drawing of Web 2.0 architecture? [Teravus makes some modifications on the drawing I send him from Dion Hinchcliffe’s presentation from his Web 2.0 Expo workshop, see original here]

Teravus: The modified diagram [now opening this post] is a great view of how it will look.

Tish: Why is the TCP stream left out of the original drawing? [For more about Transmission Control Protocol (TCP) is one of the core protocols of the Internet Protocol Suite see here.

Teravus: It is left out because the person who made this diagram had web pages in mind.  Static large files, or small changing files. In the the drawing the fact that TCP streams are smaller then HTTP is on purpose.

Tish: I have heard different opinions on the percentage of the communications for virtual worlds that can be done over HTTP?

Teravus: The fact is that the biggest usage of communications in virtual worlds is transmitting images that’s the number one bandwidth usage. So, if we’re counting by ‘usage’ I say 91%.   If we’re counting by services that use http.   I say probably 75%  I definitely think that http should be evaluated for use on new things ‘first’. But, there are a few places where HTTP doesn’t shine.

I am skeptical about replacing things in the UDP with HTTP  thinking that they’ll ‘perform better. [For more about User Datagram Protocol (UDP) another of the core protocols of the Internet Protocol Suite see here.]

I think there’s been a huge test going on now and for the last 5 or six years with regards to the UDP protocol and it really has performed admirably.   In the last year and a half, I’ve seen attempts to convert several things to HTTP that have failed, and failed somewhat spectacularly sometimes.  In the end the items get reverted back to the UDP protocol. One such item that sticks out in my mind is CAPS(HTTP) based inventory retrieval. The capability to do that in the client has been available since before February. And, it’s been turned on and off on ‘Agni’ at least once in the process. Additionally, we (OpenSimulator) enabled http inventory, and, the  inventory failures rose pretty steeply.

I think some services are really just not ‘right’ for HTTP.. . particularly where a ‘poll’ methodology is used, or, the data is significantly dynamic enough that it makes caching useless.

Anyway, as far as the future is concerned, I do want to see some services over HTTP. Other services, it would be more appropriate to have a TCP stream. Stock market data, for example, uses a TCP stream. The Scalability of the stock market, is just one example of a scalable TCP stream.

Tish: So you see TCP  as the communications protocol that would do the work for the parts of virtual worlds not suitable for HTTP. At least that is how you have shown it in our Web 2.0 architecture drawing. But should there also be a UDP stream?

Teravus: For the virtual world of tomorrow? .. probably not.

Tish: Why not?

Teravus: You have less control over the quality of service when it’s delivered over UDP then TCP.

Tish: What is the exact relation between TCP and UDP.  My understanding is UDP a lower level protocol.

Teravus: TCP offers guaranteed delivery through flow control, while UDP does not.  One of the failures of UDP, is the ‘resend’ technology we’ve put on top of it to try and make it reliable.   TCP does this automatically and better then we could at a lower level but it does also cost up to twice the bandwidth depending on what is being sent. HTTP is a layer on top of TCP.

Tish: So just like the HTTP/TCP discussion there has to be a TCP/UDP boundary discussion …so it is HTTP then TCP then UDP and the boundaries have to be worked on.

Teravus: Those are the orderings in my mind…  probably if UDP uses any..  it should use less then 0.5%.

Tish: And the current Second Life architecture what does it use if it isn’t using HTTP? [to see the work of the Architecture Working Group on the future Second Life architecture here]

Teravus: UDP or HTTP

Tish: and TCP?

Teravus: Well, TCP is a layer under HTTP.  As far as I know, SL doesn’t use TCP streams directly

Teravus: Instead, it uses HTTP polling.  This is one of the places, that I’ve highlighted where it doesn’t shine.

Tish: Polling does sound slow?

Teravus: Polling is essentially..     (connect) Got any data for me? No?(disconnect), (connect) Got any data for me?  No?(disconnect).

Tish: So what is the path to standards for this then?

Teravus: Distilling what we know works and what we actually intend on supporting as far as adoption under these standards.

Tish: Where does MPEG-V fit in?  Have you read their document yet?

Tervavus: MPEG-V is interesting reading…     but is there any working example? I have just the overview. But I’ll read it over to have a better determination of how to ‘keep it in mind’ for the future. It looks like they’ve only really defined the requirements of the MPEG-V spec. The MPEG-V spec looks quite far reaching..  but  the documents so far are requirements and marketing talk aimed toward business people – obviously intended to get more people interested in working on them.

But I have a feeling that any format with MPEG before it will be onerous to support. ..for me it’s too early to tell. It’s quite far reaching…it isn’t anything like ’signal processing’ which the MPEG group is most famous for.

Tish: The whole top down approach of the MPEG-V initiative seems counter to Web 2.0 principles to me.

Teravus: Well, remember..  that even if there’s a virtual world format war (reference to DVD-HD vs BlueRay) we still need to win over the rest of the web.

Tish: Yes and don’t you think the way to win over the web is to use as many existing standards as possible?

Teravus: Well, it’s to use as many existing standards as ‘fit’ though.. KISS, as always (K)eep (I)t (S)imple (S)tupid if we have 30 different internet standards..     people looking at it will @.@

Tish: But it is just lack of documented protocols that has created isolation from Web 2.0?  And really doesn’t it boil down to standardizing that small percentage that is outside HTTP – the TCP and UDP stream that we talked about earlier where the real time stuff that virtual worlds bring to the web happens?

Teravus: no..  actually the HTTP standardization is just as important.

Tish: You mean even though SL used HTTP it isn’t standardized?

Teravus: Not documented specifically.

Tish: And OpenSim is that documented?

Teravus: Not well enough probably to define a standard.

Tish: Is AWG (Architecture Working Group) doing the documentation?

Teravus: working on it..

2) Licensing Issues

Tish: It sounds like some of this work has to go on across client and server.  Are we running into the issue of BSD for OpenSim and GPL for the Second Life viewer?

Tervaus: Well, some of the issue here is license choice.  One of the reasons that libOMV was able to achieve what they did was they did it /before/ the client was open sourced.

Tish: So open sourcing the client actually became an obstacle!!???

Teravus: I don’t think so in a whole.  I think it was great for the community.  I do, however think that C++ UDP stacks will be scrutinized more for GPL license violations because, of course, the client is GPL and C++ .

Tish: It is my understanding that Linden Lab is open to discussions on making the licensing more efficient for the open source community?

Teravus: Well, the client, in a whole, should not be changed as far as the license.   JUST the things that they expect people to adopt should be made more open. If they expect people to adopt PRIMs, then there should be an efficient implementation available for anyone to use..   at the very least, in LGPL format. Otherwise, the die hards are forced to re-implement them from scratch, and most people will just choose something more open.

Tish: Has anyone ever put together a list of the parts that need to be LGPLed?

Teravus: Well, I think it’s there in a few places.  There is at least one jira open on it.

Teravus: A few that come to mind for me..   is the UDP stack and the prim to mesh/UV code.   I think there are some things that can definitely be improved about the UDP Stack.  There are some things, (images come to mind), that would be better over HTTP

Tish: Do you think if the UDP stack were L GPLed that would be a significant help to integrating OpenSim better with the web?

Teravus: Well, it would certainly be adopted by more clients. GPL + (your own code) = GPL Licensed client. LGPL linked library + (your own code) = Your own license.
You still need to mention that you used LL’s UDP stack, and provide the source code for it at request.

The general client itself should remain GPL, it’s better for LL that way.  Just the items that they want people to ’standardize’ on. It would help..   if it was at least LGPL

Tish: And the value to  LL on LGPLing these parts is it helped spread their basic technology while protecting the rest of their viewer?

Teravus: It furthers their goal of standardization on their systems because it allows more people to adopt it for their own uses without worrying about GPL-ing their own client.

Tish: It is hard to standardize without access to the low level parts of the client right?

Teravus: The general population of Developers..     will want a libX that they can plug into their application for communicating.. .  libY to deal with object data..

Tish: Hence your requests for LGPL were  UDP stack and  the prim->mesh/UV

Teravus nods

Tish: and at the moment they only have openmv?

Teravus: That’s the only ‘truly’ open standard right now as far as the LL technology is concerned. OpenSimulator’s use of that data..   could also be seen as a standard..

Teravus: But we have not published anything beyond code..   neither have they  really..  technically..  but their organization of the way things work is very very clear

Tish: What are the most significant limitations of openmv?

Teravus: Probably..  just it not being in c++.   c++ has it’s benefits and it’s pitfalls.  Changes in c++ usually take longer then ones in C#.  But, of course c++ is always faster.  With libOMV It isn’t always clear about what packet is used when.  However, with some experimentation, you can figure it out in 30 minutes or less..

Usability

We didn’t spend much time discussing some of the innovation in usability that this architectural integration into Web 2.0 will enable (more to come on that!). But, Teravus mentioned one interesting use case he is working on.

Teravus: You might also stick a ‘cloud renderer’ into the graphic [Tervaus was looking at the diagram (from Dion Hinchcliffe) that opened my previous post on “Web 2.0 to OpenSim Made easy”  click on the thumbnail below].

Some people have discussed having a ‘video stream’ that’s rendered on the cloud and providing that to flash clients would be the best solution to it for them.

The cloud renderer is for organizations that have large pools of servers with GPUs so would allow for very powerful rendering. The servers can render the scenes and stream them to the low end browsers. It would allow extremely high quality rendering for really low end browsers..  such as ‘cell phones.’

Tish: Is that possible now on OpenSim?

Teravus: Nope.  But it’s something that in the future, I intend on working on. It would essentially be a video [streamed to low end browsers].

Tish: Is that different from what Vollee is doing? The mobile client for SL?

Teravus:  It appears that they are, indeed, pre-rendering the client’s view and streaming it to the mobile device

Web 2.0 surpasses all previous technologies in its ability to “explicitly leverage network effects” (a definition of Web 2.0 from Tim O’Reilly). But, while virtual worlds pass another classic litmus test of Web 2.0 – two way participation, they have been, up to this point, largely cut off from Web 2.0 power/network effects.

Persistent immersive virtual worlds, led by Second Life, have done well as niche markets but they remain relatively isolated from Web 2.0, even though they bring something  vital and new to the internet – real time interaction and dynamic melded states – in contrast to the current web’s large static files, or small changing files.

The slide opening this post is a modification of a slide from Dion Hinchcliffe’s presentation from his Web 2.0 Expo workshop – Building Successful Next Generation Web 2.0 Applications. Virtual worlds are not anywhere to be found on the original. So I asked Rob Smart, IBM, who has just added JSON support to OpenSim to draw OpenSimulator into this picture. In my interview with Rob, later in this post, he discusses the relationship between virtual worlds and Web 2.0 and how JSON is an important step towards virtual worlds taking up a place in Web 2.0 architecture.

When people think of the current architecture of Web 2.0 virtual worlds do not come to mind. But we are on the cusp of a big change in this regard.  Linden Lab and OpenSim, in the Architectural Working Group, AWG, have been working on trust negotiations and the standardization, documentation and use of http (REST enabling).  But more work remains on standardizing and documenting where TCP and UDP streams have to be used to create the immersive real time interactions that are the heart of what virtual worlds bring to today’s web (see my upcoming interview with Teravus Oursley, OpenSim, for more on this).

There is a complex network of connections through identity (1st and 2nd life) that have enabled virtual worlds to implicitly leverage the social networks of  Web 2.0 (see botgirl’s lovely illustration of this above)  The slide above is from W. James Au’s “The Post-Hype State of Virtual World Marketing: What Works, What Doesn’t and Why.”

Mryl (beta) is an application hoping to streamline these linkages with a social gateway for virtual worlds that will provide what  KZero terms “outeroperability”.  In this vein, Second Life developers have produced a number of interesting high level communications applications, including Chatbridge from Intersection Unlimited, to link Second Life better with the web. I will moderate a panel for Orange Island Innovation Week, Wednesday, Oct 1st, 12 noon PDT, Innovative Communications Devices, with Beyers Sellers, Chase Marellan (Chatbridge), Kevni Koolhaven (Learning Tree International).

But, it is the low level architectural integration of virtual worlds into Web 2.0 (along with improved usability and new User Interfaces) that will weave virtual worlds into the fabric of  Web 2.0 and  our everyday lives.  But unlike Eric Schonfeld of TechCrunch, I see interoperability work (see OpenGrid Beta), and the production of standard protocols (see Open Grid Protocol, OGP) that interoperability work helps negotiate, as an important part of the process.

Immersive virtual worlds are still a long way from mainstream.  I attended the Forrester Business and Technology Leadership Forum in Orlando last week to help Oliver Goh, business development executive at Implenia, talk about delivering results with virtual worlds. We found the audience, while familiar with many aspects of Web 2.0 and its business value,  had relatively little direct experience with virtual worlds. But, the interest and excitement with this technology was very apparent.

Architectural integration of  virtual worlds in Web 2.0 and the standardization of protocols (using existing web standards where possible) will change the picture, creating new opportunities to improve usability, create specific clients for particular needs, facilitate mashups, and leverage network effects, and more!  And, JSON support for OpenSim is an important step as it allows virtual worlds to explicitly begin talking the language of Web 2.0.

Rob Smart is an Emerging Technology Specialist located at IBM Hursley where he works as part of the IBM CIO office Metaverse Initiative. In Second Life he is known as Yossarian Seattle and became known to some as the inventor of the translation HUD, which was his second foray into integrating Virtual Worlds with Web applications. The first project was enabling some of IBM’s messaging products to publish events into Second Life, including creation of an RSS Viewer for Second Life.  Recently, Rob has been working with clients integrating their internal IT services with various virtual world platforms. His interests now extend to the OpenSim project, with a focus on integration of enterprise data and common web APIs into OpenSim.

Interview with Rob Smart, IBM

Tara5 Oh – on the right (me, Tish Shute)  interviewed Yoassarian Seattle (Rob Smart, IBM) in Second Life outside Andy Stanford-Clark’s remote control house on Hursley island  (for more see here)

Tara5 Oh: I am interviewing you from the media lounge at Web 2.0 Expo and coincidently it seems JSON is the hot standard here, in fact, the hottest it seems other than RSS for its ubiquity.

Yossarian Seattle: Yes, well the popularity of JSON stems from increase of AJAX enabled websites that need to frequently pass data between server and web browser and have the javascript in a web-page understand that data. It provides a simple, lightweight way of serialising your server-side objects and doesnt require lots of extra coding in the browser like XML data does.

Tara5 Oh: As virtual worlds are still isolated from many of the network effects of Web 2.0, at the moment could you explain how integrating JSON support to OpenSim is “Web 2.0 made easy for OpenSim?”

Yossarian Seattle: JSON was created to make data exchange from browser to server easy. We want that same exchange between VWs and web servers to be equally as simple. However JSON was written with javascript in mind as you can call a simple eval() function and that’s it, you’re done and you have a nice object to use in the browser. So as a result lots of these nice service APIs out there in Web 2.0 land talk JSON, e.g. Google Translation service, flickrs image querying etc. Also our internal IBM web 2.0 systems talk JSON.

But Second Life and OpenSim so far have poor string handling functions which meant that in LSL, in particualar, parsing anything more than a simple piece of JSON was just not an option.

Lots of coders and developers in Second Life have to run PHP and other scripts on external web servers to act as an intermediary stage in calling thse Web 2.0 APIs.

Thats a real pain, and means you need to have a server somewhere and up full time if others are to use your scripts. Whereas now, with this osParseJSON. function you can forget all that hassle and go straight to the source from OpenSim.

Its a simple but powerful enabler of Web 2.0 technology. I expect it will take people a while to find it and start using it, but it just widens the accessibility for those people who get into scripting in OpenSim.

I’m planning to do a similar thing for XML parsing, but its a bit lower on my priorities at the moment. JSON parsing gives a good quick win so to speak

Tara5 Oh: I just sent you a couple of slides ‘cos one thing I have noticed here at the Web 2.0 Expo is that the understanding of where OpenSim might fit into the architecture of Web 2.0 is vague to zero.

Can you sketch something that relates OpenSim into current understandings of Web 2.0 architecture?

Yossarian Seattle: Really in that first diagram with the APIs etc OpenSim just fits on the level of the web servers. And actually that diagram is a bit wrong as there should arrows between the web servers as sites should really be connected to each other.  I’ll stick in here

OpenSim is being REST enabled. At the moment its access to assets, clothes, objects, etc. from the asset servers. But there is no reason that REST interface cannot give access to people logged on, object positions sim layouts etc.

Tara5 Oh: Could you explain the difference between the power of REST for virtual world technology in relation to the power of JSON?

Yossarian Seattle: So REST is really just calling a web URL. You use the tree structure of the URL to indicate your asking for different data. Whereas JSON is an encoding for the actual data that’s returned to you. So they are complementary really. But there has already been some discussion within the OpenSim community about introducing new APIs to OpenSim that allow different clients to connect.

I personally think that VWs are too siloed currently. At the moment in VWs it’s pretty much one world one client. Providing REST or other interfaces to the world data opens up the possiblity of a wider range of clients accessing those worlds. And when i say clients i’m talking about flash interfaces, browser interfaces or other 3d interfaces such as Unity3d clients.

Tara5 Oh: Could you tell me more about Unity3d?

Yossarian Seattle: Unity3d is a game engine. It’s a very flexible engine and adheres to a lot of the 3d modeling standards etc. One of its most interesting features is the ability to deploy the games/applications you make as web brower plugins (as well as windows/mac stand alone). I’ve been messing around with it for a while now, I can show you some demos while you’re over at the VW conf in London.

Tara5 Oh: Another theme at this conference, raised by O’Reilly in his keynote, is that the future is “world to web,” e.g ., sensor projects etc.

Yossarian Seattle: Ah well that’s another favourite topic of mine with regards to VWs Hursley is the home of realtime messaging technology.

At the moment as I say there is pretty much one VW client to VW server and because the only library to acces SL and OpenSim is openmv( formerly libsl) that restricts new clients to being written in c#  There isn’t a java library, a flash library , a php library a ruby library etc.

So if in OpenSim we add new connectors, REST ones, talking JSON or XML then we enable lots of new client types and VWs become another mashable service in the Web 2.0 world.

Its about making it easy to get information in and information out. Web 2.0 sites don’t do realtime very well, whereas VWs do. VWs are the real time space that the web often tries to provide but kind of half fudges. Web Servers aren’t built to deal with realtime asynchronous data.

Its interesting how you mention Web 2.0 not really acknowledging Virtual Worlds as when I read the terms of service for a lot of the APIs they’re very specific about use from other web sites but they often dont cover the use of the API from other applications.

Tara5 Oh: Really?

Yossarian Seattle: Yes.

Tara5 Oh: What does this mean?

Yossarian Seattlee: It doesn’t necessarily have any significance for some services. But there is often specific text saying for example that you must use a particular piece of HTML on a page and show the API owners logo etc

I think as time goes on though and more people connect to Web 2.0 services from within VWs then they will be acknowledged as a valid service consumer, after all VWs are platforms that provide novel ways for people to display and interact with data.

Tara5 Oh: I know Hursley and other IBMers have done some nice use case of RL data integration in OpenSim and Second Life. What is your favorite for illustrating the power of Virtual Worlds to bring realtime world to web experiences to Web 2.0

Yossarian Seattle: Andy Stanford-Clark’s remote control house on Hursley island is still a favourite.

I did a hook up ages back with a messaging product MQTT and Second Life. I’d like to revisit that work and extend it. i’m interested in propagating events between platforms whether they be VWs or Web sites.

Tara5 Oh: I am amazed how little play virtual worlds have here at the Web 2.0 Expo.

Yossarian Seattle: Virtual Worlds live somewhere between the gaming world and the web 2.0 world. We see it with the flash social worlds too they edge more towards gaming.

Tara5 Oh: What do you think are the gains of virtual worlds getting more integrated with Web 2.0?

Yossarian Seattle: Virtual Worlds are a platform and and its often said by some that they’re not interested in taking part yet as they haven’t yet seen a killer app for Virtual Worlds. Some of that view stems from the fact that VWs are very isolated it’s hard to get content in and hard to get it back out again.

Virtual Worlds are the shared realtime spaces of the Internet, up until now this position has been filled by IRC chat rooms and instant messaging apps. Neither of these forms lend themselves particularly well to group interaction. VWs are streets ahead in terms of rich social interaction and sharing of content and experiences.

Tara5 Oh: You mentioned you just started working on OpenSim development and becoming part of this growing effort.

Yossarian Seattle: Yep thats right. There is a very vibrant community around OpenSim.

Tara5 Oh: Why did you decide to put your energy into OpenSim at this time?

Yossarian Seattle: I’m now working for IBMs CIO office Metaverse initiative and investigating all of the relevant VWs is one of our remits. OpenSim is my chosen focus.

Tara5 Oh: What is CIO?

Yossarian Seattle: One of the IBM CIO office responsibilities is to look at and provide technologies and tools that improve the productivity of IBMers world wide. But as you know IBM has several people working on the OpenSim project and there is an interested community internally. I’m looking at how we can hook up OpenSim to our existing web 2.0 services internally.

Tara5 Oh: What kind of internal Web Services?

Yossarian Seattle: We have a number of internal Web 2.0 based systems that provide APIs for data sharing, things like Blogcentral our internal blogging platform, Fringe which contains customizable profile information, Beehive is a social networking platform helps people share their interests, track and schedule events within IBM. We also have a platform called TAP (Technology Adoption Program) where people can share services and applications they have created with other IBMers. In addition we have Cattail, a file repository that allows easy sharing and tagging of all types of file. There are many more useful internal services than this even all of which could be integrated with OpenSim.

The nice thing is though that OpenSim affords that flexibility to integrate it with our products and with existing web systems, and provide value back to the community at the same time.

Tara5 Oh: So do you have any thoughts about the path to standards for virtual worlds?

Yossarian Seattle: In terms of standards I think it’s a case of look at whats out there and successful at the current time not just in terms of 3d models, but in terms of real time chat protocols like XMPP things like JSON, REST as well and pick those for the relevant components

The reason for this is every time you introduce a new standard, you have to wait for the communities to catch up and write language specific APIs for that standard.

[Better to use existing ones where possible and give the communities that will build the tools and the extensions a head start.

Tara5 Oh: This is also some of why top down standards like MPEG-V have issues?

Yossarian Seattle: Yep, standards often work best when they’re bottom up, like JSON.

As I mentioned before because the messaging structure currently for OpenSim and Second Life is proprietary (although open) and the only library is libsl (openmv) thats stopped a lot of potential innovation by restricting client/bot developemtn to the c# language.

Tara5 Oh: why is client/bot development restricted to c#?

Yossarian Seattle: Because currently the only library you can use to talk the Second Life libsl (openmv) is written in c#

Tara5 Oh: What do you see as the way through this obstacle?

Yossarian Seattle: If for example the messages that went between your SecondLife client and the OpenSim/SecondLife servers was a standard protocol which had a bunch of libraries for a variety of languages. Then you could start logging into VW servers from all kinds of clients

Tara5 Oh: Aren’t there plenty of standard messaging protocols to use?

Yossarian Seattle: Yep, but at the moment they’re not being used. There are some technical reasons for that. like reducing the amount of data to be downloaded etc. But there’s a balance to be had somewhere.

Tara5 Oh: But in a modular architecture like OpenSim what is to stop them being implemented?

Yossarian Seattle: There isn’t anything to stop them being implemented in OpenSim Which is why i like it

Tara5 Oh: I hear a lot about people wanting to change the physics in OpenSim/Second Life (the linking to the physics simulation in particular). Do you have thoughts on this or is it not on your agenda currently?

Yossarian Seattle: There are a few different physics modules already. Though to be honest i don’t think its the most important area to focus on, for me at least.

But obviously a high end physics engine is going to benefit anyone who wants to do any kind of simulation.

And that’s the beauty of Open Source, someone else will have that as their priority.

Yossarian Seattle: I think there’s a lot of work that needs to be done around ease of adoption still. i’d like it to be easy for people to write new clients for OpenSim.

When we get to that stage then people can produce simplified cut down clients to suit their precise need, so if you’re a retailer and just want to showcase products and let people shop you have a UI to reflect that.

Tara5 Oh: What about the OpenViewer project?

Yossarian Seattle: It’s a wider problem and piece of work. Well notice that openviewer is written in c# That’s because they’re limited again to using libsl (openmv). libsl recently changed it name by the way which is why i’m bracketing it.

Tara5 Oh: So it doesn’t address the underlying issue of messaging and open API’s for OpenSim.

Yossarian Seattle: Not really. But they have made the wise choice of releasing it under a free BSD license, which will enable more people to work on the project.

Tara5 Oh: Intel is working on breaking out openmv into smaller building blocks and basic types. How will this contribute to efforts to integrate OpenSim with Web 2.0?

Yossarian Seattle: Yes they recently hired John Hurliman who wrote a lot of it. i’m following what they do with interest.

Tara5 Oh: John wrote the original openmv?

Yossarian Seattle: He started the project back in 2006 .

Tara5 Oh: How will the work he is doing on openmv now help with the goal of making it easy to write new clients?

Yossarian Seattle: Well if they provide libraries in different languages that would be a good start and breaking it into chunks would allow anyone writing a client to pick and choose between the function they enable in their custom client.

However I’m not sure that’s tackling the root of the problem still.

Tara5 Oh: ‘cos the root of the problem is the messaging protocols which restrict you at the minute to C# for the client?

Yossarian Seattle: The standards need to be applied at the server end, to make it truly accessible.

Tara5 Oh: And these messaging standards need to allow for more than C# development?

Yossarian Seattle: exactly.

Tara5 Oh: well is seems like something quite doable, just time?

Yossarian Seattle: and careful thought

Yossarian Seattle: A lot of people are focusing on issues such as object portability in VWs but i’m not sure those are the ones to be concerned about right now, the games industry seems to have settled on collada as a standard for that. These VWs platforms are complex beasts and the games industry has already solved a certain amount of problems. However in terms of social interactions the VWs industry is ahead, a blend of games and social media.

Tara5 Oh: But games platforms have not solved either the web 2.0 effects or the web to world have they where things get most interesting now?

Yossarian Seattle: No and the games industry is playing catch up in that sense.

Yossarian Seattle: Little Big Planet will be the game that brings user created content into the mainstream for 3d worlds.

Yossarian Seattle: did you read this article? http://eightbar.co.uk/2008/09/10/moving-cubes-from-world-to-world/ that’s not a hack or anything in there.

That’s a full publish subscribe messaging client embedded in unity3d, realtime events across worlds.

Tara5 Oh: What do you think are the most interesting  world to web ideas that Andy’s house points too?

Yossarian Seattle: Well the fact that the communication is two way, both in and out of world and also that its real time. when something happens in Andy’s real house it happens here too.

Tara5 Oh: Yes I am very interested in the development of  the paraverse!

Yossarian Seattle: There is a personal project Peter Finn has been looking at in IBM, which is actually called Paraverse and is taking real world data including geospatial mapping information and applying it in OpenSim.

Unfortunately our interview ended here, at a very interesting point (I had to go to a panel at the Web 2.0 Expo, NYC). But  James Governor’s post/essay – a superlative ode to the paraverse - prompted by his first look at Microsoft ESP visual simulation platform produced an interesting debate on the potential of the Paraverse in the comments that includes a response by Rob. So check it out!

Metanomics host Robert Bloomfield interviewed Second Life founder and Chairman of the Board, Philip Rosedale, at the Second Life Community Convention in Tampa, Florida.  The Rosedale interview is available here (pictures above are Philip Rosedale and his avatar).

Rosedale talked about Linden Lab’s long standing commitment to open source and open protocols in one segment of this interview and Robert asked me to post a brief reaction. The full interview covers a wide range of topics and Robert has gotten responses on different parts of the interview from Wagner James Au, Christian Renaud, ‘Bettina Tizzy,’ Nic Mitham and ‘Dusan Writer,’ and Benjamin Duranske as well.

A System Without an Owner is A beautiful Thing

While Philip Rosedale’s comments may not, at first glance, appear to be saying anything new, they are in fact a very cogent summary of the important and crucial role Linden Lab has played, and continues to play, in moving virtual worlds out of their walled gardens and bringing them closer to that beautiful thing – a system without an owner.

Only a system without an owner can unleash, for virtual world technology, the kind of creative, world changing power that we have seen on the 2D web from http and html.  Anyone with even a vague idea of the history of the internet understands that it is only through openess, open source, open protocols, open standards, and open APIs, that we will get from here – the alpha days of virtual world technology, to their coming of age of age as a mainstream phenomena.

It is very much to the credit of Linden Lab that, as Rosedale says, they have never been afraid of openess: “I don’t think that the open grid will impact our revenues any more than open sourcing the client,”  he says. While there have been criticisms of licensing choices and ways Linden Lab handles contributions back to their viewer from the community, I think that overall Linden Lab has made very important and visionary moves, first to open source, and now to open protocols.

Open sourcing the viewer at a relatively early point in Second Life’s development created an enormous opportunity for the rapid development of an open source re-engineering of the server side, OpenSim.  OpenSim with the Second Life viewer is the most complete, open implementation of a persistent virtual world.  Without the head start from the open source Second Life viewer, and the connection to the thriving developer community of Second Life, the light speed progress of OpenSim would have been considerably more difficult.

Now OpenSim is getting closer to breaking free from the Second Life viewer. And, standard messaging protocols between client and server are, perhaps, the next step. Rob Smart, IBM, discussed this with me recently (see my upcoming interview with Rob Smart, “Web 2.0 Made Easy in OpenSim,” and see his post by this title for more).

As, Rob Smart, IBM, notes, “If, for example, the messages that went between your SecondLife client and the OpenSim/SecondLife servers was a standard protocol which had a bunch of libraries for a variety of languages, then you could start logging into VW servers from all kinds of clients.”  (for more see my upcoming post, “Interview with Rob Smart, IBM: Web 2.0 Made easy for OpenSim.”

Open Standards Will Emerge From Rough Consensus and Working Code

There are some that subscribe to the view that standards will arise in a virgin birth from an ivory tower, i.e., professors and captains of industry, removed from open source developer communities, will produce long documents that describe all of the fields, and every one of the messages, and all the APIs in detail prior to implementation.

But as, David Levine, IBM. Mike Mazur, 3Di, Mic Bowman, Intel, Justin Clark-Casey, and Adam Frisby, Deep Think/Sine Wave cogently argued, on the “Open Source and Interoperable Virtual Worlds” panel at the Virtual Worlds Conference and Expo in LA, this top down approach to standards, (or “vapor standards”), does not, typically, produce good results. For more on the the virtues of creating standards from “rough consensus and working code” as opposed to top down there is a full recording of the LA panel here.

Thus, in my view, Linden Lab’s current focus on open protocols, OpenGrid (for more see here), and interoperability is another key step towards the creation of open standards for virtual worlds. And Linden Lab are again leading the way in creating an environment that fosters innovation.

OpenGrid creates a testing ground where protocols can be worked out, and it enables the kind of heterogeneous ecosystem to develop that can nurture the creation of standards. I  agree with Rosedale when he says content makers will have an important role in driving interoperability and standards. The creation of standards is certainly a social as well as technical process. And as Rosedale notes content creators will have compelling reasons to move their content around in an open metaverse.

David Levine’s (IBM), described in detail in LA (again see recording here) the importance of interoperability and parallel innovation for the creation of standards. OpenSim has already produced an extraordinary amount of innovation, realXtend, Tribal Media and more. Also see my interview with Mic Bowman, Intel, for more on the role of open source/open standards in fostering innovation and in moving virtual worlds into “the fabric of everday computing.”

While Linden Lab only have a small team working on OpenGrid, it is a vital one.  And, with MarkLentczner (Zero Linden in Second Life) leading the Architectual Working Group for Linden Lab, and a collaboration with IBM led by David Levine (Zha Ewry in Second Life) driving the interoperability effort, plus the OpenGrid project, Linden lab has a high powered, agile, lean, machine working for an open future.

So with no more ado, here it is: Robert Bloomfield’s interview with Philip Rosedale!

Rosedale on Open Sim:  Pandora’s Box Was Already Open

Introduction from Robert Bloomfield

Naturally, a major topic of my interview with Philip Rosedale was on the implications of OpenSim and the Open Grid project, which both involve creating open source server-side implementations of virtual worlds that can replicate Second Life’s funcationality.  As a relative newcomer to this corner of the tech industry, I still find myself asking what a company would essentially create its own competitor.  Here is what Philip had to say; I have asked Tish Shute of UgoTrade to comment, as one of the people who has covered the OpenSim/OpenGrid movement with more detail and passion than just about anyone.

PHILIP ROSEDALE: I just really hold true to the strategic belief that there’s going to be a tremendous amount of consolidation and interconnection between these worlds because the content development process is so challenging that the content developers are going to push us all together. They’re going to say, “Give me a file format. Give me an interchange format. And let me move that chair from this grid to that grid. I’ve got to be able to do that because I’ve got a customer here who wants to buy it.” And so I think that that consolidation is going to happen, and it’s going to happen earlier than people would have thought.

ROBERT BLOOMFIELD:  And this is looking at the success, the energy around OpenSim, open grid.

PHILIP ROSEDALE:  The energy, yeah. I think, at this point, we’ve got an appropriate level of energy – I think that’s exactly the right word – around exploring how quickly we can generalize all this stuff and open and interconnect everything together. I really think that’s going to continue.

ROBERT BLOOMFIELD:  [D]o you feel like you might have opened Pandora’s box and that it’s not really under your control now?

PHILIP ROSEDALE:  I think that Second Life has, in many ways, not been under our control from the beginning and that it’s been a basic operating assumption that to create the kind of incredible place and business opportunity, and social opportunity more broadly, that Second Life would require a certain lack of control. And that was true with the content from day one.

So for us, oh, we open-sourced the client a while ago, and now we’re trying to do the same thing with respect to operating standards to interconnect grids. This is a pretty logical progression, using worlds that we’re pretty familiar with. I mean we’ve always felt that, if you have a compelling use proposition, which certainly Second Life does, in other words, if there’s real utility, real fun or real business or real whatever in what people are doing, then there should be a way, as a company, to be open, global and still make money on an hour-to-hour or a user-to-user basis or whatever on what we’re doing. And the economic aspects of the business have been fantastic from the very early days, and we don’t really even worry about them.

Our ability as a company to find a way to make a reasonable amount of money per hour that people spend in Second Life, it’s really never been that much of a problem. It’s actually been fascinating as we’ve changed pricing and as we’ve changed the ways that we make money. Introducing new ways of making money –  like selling currency on the LindeX – it’s been amazing how stable our revenues have been as a function of usage hours. It’s one of the things that we sometimes marvel at. It’s almost an emergent effect, if you will, that the company’s business, its operating revenues are really very stable.

ROBERT BLOOMFIELD:  Even though they’re coming from different streams.

PHILIP ROSEDALE:  Even though they’re coming from different streams. And sometimes the requirements of the platform and decisions that we make will really substantially change the nature of those streams, but when you put them all together and you divide them by the number of usage hours, it’s like a constant. It’s almost a magic number. And it’s a magic number that allows us to be profitable, and therefore, is certainly adequate to make a business in the future. I don’t think that continuing to open Second Life up as we have been is going to impact that. Again, I just think there are so many opportunities to make money that we shouldn’t have to worry about that too much in the company. And, again, I think that’s a lot like the early internet. I mean if you step back and look holistically at the internet – you look at PayPal, the payment systems, auction systems, transaction systems, posting, naming – you look at all the businesses that comprise the internet, well, those are all the kinds of businesses that we as a company can be in, in this emerging market. There’s no business that’s denied us. We are in the hosting business. We can continue to be in the hosting business long term, putting servers up and providing access to them.

We can certainly be in the naming business. We’re in the currency and transaction support business. It’s funny, it’s something that’s often discussed. We worry much more about improving the scalability, stability and the usability of the system: reducing that initial user experience, reducing the time associated with it, making it easier. That’s got to be the lever that drives more growth in the overall industry, more revenues for us. So it’s really all we worry about. But I don’t think that the open grid will impact our revenues any more than open sourcing the client did.

Tim O’Reilly, founder of O’Reilly Media addressed the audience of the Web 2.0 Expo with a series of challenging questions. I felt happy that so many people I know are already answering this call “to do something worthy” with Web 2.0 and “to make technology that matters.”  However, many of these people were not at the Web 2.0 Expo.  This is, in part, because as O’Reilly pointed out:

if you look at the focus of a lot of what you call ‘Web 2.0,’ the relentless focus on advertising-based consumer models, lightweight applications, we may be living in somewhat of a bubble, and I’m not talking about an investment bubble. (It’s) a reality bubble.”

But as I explored the conference and expo, I did find friends, old and new, dedicated to figuring out how to use Web 2.0 to make a better world.

Caroline McCarthy has an excellent post, on CNET news on the message of Tim O’Reilly’s keynote address.

If you have read Ugotrade before you already know the threads I have been following re the potential for virtual worlds for positive global development and to reduce the carbon footprint of business, so some of McCarthy’s comments caught my attention:

There’s an inherent irony in what O’Reilly said, given the fact that massive conferences like the Web 2.0 Expo are packed with the trendspeak and hype that birthed SuperPoke-like entertainment, and certainly aren’t helping the environment by distributing tons of press kits and swag–not to mention flying in hundreds of attendees in a massive spurt of carbon emissions.

To be fair, O’Reilly Media has been printing fewer event programs and encouraging conference goers to recycle, and it has used carpeting made of post-consumer material.

There is clearly a lot that needs to change, and perhaps the tech industry trend of large-scale conferences is part of it. We’ll see whether Silicon Valley’s leaders and moguls are willing to do what they think is right, rather than what they think is profitable.

But, as Tim O’Reilly pointed out, the huge problems we face today create an enormous amount of opportunity for us to find creative solutions.

“We are going to figure out how to make a better world using the power of the web.” O’Reilly

Virtual World technology will soon, play a major role in re-imagining these tech industry large-scale conferences. There is an talented and dedicated community of open source developers working hard to take this nascent area of Web 2.0 technology mainstream through open source, open standards, and open API’s. I am so proud to be part of this global community!

Virtual Worlds were only a very small part of the Web 2.0 Expo. W. James Au’s “The Post-Hype State of Virtual World Marketing: What Works, What Doesn’t and Why” was the only panel I noticed focusing on Virtual Worlds in any of the main tracks. This reflects the relative lack of integration of virtual worlds into Web 2.0.

One of my passions is to make this isolation of virtual world technology from Web 2.0 a thing of the past (see my upcoming post on Rob Smart’s, IBM, integration of JSON support into OpenSim – which is a vital step towards Web 2.0 made easy for OpenSim).

But, as W. James Au pointed out very eloquently, this notion of isolation is really not accurate, even now. For example, Second Life communities interact in myriad and powerful ways with other social media communities on the web despite the current  lack of common protocols that have kept immersive virtual worlds architecturally cut off from some of the networking effects of Web 2.0.

But, for all of us living here in the US, O’Reilly’s most important message was simple and fundamental. So let’s reblog, retweet, plurk, friendfeed, facebook, send it out on notecards in SL, make machinima, and spread the word in every way available to us.

“Bad politicians are elected by good people who don’t vote.”

“There is no reason in 2008 to do shit you hate, ‘cos you can lose just as much money being happy as hell.”

Quote and pic above from Gary Vaynerchuk’s keynote, Web 2.0 Expo, 2008 (also see Execute on Being you).

“Do something you love,” Vaynerchuk

I personally can’t wait until the potent mix of real time interaction in immersive spaces is combined with the networking effects of Web 2.0. Not just because this will unleash an awesome new wave of innovation and creativity but the early adopters I have met in immersive virtual worlds, and the phenom developers in the rapidly growing open source communities of this emerging technology, have passion, do stuff they love, and do stuff that is worthy, w00t!

And further: “Do Something Worthy,” O’Reilly

While speakers and exhibitors from virtual worlds were scant in the main hall and panel tracks, Second Life had a strong showing in the  “Do Something You Love,” “Do something that need’s to be done” zone of the Not For profit strip. “Create more value than you capture” (from O’Reilly’s keynote) is the natural heart of their mission.

Below is the awesome Evonne Heyning ( InKenzo in Second Life) of Amoration.

What is Amoration?
AMO:  The root of love
~ation:  The state of being, practice and study
Amoration:  living in love, a practice of open engagement
Our Mission:  To Create a Culture of Conscious Compassion

Evonne addressed the problem of the Not For Profit’s being in an out of the way strip in the Expo hall with some very tasty peanut brittle.

In Contrast, the MS Surface crew, in a prime location, playing with super shiny things, did not have to do much to keep a crowd at their table!

I IMed my friend Kyle Gomboy (G2 Proto in SL) from the Microsoft Development Community in Second Life and in OpenSim to ask him if the Microsoft  .net, technet, and c#  developers in OpenSim had any plans to integrate Surface with OpenSim.  I saw the Surface/Virtual Earth integration  and realized Surface with OpenSim would be hotness for a small company looking to develop a vertical for hospitality, medicine, or education. At between 12 to 15K with SDK and two days training, Surface is priced in a range a small company can probably afford.

G2 also came up with a thought that would bring shiny together with worthy when he mentioned to me how great it would be to see Surface in every public school library in the country.

See Rik Riel’s blog for a nice video of the MS Surface demo in the expo hall.

“The Making of Second Life,” W. James Au’s book signing

From left to right, Rik Rik Panganiban from Global Kids (Rik Riel in SL), Joyce Bettencourt, Rhiannon Chatnoir, in SL, W. James Au, (Hamlet Au in SL),  Evonne Heyning ( InKenzo in Second Life), Jennifer Schlegel (Schlink Lardner in SL).

And further more: “Make technologies that matter,” O’Reilly

One of the highlights for me of w2e was getting an inside look at the Interop NOC and meeting Bill “WEJ” Jensen the Troubleshooting Lead of the InteropNETteam (WEJ center sitting at the Mac).

The Interop NOC is a “real” world work of art  – “the largest temporary network in the world” where voluteers have come together with industry leaders to take on the ultimate network challenge – “creating a completely interoperable network using the industry’s most cuting edge technology.”

If you have been reading Ugotrade you will know I have been following the work of Michael Osias, IBM,  (Illuminous Beltran in SL) who has been creating virtual network operation centers (VNOCs) in OpenSim (for more see  here).  I am looking forward to introducing “WEJ” to Michael’s work which I believe foreshadows a new era for software – along the lines Gelertner first envisioned in 1992. Michael follows the Gelertner vision pretty closely.

Gelertner talks about software as an embodied information machine. And, as virtual worlds come of age so will this notion of software as 3d info machines that we can walk around, tinker with, and hang out in with other avatars and agents in real time.

Mirror Worlds will transform the meaning of “computer.” Our dominant metaphor since 1950 or thereabouts, “the electronic brain,” will go by the boards. Instead people will talk about crystal balls, telescopes, stained glass windows, wine, poetry, or whatever – things that make you see vividly.

Software today offers assistance to the specialist (in everybody) not to the citizen. The mere citizen deals with the increasingly perilous complexity of his government, business, transportation, health, school, university and legal systems unaided. Mirror Worlds represent one attempt to change this state of affairs (Mirror Worlds, David Gelertner 1992).

More on VNOCs in an upcoming post.