UgoTrade

Only a two weeks after debuting their first Hands Free 3D video showing the possibilities for navigating Second Life “hands free” without a mouse or keyboard, Mitch Kapor (MitchK Linden in Second Life) and Philippe Bossut have a new demo out - Hands Free Object Editing in Second Life.

Philippe points out on the Hands Free 3D blog that they have already seen a lot of interest in their “hands free” project even from the main press (see this article from the NYT). Hands Free 3D, a project of Kapor Enterprises, is creating a prototypical interface using the 3D Camera designed by 3DV Systems to control virtual worlds like Second Life.

Mitch Kapor told me, they are now working “so that avatars can directly mirror body language and facial expression.”

Mitch very generously gave me an interview in which he not only describes his project to explore how:

the camera could be a central device to a whole new kind of interface the way the mouse became the central piece of hardware that enabled the whole graphical user interface and it enabled the transition from character based computing DOS to the GUI.

But also, Mitch shares some of his thoughts on the future of Second Life. A full transcription follows in this post.

“Moving From Science Fiction to Science”

Mitch explained to me he began to get excited with the idea of Hands Free 3D when he realized:

we had a shot at moving from science fiction to science as it were actually making some of this stuff work that people have been talking about for a long time

As Gwyneth Llewelyn points out much of the so called virtual worlds industry has backed off the bigger vision of a unified metaverse and is retreating into a more limited vision of a multitude of closed and controlled virtual worlds (see Digado’s post Raising Kids in Virtual Worlds and this video from fastcompany.tv to see how this controlled/controlling vision for virtual worlds plays from Disney’s point of view).

But while a bigger vision for virtual environments with a revolutionary role in adult life may not not be interesting to marketeers at the moment, it has a momentum that cannot be stopped. Mitch Kapor made a prediction during the interview that I wholeheartedly agree with:

the big vision of 3D is in the process of happening. It will be very transformative and anybody who is not counting on that happening, is likely to be run over by it.

I got very excited when I heard about the Hands Free 3D project because developing a natural interaction between people and virtual environments to me is one of the “it” projects for immersive 3D.

The dialogue between science fiction and science is of course the ongoing story of the metaverse. And seeing Iron Man which is alive with new possibilities for “seamless interfaces between people bits and atoms” made me think of how very exciting this new chapter in metaverse development is.

The Tangible Media Group, MIT, founded by Hiroshi Ishii has pioneered new couplings of the physical and the virtual. And, alumni John Underkoffler’s vision is definitely in play in Iron Man. Underkoffler’s exact credit flew by me too quickly - but he was clearly a futurist for Iron Man. Matt McGann points out that there is a very cool article about his work on Minority Report here.

Oh I cannot mention Iron Man without noting Iron Man in Second Life (see Massively) and Annie Ok’s latest great machinima!

And, Click on the screen shot below or here to watch the “Hands Free 3D: Second Life Object Editing Demo”

Interview with Mitch Kapor

Tish Shute: How did you get the idea to focus on Hands Free 3D out of all the possible areas you could have begun R&D in?

Mitch Kapor: You were asking me where did the idea come from? It originated in the fact that this kind of difficulty - of creating a natural, easier user interface - that we’ve had is characteristic of virtual world interactions.

There are things to be done about that at every conceivable level. From fixing all the little bugs to a bigger initiative. I was doing a thought experiment about what would really make a virtual world fundamentally easier to use.

I didn’t have an answer, but somebody had mentioned to me - one of the other investors in Second Life - that there are two Israeli companies working on 3D cameras. I had read about and heard about lots of things but this caught my attention. And I started to ask some questions about it. I had seen the video that Johnny Lee shot with the Wii on YouTube.

That had begun to prepare my mind to think about how you could use new types of input devices to control virtual worlds. So when I heard about the cameras I said this is really interesting and I started to make some phone calls and inquire. The Idea came to me that you could use the camera … the camera could be a central device to a whole new kind of interface the way the mouse became the central piece of hardware that enabled the whole graphical user interface and it enabled the transition from character based computing DOS to the GUI.

One of the other things is that I’ve now been around long enough, 30 years - active and professional - that I’ve seen many things come and go and I have a feeling for patterns. So I was fortunate in actually being able to get hold of a prototype of one of the cameras to do some experiments with it.

Tish Shute:
They’re not yet released generally are they, later this Summer, right?

Mitch Kapor:
Well .. it’s unclear. Sometime in 2008 or 2009. There will be multiple manufacturers. They have somewhat different approaches as to how they’re going to go to market. I’d say it’s all being sorted out soon. Everybody I’ve talked to is quite certain that by Christmas season of 2009 at the latest, they’ll be available in high volume at low cost.

Tish Shute:
I just got so excited when I saw you doing this because I think, basically, in terms of free form 3D programmable space which is how I’ve come to see Second Life now, it’s the future. Everyone’s been complaining that the problem with free form 3d programmable space for a mass audience is the difficulty of the interface. So there seems to have been this big retreat back into 2.5D, 3D chat rooms - plugins to Facebook etc. It seems like a step backward to me.

Mitch Kapor:
I think it’s inevitable that we’re going to get fully interactive 3D. It’s all a question of how we’re going to get there and how long it takes. It’s understandable why, for commercial reasons, people do more incremental things, but those are only going to get you so far.

Tish Shute:
Well it seems to me ideas about the evolution of 3D are to some degree being driven by marketing on the web forces at the minute. I suppose the thinking is that you can get these 3D chatrooms up easily and they are more amenable to marketing than a freeform 3D space like Second Life.

But my question is why you didn’t decide to go to game controllers? I suppose this is where a lot of thinking goes because all the kids have already a high level of skill with these?

Mitch Kapor:
Well, I’m not a gamer. It seemed to me that the possibilities with a camera to do the imaging and to be able in real time, to extract out a 3D model of the scene and the objects in it, is fundamentally just incredibly powerful. It feels like the right direction if you can develop it. What I was pleasantly surprised by was actually creating the first demo was pretty straightforward.

Tish Shute:
How did you prevent every random motion being sucked into the program?

Mitch Kapor:
It turns out that the cameras are pretty sensitive. They can detect relatively small motions like the resolution at a distance of 5 to 10 feet is a half a centimeter. That would be one part in several hundreds. maybe one part in a thousand. So it can detect slight motions. I don’t know the details of the software that the camera came with and that Philippe wrote. One of the other advantages is that Philippe, who is the engineer that did the work, has a PhD in computer graphics. And, he has been around the block quite a few times, and had a whole bag of tricks. I know that he spent some of the time writing filtering code to filter out noise in the signal and so on.

Tish Shute:
Do you have to be particular about where you stand at the minute? Can you smoothly go back and forth between when you have to type and things like that?

Mitch Kapor:
No, I’m not anticipating problems. We have another video coming up very shortly where we show object editing. The object editing isn’t as sexy as we would like it because it has to use the existing interface. They’re having to emulate keyboard and mouse. The point is that we have the concept of a control plane, a vertical plane, in front of you, that if you put your hand out so it crosses that imaginary plane, then it interprets what you do as controlling the mouse.

If you push through to the far side than pull it back it doesn’t. That actually works quite well as a gesture. And you get visual feedback when you’re in the control plane, it lights something up, so you can see - OK. It’s sort of like when you’re using the mouse to target an object you can tell tell when a mouse is inside a clickable button. Similarly there’ll be some kind of control zones. When your hand or other body part is in that you’ll get some feedback in the same way that a button highlights to indicate I’m clickable, or you’re over me. It’ll be a similar kind of thing.

Tish Shute:
But you have to avoid ending up with a mapping that’s more difficult to learn than the original one, don’t you?

Mitch Kapor:
I agree with you, but on the navigation and flying, we’ve had people learn to use this in less than 30 seconds. We just stand them up and say lean forward, lean back, stand up, lean to the side, raise your arms, and they’re moving, they’re flying, they’re walking.

Tish Shute:
And you don’t get a problem with the casual motion?

Mitch Kapor:
No. And this was just our first shot at this.

Tish Shute:
I know! I was really impressed that you could actually have done that in 3 weeks.

Mitch Kapor:
I think the start to finish time was a couple of months including the fact that Philippe had never seen the Second Life viewer code. So, he started like any other developer, just downloading and building the Second Life client. And, we never had a camera before! Ha!

Tish Shute:
But this is the great beauty of Second Life - the power that people have to do so many amazing things so rapidly.

Mitch Kapor:
He’s already re-written the code once. We’re totally prepared to give the code to Linden. It’s a little premature because the cameras’ aren’t available, but if the cameras’ were available, we would just donate the code. The nice thing is it’s actually pretty clean. It interfaces to the client at just a couple of points. We’ve isolated the dependencies.

Tish Shute:
But that’s my other question. If you donate the code will it be open source so that other developers could get involved? I know lots of people …

Mitch Kapor:
This stuff, the demonstration stuff, absolutely. That’s the intent. The purpose of this whole phase was just to test what we could do and to promote or evangelize the use of the camera. Get people excited. We’re thinking about what we might do with it.

I’m actually incredibly excited about the thing Philippe is working on now which is to use the camera so that avatars can directly mirror body language and facial expression. So that if I’m sitting in my chair and I have my arms crossed, my avatar will cross it’s arms. If I tilt my head to the side or smile or frown, the avatar will do the same thing. We’re quite optimistic that we can do something compelling in pretty short order, like less than a month.

Tish Shute:
Wow! That is really, really exciting. I think that has just been something people have been talking about a lot recently - to have gesturing and expressions transmitted to the avatar ..

Mitch Kapor:
The reason I get so excited is cause when I started believing we had a shot at moving from science fiction to science as it were actually making some of this stuff work that people have been talking about for a long time.

Tish Shute:
So the plan is to make your work part of the open source community and …

Mitch Kapor:
I don’t have a plan yet. I would say anything we’re doing in this phase we’re happy to give away. At some point I think things are going to become clearer as to the availability of the cameras, what Linden is going to build in, and then businesses that might be built off of what we’re doing.

But I’m very confident that the kinds of things we’re doing now and in the short term are just going to become part of the standard repertoire of things you can do in Second Life in code that’s available to developers.

I don’t have the exact road map.

Tish Shute:
I heard your recent talks in Second Life and how you were very interested in seeing how Second Life could become more of a business tool. I’ve talked about what Second Life and its “cousins” offers in comparison to other open source platforms like SUN’s Project Wonderland and the Croquet platform Quaq. For example, Second Life is a free form 3D programmable space that’s really accessible and easy to develop in.

But in Qwaq you can drag and drop documents in from 2D applications easily, and Wonderland has some great telephony/audio development. I’m totally psyched by what you’re doing because it has the potential to make the free form programmable space of Second Life more widely useful, and it could be bring much innovation to business communications.

I see a future in interactive data visualization, for example, the idea that Ben Lindquist of Green Phosphor has been developing, i.e., that you can actually model business processes dynamically in a collaborative environment. What are your thoughts on Second Life’s potential in business applications?

Mitch Kapor:
One thought is that a more general platform, more general purpose, more open, in the long run, all other things being equal, will be superior to more limited, less capable, more closed platforms, for building any kind of application.

And at the moment, Second Life is the most general and most open platform. So all other things being equal, which usually they’re not, Second Life should be viewed as superior by people who are building a variety of applications.

But there are clearly some things that need to happen. Well let me put it this way some of the other platforms have temporarily at least moved further ahead in enterprise related applications by developing collaboration capabilities.

So the imperative is for Second Life to provide comparable capabilities. It has to do that, in terms of fundamental stability, reliability, in all respects. If it does that then it’s actually going to win on it’s own merits.

Tish Shute:
I absolutely agree with you because in terms of ease of use, it’s the only dynamic networked general simulation platform around. There’s no one else close.

Mitch Kapor:
It’s also I think highly scalable in ways that some other things aren’t. Even though it doesn’t have as many 9’s in uptime as it needs to have, there have been recent signs of more progress. I guess the HTML on the prim stuff is rolling out finally or at least the first version of it.

I think it’s ended in beta now.

It’s not the full thing. But it’s a huge step. That’s going to help a lot.

Tish Shute:
Plus the fact it seems Linden Labs moving towards a more heterogeneous idea of a grid where there’ll be the potential to connect behind the firewall worlds with the main grid .

Mitch Kapor:
I also know that there are some third parties that have done that. They’ve sworn me temporarily to confidentiality. But they have done some very impressive stuff with integrating the web with Second Life in ways that you can for instance in a web interface just go and grab a PowerPoint. In your Second Life window. The power point will just show up. So there is a kind of work around to using the familiar web to get your intercollaboration stuff working. There’s progress. It’s going to be some time before it all sorts itself out.

But to come back to the camera as a more natural interface, I think for personal interaction, is important. It’s going to be a breakthrough.

Tish Shute:
It’s a huge breakthrough also to have the avatar related to your real life gestures. It’s a huge leap forward. When you introduced it at metaverse meetup that really got people’s attention. I have a question. Have you thought about going even further with the thought driven game controllers?

Mitch Kapor:
At some point I intend to take another look at that. I have the feeling that your not doing anything highly profound. Kind of a cute hack.

Tish Shute:
Again they’re not available, I would guess they would give some to you though.

Mitch Kapor:
From looking at earlier incarnations of this stuff I think what they can pick up on is very superficial. So I’m not sure that they’re going to be that interesting cause we really don’t know how to do, without some invasive type of surgery,

Tish Shute:
You can do it with very very complicated brain scanning you can do a lot more, but I agree. Although I did see the Japanese University was using them for severely disabled people. Looked like they were doing some interesting things.

My question is, this is something you mentioned in one of your talks in Second Life, you thought some of the steps forward to make Second Life truly a player in the business world, would be changes on the server level. Were you thinking more about the moves that are going on towards open source and making a heterogeneous grid?

Mitch Kapor:
Yes. I was thinking about letting people run it behind the firewall, and also it’s not just putting it behind the firewall, anytime you’re talking about an enterprise application, the enterprises want to integrate all of their existing IT systems. They already have these very sophisticated systems for managing say identity, and having easy integration of those thing with Second Life identity management is not glamorous but very important.

Tish Shute:
This brings to mind another question. I know I have some ideas about what Second Life really brings to the table for business. No one else has taken on working with dynamic melded states on the internet in 3D to the degree Second Life has. That’s sort of, to me, the essence of it - having groups of people working around 3D objects that can be updated on the fly and modeled on the fly.

Mitch Kapor:

If we do things well there will be a good level of interoperability and all of the open source work and the reverse engineered clones will actually be a good thing.

Second Life is, and I’ve probably used this line, faced with insurmountable opportunities on all sides.

Let me ask you a question. I’ve read your blog, or some of it, but what do you actually do?

Tish Shute:
I spend a lot of time on my blog at the minute!!! You can tell I have kids and dogs driving me crazy [dog is barking in the background], which is exactly why I took this up a year ago. I worked in film and special effects for the early part of my career.

When I had my kids and dogs and all of that it got to be just too much to do 24/7 film production. My son’s nearly 9 now. I tried academia for a while, then I just said forget it..too hard to be in a medieval guild as a second career!

And I actually a year ago when I started looking at this (Second Life) I thought my goodness this is what we sat around and talked about every night when we were doing multiple pass motion control photography in the eighties. And so I started writing about it and that took a life of it’s own. And now it’s become a little ridiculous because it’s an excessively time consuming hobby!

Mitch Kapor:
Are you in New York or the UK?

Tish Shute:
Yes. I’m in Manhattan.

Mitch Kapor:
The reason Second Life has gotten as far as it’s gotten is because of people like you who have become inspired and become obsessed and feel the possibilities and feel them to be so utterly compelling to cause some rearrangement of life priorities.

Tish Shute:
It’s interesting cause it’s like every week I say “Oh I really can’t spend all this time writing!” Then I see something, like this week I saw all the new wave of 3D chat rooms coming out. And it just got me going again! I just can’t bear to not to have a voice because when you see the big picture you want the really innovative stuff to move forward. That’s why when I saw your work on hands free 3D, I said: “Oh my goodness, someone’s taking it the next step. And as you say there isn’t a path that’s clear. There’s no guarantees. But its a path worth traveling, in my view!

Mitch Kapor:
I firmly believe, I have complete conviction, that all of the 3D, the big vision of 3D is in the process of happening. It will be very transformative and anybody who is not counting on that happening, is likely to be run over by it.

Tish Shute:
Right. Of course you’re much more knowledgeable of this aspect of it but in terms of business applications, has anything interesting happened in a long long while?

Mitch Kapor:
There are some interesting things that are happening, I just learned this by accident, that are being kept under very close wraps. There’s at least one consultancy that is doing extremely well with very large prestigious global corporations. They have done a lot of development of this integration of web with Second Life. Their clients are shy. They do not want public exposure at the moment because of the backlash against the overhyping of Second Life that happened last year. I was very heartened to hear about this. I think it’s going to start coming out in the next few months what some of these companies are doing.

Tish Shute:
I agree. Many of the interesting things I know about I can’t write about either because there’s no interest for people developing business applications to have a lot of web publicity about it in the early stages.

Mitch Kapor:
Right. I think we’ll be in this phase for a while. But then we’ll get out of it.

Tish Shute:
In terms of specifics about business application, do you have any dreams for Second Life?

Mitch Kapor:
I would like to just personally have a really good meeting application. Just simple like when you and I want to get together and meet in world, I would like that to be easy, bullet proof, convenient, natural. I’m imagining that we both have cameras, so that we can see each other and you get body language and you get a sense something like what you would get in a face to face meeting. And I want people to have the ability to easily get more realistic avatars, if that’s what they want. And actually there’s a lot of good technology around that now. Where you can just basically take a picture or two with an ordinary digital camera, upload it and get back something that pretty much looks like you.

Tish Shute:
What do you think are the biggest obstacles to this kind of free form 3D programmable space?

Mitch Kapor:
There’s a lot of software that has to be written to bring out its full potential. And not just by Linden or any one company. It’s really a collective effort that is the work of a whole generation.

It’s comparable to all of the work that went into making the ecosystem of the personal computer. Or for that matter the ecosystem of the internet. It requires having the right architecture, it has to stay open. If that can happen I think it’s mostly just a matter of time and some patience.

It is going to happen. There are lots of individual challenges. Tons of problems to solve. I’m not a technological determinist, but at this point I don’t think anything can hold it back.

In a way though having lived through the onset of the internet, while it has changed things a lot, and in certain ways it would be very difficult to imagine life without it, it also has left things the same. I mean people bring all of themselves and their issues into every technological medium. The drama gets played out in a different ways, but it’s neither going to be a good thing or a bad thing. It’s going to be some of both. And so the question is, to me, how people of good will who want to make the world a better place are going to use whatever new things get created in a positive way.

Tish Shute:
I know Mark (Zero Linden) heads up a lot of interoperability work in his office hours and other meetings. But I got a couple of emails this week saying that all these groups that are working off of either clones or reverse engineered, and there are so many of them, and some are under wraps too, need to actually meet on an even more regular basis?

Mitch Kapor:
That’s true. I guess it’s much more desirable for people to meet and talk, and if they don’t for awhile, you get more noise in the system. It just will take longer to put things back together.

Tish Shute:
That’s what I was thinking, that it’s become pretty clear to me that cooperation, if it is going to happen, has to happen around the clones and the reverse engineered versions of Second Life because other platforms are not prioritizing interoperability at the moment, that I know of.

Mitch Kapor:
People will call - this and that should be happening but my view is that the ecosystem is still sufficiently underdeveloped that there is a risk of attempted premature standardization.

If you look at the history of things, It’s very important for there to be working instances before anybody attempts to standardize anything.

There’s a lot to be learned in the early history of the internet. pre-history, from the 60’s up through the 80’s — when the basic protocols were being developed. There’s some very smart people working on that and a certain amount of looseness is actually quite important now.

There’ll be people who want to prematurely standardize and get everybody together and all you’ll wind up with is a massive crud.

I thinks that the power of the open systems is so much greater than the walled gardens Also the open source ethic is so deeply established in large parts of the development community, even in enterprises, that overall I’m not too worried about it.

When the functionality of whatever it is, is that well known and well understood, that’s the period in which the open source alternatives can really flourish. When there’s still a lot of evolution in functionality, and design in the user experience, open source techniques can become too slow.

So it’s going to be somewhat chaotic. I think we have to embrace or at least make peace with a certain amount of chaos right now and the understanding that it’s likely to settle down. The chaos is not the last word.

Next week there will be a key event for Virtual Worlds:

Federal Virtual Worlds Expo: Implementing the Future. The picture above is from the November event.

The Federal Consortium for Virtual Worlds is a newly formed group of federal government employees and contractors interested in exploring the use of virtual worlds in government, sharing best practices and policies, creating shared repositories, and networking.

April 24-25, 2008

National Defense University Washington, DC

If you cannot attend the meeting physically, you can attend through Second Life free of charge at the following NOAA SLURL: http://slurl.com/secondlife/Meteora/199/138/28

There will be a series of pre-conference workshops in SL, April 23rd, led by Pathfinder Linden, John Lester, Sue Singer, Aimee Weber

One of the speakers will be Jeanne Holm—NASA. Jeanne’s talk title: “Exploring the Heavens on Earth” inspired me to write an upcoming post, looking at the work of NASA and space enthusiasts in the the “free-form three-dimensional programmable space” of Second Life.

Juha Hulkko’s Second Life avatar, Jussi Hudson relaxing in his “mirror house” in Second Life - photo by Noelani Lightfoot.

The story of Juha Hulkko’s vision for realXtend, the future of virtual worlds and the next generation of the internet, is fascinating and several conversations are combined in the following interview. You can read more about realXtend’s road map and accomplishments to date on their web site. But to introduce the interview here are some of the key points about realXtend first.

Juha emphasized to me that realxtend is a joint effort with many talented people all over the world. He stressed that his personal role is to help and support these people and that it is their talent and skill that is making this vision happen. Juha especially wanted to emphasize the importance of Linden Lab and the OpenSim project. Their innovations and innovative work are the basis for realXtend too.

They are role models to all who are building the future web and they have already made history in computer science. RealXtend is one important spin-off from their great work.

Overview of realXtend

Click the screen shot or here to play the realXtend avatar technology demo also see the realXtend technology video here.

The emphasis of realXtend is real life applications: enterprises, public sector - e.g. environmental management and protection, interactive and community based (socially networked) communication, extended web, integrated communications - multimedia and 3d.

Open Source: basic servers and clients are platform and open source, platforms have no long term business value. Rex sees the value is in content, application development and support, hosting and better customer access/service.Value change innovations: e.g., content directly from content creators to end users (no middlemen needed).

Rex sees a new roles for telecomm, music, movies, advertisement creating huge possibilities that will change web. Social innovation, public and enterprises should inform the direction of the next generation of the internet. Rex takes a pragmatic approach, i.e. that is looking for real needs and keeping eyes open for better solutions: “We try to avoid “not invented here” syndrome all the time in our own doings.” The vision is for a smooth transition from traditional web to extended web (no revolutions are possible, nothing changes over night).

The Vision from realXtend is a robust OpenSim server (quality of service core issues, branches from OpenSim are maybe needed).

Separate avatar servers with identification servers (to enable same avatar usage in different applications), multi platform clients (from light, browser based to heavy duty 3d+), customizability of clients. This Rex points out is very important as clients have to be modified from servers in real time too, (as in Rex). In the realXtend vision serious applications will coexist with social applications (like SL), connected worlds, neither alone is enough.

Funding: Funding for realXtend is through a private, nonprofit foundation, 3 year program is under preparation.

Interview with Juha Hulkko

Picture of Jussi Hudson (Juha’s SL avatar) by Noelani Lightfoot the proprietor of Quixotic Photography in Second Life (see more of her great work here).

Tish Shute: Could you tell me about the vision behind realXtend and how you got involved with virtual worlds and with developing the next generation of the internet?

Juha Hulkko: My fathers fathers and his brother were the first radio amateurs in late 1920s, as schoolboys they built small transmitters / recovers and started to make contacts all over the world first with Morse code (cw). My father and brother started as school boys. I started actively after returning from army 1975 and been more or less active since that.

Ham radio operators were the first new era global community. The first ones started in beginning of 1900 century. Normal people with a technical background built simple radios with antennas and were able to make friends around the world. They were pioneers of modern wireless communications. And social aspect was there already since the beginning. In North Finland schoolboys could make friends from USA to NeXeland and keep contact with them on regular basis.

That was the first internet to me, and that has inspired me a lot to be a part of helping people to build these global platforms to enable people to be better connected, and to enable real global friendships and understanding between people. Without communication, there will not be deep understanding and respect of other cultures and people. This is almost totally missing from my generation (except radio amateur operators ). We just assume we understand global things by listening to the news or other people’s stories. That has very little to do of understanding global things. Our new generation have real friendships around the globe and that is far beyond what my generation ever had possibility to create.

Juha Hulkko’s home in Oulu, Finland in realXtend and Juha at home in “real” life

I’ve been in the communications area all of my life and I’m still involved in that, but not actively anymore at the corporate level. I’m board member of a couple of companies, yes. But, when I stepped out from my position in my company one and a half years ago, I decided to explore my life long interest in the internet. I’ve been using the internet in my work, of course, but I’ve never had time really to explore the possibilities for innovation. I was just ordinary internet user. But, I was exploring and researching things and then I ran into Second Life.

I got quite interested very suddenly because I realized that this is something which is really important. I realized very soon that it is, and I still think it is, one of the greatest social innovations in decades. I could compare it to the internet when the internet started or when this wireless communication mobile phone boom started. It’s kind of same kind of revolution which will happen with virtual worlds. This is not technological innovation, it’s social innovation. This will make the World Wide Web interactive, social, so you can create your own interest groups there, and then combined with 3D and all the media things, it’s all this combination, it’s a huge thing.

But very soon I realized that Second Life was kind of hopeless for integrating real world applications. It was a closed world which means that 90% of the people I met, they wanted to isolate real life from Second Life. That was something I was thinking about quite a lot. I understand the motivation behind that, but somehow I realized that this is not really something with a big interest for me. I started to think very fast how this virtual world, in this case Second Life, really could be a kind of extension of real life.

Juha and Jussi outside their homes in RL and SL. Second Life picture by Noelani Lightfoot.

I understand completely why some people want to isolate their experience from real life. But I was thinking more of how I could utilize these virtual worlds for real life needs. I started to speak with people and when I meet new people I ask them one very simple question: What is your dream? Why are you in second life? And over 90% of the people answered me that their motivation is to explore and have fun. Look around, see different things and have fun.

And I said to those people, that’s great, you should do that. But, I also located some people who I saw that they had things they wanted to do connected to real life. They can be really simple things. For example, you are designing clothes in real life and you want to create them in Second Life and sell them to people and see what goods are good to sell there in Second Life, and in a real life shop. This is a very simple example. This is the way I started to find very eager active people and I made friends with them.

I also realized that one problem which I don’t personally like very much. In Second Life they are selling land to young people with very, very high prices. So I bought some islands there and I started to invite these young people to the island and said to them here is free land, do whatever you want. Of course there were some people who weren’t willing to do something useful. But I said OK but I hope that you would be part of our thing but regardless, if you get some sale wherever in real life or second life still I’m not asking any rent or any revenue.

I wanted people to do what they want but I wanted at the same time those people to be part of our Arkala community. Still things started to develop and we created a community of virtual friends called Third Eye. The idea behind Third Eye was we wanted to break the barrier between Second Life and Real Life. We started to think of different kinds of technical solution, e.g., sending sms between Second Life and “real” life. We can even control a huge radio station in real life from there (see picture below ). That’s the magic motivation for me. I have been in telecommunications, I have been an entrepreneur for 25 years so I have no business interest for myself. I just wanted to see can you really break these barriers.

We had these communication concepts we made a project for vanishing species. These animals are vanished from Real Life, e.g. The Chinese River Dolphin which is extinct. You can see it on our island and communicate with it. We wanted to show people what it means when we are killing animals, whole species, in Real Life. By seeing them here in Second Life we hope people will appreciate more what is being lost and start thinking.

When we called the Chinese River Dolphins made by Nicolas Sordello a student of Ecole Nationale Supérieur des Arts Décoratifs de Paris came to us. Picture by Noelani Lightfoot.

“Save the planet” sounds a naive approach, but behind it is a growing worry about the state of our planet. That is why my dream is to use Virtual Worlds as tools to enable worldwide open communication on environmental issues. Not communication dictated by any governments, big companies or any single environmental institute. All people and organizations have equal opportunity to create change and right to contribute and offer their opinions and share their solutions with other people who are interested on these issues. It is a very powerful tool, I believe. We simply should leave this planet behind in better condition to future generations, better than the conditions we were born into. We need to start the wheel turning in a good direction. It is our generations responsibility and legacy to future generations.

Kilpisjarvi - place of great beauty where Juha tries to spend as much time as possible

I have been in global business for many years but I like the people I have met in Second Life this is something completely different they do this for love and not just for money. But pretty soon I realized we need an open platform and it has to be scalable so that you can create your own virtual world on your laptop or desktop.

I found a group of young talented people. As you know we have much technical talent as you have met some of them from Admino and Ludocraft (see my interview with Tony Manninen of Ludocraft here). We have many people from wireless telecommunications, a big base of computer scientists, engineering. I approached these people as said that in 6 months time I wanted to have these things happen. And now we have done this server concept which is scalable up from one pc all the way up to a big ISP hosted platform, so it is following very much the basic structure of the WWW not the Linden model of the closed world.

I understand why they did this and they have done a lot with this model of micropayments. But for real life applications it is not so much how much people are selling virtual goods to each other but how companies are selling content and services from virtual world to real life people. That is more realXtend’s focus - real life applications. We don’t want to compete with Linden we want to coexist with Linden. RealXtend is a free platform so of course anyone can do what they want with it, but it was originally designed to serve real life needs.

Kilpisjarvi Reindeer

A tour of Arkala Island

Over the last few weeks I have been fortunate to meet many of the talented people who are part of the realXtend team (see my post here about Tony Maninnen of Ludocraft). It was a great delight to be show round by Prof Lunasea the Second Life avatar of Professor François Garnier of Ecole Nationale Supérieur des Arts Décoratifs de Paris who is doing some extradordinary work with his students on Arkala - check it out both on Arkala Island and see their awesome films here.

A screenshot from a short film by de Grégoire PIERRE.

Spade Dagger the Second Life avatar of Tommi Hollström of Admino showed me some of the work Admino has been doing as the part of the research and development for the reaXtend project. Admino Ltd is a company established by two brothers, Hannu and Tommi Hollström. Company started in June 2003. In the beginning Admino was concentrating in ERP systems and Microsoft products. Admino Ltd was actually third business that the brothers were working together. Tommi show me the Second Life control center for Radio Arkala.

This is 3d user interface for a real ham radio in arkala radio station. The user can use real radio, change frequency, hear radio, and talk. This is an easy way give anyone chance to try it out. This was first project where we did to extend reality :-). But there are other talented people involved too. Arkala Island is an island of innovations.

Radio Arkala in Second Life and Real Life

Spade also showed me a project to sell mobile content using L$.

When the user buys content and types in a phone number he gets content directly to mobile phone. The benefit of this from a business point of view benefit is we don’t use visa or sms payment so we save about 30% mobile operator payment in Finland. Usually the operator takes about 30%. We also are also researching selling videos or renting videos.

With OpenSim server we can male better integration examples for mobile content selling systems. So we can use those databases directly and maintain preview pictures prices, product codes there. Now we have import all preview pictures to SL objects and maintain those here. Its a big problem when we have for example 10000 products.
But the good part about SL is that here is a global currency and it is good for micro payments.

Our biggest problem now in SL is we cannot offer extranet or intranet solutions (we have lot ideas in that area), but
but most big companies won’t put their intranet to SL servers. So we need overall security and a global avatar server, DRM, payment integrations, paypal, itunet, ovi etc. We don’t need make own currency or payment. For example, if we make “3d interface” to itunes, we just need api to use itunes payment.

The Key Issue is the Avatar

Juha Hulkko: One important thing we saw was - not only did everything have to be open, but that the key issue for this was the avatar. You always have to generate a new avatar for a specific game/virtual world.

The rational behind our avatar server which we are going to publish tomorrow is that for human life for hundreds of thousands of years the interface for how people communicate with each other has evolved in a certain way. We speak to people a certain way, we have body language which we use when we communicate with each other. But during the last 30 years, thanks to Mr. Microsoft and some big companies, they have taught us to sit in front of screens and use computers in a very very certain way. You have to use menus and you have to have to try to figure out what is the reasoning behind this web page before you can buy something, or how you can find information. But we simply saw that why we should not try to reinvent the wheel because Mother Nature has already developed a human interface which we use in real life. And that’s why we decided that it’s time to create completely new avatar concepts.

So if you want to be in the World Wide Web in the future you have to be there and communicate with surroundings as much as possible in the same way you do in real life, because that’s something everybody knows, you don’t have to learn. Of course, children have to learn to speak. But to get back to the natural human interface is why we wanted to make this global avatar project.

We started from very basics. For example, we started to design the avatar for 220 bones. But this physical part is only very small fraction of human life. It’s like in human life you can see people but that is not everything. So we are adding a lot of new features like mimicry. It means if I’m sitting with my laptop I have a small camera here if I smile, my other person smiles. If I cry the avatar cries.

This is very simple example. But you know from artificial intelligence research there are many people in the world who are now looking at these kind of things and looking at how people really interact in real life. We try and make everything open technically so everybody can add new kinds of features to our avatar. For us really the basic idea is to make this kind of avatar platform that enables everybody, who wants to, to embed different kinds of things in this avatar. We are doing this to create a defacto standard for virtual worlds which is based on open interfaces so that people can do what they want with it.

We have built the technology for monitoring almost everything, almost everywhere and we are making 99% of it open-source. Thanx to Bruce Sterling who inspired us, we called our technology “OpenSpime.” This is a concrete opportunity to monitor the earth, and everyone on this planet will be able to contribute to this.

OpenSpime have prepared a concept video of CO2 monitoring (still from the video above) and Google maps mashup via their OpenSpime infrastructure. For more see, co-founder, David Orban’s post OpenSpime: What do you know about your planet?

Instrumentation versus Surveillance

Surveillance is all about when people in authority know a lot about you. Instrumentation is when you know a lot about the world. And it allows you have more agency. When people know a lot about you it takes away your agency.
Cory Doctorow, Craphound.com, boingboing.net

OpenSpime and SL Data Viz project (a project in Second Life “where people can contribute, review and copy open-source data viz tools” which OpenSpime will participate in) are taking up the challenge that Bruce Sterling made at end of his visionary book, Shaping Things. Sterling gave an imperative to humankind to start to make “decent technology” - social software entities that can answer questions. Questions about our world. Questions about objects. Not the profit-centric questions - serious questions. (Shaping Things).

The protagonists of this new era of social software - the narrators of the instrumented world to come - are Spimes.

Sterling invented the term by compressing ’space’ and ‘time’. Spimes are aware of their environment, they know where they are, and when they are, and keep track of some parameter around them. Sensing, memory, and ubiquitous communication enable spimes to accurately map the physical world around them. The progressive saturation of the world with spimes is creating what is called the Internet of Things.

The closing words of Sterling’s great visionary book on “spimes” and “the internet of things” are:

Its not enough to think about that, or even write about. If it is to be any use to humankind, it will have to get done.

OpenSpime is doing it! SL Data Viz project is doing it!

If you are not yet in tune with “the internet of things” here is a music video Royksopp’s “Remind Me” that Sterling suggests is some kind of “spime” theme song. This is the third time I have posted this link but Spimes deserve all the air play they can get!

Our technology enables individuals and corporations to better understand their environment, through the use of a series of GPS-enabled sensors. We provide a set of open APIs and communication protocols to manage the data collected.

See an interview with the founders of OpenSpime here.

With free hardware, free software, open APIs and communication protocols, OpenSpime’s business model is about the provisioning of the SpimeID identification numbers for the trusted communication of validated data streams between spimes and the OpenSpime servers.

Here are David’s responses to questions about the revenue model:

The revenue model is based on the sale (in large quantities to the hardware makers) of the certified SpimeIDs. Anybody can build spimes that conform to our open specifications, but if you want us to validate the data that your spime collects, and aggregate it together with other trusted sources, than you have to pay €1 per device to get the ID. Ah, and we already have LOIs for several hundred thousand IDs to be built in devices. [smile]

The estimates are that there will be ten or more spimes per person in the developed countries within ten years. That is approximately 25 billion spimes, and counting, since by 2020 the number of people who live in countries we define developed will also increase. That’s a
market for you! And we want to define the way the data generated is managed.

What comes after humans twittering to each other? Spimetalk. And spimetalk is going to be several orders of magnitude more intense than any human2human or human2machine communication before that.

From Spimes to Mirror Worlds

Agents, avatars and spimes will eventually hang out together in Virtual Worlds, interacting in real time in networked virtual environments built of live data and 3D info machines.

Melanie Swan points out:

Virtual Worlds have been used for architectural builds and interaction and the next obvious step is making them alive with data, streaming in data and representing it visually. Data visualization in Second Life is starting to take off with an open-source movement to make open-source building-block tools available to the community and developers and end users creating specific-purpose enterprise and science applications. There is a community wiki and a “Data Visualization” group in Second Life.

The picture above is of a Bashiba Panorama - “a commercially available ambient data panorama that dynamically reflects the overall “mood” of a rich information environment. The so-called “data atmosphere” reacts in real-time multivariable changes (e.g. stock market data), that are then mapped to visual counterparts (e.g. ocean waves, sun strength, wind speed, rain). the resulting information display can be interpreted ‘without cognitive effort’.”

“once the stock market opens, our 3D simulation comes to life, & people start ‘breathing’ business information.”

The streaming Bashiba Panorama in Second Life is an experiment in collaboration with Melanie Swan (MS Futures Group) and powered by the Ohio University VITAL Laboratory streaming server (many thanks to Dr. Chang Liu + Stephen Carroll). Look for Bashiba to come to Second Life in full immersive 3D!

For videos of Bashiba in SL see here and here.

3D Command Centers the killer app of the 3D Internet

A Mobile Tactical Data Comm Unit built by Illuminous Beltran in OpenSim

Data visualization will develop increasingly into and become inextricable from operation/command centers - the 3D information machines that will reinvent the relationship between humans and the up to now invisible but most crucial layer of modern society - software. For more on Illuminous Beltran’s, (a.k.a Michael Osias, IBM) virtual operation centers/command centers in Second Life/OpenSim see here.

These 3D command centers are being used for energy management, virtual network operations centers, and for building automation (see the work of Eolus). But, as long as our society is wedded to war, the “killer app” of the 3D internet will also be a killer app in more ways than one. The instrumentation of society has grown up hand in hand with surveillance and military technology. But it is up to us, as a society, to grow the peaceful and people centered aspects of this technology.

The nascent spiming technology of RFID had its roots in military innovation.

In 1946 Léon Theremin invented an espionage tool for the Soviet Union which retransmitted incident radio waves with audio information. Sound waves vibrated a diaphragm which slightly altered the shape of the resonator, which modulated the reflected radio frequency. Even though this device was a passive covert listening device, not an identification tag, it has been attributed as a predecessor to RFID technology. The technology used in RFID has been around since the early 1920s according to one source (although the same source states that RFID systems have been around just since the late 1960s).^[2][3][4][5]

Similar technology, such as the IFF transponder invented by the United Kingdom in 1939, was routinely used by the allies in World War II to identify aircraft as friend or foe. Transponders are still used by military and commercial aircraft to this day. (for more see Wikipedia on RFID)

But, Gelertner’s vision in, Mirror Worlds 1991, is the transformation of computers into seeing machines that will empower people to understand and work with the machinery of their society. And with projects like OpenSpime everyone can contribute to the task of asking important questions about our world. And as these questions are increasingly incorporated in virtual operation centers, we will be able to interact with the previously invisible machinery of our modern world - tinker with it, hang out in it with other avatars, and agents, and command it in new ways.

Life2.0 Summit Spring ‘08

Life 2.0 is the leading event on virtual world application and business development. Produced three times yearly by CMP in Second Life, the six-day virtual event draws a fully-registered, global audience of over 1000 software architects, creatives, CMOs and key executives seeking to harness the power of virtual reality for marketing, application creation, commerce, education, and to connect with the Net’s fastest-growing, smartest, most-engaged communities.

Also check out CMP’s John Jainschigg’s (John Zhaoying in Second life) post, “Life 2.0 - As green as Five Brazilian Households,” which demonstrates “that importing certain kinds of real-world activities into virtual reality saves a bagload of carbon. Or put another way, that it enables us to enjoy the benefits of global human community at small cost to the planet.”

Eben Moglen to join IPPI panel at Life 2.0 in SecondLife

The Kickoff Symposium will Discuss Opensource, IP and Privacy/Identity in Virtual Worlds On Sunday, March 16, at 1 PM PST.

Life 2.0’s IPPI (Intellectual Property, Privacy and Identity) symposium will kick off with a rousing panel discussion on Opensource, IP and Privacy in Virtual Worlds — including special guest Eben Moglen, Director, Chair and Chief Counsel of the Software Freedom Law Center. Moglen, professor of law and legal history at Columbia, is a pioneer of the opensource movement, former general counsel for the Free Software Foundation, and one of the architects of version 3 of the GNU GPL.

The panel (still growing!) will be moderated by Tish Shute (Tara5 Oh), who blogs on virtual worlds (hey that’s me!), and CMP’s John Jainschigg (John Zhaoying).

Also see, A Conversation with Eben Moglen on Second Life.

There will be a Data Viz panel at Life 2.0, Monday, March 17th, 2pm to 3pm PST, organized by Melanie Swan (more on this soon). See the impressive list of data visualization tools that SL Data Viz project has already gathered together below.

Directory of Data Visualization Tools in SL:

(excerpted from the sldataviz wiki)

Interactive Data Exhibits:

• Embodiment Island Data Visualization Exhibit Hall, SLurl, contact Mark Dubin/ThreeDee Shepherd to exhibit material
• NOAA real-time U.S. weather sim, SLurl
• Daden Prime real-time U.K. weather sim, SLurl
• Daden Prime real-time LAX Air Traffic Data, SLurl

Tools - scientific:

• CAIA - Cheminformatic Assisted Image Array visualization laboratory. Status: Available to view (Gus Rosania/Caia Alter), SLurl
• Hiro’s Molecule Rezzer - Rezzes molecules from a notecard. Status: OpenSLedware. (Andrew Lang/Hiro Sheridan) ONREZ SLurl
• Histogram (20 period, real-time, on-demand stock market data), SLurl. Status: Open-source download available (Melanie Swan/Xantha Oe and Eloise Pasteur)
• Orac - Takes smi, InChI, or inchikey as input, queries three web services and rezzes the returned minimized structure in SL. Status: Avaliable upon request. (Andrew Lang/Hiro Sheridan)
• StoryMachine - Generic visualization tool for dynamic interactions. Status: In development (Peter Miller/Graham Mills), SLurl
• StoryMachine - PubMed Search. Status: In development (Peter Miller/Graham Mills), SLurl
• Protein Rezzer Toolkit - In-world rezzing of 3D protein backbone structures based on specially parsed PDB files. Status: In development (Peter Miller/Graham Mills; based on scripts by Troy McLuhan).

Tools - enterprise:

• Airline flight booking - connecting Second Life to a SAP NetWeaver system to create a 3-D Business Process (Bjoernaeon). Also see Craig Cmehil’s blog.
  
• Green Phosphor, New Jersey, USA - enterprise data visualization applications (Ben Lindquist/Arkowitz Jonson, David Cooper/Coop Upshaw)

3D information machines, Bashiba, building automation in virtual worlds, energy monitoring in virtual worlds, instrumentation versus surveillance, intellectual property, internet of thins, killer app of the 3D internet, Life 2.0, Mirror Worlds, open source data visualization, open source data visualization tools, openspime. CO2 monitoring, privacy and identity, RFID, Shaping Things, software freedom, spime, spimes, spimetalk, virtual command centers virtual network operation centers

Tony Manninen, the CEO of LudoCraft games studio (the client side development division of realXtend) who has being doing all this amazing recent development on OpenSim, has a vision for Avatar 2.0 that he is bringing to OpenSim. The possibilities for the future integration of realXtend features (that include meshes and the ability to import proper 3D models) with Second Life is currently under discussion - more on this soon.

We have tried to keep the rexviewer as compatible as possible. We totally appreciate what Linden Lab has done and we are trying to do our best to co-exist with their beautiful social innovation.

When I interviewed Tony (see interview later in this post), I was very excited not only by the scope of his vision and his devotion to enhancing the user experience and possibilities of virtual worlds, but also by his absolute determination to manifest this vision in code with the the utmost speed

The next release of realXtend server and client will be published on 29th of February, 2008 at 4pm (Helsinki time). The features that will be in this release and a roadmap for the future are here.

Virtual Worlds that make our “real” and “virtual” lives better

What do I mean by an exciting vision for virtual worlds? There are many visions for virtual worlds floating around and some of them are not so appealing to me, like the notion of a virtual world as a walled garden/company town put forward by the CEO of There.com, Michael Wilson.

When Michael Wilson was interviewed in Second Life by Beyers Sellers (Robert Bloomfield, Cornell University) for Metanomics this week, he gave the impression that creating a controlled environment for marketing is the number one priority of There.com.

In my view, which seemed to be shared by many of the Second Life audience participating in the group chat, this notion of designing virtual worlds as “big brother houses” is the epitomy of “the metaverse gone wrong.” You can see the full interview here on SLCN. Also see here for Christian Scholz’s (Tao Takashi in Second Life) excellent summary and analysis of a virtual world according to There.

I hope there are some more positive aspects to There.com than this interview conveyed.

Despite ups and downs and growing pains, Philip Rosedale’s vision for Second Life has always, in my view, had at its heart the motivation to make people’s “real” and “virtual” lives better. See The Making of Second Life by Wagner James Au for a fascinating look at the early days.

While Linden Lab is still working hard to break free of the “walled garden” model, with OpenSim and realXtend, the expansive vision that they have pioneered has been open sourced even before they have finished their own open sourcing project. And now, an open source development community of genius and depth (including Linden Lab) is rising to the challenge of taking this vision of making our “real” and “virtual” lives better to the next step.

There are several Open Source virtual world platforms available now, including Ogoglio, Croquet and Sun’s Wonderland. In the news today is the Open Virtual Worlds Project from The New Media Consortium (NMC) and Sun Microsystems which as VWN points out is similar to efforts announced by the Immersive Education Initiative to create an Education Grid across Second Life, Wonderland, and Croquet.

But OpenSim stands out from the pack because it inherits from Second Life an awesome set of tools to facilitate user generated content. This emphasis on user generated content is, of course, key to how Second Life has become the largest and most highly developed 3D immersive world to date. The fact that OpenSim is a platform built for people to build in was of one of the reasons that realXtend chose to work with OpenSim after they investigated all the available open source options. Also important, Tony noted, was the depth of the development community in OpenSim and Second Life.

Philip Rosedale speaking in Second Life at the MacArthur Foundation event on The Future of Philanthropy in Virtual Worlds

The Future of Ubiquitous Game Design

There are many inspiring alternative visions for virtual worlds emanating from Game Developers Conference 2008 which have been percolating through the blogosphere. Particularly interesting, to me, are the new possibilities for avatar expressiveness and some interesting ideas on the future of ubiquitous game design.

I was thrilled to discover that Tony Manninen has been doing research on avatar expressiveness since 2000 (see his papers here). Rich Interaction in networked virtual environments, avatar expressiveness, and the future of ubiquitous game design are his forte. How very cool!

Tony’s vision is to take social gaming to the next level and to produce games that are so heavily collaborative that they reach deep into our experience of the pleasures doing things together, our enthusiasm for team sports, and childhood memories of the enjoyment of playing together. He has been exploring these ideas of team play and community in earlier LudoCraft games like AirBuccaneers.

Ray Kerzweil’s keynote GDC 2008 laid it on the line: “Games are the harbinger of everything.”

But some of my favorite GDC 2008 quotes are from Jane McGonigal (see her blog post “Reality is Broken” - My GDC Rant) and in Liz Lawley’s copious notes on “Pouring Fuel on the Fire: Game Designers’ Rant.”

Jane McGonigal: I’m not mad at game designers. Compared to the rest of the world, we have it all figured out. Our medium kicks all other media’s ass. We make more people happy than any other platform or content in the world. (If you don’t believe that, you’re not paying attention to what’s happening.) We’ve won. Games have won. As an industry we’ve spent the last 30 years learning how to optimize experience. Brains, bodies (recently), and hearts are all engaged. That’s the good news.

The bad news is we rule the virtual world only. Reality is broken, and we’re not fixing it, we’re offering alternatives to it. We offer better experiences, better socialization, in virtual experiences. That needs to start changing. If reality is broken, why aren’t game designers trying to fix it? It’s our responsibility to design systems that make us happy and successful and powerful in real life? We have the power and the responsibility.

Jane shows an image of her favorite “I’m not good at life.” graffiti (from Liz Lawley’s notes).

Tony Manninen is an innovator committed to bringing the power of games into arenas we are yet to dream of.

Jane McGonigal has some nice examples to stretch your mind in this direction. For example, the Sniflabs collar remembers other dogs you’ve encountered. “We can play games with our dogs. What if I could play an MMO with my dog?”

Why will it make our “real” and “virtual” lives better to take play into every nook and cranny of our experience? Because, as Kurzweil points out, “Play is how we principally learn and principally create.” Jane McGonigal goes further:

I have spent the last year doing research on happiness. Instead of trying to figure out what’s broken, these people are trying to figure out what makes us happy. Every positive psychologist has found the same thing. Happiness is 1) having satisfying work to do, 2) the experience of being good at something, 3) time spent with people we like, and 4) a chance to be part of something bigger than ourselves.

“If you are a game designer you are in the happiness business.”

Jani Pirkola, realXtend project manager, works on innovation in the offices of LudoCraft.

LudoCraft = The Art of Designing Games and Play

Ludo - Theories of game and play

Craft - Art of game design and development

Reinventing the technologies of expression and experience.

At the Metaverse Roadmap meetup, Mitch Kapor pointed out “3D cameras would make virtual worlds easier to use.” And Bob Moore (on left above, me on right) asks in this excellent post, Is “free gesticulation” for avatars here yet?

When I asked Tony this question he replied:

“Our avatar will have enough “bones” for full facial expressions, etc. When the actual base architecture of the avatar is fully functional, there’s a possibility to use webcam, voice, or other input devices to control your avatars facial expressions. It can be true 1:1 mapping, but of course it can be something else as well. You can be yourself, or, you can change your “output” to something else.”

Tony, through realXtend, is reinventing the technologies of expression and experience.

Man Upside Down from The Book Of Urizen, William Blake

Interview with Tony Manninen, LudoCraft

Me: Second Life as a platform has been pretty much ignored for game development up to this point. Do you imagine transforming OpenSim and Second Life into platforms suitable for MMOG?

Tony: I am running the company and also making sure the realXtend development reaches the required quality and performance standards you would expect from MMOGs.

We’d definitely love to make games for Second Life, but at the moment the end-user experience is not exactly what you would expect from a game system. Therefore, we’ll pay heavy attention to things like responsivity, graphics quality, frame rate, etc. If we manage to keep up the momentum of realXtend development, then I’m sure there will be some interesting games spawning up in the near future…

Developing a sophisticated game engine is not an easy - or cheap - task, so there’s a loads of challenges ahead. But I truly believe that is the only way forward. With game-like interfaces and features you’ll be able to get much more heightened experiences.

Me: Have you worked out anything with LL yet re keeping the realXtend viewer compatible with SL?

Tony: We have tried to keep the rexviewer as compatible as possible. We totally appreciate what Linden Lab has done and we are trying to do our best to co-exist with their beautiful social innovation.

Me: But will you be able to work out a licensing deal under the GPL so that they can integrate your code into their browser?

Tony: The whole licensing scheme is still undergoing some serious thinking processes. We will try to find the best possible option in order to satisfy the needs of hungry virtual world adopters. GPL has its challenges. But, on the other hand, everything invented by a man can be re-invented by another. I am sure there will be a fruitful solution to the licensing issue. At the moment we are concentrating on releasing all the code to the general public, so that all the enthusiastic developers out there can join the forces and increase the momentum.

I’m pretty sure the pieces will eventually find their right places. I am really happy about the response we’ve got from Linden Labs - it’s great to think we might be able to give something back for them.

Me: Is it safe to say the licensing issues are on the table and being worked out?

Tony: There’s definitely some serious working-out being done, so I suppose it is safe to say that.

Me: I know Will Wright creator of Sims online and Spore has spoken a lot on the spiraling costs of content production and that diminishing returns for content development at these high costs. He has gone to procedural programming with Spore to take gaming on another path. But, it seems to me that you are taking another approach by trying to bring SL up to gaming standards - is this correct, or are you doing something different?

Tony: I guess that’s quite correct assumption - at least in relation to LudoCraft. This is not necessarily a conscious decision. It’s more like the costs of licensing a decent game engine are generally so high that they more often than not fall out of reach of start-up companies and small developers - let alone universities, communities, etc.

We have tried to find a suitable platform for our collaborative games, but since there were no perfect solutions, we decided to try and make one. Not alone, but by joining forces with our partner company Admino and several other keen developers.

The OpenSim merger will increase the development base even further, so there’s a great chance we’ll actually pull this off.

Like I said, there’s loads of challenges, huge amount of work and some design issues involved. But the Open Source communities have proven themselves earlier, so why not now.

Me: What do you think are the chief design issues to be addressed?

The realXtend project includes LudoCraft and Admino, plus we have several sub-contracting developers doing work with us.

The main issue is the divide between social 3D worlds (like SL for example) and MMOG. The gamers tend to avoid these social virtual worlds for obvious reasons [the quality of the experience from a gaming POV]. However, if we manage to develop a platform that can serve both purposes, then I’m sure things will change.

The main design issues, therefore, are the performance, audiovisual quality, rendering, frame rate stability, responsivity, interaction, etc.

Numerous issues that are not necessarily critical in purely social virtual world, but are absolutely essential in any multiplayer game environment.

Plus, if well taken care of, these issues will boost the whole end user experience in the non-gaming situations as well.

Me: You have gone a long way with the rendering and meshes what will be the next most important features and when do they arrive?

Tony: Inverse Kinematics and procedural animation are essential features if we want to have truly expressive and adaptive animations. We see avatars as main tools for self-expression. We’ve researched the issues since 2000 and we believe we are on the right track. With flexible and powerful expression potential and accurate controllability, the users will be able to communicate with their avatars so much more than is possible nowadays. The concept of Rich Interaction is something we will utilize here and it will be interesting to see the results when the system is actually usable by the general public.

Vehicles and projectiles are really important so they sit heavily on our roadmap.

Me: Are you going to do 3D face mapping etc - I know that this is getting close to doable now?

Tony: Oh, the required list of features is endless. There are several key features in terms of game development, but also some interesting stuff for more serious applications.

Our avatar will have enough “bones” for full facial expressions, etc. When the actual base architecture of the avatar is fully functional, there’s a possibility to use webcam, voice, or other input devices to control your avatars facial expressions. It can be true 1:1 mapping, but of course it can be something else as well. You can be yourself, or, you can change your “output” to something else.

Me: OpenSim is very attractive for a number of vertical applications - building automation is one you mention. How focused are you on getting the OpenSIm platform ready for the huge mirror world, 3d info machine/command center market?

Tony: In terms of the mirror world, we already have some interesting pilot projects that target those features. It may be less juicy than making cool games, but it provides some highly interesting scenarios.

Me: What are your pilot projects?

Tony: I don’t think I should go into details at the moment. We will include a list of initial projects and collaborations for our next announcement, which is due 3rd of March (latest). We want the third parties to have their saying first before making these public.

Me: The realXtend avatar server is a new way of connecting sims - could you tell be more about your vision there?

Tony: The main vision is: there should be only one. Only one avatar, one user account, one login, etc.

It is ridiculous that I need to have an avatar in SL, character in WOW, nickname in IRC, and whatnot. I mean, it’s ok if I want all these different identities, but what if I would be, eh, me in all these worlds, perhaps doing business, shopping, hobbying, and so on. There should be an option for me to jump from world to world without the hassle of logging in and logging out

Me: And how will the work you have done on the avatar server alleviate this problem unless SL, WoW and other cooperate on interoperability?

Tony: Think of it more like the 3d web. realXtend/OpenSim is like the Apache of virtual worlds, rexViewer is the Mozilla or Firefox of whatever. When “surfing” the web, you are not constantly required to prove and change your identity when loading different pages.

Me: But because there is no equivalent of http for virtual worlds yet, how will this work re the other worlds?

Tony: Ah, that’s the critical question: it won’t. There is some interoperability work going on in terms of creating the standards. The problem is, this work tends to be slow. We are not willing to wait that long. We want to see some action. That’s why we thought that we’ll start the wheel rolling and then see what happens.

Me: So at least now OpenSim worlds and hopefully Second Life will be able to connect this way?

Tony: The openess is the key to the lack of unified protocols. When the interfaces and APIs are open and transparent, people are free to develop applications, converters and bridges between different systems.

Right, at least OpenSim is now reachable, plus we want to get the teleport between SL and OpenSim/realXtend up and running, at least in the quick-and-dirty way so that people will have a chance to experience the feeling of world-to-world hopping.

Me: So in terms of this experience of avatars hopping between OpenSim worlds how do you think this will change the need /or not for large grids like SL or any of the new ones like OpenLife that are springing up?

Tony: I think there’s still need for various technical solutions like there is need for lighter and heavier websites. There are different applications, different scenarios and different user requirements. It is still really enjoyable feeling to be able to actually walk - ok, virtually at least, from one server to another. With hopping there may be some loading involved (like when surfing the web). So I guess the big grids won’t go anywhere. We’ll just have more modalities in terms of servers/worlds.

Me:So what are your expectations for concurrencies on OpenSim in the near future?

Tony: Hmmm… I’m not sure yet. OpenSim is an excellent initiative and there’s already a substantial user base. There are a lot of work to be done in terms of integrating realXtend codebase to the OpenSim, but since the visions and missions of the people involved are aligned, the hard work is a minor detail in the process.

avatar 2.0, avatar expressiveness, Croquet, free gesticulation for avatars, game design and development, GDC 2008, global avatar mesh, home automation in virtual worlds, import of 3D models to Second Life, Linden Lab, Ludocraft, MMOG, Ogoglio, Open Source, open source virtual worlds, OpenSim, realXtend, rich intertaction in network virtual environments, social gaming, Suns Wonderland, teleports between realXtebd and Second Life, the art of designing games and play, There.com, ubuqitous game design virtual worlds as company towns