This post, and my previous post about integration of OpenSim into Web 2.0, explore how immersive virtual worlds, through a full architectural integration into Web 2.0, will become part of the fabric of everyday computing.

The diagram above shows where OpenSim sits in Web 2.0 (click on the diagram to see a readable enlarged version!). The following interview with OpenSim developer, Teravus Ousley, describes some of the work being done to create documented protocols that will make OpenSim fit seamlessly into Web 2.0 architecture.

OpenSim is in the news a lot these days, explicitly as in the case of the announcement last week by 3Di of their “3Di OpenSim” Standard (for more see here and here), and implicitly with the launch of ChinaQ.  Adam Frisby, OpenSim, pointed out to me if you download the ChinaQ client that it is based on OpenSim, it connects nicely to OSGrid too. There is speculation the client is a rebranded version of the realXtend viewer (which is based on the open source Second Life viewer) as all the version numbers are the same.

So OpenSim is not only attracting the interest of business giants like IBM, Microsoft and Intel, it is becoming the architecture of choice for virtual world initiatives from Chinese and Japanese telecoms (see here and here for more). Also, see the press release about Nokia and the City of Oulu, Finland, joining as supporters of realXtend.

But, as Raph Koster in his post commenting on 3Di’s OpenSim announcement notes, the question how immersive virtual worlds can go from strong niche or enterprise markets to mass adoption in consumer markets must be answered.  As Raph points out, Lively, Whirled, SmallWorlds, Vivaty, and yes, Metaplace have a very different architecture that they hope will attract broad consumer markets.  (I did a long interview with Raph on this at The Virtual Worlds Conference and Expo in LA which I will post as soon as it is transcribed, so more on this soon!).

Architectural integration into the heart of Web 2.0, I would argue, is the key to mass adoption for immersive virtual worlds. While architecture alone will not guarantee the necessary breakthroughs in usability for widespread consumer adoption, it will create the ideal conditions for the innovation through which usability obstacles will be overcome, and the enormous potential for immersive, real time interaction over the internet will be realized.

Interview with Teravus Ousley

Tish: What has been  the most fundamental problem re virtual world architecture that has kept immersive virtual worlds isolated from web 2.0 to date? 

Teravus: a lack of standardization, licensing issues, and the difficulty of entry into the industry.

1) Standardization

Tish: In order of importance what in your view are the priorities for standardization?

Teravus: Probably the same order that OpenSimulator was tackled in, basic connect (current state of OGP – Open Grid Protocol).  Basic Service (interaction standards).  Advanced connect/mashup/aggregate extensions.   Preferably people will have working code in the various spaces there to use freely under various licenses..

Tish: Can you show me where OpenSim will fit in this drawing of Web 2.0 architecture? [Teravus makes some modifications on the drawing I send him from Dion Hinchcliffe’s presentation from his Web 2.0 Expo workshop, see original here]

Teravus: The modified diagram [now opening this post] is a great view of how it will look.

Tish: Why is the TCP stream left out of the original drawing? [For more about Transmission Control Protocol (TCP) is one of the core protocols of the Internet Protocol Suite see here.

Teravus: It is left out because the person who made this diagram had web pages in mind.  Static large files, or small changing files. In the the drawing the fact that TCP streams are smaller then HTTP is on purpose.

Tish: I have heard different opinions on the percentage of the communications for virtual worlds that can be done over HTTP?

Teravus: The fact is that the biggest usage of communications in virtual worlds is transmitting images that’s the number one bandwidth usage. So, if we’re counting by ‘usage’ I say 91%.   If we’re counting by services that use http.   I say probably 75%  I definitely think that http should be evaluated for use on new things ‘first’. But, there are a few places where HTTP doesn’t shine.

I am skeptical about replacing things in the UDP with HTTP  thinking that they’ll ‘perform better. [For more about User Datagram Protocol (UDP) another of the core protocols of the Internet Protocol Suite see here.]

I think there’s been a huge test going on now and for the last 5 or six years with regards to the UDP protocol and it really has performed admirably.   In the last year and a half, I’ve seen attempts to convert several things to HTTP that have failed, and failed somewhat spectacularly sometimes.  In the end the items get reverted back to the UDP protocol. One such item that sticks out in my mind is CAPS(HTTP) based inventory retrieval. The capability to do that in the client has been available since before February. And, it’s been turned on and off on ‘Agni’ at least once in the process. Additionally, we (OpenSimulator) enabled http inventory, and, the  inventory failures rose pretty steeply.

I think some services are really just not ‘right’ for HTTP.. . particularly where a ‘poll’ methodology is used, or, the data is significantly dynamic enough that it makes caching useless.

Anyway, as far as the future is concerned, I do want to see some services over HTTP. Other services, it would be more appropriate to have a TCP stream. Stock market data, for example, uses a TCP stream. The Scalability of the stock market, is just one example of a scalable TCP stream.

Tish: So you see TCP  as the communications protocol that would do the work for the parts of virtual worlds not suitable for HTTP. At least that is how you have shown it in our Web 2.0 architecture drawing. But should there also be a UDP stream?

Teravus: For the virtual world of tomorrow? .. probably not.

Tish: Why not?

Teravus: You have less control over the quality of service when it’s delivered over UDP then TCP.

Tish: What is the exact relation between TCP and UDP.  My understanding is UDP a lower level protocol.

Teravus: TCP offers guaranteed delivery through flow control, while UDP does not.  One of the failures of UDP, is the ‘resend’ technology we’ve put on top of it to try and make it reliable.   TCP does this automatically and better then we could at a lower level but it does also cost up to twice the bandwidth depending on what is being sent. HTTP is a layer on top of TCP.

Tish: So just like the HTTP/TCP discussion there has to be a TCP/UDP boundary discussion …so it is HTTP then TCP then UDP and the boundaries have to be worked on.

Teravus: Those are the orderings in my mind…  probably if UDP uses any..  it should use less then 0.5%.

Tish: And the current Second Life architecture what does it use if it isn’t using HTTP? [to see the work of the Architecture Working Group on the future Second Life architecture here]

Teravus: UDP or HTTP

Tish: and TCP?

Teravus: Well, TCP is a layer under HTTP.  As far as I know, SL doesn’t use TCP streams directly

Teravus: Instead, it uses HTTP polling.  This is one of the places, that I’ve highlighted where it doesn’t shine.

Tish: Polling does sound slow?

Teravus: Polling is essentially..     (connect) Got any data for me? No?(disconnect), (connect) Got any data for me?  No?(disconnect).

Tish: So what is the path to standards for this then?

Teravus: Distilling what we know works and what we actually intend on supporting as far as adoption under these standards.

Tish: Where does MPEG-V fit in?  Have you read their document yet?

Tervavus: MPEG-V is interesting reading…     but is there any working example? I have just the overview. But I’ll read it over to have a better determination of how to ‘keep it in mind’ for the future. It looks like they’ve only really defined the requirements of the MPEG-V spec. The MPEG-V spec looks quite far reaching..  but  the documents so far are requirements and marketing talk aimed toward business people – obviously intended to get more people interested in working on them.

But I have a feeling that any format with MPEG before it will be onerous to support. ..for me it’s too early to tell. It’s quite far reaching…it isn’t anything like ’signal processing’ which the MPEG group is most famous for.

Tish: The whole top down approach of the MPEG-V initiative seems counter to Web 2.0 principles to me.

Teravus: Well, remember..  that even if there’s a virtual world format war (reference to DVD-HD vs BlueRay) we still need to win over the rest of the web.

Tish: Yes and don’t you think the way to win over the web is to use as many existing standards as possible?

Teravus: Well, it’s to use as many existing standards as ‘fit’ though.. KISS, as always (K)eep (I)t (S)imple (S)tupid if we have 30 different internet standards..     people looking at it will @.@

Tish: But it is just lack of documented protocols that has created isolation from Web 2.0?  And really doesn’t it boil down to standardizing that small percentage that is outside HTTP – the TCP and UDP stream that we talked about earlier where the real time stuff that virtual worlds bring to the web happens?

Teravus: no..  actually the HTTP standardization is just as important.

Tish: You mean even though SL used HTTP it isn’t standardized?

Teravus: Not documented specifically.

Tish: And OpenSim is that documented?

Teravus: Not well enough probably to define a standard.

Tish: Is AWG (Architecture Working Group) doing the documentation?

Teravus: working on it..

2) Licensing Issues

Tish: It sounds like some of this work has to go on across client and server.  Are we running into the issue of BSD for OpenSim and GPL for the Second Life viewer?

Tervaus: Well, some of the issue here is license choice.  One of the reasons that libOMV was able to achieve what they did was they did it /before/ the client was open sourced.

Tish: So open sourcing the client actually became an obstacle!!???

Teravus: I don’t think so in a whole.  I think it was great for the community.  I do, however think that C++ UDP stacks will be scrutinized more for GPL license violations because, of course, the client is GPL and C++ .

Tish: It is my understanding that Linden Lab is open to discussions on making the licensing more efficient for the open source community?

Teravus: Well, the client, in a whole, should not be changed as far as the license.   JUST the things that they expect people to adopt should be made more open. If they expect people to adopt PRIMs, then there should be an efficient implementation available for anyone to use..   at the very least, in LGPL format. Otherwise, the die hards are forced to re-implement them from scratch, and most people will just choose something more open.

Tish: Has anyone ever put together a list of the parts that need to be LGPLed?

Teravus: Well, I think it’s there in a few places.  There is at least one jira open on it.

Teravus: A few that come to mind for me..   is the UDP stack and the prim to mesh/UV code.   I think there are some things that can definitely be improved about the UDP Stack.  There are some things, (images come to mind), that would be better over HTTP

Tish: Do you think if the UDP stack were L GPLed that would be a significant help to integrating OpenSim better with the web?

Teravus: Well, it would certainly be adopted by more clients. GPL + (your own code) = GPL Licensed client. LGPL linked library + (your own code) = Your own license.
You still need to mention that you used LL’s UDP stack, and provide the source code for it at request.

The general client itself should remain GPL, it’s better for LL that way.  Just the items that they want people to ’standardize’ on. It would help..   if it was at least LGPL

Tish: And the value to  LL on LGPLing these parts is it helped spread their basic technology while protecting the rest of their viewer?

Teravus: It furthers their goal of standardization on their systems because it allows more people to adopt it for their own uses without worrying about GPL-ing their own client.

Tish: It is hard to standardize without access to the low level parts of the client right?

Teravus: The general population of Developers..     will want a libX that they can plug into their application for communicating.. .  libY to deal with object data..

Tish: Hence your requests for LGPL were  UDP stack and  the prim->mesh/UV

Teravus nods

Tish: and at the moment they only have openmv?

Teravus: That’s the only ‘truly’ open standard right now as far as the LL technology is concerned. OpenSimulator’s use of that data..   could also be seen as a standard..

Teravus: But we have not published anything beyond code..   neither have they  really..  technically..  but their organization of the way things work is very very clear

Tish: What are the most significant limitations of openmv?

Teravus: Probably..  just it not being in c++.   c++ has it’s benefits and it’s pitfalls.  Changes in c++ usually take longer then ones in C#.  But, of course c++ is always faster.  With libOMV It isn’t always clear about what packet is used when.  However, with some experimentation, you can figure it out in 30 minutes or less..

Usability

We didn’t spend much time discussing some of the innovation in usability that this architectural integration into Web 2.0 will enable (more to come on that!). But, Teravus mentioned one interesting use case he is working on.

Teravus: You might also stick a ‘cloud renderer’ into the graphic [Tervaus was looking at the diagram (from Dion Hinchcliffe) that opened my previous post on “Web 2.0 to OpenSim Made easy”  click on the thumbnail below].

Some people have discussed having a ‘video stream’ that’s rendered on the cloud and providing that to flash clients would be the best solution to it for them.

The cloud renderer is for organizations that have large pools of servers with GPUs so would allow for very powerful rendering. The servers can render the scenes and stream them to the low end browsers. It would allow extremely high quality rendering for really low end browsers..  such as ‘cell phones.’

Tish: Is that possible now on OpenSim?

Teravus: Nope.  But it’s something that in the future, I intend on working on. It would essentially be a video [streamed to low end browsers].

Tish: Is that different from what Vollee is doing? The mobile client for SL?

Teravus:  It appears that they are, indeed, pre-rendering the client’s view and streaming it to the mobile device

OpenSim was in the spotlight at Virtual Worlds Conference and Expo, LA 2008. OpenSim won a conference Innovation Award and the OpenSim booth was a hub of activity.

At one time I saw conference attendees from Microsoft, Intel, and IBM all in conversation at the OpenSim table. A video of the OpenSim integration with Lotus SameTime was a centerpiece to Colin Pariss’, IBM,  keynote interview with Erica Driver of Think Balm. And, just to back up what might seem my own seriously partisan opinion, I will note Wagner James Au made OpenSim the first of his top three picks from the conference for GigaOM

And, if you want to know why OpenSim is sizzling hot these days and you can manage some serious gearhead discussion here is an audio recording of our “Open Source, Interoperable Virtual Worlds” panel.

I was facilitator for a stellar group of  virtual world uber geeks/phenom developers from OpenSim: Adam Frisby, OpenSim/SineWave, Mic Bowman, Intel, Justin Clark-Casey, Fashion Research Institute, Mike Mazur, 3Di and David Levine, IBM.  See Dusan Writer’s blog for more on the heights of geekdom that were reached in the discussion, and for some interesting thoughts on the intersection of code, standards and social policy.

I open this post with a picture of Adam Frisby, Joshua Meadows, myself and Tess Linden enjoying some pre-conference Venice Beach time. Thanks Tess for showing us that you can be a phenom developer and get beach attire right!

But, the heart of our panel’s message was straightforward and powerful (yes I am obviously caught up in campaign season and on a message kick) . But in case you missed the LA Virtual Worlds conference here is the essence/message of our panel again, from my perspective, and in my words, and in short (relatively!!).

Open Source and Interoperability are the Basis for Innovation in Virtual Worlds.

This picture is from Adam Frisby’s blog of OpenSim “booth babes”  with the  Innovation Award.  They were dubbed “booth babes” by Dusan Writer in counterpoint to David Levine’s comment on seeing this photo, “What a bunch of nerds!”  David is in the center. And from the left Justin Clark-Casey (justincc), Adam Frisby (afrisby), Charles Krinke (ckrinke), David Levine (zha ewry), Adam Johnson (adjohn), Mike Mazur (mikem), Jeff Ames (chil1ken)

Basic building blocks and interoperability work will make Virtual Worlds part of the fabric of everyday computing

Big players like IBM and Intel are often asked why so much interest in OpenSim, in particular, when their goals obviously reach well beyond one platform. The answer to this, panelists explained, is that OpenSim is providing the best environment currently available to explore what will be the basic building blocks similar to http that will enable virtual worlds to emerge as a mainstream phenomena.

In conjunction with the open source Second Life viewer (despite the need for licensing changes and a new modular architecture which was discussed) OpenSim is the virtual world test kit supreme. The modular open source architecture allows for components to be disaggregated and reaggregated into new formations easily and rapidly. As I have mentioned before,  Stefan Andersson, Tribal Media AB,  has a catchy way of putting this,  “OpenSim is a rapid prototyping testbench for wannabe paradigm shifters.”  Stefan, one of OpenSim’s founders, notes this has been the heart of OpenSim from the start.

Interoperability work is the path to achieving virtual world standards

Achieving standards will take virtual world innovation mainstream.  The innovation emerging from open source to produce basic building blocks, and the standards emerging from interoperability efforts, panelists explained, will be the magic brew that transform virtual worlds into part of the fabric of everyday computing.  As Mic Bowman, Intel, describes it (interview upcoming in Ugotrade). The goal is that virtual worlds will ultimately be taken for granted much as when say “browse the web” – we take the “web” for granted (it is the applications YouTube, Flickr etc that gets our attention).

Our panel argued strongly that achieving virtual world standards will emerge from a combination of “rough consensus and working code” rather than “Vapor standards” – worked out by committees in the abstract. Both David Levine, IBM, and Adam Frisby noted “vapor standards” have been lost causes in the history of the internet.

Interoperability work allows for these basic building blocks of virtual worlds to be expressed across various computational fabrics which creates an environment for what David Levine describes as “parallel innovation.” David explains:

The point about parallel innovation is that there are several ways to evolve things. Open Source, tends to foster the addition of features, in common to the main trunk of development. Its very much about everyone building together. Sometimes, we need to or want to build stuff which works with that main trunk of development, but doesn’t share code, or has to fork. That’s where standards come into play.

While the OpenGrid beta has begun with Linden Lab and OpenSim, OpenGrid is just the beginning of much more extensive efforts to test and create protocols across different computational environments that will include increasingly different models of virtual worlds from OpenSim and LL’s model.

So another key message from our panel is: “Heterogenity fosters the creation of standards and more innovation.”

Metaplace and Second Life TM/OpenSim in Dialogue: Play Metaplace on a Prim!

One of the highlights of Virtual Worlds, 2008, LA, for me, was a forty minute conversation between Raph Koster, Tess Linden and myself about Metaplace (the pictures from the Metaplace blog are screenshots from worlds Metaplace testers have made.

I will have a more detailed post on this interesting dialogue soon.  I think Raph illuminates some of the subtleties of what, Dusan Writer noted, came across as somewhat of a Metaplace v Linden Lab debate on the Future of Virtual Worlds panel. As Raph explained, one of the the things they like to say at Metaplace is that, “virtual places are going to be first class citizens of the web next to tags, audio, video, images….that’s a very different kind of picture of where this kind of stuff is headed”  (see Raph’s comments on the panel here).  But as a little warm up to more on Metaplace soon, here are some of the remarks Raph made about integrating OpenSim with Metaplace hehe!

If somebody took an OpenSim server and made it speak the tag language out.  Then every Metaplace client would be able to talk to the OpenSim one. And visa versa, if OpenSim can open a  bog standard gpt socket, because we don’t even use a binary protocol. We use absolutely bare bones of the web. If you can connect by telnet you can connect to Metaplace. So that means an OpenSim client could become a Metaplace client, parse the tags, and play them on a prim.

RealXtend bring a cool new demo to LA

In the picture above Ryan McDougall now senior software developer with RealXtend, Tomi Kujanpää (LudoCraft/realXtend Art Director & Avatar Specialist), Hannu Hollström (ADMINO technologies) have a drink with Gene Yoon (Ginsu Linden), VP of Business Affairs, Linden Lab at The Metaverse Mixer Meetup at the Hotel Figueroa.  Cheers!

Kudos  to Linden Lab for their big picture thinking demonstrated in the work they did open sourcing their viewer at a time when few other companies would have made such a move.  And, more recently, for their efforts on interoperability with the OpenGrid initiative.

And, congratulations to RealXtend who have pushed the envelope on innovation with the open source Second Life TM viewer and OpenSim server this year. RealXtend showed up in LA with another awesome “undersea” demo that will be public on September 23rd (screenshot below from Ludocraft Ltd.)

Also more exciting news from RealXtend this week. Mac’s Virtual World reports:  “OpenLife will be the first grid based on the OpenSim technology to use RealXTend features as far as I am aware of.”

Augmented Reality will bring Virtual Worlds out to the mainstream.

For an important look at how instead of more people going into virtual worlds, virtual worlds will be coming out to meet the people soon, see the Augmented Reality Panel video above. For more see Roo Reynolds and David Orban’s blogs.

Eben Moglen – Open Source, IPPI panel in Second Life

Life 2.0 Summit Spring ’08 will kick off with the Open Source, IPPI (IP and Privacy/Identity) in Virtual Worlds On Sunday, March 16, at 1 PM PST, with special guest Eben Moglen (his avatar pictured above).

The event will be held in the CMP Amphitheater at CMP 1, 2, 3, 4 ( SLURL ). To attend the Second Life events or watch video you must register for Life 2.0 here.

Eben Moglen, is Director, Chair and Chief Counsel of the Software Freedom Law Center. Moglen, professor of law and legal history at Columbia, is a pioneer of the opensource movement, former general counsel for the Free Software Foundation, and one of the architects of version 3 of the GNU GPL.

Zero Linden

Eben Moglen’s co-panelists on Sunday will include Zero Linden, a.k.a Mark Lentczner, Linden Lab. Zero is one of main the architects of Second Life’s evolving infrastructure. Zero recently published the first draft of Second Life Grid Open Grid Protocol a.k.a. SLGOGP a important step forward on the path to opening up the Second Life grid (see Tateru Nino’s post on Massively, Tao Takashi’s post at mrtopf.de, and mindblizzard).

The brilliant and very elegant Zha Ewry (a.k.a David Levine, IBM Research) will be joining the panel from JFK airport while he waits for his flight to San Francisco. David Levine and Eben Moglen had an interesting conversation back in December that you can find on Ugotrade here. They explore some of the problems of defining digital public space and issues of privacy on the internet, offering many suggestions on how to implement online privacy enhancing technologies and insights as to how we could design the next generation of these technologies in responsible ways.

Also Zha was a interviewed recently on Metanomics with Beyers Sellers (a.k.a Robert Bloomfield). This interview is highly recommended as some of the key issues facing Second Life’s Architecture Working Group (AWG) and a future Open Grid “that will ultimately allow the cohesive operation of both Linden-operated and non-Linden-operated Second-Life style simulators and grids.” (see Massively for more) are unpacked.

Download the video (Quicktime)
Download the audio (MP3)
Read the transcript
Metaversed video archive at SLCN

Also, OpenSim and Linux guru Sean Dague (IBM) will be a panelist. Sean Dague has been a member of IBM’s Linux Technology Center since it’s inception in 2001. He has worked on numerous Open Source technologies over the years including: Cluster Management (SystemImage and OSCAR projects), Hardware Control (OpenHPI), Virtualization (Xen), and now Virtual Worlds with OpenSim. Sean has been an active member of the OpenSim project since July 2007.

Sean’s avatar (picture below) is Neas Bade.

Tara5 Oh (below – that’s me!) will moderate with CMP’s John Jainschigg (John Zhaoying).

See you there! (SLURL)

The excitement in the open source virtual world developer community has been bubbling over since realXtend brought many OpenSimulator developers over to Finland to reveal the extraordinary amount of work they have done “extending” the function of the OpenSimulator project in just 4 months.

See here for an account of this amazing meeting in Oulu, Finland, by one of the participants from the OpenLife grid. Pictured above is a screenshot of the new mesh avatars being conformed inworld. Adam Frisby explained to me:

Meshes behave just like another prim type – you can drag, scale, and rotate meshes around using the same method that you use on primitives and object groups. You can link meshes with other primitives, save them to inventory and use them in pretty much the same way you use primitives.

Second Life is built with prims, so I asked my friend, Zha Ewry, for a 101 on meshes versus prims. “A prim, is a solid, defined by an equation – a cube, and such with permutations, like twist and taper, and cut. Whereas a mesh, is a series of triangles in 3d which conform to the shape of the object.”

Meshes are groovy because “they can do arbitrary surfaces. You can lay a mesh over any shape, add more triangles, increase resolution. A prim you can’t make any shape, only what the math bends it to, so you blend them into shapes by composition.”

SL originally went with prims and not meshes “because they are more compact, lower cost to render and easier to map to the physics model.” But, “many 3d programs build meshes by default and nothing builds prims,” so the introduction of meshes facilitates one of the key new realXtend features – support for proper 3D models.

I asked Zha if meshes would have a negative effect on concurrency. Zha said the answer to that question is not entirely clear yet, “but its not as bad as it seems because its prim+texture vs mesh=texture.”

Zha explained that a lot of people think of meshes as much more expensive when its not prim vs. mesh but rather, prim+texture vs mesh+texture so the difference is smaller, “and even more to the point, for many things, its Prim+prim+prim+prm+textures vs mesh+texture set. Often, even for small objects, you can do several prims worth of detail in a mesh.”

I have heard from OpenSim developers that the expectation for OpenSim is for concurrencies of 100 at least in the next six months.

Two realXtend companies, Admino & Ludocraft, have been dedicating 20 personnel (programmers, designers and content creators)‘ to extending’ the function of the OpenSimulator Project, both server side and client side respectively. The project is backed by the vision and generosity of Juha Hulkko, who has another fascinating project Radio Arkala (also in Second Life on Arkala Island).

Click on the screenshot above for the realXtend video demo. There is a full list of realXtend features on their site here. RealXtend give a summary of some of the most important functionality they have added.

The most important realXtend-specific development areas were the media tools such as VoIP and document sharing, OGRE renderer support and, of course, the separate realXtend avatar storage / authentication infrastructure that enables seamless transition from one world to another.

Having support for proper 3d models as well as simpler primitive type objects is an asset in many different ways. Most importantly, just about all the professionals and advanced amateurs who create content suitable for virtual worlds like to work with proper 3d modeling software. There’s already a lot of existing material that can now be directly transferred to the virtual reality if for example a movie maker wants to create a virtual world based on an animated film and the same goes for computer game developers. Thanks to proper 3d model support, practically all of their material should be easy to transfer from games to realXtend based virtual worlds and vice versa.

Another central feature is the global avatar architecture. A network of interconnected virtual worlds simply doesn’t exist until users can take their virtual representatives with them wherever they choose to go. Architecturally this means separating the avatar and their asset storage functionality from the virtual world server. Avatar service will provide the users with identity and authentication independent of the world they happen to be in at any given time. In practice the role of the avatar service will be somewhat similar to what email is today. Security is naturally a major issue whenever someone’s identity or asset ownership is concerned and it has been an important consideration in the design of the avatar architecture.

The picture below is of the web page as a texture feature.

Under testing for the next release (29th of February 2008)

• Free-form non-humanoid avatars
• Global avatar mesh, skeleton, textures, attachments and animations
• Single sign-on to multiple worlds for teleporting
• Avatar generator
• Avatar attachment tool to help set 3D meshes to different bones
• Unlimited amount of attachments per bone
• Teleports between realXtend and Secondlife
• Avatar storage to move avatar appearance between realXtend grids/worlds
• Mesh tool to scale and set pivot of 3d models
• Server launcher and configuration application
• Home automation example using X10 technology
• Bot with Python scripting
• Media library for world builders
• Server status window

Pictured below is the the location of this historic meeting which was also attended by Chris Collins of Linden Lab. I did hear some very interesting anecdotes about the visitors introduction to Finnish culture – flaming drinks, saunas, and hot metarati rolling in snow!!! But, the focus of this post is the road map for an open standards virtual world’s future, so I will leave it to others to tell those tales.

What follows is an extract from realXtend press release, and then I move on to an interview I did with Adam Frisby who was in New York last weekend. He shared some of his thoughts on the future of the Open Simulator project with me on another snowy weekend of skiing and snowboarding in upstate New York (lots of snow again but no flaming drinks, just Ribena, Mario Galaxy, and snowball fights with Ugotrade Jr).

Oulu, Finland – even a blizzard couldn’t delay the meeting

Team up to make open source Virtual World standard

Open source virtual world developers OpenSim are building a global standard to power the 3D internet, now joined by Finnish developer’s realXtend, OpenSim gains many new key features and technologies.

OpenSim is a free and open standard virtual world server founded at the beginning of 2007 by a small group of developers. The OpenSim platform can be used for creating and deploying immersive 3D Virtual Interactive Environments. The realXtend project’s mission is to assist in bringing forth the next generation of virtual reality development by focusing on interoperability technologies, usability and real-life application support. realXtend brings to OpenSim professional experience and development in the realm of 3D engine programming, voice over IP and network support, and is contributing many new features such as improved 3D graphics and voice chat.

“I welcome the realXtend project to join us. It is my pleasure to see that the OpenSim gets a huge professional code contribution. The future of the OpenSim project looks bright as the developer base continues to grow with giant leaps like this”, says OpenSim project manager Darren Guard (aka core developer Michael Wright).

realXtend is contributing all the server side code for their developments to OpenSim and continues to do so from now on. This collaboration of two projects with a similar vision enables realXtend and OpenSim to focus on common issues, and solve them more quickly – leveraging experience and knowledge from each party. realXtend project manager Jani Pirkola comments on the joining as “I see this as a great possibility to quickly make OpenSim the global de facto standard and to significantly speed up the global technology development in this area. Our common goal is to create the best open source virtual world server platform, and to continue the rapid evolution of OpenSim”.

Interview with Adam Frisby of OpenSim

Snowball fight in the Catskills while “snowed in with Open Sim”

Me: Could you tell me more about the “avatar server?”

Adam: The “avatar server” improvement allows you to sign in remotely from another region and bring your avatar & attachments across with you. It allows you to sign onto a OpenSim with an address like: adam frisby@mygrid.com — and then bring across your avatar & appearance. Inventory/etc is going to come in with this sign on method as well.

Me: You also mentioned to me how this would really change the concept of griding and introduce a different model to the Second Life grid.

Adam: Right, that’s tied in with that [the avatar server] a little – you don’t need a big central grid in the way they are operating. Their philosophy has been more akin to the world wide web — lots of interdependent sites that can link to each other. The avatar server is the first step towards that, since it allows you to login to multiple independent installations with one account.

Me: Do these changes, potentially, signify more of a fork from Second Life or not?

Adam: Definitely. This is the first time we’ve broken compatibility with SL feature-wise and implemented things that go far beyond SL.

Me: Will the Rex viewer work with SL?

Adam: You can still connect to SL with the Rex viewer, but your missing lots of nifty functionality, like the new rendering engine.

Me: What do you think are the implications of that and do you think the fork will continue to get wider or not?

Adam: Well, the subtle irony of it is that Linden Lab can’t take the changes back because of their client being GPL’d and they still rely on non-GPL’d components, i.e., FMOD and other bits and pieces. They can take BSD [all the realXtend server side work is BSD, but the client source code is GPL’d at the moment.
If we released the patches as BSD licensed, which RealXtend intends to try to do, then yes they could take those ones back.

Me: So why the conflict exactly?

Adam: LL use a set of proprietary components in their official release, for example the Kakadu JPEG2000 library – which means their official release is actually GPL-incompatible. RealXtend have to extract out any bit of Linden IP or derivative work in order to be able to release it under a more permissive license. (The GPL is actually fairly restrictive). In copyright law, you can generally make a open source license more restrictive, but never the other way around – it’s one of the reasons that OpenSim is licensed under one of the least restrictive Open Source licenses. If LL took the GPL’d code RealXtend had done back, they would be under the terms of the GPL via RealXtends IP.

Me: So taking the client code back could cause problems?

Adam: Well, if they took it – yes. But unfortunately they really can’t. They could in their pure-open-source release, but not in their mainline release.

Me: What do you think is the best path forward re interoperability?

Adam: I think the best path forward for a standard is to release everything to either the public domain, or as close as you can get. It allows all parties to use the code for any purpose whatsoever.

Me: So you mean no GPL no BSD?

Adam: BSD is fairly close legally – it’s effectively public domain with a “Don’t sue us” clause. There’s a small group of licenses which are essentially public domain:
- New BSD (what OpenSim/libsl are under)
- MIT
- X11
and a few others.

Me: Can Linden Lab switch from GPL to BSD?

Adam: They can – as far as I know they have never accepted anything into their codebase which would prohibit it. They have been discussing lately switching a few key files to BSD, which would definitely be appreciated – but the whole client would be better. A lot of people do – I’m not going to go into names (you can if you want), but there’s a fairly large group of people who dislike the GPL. At least for this kind of standards setting thing.

Me: So what is the road map for OpenSim now? What do you see as the 3 month goals to be, and then 6 month and then one year?

Adam: 3 month goals – we’re going to try and align the two codebases we have now into one seamless codebase that works well both with the official SL client, and with the improved Realxtend one.

We’ve also got a few short term goals in terms of stability, getting features working properly, and maybe with a little luck, doing a bit of work on our client networking. Of course – anything I say has to be taken with a grain of salt, open source means people can work on what they like effectively.

In six months time, I’d like to have the majority of the features in the official client supported, and to be reasonably stable enough for large scale deployments.

I think that might take longer than 6 months, but you never know.

A years time, I’d like to be working towards next generation features – what sorts of things we think the standard virtual world must support, better support for linking environments has to happen over the next year – and we definitely need to be well on our way towards a really good standard infrastructure that can scale to the size of the web today.

Me: Re the discussion of standard features – what do you think is the best forum for this discussion? And, has realXtend committed to more development?

Adam: They have committed to working on this until their backing runs out, but it sounds like they might have found a way to continue working for a good while yet.

Me: And, the forum for the discussion of standards – what do you think is best for that?

Adam: I think that’s going to just happen on it’s own.

A lot of the standards bodies that have been setup seem to be moving no-where, and while we are all happy to participate in them – I think more is going to be achieved by doing things, testing them, seeing what works.

Me: And sorting out a more permissive licensing situation seems like it would help?

Adam: It would help – because it means we don’t need to spend so much time reinventing the wheel. We have to reinvent everything LL has done to be able to license it permissively.

Adam: The Rex serverside changes – those are all BSD, and we’re incorporating it back. Their viewer changes – those are GPL’d by necessity, and we’re not touching those. The RealXtend company has two different companies working on it to avoid any IP contamination. The serverside stuff was done by Admino, and the clientside by Ludocraft.

We’ve got two pieces of software here:
- OpenSim
- Rexviewer
and the server side changes to list them are the avatar server and….. and all the meshes etc. are on the client side with Ludocraft. The server side changes are things like the Python scripting, backend support for the meshes, avatar server, etc. The client side changes are the new rendering engine, client side mesh support, shadows, improved lighting, etc.

Me: Does Windlight goes out if Rex comes in?

Adam: Yes, Rex are looking at integrating Windlight properly into their viewer at the moment. But they can integrate that in without too much of a problem because that’s strictly a client side thing, and that’s already GPL’d.

Me: My friend Ben Goertzel of Novamente who is developing Artificial General Intelligence projects in VWs wanted to know if the skeletons allowed “the ability to script fine-grained control of skeletons controlling characters of different types.” He wants to control characters via inverse kinematics not just by launching pre-fab animations because without that you can’t have naturalistic or adaptable motor control. Will the mesh avatars allow that sort of thing?

Adam: IK is on RealXtend’s feature list – they haven’t implemented it yet, but they are working towards it. What your friend might appreciate is that you can have completely customised bones under the realxtend avatar.

The default skeleton has 220 bones in it, but you can make your own custom skeletons for things like anthropomorphic avatars.

One of the avatars we got shown was someone wandering around as a giant collection of mushrooms.

Me: Can you control the angles of motions of the joints?

Adam: Yeah, you can.

Me: Cool! Ben is going to be so happy. AI just moved a big step forward!

Adam: Shiny.

Me: Ben also said if I want to stress you are ask if you can implement realistic physics of fluids for him

Adam: We’ve got a physics engine that supports it, but client will take a lot of work!

Click on the screenshot below to download your own OpenSimulator and get started. I have. You can download the realXtend client at realXtend.