This post, and my previous post about integration of OpenSim into Web 2.0, explore how immersive virtual worlds, through a full architectural integration into Web 2.0, will become part of the fabric of everyday computing.

The diagram above shows where OpenSim sits in Web 2.0 (click on the diagram to see a readable enlarged version!). The following interview with OpenSim developer, Teravus Ousley, describes some of the work being done to create documented protocols that will make OpenSim fit seamlessly into Web 2.0 architecture.

OpenSim is in the news a lot these days, explicitly as in the case of the announcement last week by 3Di of their “3Di OpenSim” Standard (for more see here and here), and implicitly with the launch of ChinaQ.  Adam Frisby, OpenSim, pointed out to me if you download the ChinaQ client that it is based on OpenSim, it connects nicely to OSGrid too. There is speculation the client is a rebranded version of the realXtend viewer (which is based on the open source Second Life viewer) as all the version numbers are the same.

So OpenSim is not only attracting the interest of business giants like IBM, Microsoft and Intel, it is becoming the architecture of choice for virtual world initiatives from Chinese and Japanese telecoms (see here and here for more). Also, see the press release about Nokia and the City of Oulu, Finland, joining as supporters of realXtend.

But, as Raph Koster in his post commenting on 3Di’s OpenSim announcement notes, the question how immersive virtual worlds can go from strong niche or enterprise markets to mass adoption in consumer markets must be answered.  As Raph points out, Lively, Whirled, SmallWorlds, Vivaty, and yes, Metaplace have a very different architecture that they hope will attract broad consumer markets.  (I did a long interview with Raph on this at The Virtual Worlds Conference and Expo in LA which I will post as soon as it is transcribed, so more on this soon!).

Architectural integration into the heart of Web 2.0, I would argue, is the key to mass adoption for immersive virtual worlds. While architecture alone will not guarantee the necessary breakthroughs in usability for widespread consumer adoption, it will create the ideal conditions for the innovation through which usability obstacles will be overcome, and the enormous potential for immersive, real time interaction over the internet will be realized.

Interview with Teravus Ousley

Tish: What has been  the most fundamental problem re virtual world architecture that has kept immersive virtual worlds isolated from web 2.0 to date? 

Teravus: a lack of standardization, licensing issues, and the difficulty of entry into the industry.

1) Standardization

Tish: In order of importance what in your view are the priorities for standardization?

Teravus: Probably the same order that OpenSimulator was tackled in, basic connect (current state of OGP – Open Grid Protocol).  Basic Service (interaction standards).  Advanced connect/mashup/aggregate extensions.   Preferably people will have working code in the various spaces there to use freely under various licenses..

Tish: Can you show me where OpenSim will fit in this drawing of Web 2.0 architecture? [Teravus makes some modifications on the drawing I send him from Dion Hinchcliffe’s presentation from his Web 2.0 Expo workshop, see original here]

Teravus: The modified diagram [now opening this post] is a great view of how it will look.

Tish: Why is the TCP stream left out of the original drawing? [For more about Transmission Control Protocol (TCP) is one of the core protocols of the Internet Protocol Suite see here.

Teravus: It is left out because the person who made this diagram had web pages in mind.  Static large files, or small changing files. In the the drawing the fact that TCP streams are smaller then HTTP is on purpose.

Tish: I have heard different opinions on the percentage of the communications for virtual worlds that can be done over HTTP?

Teravus: The fact is that the biggest usage of communications in virtual worlds is transmitting images that’s the number one bandwidth usage. So, if we’re counting by ‘usage’ I say 91%.   If we’re counting by services that use http.   I say probably 75%  I definitely think that http should be evaluated for use on new things ‘first’. But, there are a few places where HTTP doesn’t shine.

I am skeptical about replacing things in the UDP with HTTP  thinking that they’ll ‘perform better. [For more about User Datagram Protocol (UDP) another of the core protocols of the Internet Protocol Suite see here.]

I think there’s been a huge test going on now and for the last 5 or six years with regards to the UDP protocol and it really has performed admirably.   In the last year and a half, I’ve seen attempts to convert several things to HTTP that have failed, and failed somewhat spectacularly sometimes.  In the end the items get reverted back to the UDP protocol. One such item that sticks out in my mind is CAPS(HTTP) based inventory retrieval. The capability to do that in the client has been available since before February. And, it’s been turned on and off on ‘Agni’ at least once in the process. Additionally, we (OpenSimulator) enabled http inventory, and, the  inventory failures rose pretty steeply.

I think some services are really just not ‘right’ for HTTP.. . particularly where a ‘poll’ methodology is used, or, the data is significantly dynamic enough that it makes caching useless.

Anyway, as far as the future is concerned, I do want to see some services over HTTP. Other services, it would be more appropriate to have a TCP stream. Stock market data, for example, uses a TCP stream. The Scalability of the stock market, is just one example of a scalable TCP stream.

Tish: So you see TCP  as the communications protocol that would do the work for the parts of virtual worlds not suitable for HTTP. At least that is how you have shown it in our Web 2.0 architecture drawing. But should there also be a UDP stream?

Teravus: For the virtual world of tomorrow? .. probably not.

Tish: Why not?

Teravus: You have less control over the quality of service when it’s delivered over UDP then TCP.

Tish: What is the exact relation between TCP and UDP.  My understanding is UDP a lower level protocol.

Teravus: TCP offers guaranteed delivery through flow control, while UDP does not.  One of the failures of UDP, is the ‘resend’ technology we’ve put on top of it to try and make it reliable.   TCP does this automatically and better then we could at a lower level but it does also cost up to twice the bandwidth depending on what is being sent. HTTP is a layer on top of TCP.

Tish: So just like the HTTP/TCP discussion there has to be a TCP/UDP boundary discussion …so it is HTTP then TCP then UDP and the boundaries have to be worked on.

Teravus: Those are the orderings in my mind…  probably if UDP uses any..  it should use less then 0.5%.

Tish: And the current Second Life architecture what does it use if it isn’t using HTTP? [to see the work of the Architecture Working Group on the future Second Life architecture here]

Teravus: UDP or HTTP

Tish: and TCP?

Teravus: Well, TCP is a layer under HTTP.  As far as I know, SL doesn’t use TCP streams directly

Teravus: Instead, it uses HTTP polling.  This is one of the places, that I’ve highlighted where it doesn’t shine.

Tish: Polling does sound slow?

Teravus: Polling is essentially..     (connect) Got any data for me? No?(disconnect), (connect) Got any data for me?  No?(disconnect).

Tish: So what is the path to standards for this then?

Teravus: Distilling what we know works and what we actually intend on supporting as far as adoption under these standards.

Tish: Where does MPEG-V fit in?  Have you read their document yet?

Tervavus: MPEG-V is interesting reading…     but is there any working example? I have just the overview. But I’ll read it over to have a better determination of how to ‘keep it in mind’ for the future. It looks like they’ve only really defined the requirements of the MPEG-V spec. The MPEG-V spec looks quite far reaching..  but  the documents so far are requirements and marketing talk aimed toward business people – obviously intended to get more people interested in working on them.

But I have a feeling that any format with MPEG before it will be onerous to support. ..for me it’s too early to tell. It’s quite far reaching…it isn’t anything like ’signal processing’ which the MPEG group is most famous for.

Tish: The whole top down approach of the MPEG-V initiative seems counter to Web 2.0 principles to me.

Teravus: Well, remember..  that even if there’s a virtual world format war (reference to DVD-HD vs BlueRay) we still need to win over the rest of the web.

Tish: Yes and don’t you think the way to win over the web is to use as many existing standards as possible?

Teravus: Well, it’s to use as many existing standards as ‘fit’ though.. KISS, as always (K)eep (I)t (S)imple (S)tupid if we have 30 different internet standards..     people looking at it will @.@

Tish: But it is just lack of documented protocols that has created isolation from Web 2.0?  And really doesn’t it boil down to standardizing that small percentage that is outside HTTP – the TCP and UDP stream that we talked about earlier where the real time stuff that virtual worlds bring to the web happens?

Teravus: no..  actually the HTTP standardization is just as important.

Tish: You mean even though SL used HTTP it isn’t standardized?

Teravus: Not documented specifically.

Tish: And OpenSim is that documented?

Teravus: Not well enough probably to define a standard.

Tish: Is AWG (Architecture Working Group) doing the documentation?

Teravus: working on it..

2) Licensing Issues

Tish: It sounds like some of this work has to go on across client and server.  Are we running into the issue of BSD for OpenSim and GPL for the Second Life viewer?

Tervaus: Well, some of the issue here is license choice.  One of the reasons that libOMV was able to achieve what they did was they did it /before/ the client was open sourced.

Tish: So open sourcing the client actually became an obstacle!!???

Teravus: I don’t think so in a whole.  I think it was great for the community.  I do, however think that C++ UDP stacks will be scrutinized more for GPL license violations because, of course, the client is GPL and C++ .

Tish: It is my understanding that Linden Lab is open to discussions on making the licensing more efficient for the open source community?

Teravus: Well, the client, in a whole, should not be changed as far as the license.   JUST the things that they expect people to adopt should be made more open. If they expect people to adopt PRIMs, then there should be an efficient implementation available for anyone to use..   at the very least, in LGPL format. Otherwise, the die hards are forced to re-implement them from scratch, and most people will just choose something more open.

Tish: Has anyone ever put together a list of the parts that need to be LGPLed?

Teravus: Well, I think it’s there in a few places.  There is at least one jira open on it.

Teravus: A few that come to mind for me..   is the UDP stack and the prim to mesh/UV code.   I think there are some things that can definitely be improved about the UDP Stack.  There are some things, (images come to mind), that would be better over HTTP

Tish: Do you think if the UDP stack were L GPLed that would be a significant help to integrating OpenSim better with the web?

Teravus: Well, it would certainly be adopted by more clients. GPL + (your own code) = GPL Licensed client. LGPL linked library + (your own code) = Your own license.
You still need to mention that you used LL’s UDP stack, and provide the source code for it at request.

The general client itself should remain GPL, it’s better for LL that way.  Just the items that they want people to ’standardize’ on. It would help..   if it was at least LGPL

Tish: And the value to  LL on LGPLing these parts is it helped spread their basic technology while protecting the rest of their viewer?

Teravus: It furthers their goal of standardization on their systems because it allows more people to adopt it for their own uses without worrying about GPL-ing their own client.

Tish: It is hard to standardize without access to the low level parts of the client right?

Teravus: The general population of Developers..     will want a libX that they can plug into their application for communicating.. .  libY to deal with object data..

Tish: Hence your requests for LGPL were  UDP stack and  the prim->mesh/UV

Teravus nods

Tish: and at the moment they only have openmv?

Teravus: That’s the only ‘truly’ open standard right now as far as the LL technology is concerned. OpenSimulator’s use of that data..   could also be seen as a standard..

Teravus: But we have not published anything beyond code..   neither have they  really..  technically..  but their organization of the way things work is very very clear

Tish: What are the most significant limitations of openmv?

Teravus: Probably..  just it not being in c++.   c++ has it’s benefits and it’s pitfalls.  Changes in c++ usually take longer then ones in C#.  But, of course c++ is always faster.  With libOMV It isn’t always clear about what packet is used when.  However, with some experimentation, you can figure it out in 30 minutes or less..

Usability

We didn’t spend much time discussing some of the innovation in usability that this architectural integration into Web 2.0 will enable (more to come on that!). But, Teravus mentioned one interesting use case he is working on.

Teravus: You might also stick a ‘cloud renderer’ into the graphic [Tervaus was looking at the diagram (from Dion Hinchcliffe) that opened my previous post on “Web 2.0 to OpenSim Made easy”  click on the thumbnail below].

Some people have discussed having a ‘video stream’ that’s rendered on the cloud and providing that to flash clients would be the best solution to it for them.

The cloud renderer is for organizations that have large pools of servers with GPUs so would allow for very powerful rendering. The servers can render the scenes and stream them to the low end browsers. It would allow extremely high quality rendering for really low end browsers..  such as ‘cell phones.’

Tish: Is that possible now on OpenSim?

Teravus: Nope.  But it’s something that in the future, I intend on working on. It would essentially be a video [streamed to low end browsers].

Tish: Is that different from what Vollee is doing? The mobile client for SL?

Teravus:  It appears that they are, indeed, pre-rendering the client’s view and streaming it to the mobile device

HiPiHi is taking up the challenge of building a managed world with the emphasis on creating a strong virtual economy and a community built around the trading and creation of virtual goods, just at the moment when Linden Lab is beginning to make serious moves to an open grid (see here, here, and here).

While HiPiHi will not focus on real life integrations or enterprise applications, they will provide APIs for enterprises to do that themselves. They will be using the Chinese micro payment system Alipay, by alibaba which unlike PayPal does not have prohibitive costs for micro transactions.

IBM is a “solutions provider” for HiPiHi helping them design a systems architecture that will facilitate running a scalable world with a strong virtual economy. The early focus of HiPiHi is on building an architecture to support the virtual economy.

Toshitaka Jiku, HiPiHi’s new CTO and Executive Vice-President notes: “Virtual goods will be housed in a server for the purpose of creating a market place that will be our vision for an ebay for virtual worlds, so these virtual goods would be portable as opposed to having avatars being portable first.” IBM is also working with Linden Lab in the Architectural Working Group (see earlier post) on scaling and interoperability for Second Life and interoperability and avatar portability is part of the long term vision for HiPiHi.

And, HiPiHi is partnering with Intel to tap more CPU power. It has often been noted that one of the weaknesses of all current game engines and virtual worlds is they do not tap the power of the new CPUs.

HiPiHi has only 40,000 users so the focus of the public beta, which began April 2nd, will be on community building. While they have a future vision of interoperability with Second Life and other platforms based on the Linden Lab technology, the focus, for now, is on building a Chinese community. But they are experimenting with a dual naming system with avatars bearing English and Chinese names because international communication is very much in the HiPiHi vision of the future.

While HiPiHi plans for some mobile integration early on, IMing friends and inventory management for example, the emphasis at the moment is building the community inworld (also note Second Life’s recent integration with mobile, “Samsung Unveils Second Life..” ). But Toshitaka Jiku, HiPiHi’s new CTO is one of the first to develop a mobile interface for SL. And, Jiku comes from NGI the Venture Capital company that is also backing 3Di, so look for interesting innovation with mobile integration in the future.

While HiPiHi is commonly seen as a mere Second Life clone, the work they are doing with IBM and Intel on the system architecture is hoped to produce some valuable innovation. They are also researching the innovations of realXtend’s client. HiPiHi has a close relationship with OpenSim through their connection to 3Di and with Adam Fisby’s company, Deep Think, that is opening offices in Shanghai. It will be interesting to see how these relationship develop over time. Xu Hui and Philip Rosedale met last year and there is a long term vision of cooperation possible. These connections if they blossomed into cooperation and full interoperability would create a very interesting step forward for positive global development through virtual worlds.

Interview With Xu Hui, CEO of HiPiHi

Xu Hui, CEO of HiPiHi with Toshitaka Jiku, HiPiHi’s new CTO and Executive Vice-President

Bjorn Lee, Senior Manager, Marketing & International Business Development for HiPiHi, did an excellent job of translating for this interview, not only with Chinese and English but with the skillful and patient way he worked with me to find the essence of some of my long stream of consciousness questions! Bjorn also contributed many insights, and Toshitaka Jiku stopped by with some interesting insights into HiPiHi’s direction.

Tish Shute: Please could you tell me about your vision for the future of virtual worlds?

Xu Hui: The global virtual world industry will be undergoing a revolution over the next few years. What HiPiHi, Second Life and the other virtual worlds have done in the past few years has really been just setting the stage, exposing the world and educating on the possibilities – kind of like a proof of concept for what virtual worlds can do and can’t do. The goal this year for most virtual platforms will be to build system architectures that can truly scale for a massively-concurrent user base, in addition to inter-operability.

Tish Shute: I know that HiPiHi has formed a patnership with IBM. Could you tell me more about this?

Xu Hui: We are working very closely with IBM in terms of building new technical infrastructure for our platform. What this means is that we will specifically be collaborating very closely on aspects such as redesigning our architecture.

Tish Shute: RealXtend has been working on interoperability of virtual worlds with “real life” and working on meshes and facilitating 3D imports. Have you been in contact with realXtend yet?

Xu Hui: We are actively researching what realXtend is doing, as with other exciting virtual world technologies; and will seriously considering integrating them with our new system architecture.

Tish Shute: What is the strategy of HiPiHi re building a community of content developers?

Xu Hui: This is one of the focuses for HiPiHi this year. We are looking into incentive systems for content creation, including monetary and non-monetary forms. For the former, our approach will be to facilitate universal trade and have a secondary market for people to exchange their virtual goods, their creations, their applications, and so on. But in China, the model will be slightly different initially as we favor an ecosystem approach where we work with third-party providers of virtual economy functions and services. An example could be in payment systems, where instead of reinventing the wheel, we could explore ways to facilitate transactions conducted through Alipay, often regarded as the Paypal for China.

Tish Shute: I am a Mac user and, of course, I am really interested to know when there will be a Mac Interface and easy to use english version?

Bjorn Lee: I am a fellow Mac user too, along with an increasing number of colleagues. So do rest assured we have an internal Mac evangelist team! Since December, we have had a very basic English version but our lack of bilingual staff has affected the development of a satisfactory support system, not to mention interface only for English users.

Tish Shute: How big is the market in China for Mac?

Bjorn Lee: It is about 1%. But of course it is out of a larger population base here in China. Despite their relative lack of scale among China users, Mac users here are very enthusiastic, grassroots, and very tightly knit. Macbook Air ads are playing heavily across Beijing too and have garnered strong brand awareness among the younger set of Chinese consumers.

Tish Shute: What are some of the goals for the public beta which is starting in early April?

Xu Hui: The Public Beta will begin April 2. For public beta what’s interesting is this dual naming system that we are developing for the very first time. The first phase is for the current Private Beta users to migrate to the new naming system before we release that naming system to the public.

We expect a fair amount of proactive user feedback in the initial days, as with all things new. But I think it’s a good step forward because it will try to bring together the English and the Chinese speaking worlds. In a “first” for the virtual world, this new naming system displays both your English (that we call international name) and your Chinese name on top of your avatar. Across many in-world interactions such as chat, social networking, and for future commerce , we are trying to break down the language barrier in the virtual world.

But we are trying to do something to move forward in trying to foster multi-cultural interaction, with the foreign audience and local Chinese audience. Because there’s a lot of demand from local audiences here who want to internationalize and meet people from overseas and the same feedback is coming from our foreign users such as Suezanne C. Baskerville who seems very keen on learning some Chinese. She would like to put some Chinese and English on her avatar too – it’s like a social “code”, you start putting Chinese words in your avatar and so you say that you know I’m friendly and I’m willing to speak to Chinese users. And so too for the Chinese because with the English names up there it doesn’t look so foreign to the foreign audience.

In the later part of the year after our new system architecture is up, we will begin to consider micro payment systems. But because we are migrating to this new infrastructure, the initial stages of the public beta will just be to get more people to use the tools and continue to gather feedback.

Tish Shute: “What is the business model for HiPiHi?”

Xu Hui: Our platform is oriented more for the individual users, that is the residents as opposed to the enterprises and the corporate residents. A lot of the features we are adding and a lot of the feedback that we’re taking is user centric. But, as for our relationship with corporate residents, we will be opening a series of programs and that includes opening our API to allow development on our platform by the enterprises. We think of it as a self-service approach, in the form of open APIs and maybe incentive programs for enterprises to kind of drive this for themselves. But we will design and customize the platform more according to our core user group which are the non-corporate users.

Tish Shute: What is HiPiHi’s relation to other virtual world initiatives, e.g., Entropia’s and other virtual world start ups in China?

Xu Hui: My starting point in responding to this is the definition of a virtual worlds in our company’s opinion is an open-ended user-directed environment. User-directed means that users would drive the content creation, the development of not just their own content, but also feed back to the company, and what they hope to see on a platform level. Open-ended also in the sense that they can have a freer rein in creating and managing their creations.

Concerning that kind of concept, as it plays right now in China, we are the only company that really does that. A lot of the other initiatives that have sprouted recently from the interested companies or other startups in this space have more of what we classify as virtual communities which means that they place real limits and constraints on the users ability to create, and actually have more control over their lifestyles in these worlds.

We will welcome other players as they enter too. We actually welcome the entry of others into this ecosystem because it helps this ecosystem grow and mature faster. And, it can only be good for the users to have so many different companies push out their products and try to reach out to them. So it’s good because then they’ll be able to make an intelligent choice and see how fulfilling a virtual lifestyle they want.

Tish Shute: How do you plan to expand beyond China and how will HiPiHi differ in other countries? I know Linden Lab has met some interesting legal challenges as they have expanded globally.

Xu Hui: HiPiHi will be the sole platform operator for China. As for regions outside China we will take a partnership approach to finding local companies which will then operate this platform. They will be licensed and hence operate this world on our behalf. Thus they will be entitled to benefits commercially and so will have to be responsible to bear the legal costs and challenges. This will reduce the amount of legal burden on our side. A US based operator of the HiPiHi platform in US will certainly have to follow US laws to be entitled to collect revenues but also they will have to handle US based law suits.

Tish Shute: Will HiPiHi have a strong ID verification system tying virtual identities to real identities as a way to try and control griefers etc?

Xu Hui: This question itself doesn’t address how we think about identity. First, we are not going to have a very strong link between real world identity and virtual identity because we feel that our focus would really be to improve accreditation of what is popularly known as a reputation system for virtual identities. So we will focus on building an attractive incentive program for avatars to view their virtual identities in our virtual world as opposed to saying that you’re going to tie this virtual identity very tightly to your real identity.

We want to create mechanisms to facilitate and encourage residents to improve their in-world reputation. But it doesn’t mean we’re not going to manage disruptive behavior such as griefing, which is already known to create problems for virtual worlds like Second Life.

We will have a monitoring mechanism for these troublemakers in our virtual world. But our intention is to let the actual policing be done by residents themselves, through self-organized groups and features we provide for them. .

Tish Shute: The next generation of the Linden Lab grid architecture will separate avatar identity from what constitutes their environment. Will you be going in this direction too.

Toshitaka Jiku: (HiPiHi’s new CTO) Our server architecture will have a different focus. Our server architecture picks out virtual goods as an item that we will separate from the others in the sense that we are going to place them on different servers first. So virtual goods will be housed in a server for the purpose of creating a market place that will be our vision for an “ebay for virtual worlds”, so these virtual goods would be portable as opposed to having avatars being portable first. These are just our first steps and it does also mean that avatars would be housed in a separate server. But the focus right now is to make virtual goods portable and enable the virtual economy.

Tish Shute: How is HiPiHi going to deal with issues of protecting IP rights? This issue has become quite a difficult one in Second Life.

Xu Hui: This is a very big question. I am just going to lay out some basic principals. We like the concepts of giving back the rights of a media creation and returning it back to the creator, enhancing the motivation and incentive systems for people to share their creations and so on. A lot of our influences come from Creative Commons – that is the first part. The second part is when IP rights are infringed we understand where we stand in the whole legislative environment. We are not a legislative body, nor can we judge or rule on certain issues of conflict. Hence what we can do as platform is to provide the data but when it comes to actually making decisions in the legislative environment we are going to rely on third-party intermediaries. This could involve bringing in real-world law makers and courts to uphold some of these IP right because we can’t do that ourselves. So we do face limits somewhat similar to Second Life.

Tish Shute: What are your goals with IBM and Intel?

Xu Hui: IBM to us is really a solutions company. They have expertise in almost every single aspect of the IP sector which makes them a very good partner for us because we considering the architecting of our systems across all areas, client, the backend, algorithms and so on. They can help because they are pretty broad in their understanding of all IP areas.

But Intel has a little bit more focus. Intel is the father of the CPU. They are still the best right now in their understanding of CPU performance and we believe they are going to continue to lead this sector. So when we work with them it is going to evolve around the understanding of the CPU unit – what kind of features and abilities are we able to extract and are going to be useful for virtual worlds. I think this is something many virtual worlds have not focused on – that is extracting value from the CPU. And where better to find out how than from the makers themselves which is Intel. So we work across a broad spectrum with IBM, but with Intel we work in the vertical, and we drill very very deep.