So here it is, as promised in my previous post, my interview with Avi Bar-Zeev! Avi is a key virtual world thinker, now with Microsoft, who has put time and energy “into trying to figure out where the real value is with 3D.”

Avi was co-founder of Keyhole, maker of Earthviewer (which later became Google Earth). Also, Avi developed technologies for Second Life, including the procedural 3D object rendering code.” The pictures above are from his portfolio (from left to right, Linden Lab’s Second Life, World Design’s Virtual Environment Theater, Disney Imagineering).

I requested an interview with Avi because it seems in the recent “cambrian explosion” of browser based virtual worlds there is a lot of confusion about what “in your browser” virtual worlds really bring to the table.

Are they a step backward into little more than 3D chat or a harbinger of a future seamless integration of fully immersive, user generated virtual worlds like Second Life into the web?

After reading Avi’s post, “Volumes of Reading,” where he points out some of the confusions about browser based virtual worlds in this TechCrunch post, notably an elision of the difference between “in-browser-window” and truly “hosted in a browser,” I became convinced that if anyone had a clear view of what was actually going on, and what the future might hold for virtual worlds, it might be Avi.

If you have read Ugotrade before you know I am a tireless promoter of open source virtual worlds, in particular OpenSim. Why OpenSim? Because, as key OpenSim player, Stefan Andersson, Tribal Media AB, puts it, “OpenSim was thought to be a rapid prototyping testbench for wannabe paradigm shifters. That is really the heart of OpenSim.”

And yes, “open and distributed will win!” IMHO.

In his post Avi made some interesting comments about Adam Frisby’s “firestarter” XBAP based in your browser viewer for OpenSim and Second Life, Xenki. So my interview started, with the questions about the significance of Adam’s work, “Creating An In Your Browser Viewer For OpenSim/Second Life.” But Avi’s eloquent answers quickly made for a wider discussion about the future of virtual worlds.

Thank you Avi for being so generous with your time!

Will the future of Virtual World’s be in the browser: Interview with Avi Bar-Zeev

1) Could you explain more of your thoughts on a truly “hosted in the browser” open viewer and what the potential and limitations for such a project are? Have you since your blog post examined further the potential of Adam’s Xenki project in this regard? Are there other truly hosted in the browser projects that you know about that have potential?


There are a number of embedded 3D viewers out there now, without naming any names (one of the caveats of working at Microsoft now is that I should generally try to avoid openly praising or criticizing specific companies, though I’ll make it clear that all opinions offered here are my own and do not represent my employer in any way). Most of those viewers are deployed as ActiveX controls, which means they are essentially compiled OpenGL or DirectX applications with their 3D output windows and input events glued to the browser window, and with an added ability to invoke some of their functions via Javascript.

The adoption rate for those sorts of approaches have been pretty low, it turns out, mostly due to “effort to install” and the issue of “trust.” ActiveX controls can do just about anything on your computer that you can, and even well meaning ActiveX controls could potentially be exploited by malicious websites via bugs, which is coincidentally why they require so much pain to install. You should really be careful about which plugins you let on your computer, because I guarantee not all of them have been thoroughly secured.

Now, any one of those plugins could take off the way Flash has. But it’s a chicken and egg problem — a critical mass is needed to attract a critical mass. Flash broke through largely because of one killer app: web video. Prior to that, it was common to hear that Flash-based websites were over-designed, pretentious and annoying. Along came YouTube and others and now everyone has Flash installed almost by default.

Being “hosted in the browser” implies a few things that ActiveX can’t handle. First, the 3D viewer should run in a sandbox, with prohibitions against accessing or writing to other parts of your computer. That’s a benefit of the XBAP method Adam picked. But “hosted” also typically implies more cross-platform portability, much like Javascript, which his chosen method doesn’t provide. XBAP does work equally well in IE and Firefox on the PC with .Net installed, so that’s a plus. But a Mac version, for example, will be more difficult to come by. To be fair, even Javascript, incl. AJAX apps, often require a few platform-unique code paths, but hopefully only for low-level logic, not the whole app.

The real “hosted” in browser projects out there right now all use either Flash, which isn’t really part of browsers but is so ubiquitous that it might as well be, or Java, which was designed to handle the sandboxing and cross-platform issues right out of the box. Flash is unfortunately still quite slow for any serious real-time 3D. And Java has some lingering problems, like startup time, which I’m still hoping to see resolved someday.

The best hope for true “hosted” 3D viewers though is for 3D to become a native element of existing browsers — OR — for someone to write a new class of browser that’s built to handle 3D and 2D equally well. It’s the best hope, simply because 3D is a superset of 2D — anything you can do in 2D can be done in 3D with some degrees of freedom left blank. 3D elements should be able to be mixed effortlessly with traditional 2D markup and vice versa, selecting, cut, paste, flow, and so on. And that’s never going to happen in a simple plug-in.


2) While Adam explained to me his viewer Xenki will have a tighter coupling than say Adobe Acrobat which opens solely in a new tab, it will not be a truly hosted in the browser and not be using javascript or canvases or anything like that since it lacks the ability to do things like access UDP network stacks, etc. So it seems cross platformness is still an illusive goal!

Is a truly hosted in a browser viewer feasible right now considering the feature sets of browsers aren’t sufficient yet to do 3D graphics or advanced networking. And do you think the latter will ever be the case?


Alas, I don’t see enough 3D support in browsers or even Flash for the very robust and performing hosted viewers just yet, but maybe soon. The best approaches for 3D from a technological perspective are still ye-old stand-alone application, where you simply have more ability to control the variables. The main impetus for hosting or at least plugging these 3D viewers into browsers is a business one — trying to break the adoption limits we’ve seen and make it a seamless web-like experience (assuming the web ever is). Why should anyone care if 3D is implemented differently from text and video when they’re just bopping around the web? But right now, they should, because 3D in a browser window just doesn’t work as well.

On the other hand, one thing the web is very good at is enabling mashups to find opportunities the original application makers never found or had time to explore. And so for the truly visionary companies who are willing to open up their 3D viewers to external mashing, they may find a more organic path to Flash-like ubiquity that no mortal marketing budget can afford.


3) Adam’s work towards a “hosted in your browser viewer” for OpenSim and SL is a separate project to a full scale Open Viewer project. (With Xenki he is aiming for a more lightweight browser-oriented client which will be more useful for people just casually visiting.)

But while doing a from scratch Open Viewer is an enormous and very difficult task, most advocates for open source virtual worlds and interoperability seem to agree an Open Viewer project is very necessary. But at the moment I think Open Viewer is in a very rudimentary stage (unlike OpenSim which is really moving along at a fast pace). What are you thoughts on possible directions for a full on Open Viewer project? How and where might energy be best directed in this regard?


I’m not convinced an Open Viewer will be any more or less successful than other 3D browser plugins, as long as it takes a similar approach. I think you could in this case try something more unique to open source — the main advantage you can exploit is to leverage other open source projects, like, say, Mozilla, with your code to either enhance Mozilla under the hood or produce some new combination, say, using the nice layout engine, JS interpeter/compiler, and network support from Moz with some new 3D rendering layers.

The main problem with getting any 3D viewer adopted by the mainstream isn’t the openness of the code, IMO, but the suitability of the technology, UI, etc..

For example, Real Video was around for a long time. Philip Rosedale in fact was a pioneer in that regard. But Web Video didn’t really explode into UCC until the quality of the experience increased to at least YouTube levels. That meant better compression and bandwidth to avoid the “buffering…” delays, better resolution, more viewable window sizes, and much simpler installation than Real Video offered (i.e., no registration and those annoying ads on my desktop every day — ymmv).

Whoever solves those problems for 3D can become the viewer of choice. And thus far, I’d say Google and Microsoft have the biggest advantages in that regard, if for no other reason than they have market and mindshare to attract lots of people at once and see what sticks.


4) You mentioned that when/if “one of the “3D chat” clients pushes into that territory” it could be a game changer? Could you elaborate more on what you think might be the magic ingredients for this?


Lively could add some real user created content, for example. They haven’t shown much interest in making SketchUp a more vital part of that experience, except maybe at the file level (i.e., create something and then publish it, unlike SL’s instantaneous collaborative editing — I’m referring more to Google Earth now, but I expect this for Lively too). Caligari has been collaborative for a while now, and it’ll be interesting to see what direction they want to go next. Vivaty just announced some more UCC tools, which I haven’t yet tried out, so I won’t comment, except to say that VRML/X3D still feels more like Real Video than YouTube.

That’s all to say that SL is still fairly alone in the degree to which it lets people craft and control the world without direct corporate control. The closest analogs for SL are actually the lego-like worlds like Roblox and, well, the Lego MMO, not other social worlds.


5) Considering your depth of experience in networked 3D environments and, that you have generously offered to work up a blog entry on LLVolume and a procedural geometry primer, do you see yourself becoming more involved with the community effort that is beginning to gather around OpenSim?

I’m a big fan of open source. I’m not as big an advocate of SL’s gridded approach to building the Metaverse, though it works well for them. I’m happy help document the open sourced LLVolume because I had a big hand in it originally and you shouldn’t have to work so hard to reverse engineer it at this point. I’m sure even Linden Lab would want you making better use of that code.

6) I totally agree with you, the future should hold richer, smarter object formats. But at the moment the thorny issue of trust management in virtual worlds seems to be the first hurdle to get over? Do you have any thoughts on this? I am not sure if this is an issue you are interested in? But, just in case, here is the chat on this issue from the AWG to show u the stage this discussion seems to be at in the recent Open Grid Beta between LL and OpenSim.

http://wiki.secondlife.com/wiki/User:Zero_Linden/Office_Hours/2008_August_05).

It’s a huge topic to cover, but I’ll take on a few aspects. One is basic security. A procedural language like Java or Javascript needs to be sandboxed to avoid compromising your PC, as would any system that delivered 3D objects along with procedural methods. But a simple declarative language might be the way to go. I’m guessing that’s one reason why the XAML stuff Adam is working with works as well as it does. Turns out, a simple declarative language could be extremely expressive in terms of 3D shapes as well, and even the term “procedural geometry” may be a misnomer, unless of course actual code or script is being delivered for each new object.

Another aspect of trust is transactional security — maintaining the integrity of who owns what. And on that front, my views might be a bit out the mainstream. Since I believe that no electronic system can ever truly prevent digital copying, we should actually be thinking about how to reward authors in a “post-scarcity” environment, where any object can be freely copied without restriction — it’s the original designs, unique patterns in the universe that we should try to reward (but again, not restrict).

For example, I’d rather see the system track uniqueness in objects, and have the ability to determine which objects are “derived” from which (and by what percentage) to create a sort of “tree of authorship.” People could copy and modify designs at will. But when any derived object makes any actual money, the system, knowing the derivation chain, could flow proportionate revenues back to each partial author. If objects don’t make money, there’s nothing to fight about. But authors never need to worry about who’s copying them — in fact, they’d actually want people to copy them.

In such a system, authors are encouraged to maximize originality and usefulness of their objects because anyone who uses or derives from your object even indirectly could potentially be a revenue source down the road — in fact, the farther down the road, the broader the base in general. Imagine I’m the original author of “the basic sphere” and see how often that would be used or modified by others.

In this case, the algorithm is the final arbiter of how similar two objects are and which came first, so as to handle cases of two people independently creating similar forms. The right kind of modeling system for this approach would work “by example,” encouraging reuse vs. reinvention. But that part is a much longer discussion.


7) Leaving the tech stuff aside for a moment, what about the user experience in a new browser hosted viewer? How should the user ‘be’ in several places at the same time, the way he is on a web site? How would you have a shared-space continuum like SL operate over several ‘pages’?

No one has really solved this yet, and it’s probably the biggest hole in virtual worlds right now. Some people take the literal shot at an answer, saying a “room” is an analog to a website. In that sense, multiple connected rooms (using, say, portals as the analog to href links) makes some sense.

But one of the advantages of 2D web pages is that they can lay out a good deal of information in a simple static (1D scrollable) format. When you start plastering content across multiple walls like some art gallery, many people get lost or frustrated that it’s not all in one easy place. The benefit of 3D should be to add spatial context, not make things harder.

And it turns out, by introducing some cute navigational constraint like walking, flying, etc.., you may actually detract from the user experience as compared to 2D. You’ve added a handicap of having to move at a reasonable pace by real-world or perceptual metrics. vs. the near instantaneous click of a mouse on your 2D screen. You’ve actually lost something in translation.

One experiment I tried for a company tried to work around that by expanding your field of view to a full 360 perceptual space using some rendering tricks. I wouldn’t call it successful, but there were some interesting notions that grew out of it for being able to simply click on what you want vs. having to walk or fly to get close enough or to move around to see something without distortion. I think that’s probably why Lively makes every thing in the world clickable/dragable, which leaves MMO users scratching their head over why you have to drag your avatar around too.

Frankly, right now, I’d be happy to see a mostly 2D web page with 3D elements popping out as needed. We can gradually build up a working vocabulary of 3D concepts in the way we built up a vocabulary of cinematography for understanding emotion in movies and of web browsing too. We tend to rely on those shortcuts without realizing. This will take some time to work out, so any standards we espouse must have the flexibility to experiment built in.


8) Do we need to ‘divide the protocol cake’ – ie, that stuff like the SL protocol, simply is too much for one protocol and a layer model is needed where several standards and protocols can coexist like the web?

Note: This question and the one above come from a chat I had with Stefan Andersson of Tribal Media. Stefan also added: “I mean, that a 3D world shouldn’t be seen as _one_ application, up in the application layer. I believe we need the “http 3.D” and the “vrhtml” which is what prompted this question.

I don’t think one protocol is ever the answer. I think it will be important to fold SL support into any attempted standard. But not everyone wants to build their world out of prims.

On the other hand, I think the other flavors of virtual world protocols could benefit from more procedural support for world-building. One of the reasons they don’t work as well IMO is that they force the client to download way too much data, especially geometry. Download times for 3D worlds need to be comparable to websites or faster (given the real-time rendering), though many currently require several orders of magnitude more data before they grant the user some navigational control. One of the best things about SL’s protocols is how the world is progressively streamed in. That’s the most important part to keep.


Anyway these questions are just for starters. Any thoughts you may have on the development of an open interoperable metaverse/VWs are interesting to me! So please feel free to give me pointers to directions you think are of interest.

You might want to look over some older blog posts of mine about Web 3D, semantics, and even scenegraphs, as they all apply to this discussion. I believe I had a post on post-DRM authorship as well.

The story of Xenki, an in your browser viewer for OpenSim and Second Life (you can download the source on Forge.OpenSimulator), began with a stream of tweets from Dr_Manhattan (Kyle Gomboy, avatar G2 Proto) – power house of the Microsoft Developer Community in OpenSim (see Project Manhattan), and Second Life (see here) – announcing the merging of DirectX and WPF in the .NET 3.51 SP1 release.

I was not the only one to notice these tweets, and the links were soon in the OpenSim IRC channels, and my friend Peter Quirk, EMC, quickly circulated them in the realXtend developers’ Google discussion group. And most importantly, perhaps, almost immediately the phenom developer of OpenSim, Adam Frisby, started coding up an in your browser viewer for OpenSim and Second Life, called Xenki (see Adam’s blog for a series of posts) that show just how fast Xenki is progressing. The picture above is of Wright’s Plaza, OSGrid. As I have been there many times, I can testify it is beginning to become recognizable as Wright’s Plaza in Xenki.

Now Zain Naboulsi, Microsoft’s Virtual Worlds Evangelist, has introduced Adam to David and Gerhard, the Project Manager and Lead Programmer respectively on the WPF Graphics Team, who Adam writes: “have graciously offered me their time to help answer some performance related questions with regards to the Xenki viewer I have been developing.”

Here are some quotes from Dr Manhattan’s first tweets:
“Why is SP1 for Visual Studio 2008 and .NET 3.5 a potential virtual
world shocker? WPF + Direct 3D love! Can you say powerhouse 3D
client?”
“You will be able to treat Direct3D content just like an image within
an application”
“A little bit of grid math translation from OpenSim, some MSSQL 2008
database coding, mix in a little .NET 3.5SP1 WPF goodness=Virtual
Viewer”
“.NET 3.5 SP1 in WPF=new shader effects architecture/API that allows
visual effects to be created & applied to any control/element in WPF.
Read about SP1 here.”

Avi Bar-Zeev, Reality Prime, comments on Xenki

Avi Bar-Zeev was one of the first people to pick up on what Adam is doing with his in your browser viewer – see this post (also see Avi’s blog Reality Prime for a lot of important posts for virtual worlds). Avi is a key virtual world thinker who has put time and energy “into trying to figure out where the real value is with 3D.” He was co-founder of Keyhole, maker of Earthviewer (which later became Google Earth ). Also, Avi developed technologies for Second Life, including the procedural 3D object rendering code.” In this post, “Volumes of Reading,” Avi begins with pointing out some of the confusions about browser based virtual worlds in this TechCrunch post, notably an elision of the difference between “in-browser-window” and truly “hosted in a browser.” Anyway Avi goes on to point out some of the weaknesses of the current crop of browser based worlds that:

still suffer from other in-browser-window issues, like what happens if I hit the back button or refresh the page, or simply scroll the window down so I don’t see my nice virtual world anymore. Or worse yet, what happens to my mouse wheel, for which, for example, the GE plugin steals control. Philip is right that it’s simply not as immersive as a full-screen stand-alone client, and web integration of 3D worlds is a much more complex issue than just placing windows on pages.

Avi reports he followed an interesting chain of progression to Adam Frisby’s blog about his work on Xenki. Now here I take some credit for being an actor in my own story as it was my comment on the Tech Crunch post that led Avi to Adam’s work on Xenki!

I was very excited when I read Avi’s post because I realized one of the best minds in Virtual Worlds was looking at Xenki with interest! So I began to formulate some questions for Avi. See my next post for some fascinating answers to a wide range of questions on the future of browser based 3D, OpenSim and the future of virtual worlds.

Interview with Adam Frisby, on Xenki

Adam Frisby: I’d like to set it up as a viable alternative viewer project – and my goal is to get a revised v 0.1.0 out by the end of the week. Functionally it will be about the same as where I am now (perhaps with some minor additional features), but the interior will be a lot cleaner leaving room for expansion down the track.

Tish Shute: What is the difference between what you have done and what Darren Guard did in some recent late night hacking? (see screen shot below from the post of Stefan Andersson, Tribal Media AB – “Using XABP to embed the Second Life Browser in a Web Page”).

Adam Frisby: He took the same base API that I’m using, but used it to embed the SL Client’s OpenGL rendering surface on it. It’s clever – but I don’t think it’s more than a clever trick because the SL viewer still has all the warts we face. Which means it’s not fantastic in the longer term.

Tish Shute: Have you decided on what rendering engine and physics you will use?

Adam Frisby: Well, physics remains serverside, but at this stage rendering will have to be done mostly in-house I’m thinking, although Francis Chung has expressed interest in helping and has made the suggestion that we look seriously at the Irrlicht engine.

Tish Shute: Is Francis Chung part of the lrrlicht engine team? And when you say rendering in house do you mean little dynamic rendering?

Adam Frisby: No, he’s a SL developer – worked on a lot of high profile projects.

Tish Shute: How does the Xenki viewer project relate to the Open Viewer project?

Adam Frisby: I’ve still got people working for me doing work on OpenViewer and that’s still making it’s own way – but I think there’s two different niches here. OpenViewer is general purpose alternate viewer – much like the mainline SL viewer. Xenki I’m aiming at something possibly a bit more lightweight and embeddable.

Tish Shute: How does using XAPB compare to using Active X?

Adam Frisby: Well, first – it runs on Firefox without anything more than .NET installed. Second – it’s sandboxed. The XBAP has a very tight cage – whereas ActiveX is unconstrained and has fully access to your system. ActiveX is pretty much universally decried these days as a really bad idea.

XBAP addresses a lot of those concerns by at least paying some attention to security.

Tish Shute: So what are the possibilities do you think for a browser hosted viewer for OpenSim/SL?

Adam Frisby: I think it makes some sense to go after the same market as Lively – to grab the instant gratification instant presence that Lively gets right. There’s no reason it should be mutually exclusive. A point I think Philip missed in his recent interview on the subject.

Tish Shute: XBAP is what enabled your sudden progress on Xenki. Can you say, in lay persons terms, what were the key factors that came together to allow you to make Xenki take off at lightening speed?

Adam Frisby: Sure – Programming libraries are basically bits of prebuilt code you can use as shortcuts. For example, Instead of having to program common infrastructure – such as communicating with your 3D hardware to display a simple scene – a library called WPF allows you to do all of that with just a handful of lines of code. The more detailed and well written the library, the faster you can develop your applications. The downside can be that sometimes they also box you into a specific way of doing things.

In this case, the WPF API (which XBAP employs) is very rich in terms of 3D functionality and allows us to create complex scenes fairly simply. It does have some drawbacks (which I wrote about), but it does lend itself to very rapid development.

Tish Shute: The Lively architects did a talk in Second Life and Lively organized by World2Worlds the other day. They said that Lively is based on Active X.

Adam Frisby: Lively uses a standard 3D engine wrapped up to render on an ActiveX control. It’s certainly a lot of work – but for what I’m doing, we don’t need to start with that level of complexity on the table.

For one, it would slow us down tremendously when it comes to simple tasks such as debugging – and mandate a large team in doing so.

Tish Shute: So do you think you will be able to get some of the same distributed functionality that is the most appealing bit about Lively for me at least!

Adam Frisby: Well, distributed infrastructure is a matter for the backend servers I think. So a modular OpenSim enables this potentially hehe!

Yeah, that’s one of the goals OpenSim is aiming for.

Tish Shute: Yes the mantra is “distributed and open will win” right!

Adam Frisby: Yep!

Here are some screen shots from Adam’s blog “of things sort of almost rendering correctly.”

and one more from Adam, “showing the detail of one of Cubey’s planes (in this case his Ornithopter – and it renders correctly!)”