UgoTrade

Mirror Worlds will transform the meaning of “computer.” Our dominant metaphor since 1950 or thereabouts, “the electronic brain,” will go by the boards. Instead people will talk about crystal balls, telescopes, stained glass windows, wine, poetry, or whatever – things that make you see vividly. (Mirror Worlds, David Gelertner 1992)

As the meaning of “computer” transforms so will software. Gelertner talks about software as an embodied information machine. And, as virtual worlds come of age so will this notion of software as 3d info machines that we can walk around, tinker with, and hang out in with other avatars and agents in real time. But, exactly how the most complex, crucial, and up to now invisible, parts of our society become embodied in all their glory is not clear yet.

The photo above is used with the permission of Dan Slater. It is taken with an experimental camera Dan built, called the Spherecam. It is a one of a kind ultra wide angle camera that records a scene in all possible viewing directions (4pi steradians). The camera uses a pair of hyperhemispherical fisheye lens to record the scene in all directions.

In Gelertner’s vision the transformation of computers into seeing machines will empower people to understand and work with the machinery of their society. Edward Tufte describes, “Beautiful Evidence,” as “how seeing turns into showing.” But for this to happen, new metaphors for representing complex information will have to emerge. We will have to imagine new ways to deal with the multiple views that are already part of complex modern software (e.g., the four plus one mode view pictured below – see Grady Booch, Turing Lecture, 2007).

And, we will have to imagine radically new ways to view software.

Software today offers assistance to the specialist (in everybody) not to the citizen. The mere citizen deals with the increasingly perilous complexity of his government, business, transportation, health, school, university and legal systems unaided. Mirror Worlds represent one attempt to change this state of affairs (Gelertner, Mirror Worlds, 1992).

Design Patterns:The soul of software architecture

I contacted Grady Booch to ask him about the role virtual worlds may have in the next generation of software design. He is one of the giants of software design and methodology (known for developing the Unified Modeling Language with Ivar Jacobson and James Rumbaugh, and the Booch method of software development). But also he is undertaking an archeology of the essential piece of software architecture – software design patterns – in a project called The Handbook of Software Architecture. The website is under construction and “this is a work in progress, and the chapters of the Handbook will be exposed as each archeologic dig for each system is finished and vetted by the original development team.”

But, I was fortunate to get a sneak preview. The Handbook, while illuminating the brilliance and short comings of software’s past is, in my understanding, about software’s future – discovering the raw materials of software’s future through a meticulous excavation of the design patterns of the past.

“Some people collect stamps, Grady collects software architectures”

The picture above, from The Handbook of Software Architecture, is of an early design drawing for Google. Google are notoriously secretive about their architecture. The “family jewels” of page rank are not elucidated! Grady pointed out the importance of the “barrels” which are the individual servers.

Contradictions of Software-Intensive Mechanisms

The cartoon/design pattern above is from xkcd, A Webcomic of Romance, Sarcasm, Math and Language – hat tip to Cory Ondrejka for this link. The next generation of software design is pushing up whole new classes of applications. The semantic web is not the last word. But, we are confronted by a set of complex challenges and contradictions. Grady points out several of these (see them in full in Turing Lecture,, Grady Booch, 2007).

1) The internet has changed the way individuals communicate and collaborate but it also creates new opportunities for griefers (for a thoughtful look at some of the issues see Identity in a new era, and for a critique of the way griefers get glorified in the media see here). Mitch Wagner of Information Week, (Ziggy Figaro in Second Life) said to me recently:

I certainly despise griefers — they seem to like to spoil things for other people through no other motivation than mean- spiritedness. A devout Christian friend says that spammers and phishers are just plain thieves, but griefers prove the existence of Original Sin.

2) The web provides unprecedented mechanisms for social networking but also new opportunities for theft, fraud and the exploitation of the vulnerable, especially children.

3) Software-intensive systems permit real time and distributed access to information but this can erode privacy and other basic human rights.

4) Email and other software-intensive mechanisms increase the velocity of communication but email and the aging of digital archives threatens the preservation of history.

5) Software-intensive systems create new forms of artistic expression but piracy can dilute the intellectual property of artists.

6) Software-intensive systems enable and accelerate scientific research but they are also at the center of a new generation of offensive and defensive weapons.

7) Software is part of the very fabric of civilization, living in its interstitial spaces but its complexity continues to grow impacting the users as well as the stakeholders in its development, operation and deployment.

“What is software?” and “What are its limits?”

Through the dedication of Grady Booch (Alem Theas in Second Life) – his archeology and anthropology of software’s past, the past is beginning to speak. The archive not only reveals a valuable history: “What worked and what didn’t?” “What was brilliant and what was a failure?” But by creating this unprecedented access to history, Grady gives us a unique opportunity to chew on the big questions and discover the cross cutting zones from which the future will emerge.

There is a grand vision in David Gelertner’s book Mirror Worlds. A vision of a software revolution in which the underpinnings of our global society, the invisible machinery of software, “becomes visible and is transformed into a beautiful, poetic experience that empowers people to understand and work with the machinery of their society.”

In Mirror Worlds, Gelertner points out, ordinary people will be able to poke around in the workings of society, business and government. You will meet software agents and other Mirror World visitors and you will be able to enter a Mirror World through any household computer.

But as Gelertner pointed out this is not “hazy science fiction” – “the tools and materials for Mirror World building are in hand, and the job is underway.” And the challenges are more social than technical. They are at their root challenges of human imagination.

Last week Grady made a presentation on Software Architecture from Second Life to a group of Canadian developers. Grady’s Second Life avatar was streamed into an IMax Theater for a live audience who watched both the stream from Second Life and a PowerPoint presentation side by side. Through a fascinating set of slides of different software architectures, I saw the process of how successful systems grow and emerge. How software architectures manage to change. How the ability to change allows them to endure over time.

Grady argued that natural forces lead to optimal developments. And, for a given domain there is a reasonably optimum architecture for that domain. Also, how domains have tendencies to grow in particular directions, e.g., How Amazon’s investment in hardware and software was applicable to some domains, e.g. Cloud computing.

Grady illuminated the characteristics of the design patterns that animate the software architecture that invisibly guides our society, the mobile phone – “wickedly complex” yet very resistant to change because customers are fickle. Mars Pathfinder – a classic example of “subsumption architecture as Pathfinder works in a semi-autonomous way, Google, E-bay, Amazon – the titans of the web centric development and web centric retail, Citibank, Visa, and air traffic control, MMOGs, and many more.

How software architectures learn

One of the insights of Grady’s archeology is that architecture that can change is architecture that endures over time. And key to this ability to be flexible and to scale, is componentization. Grady pointed out how game architecture and virtual worlds are increasingly discovering componentization. Grady links increasing maturation of architecture to increasing componentization. A quick look at these two design drawings of the Second Life architecture – one of the architecture today, and the other the new design Zero Linden (Mark Lentczner in Real Life) presented, in September, 2007, show this trend (see more of the new structural design drawings here).

Zha Ewry (David Levine, IBM research) explained to me some of the challenges of introducing an increased separation of concerns and componentization to Second Life architecture while maintaining the essence of Second Life, collaboration and dynamic content.

To a point yes, we’ll see more and more componentization. But.. there are some deep limits, driven off of the need to do state melding.

State melding is the dynamic state updating – what makes Second Life the amazing creative, social space it is. The old web works by exporting state on a per state basis. But Second Life takes inputs from 10 – 40 AVs, and 40 times a second spits out a new state. I asked Zha what are the key ways the new architecture design is different from the old?

Separating out a bunch of non state melding activity from the state melding to start. That’s the agent domain

Some examples of non melding activity are Inventory, IM , Estate management, Profiles, Search and Map. As Zha pointed out:

Anything that doesn’t involve generating the next frame of the sim’s state, or has stand alone capabilities based utilities, e.g, buying Linden. The agent domain can fetch you a capabilities (a short term secure access to a web service) to the lindex . Right now, they all route via the sim.

Many Second Lifer’s favorite design drawings from the next generation architecture for Second Life displayed on the Architectural Working Group wiki are the ones showing the designs for running your own region from a home computer.

3D Information Machines

The tools of modern software such as Rational and Eclipse with complex UML models are currently deeply lodged in the 2D realm of the user desktop. But, the third dimension is crucial to Gelertner’s vision for the future of software:

You set up a software mirror wherever you like, then allow some complex real-world system to unfold before it. The software faithfully reflects what it going on out front. But this is a three dimensional kind of reflection: The program reaches out and engulfs some chunk of reality. Like a child- sized play village modeled precisely on a real town and tracking reality’s every move, the Mirror World supplies a software object to match and track every real one.

When I wrote to Grady requesting an interview I asked him the following question:

Do you think one day we can dispense with all those 2D docs and replace them with living 3D software that we can collaborate on in real time?

Grady replied:

and why do you suppose we are not already living in such a world but don’t have the ability to see it! (hehe..one of my fav books is “Better Than Life” by Grant Naylor; it’s the sequel to the book “Red Dwarf”)

The topic of the virtualness of what we call “reality” has been coming up a lot in the WoK forums, led by Piet Hut and Steven Tainer, that I have been attending in Qwaq. And, as the worlds we call virtual become increasingly “real” in ways we have not yet imagined, what we call “real” will be experienced as increasingly virtual. So things are definitely getting very interesting.

In a few years there will be enough computation cycles for ray tracing and avatars will be more of a “real” person tied to bodily movements. Already the riddle of multi threading is the theme of much discussion and talk in Second Life (see this series of Second Life presentations by Intel’s multi threading gurus.)

These mirror worlds will become increasingly rich and, in Gelertner’s vision, they will mark a new era in humankind’s relationship to the human made world. And, “They change that relationship, for good.”

While some would argue that 2D repositories can be transformed usefully into 3D architectures there is a disruptive discontinuity in the phase shift from 2D programs to 3D as 2D respositories do not have a 3rd or a fourth dimension.

Another approach is to root the design process in 3D from the start. From this perspective the sim itself acts as the OS or middleware. Mirror Worlds are native to the 3D environment. And, prototypes for Mirror Worlds, small scale examples for the moment, are already appearing in Second Life and OpenSim.

“Beyond wickedly cool” – A VNOC (Virtual Network Operations Center) in SL

“Beyond wickedly cool,” was Grady’s Booch’s assessment of the 3D info machines of Illuminous Beltran (Second Life avatar), a.k.a Michael Osias, IBM. These are not merely visualizations they are assemblies driven by “live” or simulated data. “When real business logic is in these machines they become more than “visualizations” and models. The become 3D information processing machines.” I asked Illuminous to describe his work for me:

Nearly every technical engineer has used a program like RAD and Visio to structure and describe the architectures they are building. These drawings may be static or updated from an underlying metadata repository, and render information at different levels of abstraction based on the phase of the project. These 2D, semi-static artifacts are shared among members of the team working on the project. Via collaboration tools such as email or content management systems, wikis, etc, these models are refined, new revisions created, and the cycle starts again. Ultimately, these artifacts need to end up in a deployed system, which involves yet another set of tools, skills, and people.

There is a thought that, using Virtual World technologies, the collaboration cycle time between design, refinement, development, and deployment can be drastically reduced. The idea is that, 3D elements of what we may consider 2D software, are built in the virtual world. What does software ‘look’ like? With the phrase ‘function drives form’, they look like what they are. Components such as sockets, servers, subsystems, and applications all have well defined structural aspects that can be represented in 3D. That is not to say, every line of java code becomes a 3D object, but rather atomic elements – be they objects or collections of objects, subsystems, or other ‘normalized’ elements that represent structure and function, but also make sense.

You could stop there, and say we have a static ’3D software sculpture’, and gain some value by having multiple avatars collaborate on the structure of the system, making changes in realtime. This provides great deal of value by allowing everyone involved in the collaboration to understand and see in realtime, the structure of the system. Taking it a step further, why not give these ‘sculptures’ behavior that mirrors the ‘real’ components? Such as data flows in and out, animations, color changes, even adding in tiny screens, consoles, buttons, network connections, gauges, meters, and even sounds. These elements can be programmed to respond as the ‘real’ component responds. Therefore turning our 3D sculpture into a ’3D machine’. Now we can determine, collaboratively, how the system evolves over time given various stimulus and behavior. Connecting these components together, like connecting wires to electrical equipment, allows composite systems to be built.

You could also stop there, with significant value in not only understanding the structure, but how the state of all components evolve over time given specific stimulus. Or, you could feed key data elements from the APIs of the ‘real’ systems to create an operational mirror image of the system. Finally, to take it a step further, you could begin to move function into the 3D machine, and begin to turn off the functions of the ‘real’ system…after all data is data, and logic is logic, no matter if the runtime is in java or the Virtual World. The data and algorithms, and the behavior of the machine, come to represent not just the ‘living’ architecture, but it becomes the ‘operational’ architecture as well. Of course not all algorithms and data volumes are appropriate for the Virtual World 3D Machines. In these cases, a hybrid of real processing versus manipulation of high performance systems via APIs is an option. This final phase, is where the design, development, and deployment, could become a single entity.

New Forms of Collaboration

The biggest problem in computational science, which is quickly becoming all of science, is to find ways to let scientists write software together. (Piet Hut)

There is much research going on in many different virtual world platforms on how virtual worlds can best be used for collaborative software design. Piet Hut is working with astrophysicists in Qwaq and Second life in a project called MICA, (Meta-Institute for Computational Astrophysics), to explore the possibilities for scientists.

The IBM Project Bluegrass from Li-Te Cheng, Steven Rohall, and John Patterson, The T.J. Watson, Collaborative User Experience group, is looking at virtual worlds for developer collaboration and to support distributed work. They are focusing on some of the complex aspects of visualization of work flow processes that are part and parcel of modern software design, and the critical social aspects of working across culture and geography.

Collaborative applications such as Rational Jazz for software development and Lotus Notes for business processes, provide team support for “heads down” work. However, as teams become more distributed, it is important to support “heads up” work–the kind of social interaction that is achieved by seeing people in the hallways when they are collocated.

A video demonstration of Project Bluegrass can be downloaded in Second Life at IBM Codestation. Bluegrass is a research project. They are considering the strengths of various platforms. But, their basic approach is the creation of a virtual world client installation, a VW plug in for the 2D collaborative environments of Lotus Notes and Jazz. At the moment, they have a research prototype using Torque which is currently available for use by IBM teams.

New Collaborative Tools Evolving From 3D Environments

Bluegrass uses virtual spaces to find ways to enhance the 2D environments in which developers work today – modern developers are dealing with a level of complexity that has already usurped the real estate of their screens. But there are some interesting experiments on design and collaboration native to the 3D environment emerging in Second Life.

These experiments, currently, may seem mere toys in relation to the needs of enterprise software development. But because they are evolving directly in the 3D environment – these under-featured solutions may contain the seeds of innovation that will kill todays giants. (see Innovators Dilemma).

The following pictures show Vision Raymaker (Marco Vanadia in RL), and JonnyBee Cioc’s, Spatial Mind Map project. Their source code is freely downloadable here and the latest version is here. It’s released under the Creative Commons license. They developed Spatial Map using this LSL editor – an off-world IDE integrated Developer Environment developed by Alphons Jano (SL Avatar), Alphons van der Heijden in RL. Vision told me he considers this the best off-world IDE for LSL. “This editor allows you to edit/compile & debug one or more Linden Scripting Language sources in its simulation environment (when you press a F key you can test almost all functionality as if you were in SL). It can update itself every time you open it via internet and offers help from an extensive help file, or the LSL wiki. “Vision (Marco Vanadia is becoming the first (perhaps!) Italian scripting mentor in Second Life.

Spatial mind map to public brainstorm about SL land joint venture

Talking about Singularity through a mind map

The spatial map mind mapping project was born on Vulcano, “an open community, where experimentation is encouraged to flourish, and the consequences of applying common sense and bottom-up self regulation, enable creativity to mature.” David Orban the founder of Vulcano told me: ”

I am very proud of this concrete example of its fertility. An idea is just a humble starting point, as we know, and before we can measure its validity, in the physical world there are enormous hurdles to clear, many of which don’t relate to the idea itself, but are inherent to the rules we are accustomed to obey. In online worlds on the other hand, many of these barriers do not exist, and ideas can evolve very quickly, as they are more easily tested in their utility. I fully expect Vulcano to give birth to many other excellent ideas as well, and my best wishes to Spatial Map and all the others is to succeed, and progress!”

Malachi Mulligan is working on another mind map project on Vulcano. And, see also the opening of the Pyramid Cafe, another awesome manifestation of Vulcano creativity (read more here in Italian and here in English).

Spatial Map Creators JonnyBee Cioc and Vision Raymaker

Software Architects Meet to Play Go in Second Life

Last Thursday, a group of software architects including Saijanai Kuhn (Lawson English in RL), from the AWG, and Zha Ewry, the IBM representative to AWG gathered on the Second Life Go sim created by Zarf Vantongerloo and BamBam Sachertorte for a great match between Zero Linden (as his alt Zarf) and astrophysicist Piet Hut. It was a brilliant and wonderful match to watch. Zero is a 9K player and Piet is Shodan level. The conversation during the match was fascinating too, and touched on the aesthetics of game patterns and the best way to improve one’s skills by replaying the games of the great masters until the patterns become internalized. Thanks Zha for these great pictures! Click here visit Go in Second Life (and see the newsblog and the website for a schedule of events).