I’m really looking forward to the O’Reilly Strata events that are coming to NYC in a couple of weeks. I’m fascinated to see where the art, science, and business of data has gone since February, when I attended the first Strata Conference in Santa Clara – a sold out event imbued with an awareness that this was an important gathering of cognoscenti working on the next big thing.

Strata in New York City is a sequence of events,  Strata JumpStart, Sept. 19th, and then The Strata Summit, “The Business of Data,” Sept. 20th & 21st, and followed by the Strata Conference, “Making Data Work,” Sept. 22nd, 23rd.

“The future belongs to those who understand how to collect and use their data successfully.”

Below is a transcript of a conversation I had last Friday with Strata Program Chair, Edd Dumbill about some of the highlights of the schedule from my perspective.  However, I highly recommend taking a good look at all that is planned through the three events because there is a depth and breadth that could not be covered in one conversation.

The video opening this post is from visual.ly.com – a start-up making it easier for people to create, explore, share, and promote data visualizations and infographics.

Talking with Edd Dumbill

Tish Shute: It seems a dialogue between the art of data and the science of data is going to be center stage at Strata NYC, and there will be much discussion about story telling with data.

Is that observation correct or is there something else going on there?

Edd Dumbill: No, I think that’s a great characterization. For the Summit, the core realization for me has been that when you have these tools for getting value from data and when you can drive what you’re doing by data, then actually, the biggest consequences are human ones, and they are organizational ones, and they are strategic ones once you have the technology in place.

So what the summit is doing is really looking at how, in a variety of industries, governments, and within disciplines within those, how the amount of data, the ease of which it can be communicated and mined is changing the way industry is shaped.

Tish Shute: Also, I noticed  that the Strata Summit Schedule (Sept 20th & 21st), and even through to the Strata Conference (Sept 22nd & 23rd), has more of an emphasis on pop culture; sports – baseball, dating – OKCupid, and Narrative Science, all have a place on the schedule, for example?

Is this the culture of New York City being reflected – interests in media and marketing, or is there something else going on?  Has the data tool stack matured since the Strata Conference in Silicon Valley at beginning of the year?

Edd Dumbill: Yes, there’s certainly a different flavor to the event because we’re in New York. And, yes, the tool stack has matured, but it is, by no means mature, and the maturity’s only coming at the lowest level.

I think there’s many years left in maturing the tool stack. But one of the beauties of big data is that once you have the data together, the algorithms to get value from it initially are pretty simple.

So, focusing on the stories of success of being data driven, particularly in the Summit, is important to us because the two questions people are asking are, “One, I’ve got data. Two, What do I do with it?”  We don’t need to make the argument that data is important anymore. But we do need to demonstrate what you can do with it.

The data isn’t necessarily big; it’s just there. It’s about having an analytical approach to your business that compliments your intuition, and compliments your vision.

“One of the most powerful ways of presenting data to people is in a story,” Edd Dumbill

Tish Shute: Yes I can see the emphasis in the schedule on how to tell meaningful stories with data. Narrative Science seem to be doing something very interesting re turning data into stories?

Edd Dumbill: Yes. They absolutely fascinate me with what they do. There’s this kind of hierarchy and sort of chain of needs right now where business is going, “We need data scientists. Find me data scientists. Train me data scientists. Hire me data scientists.” And the data scientists are all going, “I need visualization. I’ve got this data, I now need to turn it back into a story that’s going to be useful to people or provide interfaces that are going to help people understand and explore this,” because it doesn’t scale to have to have an interpreter all the time between the data and the results.

You need to be able to present it in a way that means something to people.

People can look at a graph and get many things out of it, maybe not even get anything at all out of it if they are not used to it. But particularly for digesting certain kinds of high-level summaries and results, if you can put the data back into prose, it makes it very accessible to people.

Tish Shute: Natural Language Generation from data really opens up so many possibilities..

Edd Dumbill: Yes, it’s interesting. I think it’s a very novel use. A lot of people would consider that the end result of their data was a spreadsheet or a graph that they are processing.

But if you turn that back into a story, I think there’s a lot of potential of helping executives understand what’s going on. It makes it possible to use language to understand the results.

Tish Shute: I am really excited to see the emphasis on stories, data design and visualization, and the way we experience data is as much part of The Strata Summit and The Strata Conference as some of the more hardcore big data challenges and analytics stuff.

Edd Dumbill: Yes. We are definitely ramping up on visualization. And I think that’s going to become more important. Having a fundamental grasp of how to use graphics and charts is still incredibly core to what we’re saying. But I’m also interested in ways that go beyond, because at least 50% of the point of visualization is to help people understand the dynamics of the data, to really augment their senses with the results of the computation.

You know, the people who are some of our best leaders, the ones who know how to ask the right questions of the data, have a sort of indefinable fingertip feel that you get for numbers when you live around them for a while. And anything we can do with interfaces to accelerate this is going to be very beneficial, whether it comes to being visual and flying through the data or hearing it in natural language.

Tish Shute: Have I missed anything in that in terms of what you’ve got on the schedule re visualization? VisualizingData.com published an ideal schedule from the visualizing data perspective. But have you added anything recently?

Edd Dumbill: Well, there’s one event which isn’t actually listed on the schedule yet, which is on Tuesday night. There’s a venue called EyeBeam in New York; we’re having a visualization showcase that evening. So there will be stuff to walk around and then a few talks, really from some of the most interesting companies doing viz and viz approaches. So that’s not up on the schedule yet, but that will be in addition. It gives a nice focus on Tuesday night.

Tish Shute: Oh, that’s super awesome. I’ll definitely go to that.

Tish Shute: I am very interested in mobile social communications and augmented reality – especially augmented reality that feels different, not just looks different, as Kevin Slavin puts it.

I am excited to see people thinking about data not just in terms of visualization, but in other ways too that we can feel it through our secondary senses as well (see Mike Kuniavsky’s talk at ARE2011, “Somatic Data Perception”).

Edd Dumbill: Yes, absolutely. That is where we view this as going. I will be incredibly depressed if I’m still looking at the world through a glowing rectangle in 10 years time.

Tish Shute: Yes, it would be! I am looking forward to see the new data start ups too.

Edd Dumbill: Yes, there are a variety of interesting startups, that I feel are particularly important in the data space. Media Sift and Data Sift, for example, Data Sift is doing a lot of real time processing on the Twitter fire hose. They provide real time analytics on Twitter, which I think is very important.

Tish Shute: In terms of using data to provision mobile experiences, real time is massively important, isn’t it?

Edd Dumbill: Absolutely. Yes.

Tish Shute: But real time data is still a big challenge, isn’t it?

Edd Dumbill: Yes. I mean right now, our focus on real time is probably at the technology level. Looking at real time, people are kind of building out the frameworks, companies like Media Sift and Data Sift creating parts of the experience.

And yes, our Where 2.0 conference will be focused more on the mobile experience.

Tish Shute: Re mobile experiences, I am very excited about Infochimps and their new geo APIs, and sensor data is becoming such a big part of the picture now too. But the Kinect has also opened up a whole set of possibilities for the future of sensor data!

Edd Dumbill: Yeah. I still think Kinect is probably one of the most exciting things going down because of the democratization of that kind of capability. Interesting things happen when the sensors become cheap, right?

When alongside a little camera in your iPad you have a Kinect sensor equivalent. That’s become extremely interesting because everybody has it with them and can do things based off it.

So the things that always fascinate me are when it becomes cheap and hackable.

Tish Shute: And if Kinect went mobile, that would be exciting?

Edd Dumbill: I think it’s entirely likely in the next couple years, yes.

The more sensors we can start instrumenting our mobile and personal devices with, I think it’s going to always result in some much more novel uses that we ever dreamed of.

Tish Shute: There was a lot of hoo-ha about Color when they launched this year. They were unable to capture a user base, but if they had issues of privacy might have come to the fore because they were really collecting more sensor data than any other app, right?

We are still waiting to see a breakthrough app in that area in terms of using all the phone sensors in ways that will really enhance a user experience rather than just the aims of data mining, aren’t we?

Edd Dumbill: Yes. I think this is one of the things where, in parallel, we’re really learning out the social and privacy implications of this kind of technology. It seems to me the focus has shifted from the tech in the second half of the year too. Frankly, everybody getting kind of freaked out about the amount of data that’s being mined and, you know, what’s acceptable use for that.

But on a slightly more prosaic level, there are some rather fabulous things being done. If you look at the Google Maps navigation experience on an Android phone. For instance, there’s some very practical applications of sensors collecting data with traffic and a variety of other augmentations going in that to actually do something useful.

So maybe we’d like to think we carry our sixth sense around with us in our pocket, and maybe we will. But we certainly can in our car right now with all the automatic rerouting and so on. That’s slightly more prosaic, but I think a lot more significant in terms of a pattern of how that can be applied.


Tish Shute: One of the Startups that really excited me in February at Strata, Santa Clara was Singly and The Locker project. They are really thinking innovately in the area of putting people at the center of their data.

I am looking forward to seeing the fruition of that work. And, while I’m enjoying Google +, it seems, we are just sort of holding up our hands and saying, “Well, there’s only one business model for data, and that is a centralized Fort Knox,” isn’t it? Or is there something that I’m missing?

Edd Dumbill: You’re right. I mean I think Google +, for instance, is rather the walled garden is a hedged garden. You know, there is a certain barrier there that I think is more about the fact that you need to put certain barriers up to actually create a decent user experience in the first place. I think user experience is one of the BIG problems with open data, and private data, to be honest.

There’s a reason we are not all writing PGP encrypted emails to each other, right? Because it’s so hard to make a UI for encryption that’s safe. Most people don’t use passwords properly. And I think a lot of the same user experience considerations come into this whole data thing.

Facebook can get away with anything they want to because have you ever tried using their privacy settings? Google, I think, more than anybody has tried to address this issue using sensible defaults, making the explanations clear. And they probably succeeded for a geek tech audience.

So I honestly think, probably, Locker’s biggest challenge, in that kind of approach, is definitely UI and giving the concept to the users so they can understand it.

But there’s certainly a very useful contribution to this conversation.

I think there are parallels in blogging, actually. There is a case where people have information they want to disseminate. And do you choose to do in on your own website, set everything up, publish for yourself, host for yourself, so you have complete control, or do you cede, for convenience, control to Blogger or Tumbler, knowing that you are being monetized somehow and that you’re playing in somebody else’s walled garden and don’t have that control?

So I haven’t really expanded that thought too much, but I think there’s something there in following that along and seeing where that actually leads.

But, you know, there is a whole technical challenge as well.

I really like the idea of being able to give permission to people. Being able to say, well, “I’m engaging you to do X,Y,Z in return for such and such. That seems like a good bargain to me. Giving up my data is a decent bargain for the services I’m getting back.” I mean that’s generally the contract we make in real life with people anyway.

That’s another thing re Google+, –why it’s a promising approach. At least in their rhetoric, they’re trying to say, well, “We’re trying to model this on the real life economy, the economy of real life interactions.”

Tish Shute: Yes. Any movement towards saying, well, “I’m not just collecting your data randomly, I’m collecting this data because I want to give something back to you that will enhance your interactions,” definitely feels like an improvement, doesn’t it?

Edd Dumbill: Yes. I think that bargain is clear. I’m just fascinated by who could be trusted and… I do actually wonder if there will be some kind of, rather than necessarily everything being decentralized like Lockers suggests, there might be an idea of a variety of inter-operating, trusted identity brokers. People who we would actually trust. Banks, right? We do that right now. Banks are pretty much our identity brokers. Who knows?

Tish Shute: I think, that is where the Locker project’s going with Singly, isn’t? Isn’t Singly the trusted broker for the Lockers, right?

Edd Dumbill: Yes. Now the question is whether you trust a startup with that or whether you’re going to trust… I mean, who knows? Trust levels are at such all-time lows with everybody right now. People in America won’t trust the government. I think Google are probably one of the most trusted brokers out there online.

Tish Shute: Perhaps, that’s interesting, isn’t it?

Edd Dumbill: I did write a piece, which kind of speculated that Google may become some sort of center brokering of social information and kind of a platform.

Tish Shute: Oh, yes, “Google+ is the social backbone” – a very thought provoking piece! It deserves an interview on it’s own!

But back to the Strata schedule! I notice you have DePodesta doing the Moneyball talk, right? What’s the 2011 twist on Moneyball?

Edd Dumbill: I think the twist on that is that the’re a lot more people can play now, really, which is why we’re having Strata in the first place. That 10 years ago the people doing this kind of stuff are McDonalds and Walmart and sports teams. Everybody, where there was large money, they could afford to gather the data. Maybe they could try this service out in making decisions based on it.

Well, we’re now in a very instrumented society where every business, every person has instrumented data about their interactions. I think the kind of resistance and dynamics and opinions that Moneyball brought up are the ones that people are going to be facing again right now as they seek to be more data-driven in what they’re doing.

It’s also very interesting to know 10 years on, what do you think? You’ve had 10 years of this, of sort of sabermetrics and so on. Have you matured in your view, have you softened?

What I’m endlessly and ultimately fascinated by is, where does this fit in the decision process and in the organization tree? Where does it mesh with vision?

Steve Jobs achieved it perfectly. He had vision and all kinds of things for his products. But Apple succeeded through a relentless operational efficiency. Absolutely relentless in their suppliers, their supply train, their manufacturing lines down to their detail. They are an utterly data-driven, process-driven organization at the same time as melding that with vision, design values and good quality. That’s a case where it worked together.

I’m eager to try and tease it out, figure out how that really works and how those things come together.

Tish Shute: And that’s another thread I see being explored at Strata, NYC. It’s not human versus machine or machine trumps human, but it’s human with machine. This is another theme, isn’t it?

Edd Dumbill: Exactly. We all operate by feedback loops. Really, what machines are doing enables us to get better quality data and in a tighter feedback loop.

Tish Shute: One feedback loop that we’re finding machines very useful for is understanding how we feel. I think that’s really interesting.

Edd Dumbill: Yes. I’m very fascinated by all the quantified-self stuff and where that can take us. At the end of the day, we have a very personal little organization to deal with, which is ourselves.


Quid: Building Software and Mathematical Solutions 
to Simplify Complex Decisions

Tish Shute: Yes! But the thing is we don’t understand ourselves in isolation, do we? I am definitely going to attend the session by Sean Gourley, CTO of Quid, on semantic clustering analysis. It seems like sentiment analysis is going big-time now, isn’t it?

Edd Dumbill: Yes. I mean, sentiment analysis is actually becoming a checkbox feature in databases now. The latest release of Greenplum has it built it. It’s that kind of level of feature that people want as social data is so important. Of course a lot of this is being driven by marketing and advertising.

Tish Shute: Yes but even re marketing data story telling has been taking some interesting and quirky turns hasn’t it?

Edd Dumbill: Yes, absolutely. I think there’s a lot of interesting research ahead of us there as well.

OKCupid Trends

Tish Shute: OkCupid is a very interesting example of data story telling that leverages our desire to know ourselves, and ourselves in relation to others.
.
Edd Dumbill: Yes. I mean they’re an example of a shift that’s happening in the PR industry, actually, which is companies understanding that telling marketing stories with data is very, very compelling. OkCupid really used that to hit well above their weight. Of course they got acquired as a direct result of that and their profile.

Tish Shute: I know OKCupid got acquired by Match.com, but you were saying they hit above their weight by using this analysis? How did that work?

Edd Dumbill: I think a lot of it’s down to their blog. That they analyze these things, publish them on their blog. It got a lot of attention, generated a lot of media stories, which brought them to Match.com’s attention. There’re millions of – well a large number of dating sites. But they differentiated themselves through the smart use of their data.

Tish Shute: Data and Games is an area I am very interested in. Zynga changed the game with game analytics and social games. And now we are seeing Rovio partner with Medio for analytics, (see Green pigs and data). But I noticed that you don’t have games as a strong theme on the schedule?

Edd Dumbill: I think you’ll see more of that on the West Coast to be honest. It’s not that we’re not interested. I just feel that the center of gravity to that topic is probably back on the West at the moment.

Tish Shute: So what’s after Zynga in terms of game analytics? A nice easy question!

Edd Dumbill: Sure. Let me predict the future for you.

Tish Shute: Yes please do!

Edd Dumbill: I don’t know, to be honest. One of the very interesting things about games is that it helps us understand the real world by modeling and playing around. I’m highly fascinated to see some more of those things played out through real life actors. There’s been some examples right out of Scavngr and whatnot. But if any of those techniques can really start to make a way into mobile technology, that’s one interesting thing.

What lessons can we take from what we’ve actually learned in game analytics that are reproducible and useful elsewhere?

Gamification is a bit of a trend right now. I am slightly skeptical… But I am fascinated by a lot of systems that are having these game elements added to them. And so the second question is, if you’re having games added to things, like losing weight or saving money or writing a book, I’ve seen that too, what can you apply from the analytics world on top of that, and learn about systems and tweak them?

I don’t have that good of an answer for you. How my game is, is not steeped in that. But I am aware that there’s probably a lot of progress in games that has yet to be applied anywhere else.

Zynga and whatnot, is kind of a space race, isn’t it, to monetize that. Space races generate technologies that can be applied in a variety of places.

What are the spinouts of game analytics that we can actually use elsewhere?

“These Bloom Instruments aren’t merely games or graphics. They’re new ways of seeing what’s important.”

Cartagr.am by Bloom

Tish Shute: Last February, at Strata, I was very struck by the new work by Ben Cerveny and Bloom on “pop cultural instruments for data expression” (also see Ben Cerveny’s talk at ARE2011).

Edd Dumbill: Yeah. I love every time the visualization comes onto a tablet….there’s an interesting back channel there.

And Google has done this in extreme to add to their great advantage. There’s a potential when you read an E-book, or you interact with the visualization of a tablet, that it can learn from your interactions.

If you read an E-book, and the book is instrumented and sends stuff back, then the book can read you at the same time that you’re reading it. That kind of collective intelligence can then be harnessed.

So what if Bloom’s pop culture visualizations are instrumented so that they know how people are using it? Well what can they learn about that? About either the quality of the visualization, about what’s interesting to data and back at the same time?

This is what the fundamental principles I think even of Web 2.0 and definitely in this era of big data that we’re in, is that the secondary signals, the exhaust from any electronic product, can be incredibly valuable.

We know that every time you run Google you are probably a part of at least one experiment that they are running to determine an optimal, and optimize their product through that. And how can you turn this up to generalize that out?

Tish Shute: I agree.  This is at the core of the art, science and business of data.  I hear your phone ringing, but do I have time for one more quick question?

Edd Dumbill: Oh yes.

Tish Shute: So it sort of follows on from my previous question.  The relationship between the crowd sourced intelligence and machine intelligence has played a huge role in making data work and  solve real world problems – Crowd Flower, for example.

Where are we at now with this relationship between crowdsourcing power of, for example, Crowd Flower and Mechanical Turk when combined with machine intelligence. Is there anything new going on here?

Edd Dumbill: What we’re actually starting to do is learn where to apply these tools. We’re reaching a point of understanding what crowd-sourcing is for, how to better design crowd-source tasks and so on in innovative uses.

One of the things I am particularly excited about is Natala Menezes who was at Amazon working on Mechanical Turk, she’s now moved to a company called GigWalk, which is a Turk platform that’s mobile.

So if you want to assign tasks that depend on people being in particular places and being able to do particular things, this is a platform for turking using that, which I think is fascinating. That’s definitely a new approach.

Tish Shute: Yes GigWalk is awesome – I saw that Photosynth is partnering with GigWalk. That is interesting – perhaps a step towards strong AR! ( see Read Write World and Blaise Aguera Y Arcas’s work on Photosynth was big news at ARE2011).

Edd Dumbill: Natala will be talking about GigWalk. I think the session is called quirky crowdsourcing. I want to call it Quirky Turks.

Tish Shute: [laughs] I like that.

Singly’s appearance at the startup showcase at Strata 2011 this week has excited thought leaders across the web since the story got out. Singly is a new startup that exists to provide oxygen and commercial support to the open source Locker Project, and new protocol TeleHash.

With some wonderful serendipity I met Singly on my first night at Strata.  The next day, I talked in depth to Jeremie Miller and Simon Murtha-Smith, two of the three Singly co-founders (see later in this post). I also had the opportunity to ask Tim O’Reilly, Alistair Croll and Roger Magoulas for some of their thoughts on the significance of this project (see below for their comments).

It was a real – pinch myself in case I need to wake up from a dream experience – for me, to stumble across Jeremie Miller with Simon Murtha-Smith sitting behind a hand written sign demoing Singly at Strata (see my  pic opening this post). As Marshall Kirkpatrick notes:

“Jeremie Miller is a revered figure among developers, best known for building XMPP, the open source protocol that powers most of the Instant Messaging apps in the world. Now Miller has raised funds and is building a team that will develop software aimed directly at the future of the web.”

Singly’s appearance at Strata began auspiciously when they won the judges choice award in the startup showcase. And following Marshall Kirkpatrick’s post, Creator of Instant Messaging Protocol to Launch App Platform for Your Life , and The Locker Project: Why Leave Data Tracking to Others? Do It Yourself, Singly have been burning up Twitter.

Singly, by giving people the ability to do things with their own data, has the potential to change our world.  And, as Marshall Kirkpatrick notes, this won’t be the first time Jeremie has done that.

“Pop-cultural instruments for data expression and exploration,” by Bloom

I was drawn over to the Singly table when an awesome app they were demonstrating caught my eye. Fizz, which is running on a locker with data aggregated from three different places is a first glimpse of one of Bloom’s, “pop-cultural instruments for data expression and exploration.”

Fizz is an intriguing early manifestation of capabilities never seen before on the web – the ability for us to control, aggregate, share and play with our own data streams, and bring together the bits and pieces of our digital selves scattered about the web (for more about Bloom and Singly, see Tim O’Reilly’s comments below). The picture below is my Fizz. In Fizz, large circles represent people and small circles represent their status updates. Bloom says:

“Clicking a circle will reveal its contents. Typing in the search box will highlight matching statuses.
This is an early preview of our work and we’ll be adding more features in the next few weeks. We’d love to hear your feedback and suggestions.”

If you are not already familiar with The Bloom team, Ben Cerveny, Tom Carden, and Jesper Sparre Andersen – go directly to their about page and you will understand why the match of Bloom and The Locker Project is a cause for great delight.

The Locker Project – a whole new way to connect from the protocol up

As Jeremie began explaining the depth and breadth of what The Locker Project is facilitating, I was utterly gob smacked. And when the penny dropped and I realized this is the whole 9 yards, bringing awesomeness to people with a whole new way to connect, from the protocol up, all I could think was, OMG finally!

Luckily I have had time to catch up with the whole team since then, and recovered my composure enough to ask some coherent questions. But I can still barely contain my enthusiasm for this project.

Singly, The Locker Project and TeleHash take on, and deliver a simple, elegant, and open solution to some of the holy grails of the next generation of networked communications. I have written on, and been nibbling at the edges of some of these grails in various projects myself for quite a while now. Even if you haven’t been reading Ugotrade, just a glance at the monster mash of my pre Strata post will give you an idea of how important I think Singly is.

My previous post raised the question of how to invert the search pyramid and to transform search into a social, democratic act. But if you are really interested in social search, I suggest staying keyed into what Singly is doing with The Locker Project!

One of Singly’s three founders, Simon Murtha-Smith, was building a company called Introspectr, a social aggregator and search product. Singly’s other founder Jason Cavnar was working on another similar project. And they came together as Singly because social aggregation and search is a very hard problem for one company to solve, and they realized that the basic infrastructure needs to be open source and built on an open protocol.

As Jeremie puts it, “We shouldn’t…(every startup that wants to do something interesting) have to spend this much time aggregating the data, building robust aggregators.”

To me what is so important about the Locker Project is that it is built on a new open protocol, TeleHash. And having the Singly team focused on supplying tools and the trust/security layer for the Locker Project will mean that developers have the whole stack they need to do some interesting stuff very soon.

I asked Jeremy to explain the relationship between TeleHash, The Locker Project and Singly.

Tish Shute: So TeleHash…

Jeremie Miller: It’s a peer-to-peer protocol to move bits of data for applications around. Not file sharing, but it’s for actual applications to find each other and connect. So if you had an app and I had an app, whenever we’re running that app on our devices, we can actually find those other devices from each other and then connect. Our applications can connect and do something.

For the entire edge of the network, basically, out there in the wild, and let those things mesh together.

And TeleHash is actually what has led to the Locker project itself.

Tish Shute: So TeleHash led to the The Locker Project and the Locker Project led to Singly?

Jeremie Miller: Singly is a company who is sponsoring the open source Locker Project…the three of us as founders, [left to right in pic below - Jeremie Miller, Jason Cavnar, Simon Murtha-Smith, ]

I took the pic above of all three founders being interviewed by Marshall Kirkpatrick of Read Write Web for his post, “Creator of Instant Messaging Protocol to Launch App Platform for Your Life.“  I think we will look back on this moment and say it was an inflection point for the web. At least I tweeted that!

Jeremie Miller: TeleHash is a protocol that lets the lockers connect with each other and share things. The locker is like all of your data. So it’s sort of like a digital person…

Tish Shute: A locker for bits and pieces your digital self?

Jeremie Miller: Yes. So TeleHash lets the lockers connect and directly peer-to-peer connect with each other and share things. Singly, as a company, is going to be hosting lockers first and foremost. But the Locker Project is an open source project. You can have a locker in your machine or you can install it wherever you want…

Tish Shute: Yes it’s often too difficult for a lot of people to set up something locally…so Singly makes it easy to have a locker right?

Jeremie Miller: A lot of people see this cool app or this cool thing they want to do, it’s something that you run in your locker that they need to be able to turn on a locker somewhere very easily.

Tish Shute: So Singly will provide the trust layer and hosting?

Jeremie Miller: Yeah, Singly is a company that will host lockers, as well as…when people build applications that run inside your lockers or use your data, you need to be able to trust them. Maybe it’s like social data and you don’t care that much. But especially once you start to get any of your transactions in there, your browsing history, your health data, like your running logs or sleeping…fit-bit stuff…then it’s much more important to be careful about what you’re running inside your locker and sharing.

So Singly will also look at the applications that are available that you can install and actually run them and look at what data they access, and look at who created them, and be able to come back and either certify or vouch for them.

And I hope in the long-run, as this grows and builds, that power users may actually be able to buy a little device that they can plug into their home network that is their locker. Wouldn’t that be cool? This little hard drive or whatever that you plug in.

Tish Shute: Wow – that would be very cool! Architecturally is TeleHash and the Locker Project related to your work on XMPP?

Jeremie Miller: Architecturally, some of the stuff I’ve learned, XMPP, in Jabber it was designed for the specific purpose of instant messaging, but it was still a federated model, in that you still had to go through sort of a central point so you could…a server that lived somewhere. So it was really optimized for like businesses and small groups, teams, as well as big companies out there; ISPs can use it.

So it was designed with that in mind—for the communication path to be routed through somewhere. And where I’ve sort of evolved over the years since then is really fascinated with truly distributed protocols that are completely sort of decentralized so that things are going peer-to-peer instead of actually through any server.

The last 10 years, peer-to-peer has gotten a pretty bad rap with file sharing.

Tish Shute: A really bad rap, yes.

Jeremie Miller: Yeah. And almost because of that, and because it’s really hard to do, that it hasn’t gotten…the potential for it’s awesome. There’s so many really good things that can be done peer-to-peer. And it hasn’t gotten used very much.

But the other side of the peer-to-peer thing that I think is critically important, look at the explosion of the computing devices around a person anymore, both in the home and on our person. We have one, two, three, four even. And the number of devices that are online for you that are yours…I look at my home network router and I’ve got 30 devices in my house on Wi-Fi. What the heck? That’s a lot of devices.

But right now, all of those devices, for me to work with them, I’m almost always going through a server somewhere, through a data center somewhere, which is ridiculous at face value. You go five, 10 years out from now, there’s probably going to be 300 devices on me in some form.

Tish Shute: So we need a peer-to-peer network just to manage our own devices?

Jeremie Miller: A peer-to-peer, yes. You know, my phone should be talking straight to my computer, or to the iPad, or to the washing machine, or refrigerator. The applications in my TV, or whatever, they should all be talking peer-to-peer. And it should be easy to do that. It shouldn’t be that the only way you can do that is to go through a data center somewhere.

[Our conversation continued, but to sum things up, for now, here is the final question I asked Jeremie which pretty much packs in everything I would like to do with TeleHash, the Locker project, and Singly tools/trust layer all in one!]

Tish Shute: How can TeleHash, the Locker, and Singly help people combine personal data from different sources – web and mobile for example, so the data locked up in our social graph on the web can be integrated with, for example, the location data and “the data wake” from our cell phone sensors, to know not only where we have been but to give us more ways to know where we are going?

Jeremie Miller: That’s a pretty packed question, but here’s my simple answer, hopefully just seeds the right discussion

Telehash is the protocol that lets the apps (mobile, sensor, or anywhere) talk to a locker as well as lockers talk to each other, it’s the chatter, moving the bits around the network. The locker is the storage for a person’s data and the crunching ability to analyze it or trigger actions from it. Singly is the company sponsoring the project(s) and helping anyone dev apps atop it. We’re going to build the platform and looking to the world to create some amazing things on top of it (we have lots of our own personal ideas we already want to create, hah!).

The Locker Project is not just “one more rebel army trying to undo these big data aggregations,” Tim O’Reilly

@lockerproject: “We’ll be posting updates on the Locker Project (https://github.com/quartzjer/Locker) here as we make progress, very awesome stuff “

During the Strata Media Conversation I asked Roger Magoulas about Singly and The Locker Project because Roger played Yentl and brought Singly and Bloom together!  Although there was not much time to discuss it, the relationship of TeleHash, The Locker project and Singly to the social network encumbents, came up, and Roger Magoulis and Tim O’Reilly gave some very insightful comments on this when I talked to them afterwards (see below).

Roger Magoulas pointed out:

“I think Singly has Facebook like aspects, but I think a better description is an app platform that integrates your personal and social network data – including data from Facebook. Sing.ly is likely to have challenges with some of their data sources, particularly if Sing.ly gains traction with users.

I like the app platform business model, although they face risks getting critical mass and app developer attention, and I like how they plan on using open source connectors to keep up with changing social network platforms. Jeremie has credibility with the open source community and is likely to find cooperating developers. The team seems to bring complementary strengths to the project and you can tell they all work well together. ”

And Tim O’Reilly went on to elaborate the awesome potential of this platform to bring something new to the ecosystem, and to comment on just how interesting Bloom’s insight into, “data visualization as a means of input and control” is.

Talking with Tim O’Reilly

Tish Shute: So will the Locker Project be able to break the lock of Facebook’s and other big sites’ control of everyone’s data. Sometimes I feel we are stuck in the era of Zyngification, where you have to do what Zynga did and leverage the system in order to gain traction or do anything with social data?

Tim O’Reilly: I don’t think that is the objective of the Locker Project —to break the Facebook lock, because I tend to agree, the value of Facebook is having your data there with other people’s data. What Singly may be able to accomplish is to give people better tools for managing their data. Because if you can actually start to abstract the data from various sites and you can set it and manage it yourself, then you can potentially make better decisions about what you’re going to allow and not allow. Because right now, the interfaces on a lot of these sites make it very, very difficult to understand exactly what the implications are.

And I think all this done right will create a marketplace where people will build better interfaces that will give people more control over their data. They’ll still want to put it on those sites, because why do you put your money in the bank? You know, because it’s more valuable being with other people’s money.

And I think that to conceive of it as one more rebel army trying to undo these big data aggregations is just the wrong way to frame it.

Tish Shute: Yes and framing the question the way you just did – that this is not just one more rebel army, might mean that the stage at Strata will be filled with new startups next year! That’s what I thought when I found out what The Locker Project and Singly was about – that we are about to see an explosion of creativity with personal and social data.

Tim O’Reilly: Yeah, sure. I mean, because at the end of the day, if you can start to extract your personal data in ways that make it more useful, you can potentially create the ability for people to build better interfaces. It’s not just Facebook.

You know, you think, “Oh, wow, I’d really like to have a management console for all my contacts.” And you go, “Well, I’m stuck with, I can use Facebook, I can use LinkedIn, I can use my address book in Outlook or Gmail or whatever, or on my local machine.” The tools are pretty primitive. And if we get a better set of tools, I think we’ll see a lot of innovation.

Now, some of those startups might well be acquired by a Facebook or a Google. But it if moves the ball forward in giving people better visibility and control over their data, that’s a good thing.

Bloom’s insight, “data visualization will become a means of input and control.”

Tish Shute: I loved the marriage with Bloom, which is interesting, because Ben and the Bloom team haven’t really talked a lot about Bloom yet, but I gather Bloom is moving to consumer facing work with data?

Tim O’Reilly: What’s really interesting about Bloom is the notion…You know people think of data visualization as output. And the insight that I think Ben has had with Bloom is that data visualization will become a means of input and control.

Tish Shute: Right, very cool.

Tim O’Reilly: So I’ve started to feel like data visualization as a way of making sense of complex data is kind of a dead-end. Because what you really want to do is to build these feedback loops where you actually figure something out, some particular atomic action well enough that you can create an application that let’s somebody actually do something with it. But the idea of visualization as a way of manipulating the data in real-time, data visualization as interface rather than as a report, it’s a small but subtle shift that I think becomes kinda cool.

Talking with Alistair Croll

Sing.ly – Join or Die from Singly Inc on Vimeo.

Alistair Croll: So I’m a big fan of Singly. They were my choice for the Startup Showcase. I think it’s certainly the right time – the team can execute on it. But the thing I like the most is I thought back to the early days of Photoshop.

So, Photoshop was a neat application that could take data in the form of an image and manipulate it. But the real value from Photoshop came from these amazing plugins. Like, there’s a company called Kai’s Power Tools that made these things that would allow you to do manipulations. Today, commonplace things that are built in. But at the time, they were things like building bubbles, and spheres, and drop shadows and stuff like that, cutouts, in amazing ways.

Another company, I think, called Alien Skin that made these things. There’s whole ecosystems of plugins. So you could go and get a plugin and transform that original data in ways you hadn’t thought of. And eventually, there was a macro language for scripting how you could do those things, and that found its way into the Photoshop environment.

But you think about the transformation of digital design from Photoshop, I think if you can take that same pattern of you create the basic ecosystem of a few tools, and then you allow people an open system on top of that, that’s unprecedented. I think it really does allow you to take ownership of that.

And then when you allow people the proper tools to federate that information. I was actually thinking of starting a company a couple of years ago based on data federation like that. But what you really want to say is I’ve got a pattern…It’s almost like a multi-channel mixer. You’ve got a band that is your health, your weight, your blood pressure, family photos, words you’ve used. You know, the more data I record when I carry my phone around with a headset of whatever, all of that stuff goes in, all my searches, everything.

And then I say, “Ah, I want to federate height, weight, and blood pressure with my doctor. I want to federate sleep cycles and nutrition with my child’s teacher,” and so on and so on. And you start to create these federated sources of data where now you have a teacher data mining, in a safe manner, the sleep and health habits of all the students along with report card information. And you suddenly realize that Johnny is borderline diabetic and falls asleep at recess.

That’s something that never would have happened. And that happens when you have tools to federate data and then compute on top of them. So this idea of, like, lifestreaming or life logging, this is a logical consequence of the whole lifestreaming movement; that whole recorded future stuff.

Tish Shute: Yes it really is a wonderful fruition to the visions of the lifestreaming movement [see this interview with David Galernter]. And best of all it sits on a new open protocol, TeleHash and the open source Locker Project that will give tools to everyone to work with these data streams.

Alistair Croll: Exactly. This is the toolset that sits on top of that stuff. Because once I’ve life-streamed everything, great, I have this bucket of stuff that I did that I never look at again. But if I can suddenly unlock that with data mining tools and analyze patterns, all of a sudden that life logging has a reason to have existed.

The biggest problem we have with data right now is we don’t have apriori knowledge of what will be useful. We could have been recording crime reports in the city of Chicago, and a year later it turns out that data is really useful for predicting diabetes in the city, but we didn’t know it was related.

So the problem, and one of the things I think that distinguishes big data from traditional data, traditional data is collected to some apriori knowledge of how it will be used. Big data tends to be collected for the sake of…it’s almost collected on faith that later on it will be useful for something.

Tish Shute: I am very interested in this idea of federation, I actually went as far as to deep dive into Wave servers because of this….

Alistair Croll: Yeah, Wave was a great example of federation, just too complicated. When it was canceled, both users [and developers] were furious.

Tish Shute: Yeah, I suppose you could see Google Wave as a bit of an Icarus project, right? I am so excited by Singly because it is coming sort of bottom-up – a very different approach…

Alistair Croll: And remember, Facebook didn’t work before Friendster. The only difference between being wrong and being too early is that too early costs a lot of money. So it may be that this is an idea that works now, but a couple years ago didn’t work.

Alistair Croll, co-chair of Strata 2011, in his post, reframed the question, “Who Owns Your Data?” as, “It’s not who owns the data, it’s about who can put the data to work.”

And I am sure there will be many more people able to put data to work, and into play, in a multitude of  interesting ways, now we have TeleHash, the Locker Project, and Singly.

Photo by Duncan Davidson.
Strata 2011 is presented by O’Reilly Media. Produced by Good Company Communications.

I think the photo above gives a good idea of how I felt on the last day at the Strata conference. Yup – like the cat who got the cream!

And in case you are wondering where AR is in this story – it is everywhere!  Below is a pic of the AR concept designs that were omnipresent in the media communications at Strata.  The one below I snapped off the job board.  But as Sophia Parafina noted,  “AR is maturing from displaying last year’s text bubbles and dinosaurs to big data overlaid on the world.”