Improving the performance of social networks with real world data

Today's online social networks have done wonders in connecting you with what matters to you, given that they have very little data to go on. And this data generally requires you to painfully input it at each site, resulting in a proprietary lock-in. We manually add our friends and interests in a binary fashion that ignores the glimpses, hints and nuances of reality. What might be useful here would be to have a 'friendly fly on the wall' - watching you, and allowing these services to understand you better and therefore serve you better. (Note: this would be primarily of interest for those of us who are realists rather than the escapists.)

As far as I see it, there are two types of data that could be useful to capture to improve the performance of these networks: digital and analogue. Google is already doing a rather good job at capturing the first type - starting with your web-browsing clickstream and moving into your email and documents. They are starting to sniff around the second type of data which is naturally harder to come by, with Checkout. However, this is where their exalted web-only existence puts them at a disadvantage against the relative neanderthals with desktop presence and the and the positively jurassic players in the telco space. Desktop apps are much better at collecting non-web data such as what iTunes is playing, and mobiles are the ultimate fly on the wall for capturing and brokering the real life data stream (we've been using the term "Lifestream" for this).

Mobile devices could come closer to improving the equation - they know where you go, who you speak to most, will know what you listen to (if you believe Tomi), what you watch, and the list goes on. In short, they have a better chance of knowing who you are than yet another social network service that is about as smart as a new born chicken. So, what irony that these are the most successful social network services -- making do without access to these multiple rich data streams, but with crumbs from the user's desk.

I've being using this rather crude sketch to illustrate this irony -- by integrating mobile-generated data into social networks, the services could be better. (Btw - this ideal of implicit data is to my mind more of a feature than a new paradigm of the web.) So, while in the PC world I am sitting alone at my desk trying to figure out who are my top 8 friends, a social networking service that I allow to have access to my implicit location, purchase, media and communication service (privacy issues being resolved with a wave of a magic wand...) could do a lot more heavy lifting on the back end and act as a broker to my actual, rather than imaginary friends. Naturally, all the implicit or supplementary data that could be relevant for such a service does not have to be mobile, it's just that much of it is likely to be.

An ohmygod moment I had today brought this home to me. I saw LastFM's events tab -- populated with concerts in London over the next couple of months, most of which i'd really like to go to. Far from this being some happy coincidence, it was purely the result of their site connecting their listings with the filter of my recently scrobbled music. I was immediately drooling at the possibilities -- what else could I listen to, watch, see, and who, could I meet. Hey, life's complicated and busy enough already, and I'm more than happy to outsource short-list making to experts, if they have enough information to go on to make good decisions.

So, what interests me now is how to move from painful, limited, proprietary data entry into every online YASN towards an open model whereby you own your data, and can plug it and to different service providers. Things need to change from the telco side (APIs, walled gardens, data & location tariffs, richer and easier developer environements...) but also greater adoption of microformats, open APIs and clearer transparency on data policies by online sites would be helpful. Will the mobile move from wireless telephone to best friend supplement - a broker to help me navigate news stories, purchase suggestions, places to go and people to see?