Improving the performance of social networks with real world data

Today's online social networks have done wonders in connecting you with what matters to you, given that they have very little data to go on. And this data generally requires you to painfully input it at each site, resulting in a proprietary lock-in. We manually add our friends and interests in a binary fashion that ignores the glimpses, hints and nuances of reality. What might be useful here would be to have a 'friendly fly on the wall' - watching you, and allowing these services to understand you better and therefore serve you better. (Note: this would be primarily of interest for those of us who are realists rather than the escapists.)

As far as I see it, there are two types of data that could be useful to capture to improve the performance of these networks: digital and analogue. Google is already doing a rather good job at capturing the first type - starting with your web-browsing clickstream and moving into your email and documents. They are starting to sniff around the second type of data which is naturally harder to come by, with Checkout. However, this is where their exalted web-only existence puts them at a disadvantage against the relative neanderthals with desktop presence and the and the positively jurassic players in the telco space. Desktop apps are much better at collecting non-web data such as what iTunes is playing, and mobiles are the ultimate fly on the wall for capturing and brokering the real life data stream (we've been using the term "Lifestream" for this).

Mobile devices could come closer to improving the equation - they know where you go, who you speak to most, will know what you listen to (if you believe Tomi), what you watch, and the list goes on. In short, they have a better chance of knowing who you are than yet another social network service that is about as smart as a new born chicken. So, what irony that these are the most successful social network services -- making do without access to these multiple rich data streams, but with crumbs from the user's desk.

I've being using this rather crude sketch to illustrate this irony -- by integrating mobile-generated data into social networks, the services could be better. (Btw - this ideal of implicit data is to my mind more of a feature than a new paradigm of the web.) So, while in the PC world I am sitting alone at my desk trying to figure out who are my top 8 friends, a social networking service that I allow to have access to my implicit location, purchase, media and communication service (privacy issues being resolved with a wave of a magic wand...) could do a lot more heavy lifting on the back end and act as a broker to my actual, rather than imaginary friends. Naturally, all the implicit or supplementary data that could be relevant for such a service does not have to be mobile, it's just that much of it is likely to be.

An ohmygod moment I had today brought this home to me. I saw LastFM's events tab -- populated with concerts in London over the next couple of months, most of which i'd really like to go to. Far from this being some happy coincidence, it was purely the result of their site connecting their listings with the filter of my recently scrobbled music. I was immediately drooling at the possibilities -- what else could I listen to, watch, see, and who, could I meet. Hey, life's complicated and busy enough already, and I'm more than happy to outsource short-list making to experts, if they have enough information to go on to make good decisions.

So, what interests me now is how to move from painful, limited, proprietary data entry into every online YASN towards an open model whereby you own your data, and can plug it and to different service providers. Things need to change from the telco side (APIs, walled gardens, data & location tariffs, richer and easier developer environements...) but also greater adoption of microformats, open APIs and clearer transparency on data policies by online sites would be helpful. Will the mobile move from wireless telephone to best friend supplement - a broker to help me navigate news stories, purchase suggestions, places to go and people to see?

Cleaning out my closet; reheating old visions

Taking advantage of the semi-annual quiet time, I've been cleaning out my flat and unearthed my business school application that I made almost exactly 7 years ago. That fact alone is pretty scary. It's always fun to look back and see what one writes on these things. Happily, we're not held accountable to these flowery missives, constructed to flatter and to please - after all, the forward looking statements about ones own future are anything but modest. Heck, this is an American school I was applying for, so darned if I wasn't going to big it up with the best of them.

Anyway, what has struck me as odd is the way that the vision that got me excited way back in 1999, is still pretty accurate as a description of my goals, and is actually getting closer. I bought the URL Global-eye.com, and had the idea of an interactive visual / geographic search engine that would let you zoom around the world and then 'rightclick' on a person or place to discover more info, and that would [abracadabra] solve world peace. I wasn't very specific on just how it worked or led to world peace, but the remarkable thing to me is i) that I would still sign up to this vision if I could, and ii) we seem to be very close to reaching it.

Here's what I wrote, as an intro to the "career aspirations" section.

10:00am, Feb 23, 2010
Flying back from New York to London, I receive a call from my 6 year old daughter to my Geva (Global-Eye Virtual Assistant) who shows me her latest toy. Global-Eye, the company I started three years after graduating from Harvard has just reached a partnership agreement with AmericaOnTime - another information company. We have just launched Eye to Eye - a project that promotes understanding and economic development around the world. Citizens, handily equiped with a Geva handset, have unlimited access to data or people, using an interactive visual search engine linked to the virtual world.

I hope that sick bags were provided in the flight, cos it is rather slimey (and that's after I edited out the bit about coming top in my class. Yeuck.) And this shows that I'm clearly failing on the social and professional goals (no kids, no startup, no Big Deals) that I set for myself. But more interestingly, perhaps they're still the right ones. I do find it amusing that I've ended up working on how to innovate around Internet services on mobile devices, and of course creating my own startup, and having kids is still part of the Grand Plan. Up to this point I had absolutely no technology experience, had been working in trade policy for the previous 5 years, and Nokia meant nothing to me apart from a Japanese sounding phone company. I think I'll check back on this aspiration on a regular basis as the time comes for yearly evaluations of my own progress.

Mashups need microformats

Mash potatoes need bangers like mashups need microformats. Ok, not the best analogy in the world, but still. Mashups hold the much-vaunted promise of personalized, tailored services that might conceivably actually be what people want to use a particularly large problem on the mobile. These two posts clearly demonstrated to me the importance of structured data in enabling mashups, and the non-coincidental fact that most mashup examples show Google Maps since the address format is ready structured.

With that in mind, two more promising areas to go next with mashups in my view would use existing structured data: music (CD metadata is abundant) and people (telephone number as contact, until FOAF or similar makes headway). On top of that, general support for open microformats should clearly be seen as an elevated priority.

Making your work seem like the customers' fun

Google's ImageLabeler is a cute game that exploits people's competitive nature - racing to tag an image with the same words as your online partner - for some spurious reward (to be on a score sheet) and results in improving Google's labelling of images. Very smart. In a similar vein, LikeBetter lets you work to divine your true nature for the promise of them acting as an online horoscope reader. In return, they can probably extract tons of useful data about personality types.