Some principles for good data management

As companies around the world wake up to the new reality that the web flattens the playing field and offers more choices for people than they had before, there is realization that being customer-centric is probably a Good Thing. The next logical conclusion therefore it seems is the need to amass vast mounds of data about their customers, mine it every which way and that, in order to better understand people's behaviour. This has been fuelled by a Google paranoia, in which companies tremble if they don't have a data centre the size of Kansas tracking every possible thing that you, me, and them could, would, should, or might do sometime, yesterday or tomorrow.

My concern is that, as with the environement, the real costs of data collection and not externalized, and so business and governments have little incentive to act responsibly with this data. In addition to my suggestion that the provider is rarely compensated adequately for the potentially interesting uses of their data, the costs of storage continue to plummet, resulting in a simple equation for most managers - grab as much data as you can and hang on to it.

In Europe, data collection has been a hot-button issue for a while with more stringent safeguards on collecting and protecting user data than in the US. (I remember during my time working at Siebel an 'interesting' meeting between one of our more assertive West coast salesmen and an official from a French ministry. The concept of the CRM system that this hapless salesman was attempting to foist on the French government to better manage their citizenry was, we were told in no uncertain terms - illegal. It is against the law in France for this government department to share data on citizens with another department. Clearly, not great news for the efficiency levels and problem solving abilities of the French bureaucrats, and not great for the salesman.) Anyway, if there's money to be made, savvy business people will gravitate to the least cost option, so probably base their policies in the most amenable jurisdictions.

This to say that there should probably be some principles or code of conduct, to use a phrase du jour, for how companies manage their customer data. Kim Cameron's laws of identity seem to be the gold standard here, and are rather related, but not the same.

In conversation recently with a smart chap at one of the great British institutitions (didn't tell him I would blog it, so won't say which one), he outlined his principles about data collection that their organization uses. I noted them down rather quickly, and here is the list (a bit mangled as I'm taking a bit of translator's license here).

1. Ownership: The data about the user is owned by the individual, and companies are able to borrow it and use it to provide better services for the user, while they have a relationship with that user.
2. Minimum: Only collect the minimal data required to deliver certain functions: i.e. "just in time not just in case". So a vendor only needs to collect enough data to know that you have enough money to pay for the goods, and they don't need to hang on to it when you're gone.
3. Modular: The service doesn't need to have an all-or-nothing approach to data for it to work -- the data can be separated and functional with different uses.
4. Tradeable: Data can be used to interact with other services, if you have agreed that it can be shared with them to improve your service. Clearly, generic data such as location has multiple uses, whereas specific proprietary application data has more limited uses.
5. Tangible: It is clear what impact your data has on the system, and whether it's being used to impact the service level that you're receiving.
6. Extractable: You can remove all elements of your data from their system. This is something that Google is apparently doing, which I think is tremendous. Like their search engine, they are happy to send people away, not try and hang onto them if they don't want to be there.

I'd be interested to see other principles relating to good data management - this is now the time to shape people's opinions towards what is still a rather abstract issue, but which is increasingly becoming the cause for debate. And profits.