One thing former Obama 2012 campaign CTO Harper Reed cannot be accused of is subtlety so his statement at the Sydney CeBIT conference last week that Big Data is Bullshit wasn’t wholly surprising.
Reed has a good point – like all IT industry buzzwords there is a fair degree of hype and BS around Big Data although his referring to it as a storage problem misses the point.
Data storage is a problem largely solved; when we’re talking about Big Data today, we’re talking more about analysing the information and managing the life cycle of an organisation’s data.
Not that these issues are new, the tech industry has been dealing with the challenges of storing, managing and analysing data since computers first appeared. In fact, that’s the reason computers were invented.
An excellent NY Times Bits blog post expands on Harper’s views and rebuts many of the myths and hype around big data.
Most important is the point that big data is not the truth, we can torture those bits and bytes to tell us anything we like.
Claims that Big Data can tell us everything or that it will conquer discrimination and make cities smarter are fanciful. It all depends on how we choose to use the data.
There are downsides with Big Data too — we live in an age where it’s easier to let the algorithm do the work and if the computer says ‘no’, then we can shrug and say “sorry it’s beyond our control.”
Letting the algorithms run our lives is one of many risks, but it doesn’t change the opportunities for businesses, governments and communities Big Data presents. If we can understand our world better, we can do smarter things.
That’s the real opportunity with Big Data and we don’t need the hype to tell us that.