Jul 162014
Big data takes our online, shopping and social media use it is the business challenge for our time

A story in the Atlantic – Why Poor Schools Can’t Win At Standardized testing – illustrates the limits of Big Data.

When Meredith Broussard tried to computerise the text book inventory of her son’s school district she found the project limited by poor systems, fragmented record keeping and siloed management.

Broussard found the records were manually collated, collected on Microsoft Word documents and emailed to an under resourced office that entered details into an Excel Spreadsheet.

The Philadelphia schools don’t just have a textbook problem. They have a data problem—which is actually a people problem. We tend to think of data as immutable truth. But we forget that data and data-collection systems are created by people.

The human factor is a key limitation with any technology; if people aren’t collecting or using data properly than the best computer system in the world is useless. Garbage In, Garbage Out is a long standing IT industry saying.

Management systems are more than computer networks, they go to the very core of an organisation’s culture which in itself is probably a better indicator of how well a company or institution will survive the current period of change.

Were the Philadelphia public school system a business it would be a very good example of a company on its way to being digital roadkill, that it’s an educational network should worry anybody concerned about the economy’s future. That’s a bigger issue than Big Data.


Jul 052014
supermarket checkouts

US medical centre chain Carolinas HealthCare has started mining patients’ credit card data to predict health outcomes reports Bloomberg Businessweek.

The idea is that by looking at credit information and purchasing records, doctors can anticipate what ailments their patients will present with.

Carolinas Healthcare’s matching of spending patterns to healthy is an obvious application of Big Data which illustrates some of the benefits that mining information can deliver for individuals and the community.

Should the project overcome patients’ valid privacy concerns, this is the sort of application that is going to be increasingly common as organisations figure out how to apply software to their mountains of information.

May 272014
understanding data with computers

For two years we were captivated by spectacular rise of the Bitcoin virtual currency. Allegations those gains were a result of market fixing raise important questions about the integrity of our data networks.

The Coin Desk website discusses how the Mt Gox Bitcoin exchange was being ramped by computer bot network nicknamed Willy.

Rampant market ramping – where stock prices are pushed up to attract suckers before those in know sell at a profit – has a proud financial market history; during the 1920s US stock boom, fortunes were made by inside players before the crash and its subsequent banning in 1934.

So it wouldn’t be a surprise that some smart players would try to ramp the Bitcoin market to make a buck and using a botnet – a network of infected computers – to run the trades is a good technological twist.

Blindly trusting data

The Willy botnet though is a worry for those of us watching the connected economy as it shows a number of weaknesses in a world where data is blindly trusted.

As Quinn Norton writes on Medium, everything in the software industry is broken and blindly trusting the data pouring into servers could be a risky move.

The internet of things is based upon the idea of sensors gathering data for smart services to make decisions – one of those decisions is buying and selling securities.

Feeding false information

It’s not too hard to see a scenario where a compromised service feeds false data such as steel shipments, pork belly consumption or energy usage to manipulate market prices or to damage a competitor’s business.

Real world ramifications of bad data could see not only honest investors out of pocket but also steel workers out work, abattoirs sitting on onsold stocks of pig carcasses or blackouts as energy companies miscalculate demand.

The latter has happened before, with Enron manipulating the Californian electricity market in the late 1990s.

When your supply chain depends upon connected devices reporting accurate information then the integrity of data becomes critical.

Like much in the computer world, the world of big data and the internet of things is based up trust, the Mt Gox Bitcoin manipulation reminds us that we can’t always trust the data we receive.

May 232014

“People are drowning in big data,” SurveyMonkey’s CEO Dave Goldberg says in the latest Decoding The New Economy video.

Goldberg sees SurveyMonkey as bringing order to the world of big data in allowing organisations to put their information in context, “We want people to ask the right questions so we can get better data.”

“Here’s a question I need to answer – how happy are my employees? what do customers think of my new product? What are my students doing at school this year?”

Growing the survey industry

One group that’s uncomfortable with the rise of SurveyMonkey, a privately listed company that’s worth $1.3 billion after a capital raising last year, are traditional market research firms who see the service as putting a powerful tool in experienced hands. Goldberg sees it as an opportunity for the market research industry.

“We’re not replacing market researchers,” says Goldberg, “most people who come to SurveyMonkey haven’t used a market researcher before. It actually probably creates more demand for more sophisticated research down the line.”

Goldberg himself isn’t from a market research background, instead he hails from the tech sector having set up LAUNCH in 1994, one of the early music streaming companies which he sold to Yahoo! in 2001 and became the company’s Director of Music.

He left Yahoo1 in 2007 and spent two years in the venture capital industry before joining SurveyMonkey as CEO in 2009.

Understanding the data

From his experience, Goldberg sees understanding data the key business skill for today’s workers, firmly believing that kids should be taught statistic rather than coding.

“Everyone is going to have to learn how to use data.” Says Goldberg, “someone was asking me the other day about sort of skills should we teach our kids to prepare them for the future and I think the thing we’re not doing enough of is teaching them how to use and analyze data.”

To Goldberg we’re still in the early days of understanding how mobile and social media are going to change business with understanding data being one of the great opportunities.

“Implicit data is really interesting but it tells you ‘what’, it doesn’t tell you the ‘why’, believes Goldberg. “We think what we do is the explicit side, we gotta ask people to get the ‘why.”


May 152014

The first Australian Internet of Things was held in Newcastle today which I MC’d and managed to give a quick presentation on my Geek’s Tour of Barcelona.

Big Data was the big message from all the day’s sessions with every speaker touching on the challenge of understanding and securing the vast amounts of data collected.

It’s interesting how the technologists — and most of the material was quite high level — have identified this as the main problem facing management with the Internet of Things.

A key take away from the forum is that the clear opportunity for entrepreneurs with the IoT lies in giving businesses the tools to understand the data.

One of the reasons for the event was to launch the Kaooma Project that aims to link local businesses to the Internet of Things. The local business angle is something that needs to be explored in more depth.

May 072014

“Know your data” is the key tip for businesses concerned about privacy says Michelle Dennedy, Chief Privacy Officer for Intel Security, formerly McAfee.

“It’s really important to go back to basics,” says Michelle. “We’re trying to do bolt-on privacy, just like we did with security years ago. I think it’s time to take a good look at the policy side, which id called Privacy By Design, thinking about it at early states and being consumer-centric.”

“We at McAfee call it ‘Privacy Engineering'; looking at the tools. methodologies and standards from the past, adding current legislative requirements and business rules then turning them into functional requirement.”

Michelle, who is also co-author of the Privacy Engineering Manifesto, was speaking to Decoding The New Economy as part of Privacy Awareness Week.

A key part of the interview is how Michelle sees privacy evolving in a global environment, “if you’d asked me in 2000 where we’d be today I’d have told you it would be like the 1500s when we were dealing with shipping lanes. We would have treaties, it would harmonised and we’d understand that global trade is a hundred percent based upon sharing.”

“We have instead decided to become a set of Balkanized nations.”

For individual businesses “know thy data,” is Michelle’s main advice. “Know what brings you risk, know what brings you opportunity.”

In Michelle’s view, businesses need to balance the opportunities against the risks and treat customers data with respect as the monetisation policies of many online platforms don’t recognise users’ costs in time and data sold.

As businesses find themselves being flooded with data, protecting it and respecting the privacy of customers, users and staff is going become an increasing important responsibility for managers.

It’s worthwhile understanding the privacy laws as they apply to you and making sure your systems and staff comply with them.

May 062014

Just how hard is it to hide from big data? ABC Newcastle’s Carol Duncan and I will be discussing this from 2.40 this afternoon.

Princeton University assistant professor of sociology Janet Vertesi decided she’d find out by trying to conceal her pregnancy from the internet.

She describes her experiences to Think Progress and the lessons are startling on how difficult it is to drop off the Internet and business databases.

While it’s easy to tritely say ‘don’t use the internet’, Janet found that using cash to avoid being picked up by bank databases raises suspicions while not using discount voucher or store cards meant she missed out on valuable savings.

For many people though dropping off the internet is not an option – not having a LinkedIn profile hurts most job hunters’ chances of finding work while if you want to participate in communities, it’s often essential to join the group’s Facebook page.

The amazing part of all is that Janet herself became a Google conscientious objector two years ago after deciding the company’s data collection methods were too intrusive. Yet she still found it hard to keep the news of her baby off the internet.

Ultimately her friends were the greatest risk and she had to beg them not to mention her pregnancy on Facebook and other social media channels lest the algorithms pick that up.

For Janet, it proved possible but it was really hard work;

Experience has shown that it is possible, but it’s really not easy, and it comes with a lot of sacrifices. And it requires some technical skill. So to that end, it’s my concern about the opt-out idea. I don’t actually think it’s feasible for everyone to do this.

So can you drop off the net? Do you know if you’re on it at all. Join us on ABC Newcastle with Carol Duncan from 2.40 to discuss these issues and more.

Filing cabinet image by ralev_com through SXC.HU