Tag: big data

A big reset button on business

Business is like a petrie dish says Internet pioneer and Cluetrain Manifesto author Doc Searls about companies on the web.

“Every large company is just another color of a spore in a petrie dish.”

For the latest Decoding the New Economy video Internet Pioneer Doc Searls discusses The Respect Network, online privacy and the future of business on the web.

Doc Searls is one of the internet’s pioneers who helped write The Cluetrain Manifesto that laid out many of the ideas that underpinned the philosophies driving the early days of the internet.

Searls’ visit to Sydney was part of the rolling worldwide launch of the Respect Network, a system designed to improve internet users’ privacy through ‘personal clouds’ of information where people can choose to share data with companies and others.

A big reset button on business

In many ways The Respect Network shows how the internet has evolved since the days of the Cluetrain Manifesto, something that Searls puts in context.

“We wrote the Cluetrain Manifesto in 1999,” says Searls. “At that time Microsoft ruled the world, Apple was considered a failure – Steve Jobs had come along and they had the iMac but it was all yet to be proven – Google barely existed and Facebook didn’t exist at all.”

“On the one hand we saw the internet, we being the four authors of the Cluetrain Manifesto, and this whole new thing in the world that basically hit a big reset button on ‘business as usual'”

“It did that. I think we’re vindicated on that.”

Resetting business

“What we have now are new industrial giants; Apple became an industrial giant, Microsoft are fading away, Nokia was the number one smartphone company and they’re all but gone.”

One of the key things with today’s markets in Searls’ view is the amount of information that businesses can collect on their customers; something that ties into the original Cluetrain idea of all markets being conversations.

With the evolution of Big Data and the internet of things, Searls sees challenges for companies using old marketing methods which rely upon online tracking. Something that’s a challenge for social media services and many of the existing internet giants.

“The interesting thing is there’s a lot more intelligence that a company can get directly from their customers from things they already own than following us around on the internet.”

Breaking the silos

Searls also sees the current trend towards the internet being divided into little empires as a passing phase, “every company wants a unique offering but we need standards.”

For Searls the key thing about the current era internet is we’re only at the beginning of a time that empowers the individual,  “the older I get, the earlier it seems.”

“Anyone of us can do anything,” Searls says. “That’s the power – I’m optimistic about everything.”

The limits of big data

A story of lost school books illustrates the limits of big data

A story in the Atlantic – Why Poor Schools Can’t Win At Standardized testing – illustrates the limits of Big Data.

When Meredith Broussard tried to computerise the text book inventory of her son’s school district she found the project limited by poor systems, fragmented record keeping and siloed management.

Broussard found the records were manually collated, collected on Microsoft Word documents and emailed to an under resourced office that entered details into an Excel Spreadsheet.

The Philadelphia schools don’t just have a textbook problem. They have a data problem—which is actually a people problem. We tend to think of data as immutable truth. But we forget that data and data-collection systems are created by people.

The human factor is a key limitation with any technology; if people aren’t collecting or using data properly than the best computer system in the world is useless. Garbage In, Garbage Out is a long standing IT industry saying.

Management systems are more than computer networks, they go to the very core of an organisation’s culture which in itself is probably a better indicator of how well a company or institution will survive the current period of change.

Were the Philadelphia public school system a business it would be a very good example of a company on its way to being digital roadkill, that it’s an educational network should worry anybody concerned about the economy’s future. That’s a bigger issue than Big Data.

 

Staying healthy with Big Data

Doctors are starting to match shopping patterns to health problems

US medical centre chain Carolinas HealthCare has started mining patients’ credit card data to predict health outcomes reports Bloomberg Businessweek.

The idea is that by looking at credit information and purchasing records, doctors can anticipate what ailments their patients will present with.

Carolinas Healthcare’s matching of spending patterns to healthy is an obvious application of Big Data which illustrates some of the benefits that mining information can deliver for individuals and the community.

Should the project overcome patients’ valid privacy concerns, this is the sort of application that is going to be increasingly common as organisations figure out how to apply software to their mountains of information.

A bot named Willy and the risk of trusting data

Allegations of Bitcoin market manipulation are a reminder of the risks in blindly trusting data.

For two years we were captivated by spectacular rise of the Bitcoin virtual currency. Allegations those gains were a result of market fixing raise important questions about the integrity of our data networks.

The Coin Desk website discusses how the Mt Gox Bitcoin exchange was being ramped by computer bot network nicknamed Willy.

Rampant market ramping – where stock prices are pushed up to attract suckers before those in know sell at a profit – has a proud financial market history; during the 1920s US stock boom, fortunes were made by inside players before the crash and its subsequent banning in 1934.

So it wouldn’t be a surprise that some smart players would try to ramp the Bitcoin market to make a buck and using a botnet – a network of infected computers – to run the trades is a good technological twist.

Blindly trusting data

The Willy botnet though is a worry for those of us watching the connected economy as it shows a number of weaknesses in a world where data is blindly trusted.

As Quinn Norton writes on Medium, everything in the software industry is broken and blindly trusting the data pouring into servers could be a risky move.

The internet of things is based upon the idea of sensors gathering data for smart services to make decisions – one of those decisions is buying and selling securities.

Feeding false information

It’s not too hard to see a scenario where a compromised service feeds false data such as steel shipments, pork belly consumption or energy usage to manipulate market prices or to damage a competitor’s business.

Real world ramifications of bad data could see not only honest investors out of pocket but also steel workers out work, abattoirs sitting on onsold stocks of pig carcasses or blackouts as energy companies miscalculate demand.

The latter has happened before, with Enron manipulating the Californian electricity market in the late 1990s.

When your supply chain depends upon connected devices reporting accurate information then the integrity of data becomes critical.

Like much in the computer world, the world of big data and the internet of things is based up trust, the Mt Gox Bitcoin manipulation reminds us that we can’t always trust the data we receive.

The Australian Internet of Things Forum

The first Australian Internet of Things Forum is launched

The first Australian Internet of Things was held in Newcastle today which I MC’d and managed to give a quick presentation on my Geek’s Tour of Barcelona.

Big Data was the big message from all the day’s sessions with every speaker touching on the challenge of understanding and securing the vast amounts of data collected.

It’s interesting how the technologists — and most of the material was quite high level — have identified this as the main problem facing management with the Internet of Things.

A key take away from the forum is that the clear opportunity for entrepreneurs with the IoT lies in giving businesses the tools to understand the data.

One of the reasons for the event was to launch the Kaooma Project that aims to link local businesses to the Internet of Things. The local business angle is something that needs to be explored in more depth.

Privacy by design

How can businesses protect customers’ privacy, Intel Security’s Michelle Dennedy discusses how to bake privacy into your organisation

“Know your data” is the key tip for businesses concerned about privacy says Michelle Dennedy, Chief Privacy Officer for Intel Security, formerly McAfee.

“It’s really important to go back to basics,” says Michelle. “We’re trying to do bolt-on privacy, just like we did with security years ago. I think it’s time to take a good look at the policy side, which id called Privacy By Design, thinking about it at early states and being consumer-centric.”

“We at McAfee call it ‘Privacy Engineering’; looking at the tools. methodologies and standards from the past, adding current legislative requirements and business rules then turning them into functional requirement.”

Michelle, who is also co-author of the Privacy Engineering Manifesto, was speaking to Decoding The New Economy as part of Privacy Awareness Week.

A key part of the interview is how Michelle sees privacy evolving in a global environment, “if you’d asked me in 2000 where we’d be today I’d have told you it would be like the 1500s when we were dealing with shipping lanes. We would have treaties, it would harmonised and we’d understand that global trade is a hundred percent based upon sharing.”

“We have instead decided to become a set of Balkanized nations.”

For individual businesses “know thy data,” is Michelle’s main advice. “Know what brings you risk, know what brings you opportunity.”

In Michelle’s view, businesses need to balance the opportunities against the risks and treat customers data with respect as the monetisation policies of many online platforms don’t recognise users’ costs in time and data sold.

As businesses find themselves being flooded with data, protecting it and respecting the privacy of customers, users and staff is going become an increasing important responsibility for managers.

It’s worthwhile understanding the privacy laws as they apply to you and making sure your systems and staff comply with them.

Dropping off the grid

Can you drop off the grid and hide from Big Data? The results of one lady’s experiment aren’t encouraging.

Just how hard is it to hide from big data? ABC Newcastle’s Carol Duncan and I will be discussing this from 2.40 this afternoon.

Princeton University assistant professor of sociology Janet Vertesi decided she’d find out by trying to conceal her pregnancy from the internet.

She describes her experiences to Think Progress and the lessons are startling on how difficult it is to drop off the Internet and business databases.

While it’s easy to tritely say ‘don’t use the internet’, Janet found that using cash to avoid being picked up by bank databases raises suspicions while not using discount voucher or store cards meant she missed out on valuable savings.

For many people though dropping off the internet is not an option – not having a LinkedIn profile hurts most job hunters’ chances of finding work while if you want to participate in communities, it’s often essential to join the group’s Facebook page.

The amazing part of all is that Janet herself became a Google conscientious objector two years ago after deciding the company’s data collection methods were too intrusive. Yet she still found it hard to keep the news of her baby off the internet.

Ultimately her friends were the greatest risk and she had to beg them not to mention her pregnancy on Facebook and other social media channels lest the algorithms pick that up.

For Janet, it proved possible but it was really hard work;

Experience has shown that it is possible, but it’s really not easy, and it comes with a lot of sacrifices. And it requires some technical skill. So to that end, it’s my concern about the opt-out idea. I don’t actually think it’s feasible for everyone to do this.

So can you drop off the net? Do you know if you’re on it at all. Join us on ABC Newcastle with Carol Duncan from 2.40 to discuss these issues and more.

Filing cabinet image by ralev_com through SXC.HU

Microsoft and the Internet of Your Things

Microsoft has come the IoT party with the ‘Internet of Your Things’ tagline

Microsoft has come late to the Internet of Things party, but it is has a good angle with it’s ‘Internet of Your Things’ tagline.

General Manager of Microsoft’s embedded systems division, Barb Edson, spoke with Decoding The New Economy about the company’s strategy with the Internet of Things.

For Microsoft, the emphasis is on the enterprise side of the business with Edson describing their strategy of “B2B2C” where the value in the IoT lies in managing the data for the businesses providing consumer services.

Most notable is the company’s IoT tagline, as Edson says; “from Microsoft’s perspective we view the Internet of Things as ‘the internet of your things.”

“Lots of companies out there talking about the Internet of Everything that there’s 212 billion devices, why do you care as business executive. You care about your things.”

Microsoft’s strategy is based on leveraging their own assets such as Azure cloud services, SQL Server and Dynamics along their customers’ existing infrastructure.

This retrofitting the internet of things to existing infrastructure is illustrated by Microsoft’s using the London Underground as its main reference site.

Connecting all 270 stations of London’s 150 year old Tube network to the IoT is a massive undertaking and one that can only be done by retrofitting existing monitoring and SCADA systems.

Interestingly the case study only look at Phase One of what appears to be pilot project in selected locations, the Microsoft spokespeople were a little unclear on this when asked.

The London Underground is only one example of millions of organisations that will grapple with adding existing equipment to the internet of things in coming years; it’s an opportunity that Microsoft has been smart to identify.

Edson however is clear on how Microsoft intends to help companies deal with the information overload facing managers, “I think the most exciting thing is we’re seeing real business problems being solved.”

Smarthomes come of age

The internet of things isn’t new, it’s just the technology has become more accessible

“After four decades the smartphone comes of age,” proclaims Micheal Wolf in Forbes Magazine.

Wolf is right to a point but he misses the key reason why the smarthome, or the entire internet of things, has become accessible – the technology has simply become affordable.

It was possible to build a smarthome two decades ago, but it was fiendishly expensive and only a few rich people could afford the technology. Today that technology is cheap and easy to install.

This is the common factor with all aspect of the Internet of Things, connecting devices has been possible since before the internet became common but it was expensive and cumbersome so only the highest value equipment – such as oil rigs – was connected.

Now it’s inexpensive and simple to connect things, people are doing it more and that is why there’s a range of security and privacy issues which weren’t so pressing when it was only a few obscure industrial devices that were wired up.

We aren’t inventing the wheel with technologies like the internet of things or big data, they already existed – they are just more accessible and that’s what’s changing business.

Context and the digital divide

Paul Mabray, founder of US online monitoring service Vintek, sees a digital divide developing as businesses struggle with social media big data and Facebook.

“This is the most difficult time in history to be a wine maker, declares Paul Mabray, Chief Strategy Office and founder of Vintank.

“Never has the wine industry been as competitive as it is today.”

Update: The Wine Communicators of Australia, who sponsored Mabray’s visit, have posted Paul’s presentation that covers this post’s theme in more detail.

Mabray’s business monitors social media for wineries and collects information on wine enthusiasts. Since Vintank’s founding in 2008 the service has collected information on over thirteen million people and their tastes in wine.

Rewriting the rule book

Social media, or social Customer Relationship Management (sCRM), is what Mabray sees as being part of the future of the wine industry that’s evolving from a model developed in the 1970s which started to break down with the financial crisis of 2009.

“In the old days there was a playbook originating with Robert Mondavi in the 1970s which is create amazing wine, you get amazing reviews and you go find wholesalers who bring this wine to the market.”

“As a result of the global proliferation of brands the increase of awareness and consumption patterns where people like wine more, those playbooks didn’t work in 2009 when the crisis started.”

With the old marketing playbook not working, wineries had to find other methods to connect to their markets and social media has become one of the key channels.

Now the challenge in the wine industry, like all sectors, is dealing with the massive amount of data coming in though social media and other channels.

The cacophony of data

“If you rewind to when social media came out, everyone had these stream based things and the noise factor was so heavy,” says Mabray.

“For small businesses this creates an ‘analysis to paralysis’ where they’d rather not do anything.”

Mabray sees paralysis as a problem for all organisations, particularly for big brands who are being overwhelmed by data.

“The cacophony of data at a brand level is just too much,” he says.

“It’s as noisy as all get go and I think the transition is to break Big Data down into small bite size pieces for businesses to digest is the future, it shouldn’t be the businesses problem, it should be the software companies’.”

A growing digital divide

Mabray sees a divide developing between the producers who are embracing technology and those who aren’t, “the efficiencies attributed to technology are obvious whether they’re using CRM, business intelligence or other components.”

“The people who are doing this are recognising the growth and saying ‘hey, this stuff actually works! If I feed the horse it runs.”

While Mabray is focused on digital media and the wine industry, similar factors are work in other industries and technology sectors; whether it’s data collected by farm sensors to posts on Instagram or Facebook.

Facebook blues

Mabray is less than impressed with Facebook and sees businesses concentrating on the social media service as making a mistake.

“I think that every social media platform that’s been developed had such a strong emphasis on consumer to consumer interaction that they’ve left the business behind, despite thinking that business will pay the bills.”

“As a result almost every single business application that’s come from these social media companies has met with hiccups. That’s because it wasn’t part of the original plan.”

Facebook in particular is problematic in his view, “it’s like setting up a kiosk in the supermall of the world.”

The business anger towards Facebook’s recent changes is due to the effort companies have put into the platform, Mabray believes; “everyone’s angry about Facebook because we put so much into getting the data there.”

“We said ‘go meet us on Facebook’, we spent money collecting the items and manufacturing the content to attract people and now we have to spend money to get the attention of the people we attracted to the service in the first place.”

Despite the downsides of social media Mabray sees customer support as one of the key areas the services. “It’s easy to do in 140 characters.”

Context is king

“Everything come back to context. There’s this phrase that ‘content is king’,” Mabray says. “Context is king.”

“Anyone can produce content. It’s a bull market for free content. We have content pollution – there’s so much junk to wade through.

Mabray’s advice to business is to listen to the market: “Customers are in control more than they have ever been in human history: Google flattens the world and social media amplifies it.”

For wineries, like most other industries, the opportunity is to deal with that flat, amplified world.

Evangelism and the makers’ movement

Salesforce’s Reid Carlberg talks tech Evangelism, the Internet of Things and the Makers’ Movement

The latest Decoding the New Economy interview is with Salesforce’s Reid Carlberg.

During the interview with Reid we cover how the Internet of Things and big data is changing business and society along with the journey to becoming a software company’s evangelist.

Reid has a fascinating story to tell about how the makers’ movement is evolving as big data and the internet of things develops.

The interview is an insight into a winding career path and how Big Data and the Internet of Things is changing business and society.

Garbage In and Garbage Out

The success of using Big Data depends upon the quality of both the data and the algorithm

UK tech site The Register reports that Google Flu Trends has been dismal failure with the service over-reporting the incidence of influenza by a factor of nearly 12.

The reason for this problem is the algorithm used to determine the existence of a flue outbreak is that it relies on people searching for the terms ‘flu’ or ‘influenza’ and it turns out we tend to over-react to a dose of the sniffles.

Google Flu Trends’ failure illustrates two important things about big data – the veracity of the data coming into the system and the validity of the assumptions underlying the algorithms processing the information.

In the case of Google Flu Trends both were flawed; the algorithm was based on incorrect assumptions  while the incoming data was at best dubious.

The latter point is an important factor for the Internet of Machines. Instead of humans entering search terms, millions of sensors are pumping data into system so bad data from one sensor can have catastrophic effects on the rest of the network.

As managing data becomes a greater task for businesses and governments, making sure that data is trustworthy will be essential and the rules that govern how the information is used will have to be robust.

Hopefully the lessons of Google Flu Trends will save us from more serious mistakes as we come to depend on what algorithms tell us about the data.