big data | Paul Wallbank - Communications Professional

Tracking the knowledge graph

Facebook Graph search is powerful and dangerous which means we have to be careful about what we like and who we become friends with

“Married Men Who Like Prostitutes” is juicy search term and the results can wreck marriages, careers and lives.

This is one of the Facebook Graph searches UK tech commentator Tom Scott posted on his Actual Searches on Facebook Tumblr site which lists, mercifully anonymised, the results.

What should worry anybody who uses Facebook is that this data has been in the system all along, advertisers for instance have been able to target their marketing based on exactly this information, Graph Search just makes it quicker and easier to access. This is why you should be careful of what you like and who you friend online.

Tom Scott has a terrific Ignite London presentation which looks at just how vulnerable an individual is by over sharing online. In I know what you did five minutes ago, Tom finds an individual, discovers his mother’s maiden name and phone number all within two minutes.

Facebook isn’t the only service we should be careful of, it just happens to be the one we overshare data with the most. When you start stitching together social media services with government and corporate databases then a pretty comprehensive picture can be made of a person’s likes and preferences.

The best we can hope for in such a society is that picture is accurate, fair and doesn’t cast us in too unfavourable a light.

In same cases though that data can be dangerous, if not fatal.

As potential employers, spouses and the media can easily access this information, it might be worthwhile unliking obnoxious, racist and downright stupid stuff. There’s a very good chance you’ll be asked about them.

What happens when software is wrong

A phone company software glitch puts one man’s life and the safety of thousands at risk. It reminds us that computers are not always correct.

The Las Vegas Review Journal yesterday told the story of Wayne Dobson, a retiree living to the north of the city whose home is being fingered as harbouring lost cellphones thanks to a software bug at US telco Sprint which is giving out the wrong location of customer’s mobile devices.

While it appears funny at first the situation is quite serious for Mr Dobson as angry phone owners are showing up at his home to claim their lost mobiles back.

Making the situation even more serious is that 911 calls are being flagged at coming from his home and already he has had to deal with one police raid.

While the local cops have flagged this problem, it’s likely other agencies won’t know about this bug which exposes the home owner to some serious nastiness.

That a simple software bug can cause such risk to an innocent man illustrates why we need to be careful with what technology tells us – the computer is not always right.

Another aspect is our rush to judgement, we assume because a smartphone app indicates a lost mobile is in a house that everyone inside is a thief. That the app could be wrong, or we don’t understand the data to properly interpret it, doesn’t enter our minds. This is more a function of our tabloid way of thinking rather than any flaws in technology.

The whole Find My Phone phenomenon is an interesting experiment in our lack of understanding risk; not only is there a possibility of going to the wrong place but there’s also a strong chance that an angry middle class boy is going to find himself quickly out of his depth when confronted by a genuine armed thief.

For Wayne Dobson, we should pray that Sprint fixes this problem before he encounters a stupid, violent person. For the rest of us we should remember that the computer is not always right.

Customer lock in as a business asset

Barnes and Noble’s problems show how high the stakes are when locking customers into an online business.

US booksellers Barnes and Noble has been struggling for years and things aren’t getting better reports the New York Times.

An important part of the New York Times story is the quote from a Forrester industry analyst,

“The problem is not whether or not the Nook is good,” said James L. McQuivey, a media analyst for Forrester Research. “What matters is whether you are locked into a Kindle library or an iTunes library or a Nook library. In the end, who holds the content that you value?”

Locking in customers lies at the heart of the Kindle and iTunes business model. Once users have a substantial investment in their book or music collections on one platform it’s unlikely they will go elsewhere as the costs, and risks, of moving are too great.

This doesn’t always end well for the customer and it gives online businesses great power which they often misuse.

Every online business tries to lock their customers into their ecosystem – Google, Amazon, Facebook and Apple are the most successful but every single social media and cloud service tries to make it hard for users take their business elsewhere.

In some respects this is no different to the phone company or bank which have historically tried to lock customers into their services, but the online social media, cloud computing and e-commerce platforms make a much more ambitious grab for their users’ data and assets like music and book collections.

The New York Times article illustrates just how critical that user lock in is to the success of online businesses. The question for us as consumers is how much we want to be locked inside the web’s walled gardens.

Big data, mobile apps and smarter logistics – why Avis is buying Zipcar

Smartphone apps are more than just a funky way of getting information. The combination of big data, social media and mobile insights offer businesses deep market intelligence.

With no bad press over New Year’s Eve it looks like hire car service Uber avoided the surge pricing traps of 2011 and the good news continues for the online booking industry with the news that Avis is buying car sharing service Zipcar.

Assuming the acquisition isn’t another example of the greater fool investment model, Avis’ purchase of Zipcar makes good sense in expanding the hire car giant’s footprint into the share car business.

Regrettably Avis use the 1980s term “synergies” four times in their media release but it does seem the businesses are a good fit both in fleet sharing and improving both company’s services.

Zipcar’s technology is another asset which Avis can use, with the car sharing service’s ability to track vehicle locations meaning better fleet management for the hire car business.

Car sharing logistics

The logistics angle of car share services is something that’s been highlighted by Uber’s CEO Travis Kalanick at various times, most recently at the service’s Sydney launch last November.

Another aspect of the car sharing and hire car booking services is their Big Data advantages which the online startups bring.

Historically, car hire companies have been reasonably good at gathering data on their customers with loyalty schemes, direct mailing and plugging into airline frequent flier programs. However they have been left behind by the Big Data boom in recent years.

Companies like Zipcar, Uber and taxi hailing apps like GoCatch have big data in their DNA, having been founded in the era of cloud computing and social media they have access to more information and a better ability to use the knowledge they gather.

Predicting the price surges

At Uber’s Sydney launch Kalanick described how Uber’s traffic volumes increase in San Francisco when the Giants win a game, the interesting thing is that the surge happens three hours before the match starts.

Insights like the traffic patterns around football games and holidays are gold to a high inventory business like hire car services. They are also important to the entire logistics industry.

This latter point is probably the most overlooked part of all with the current rush into social and mobile based apps – the market intelligence that these services gather.

While it’s tempting to dismiss that market intelligence as just monitoring who likes cats or cheeseburgers, the application of that data is transforming supermarkets, airlines and even concert venues.

Avis seem to have understood that it will be fascinating to see how they will use Zipcar’s data and whether their competitors will figure out the importance of what these services offer.

Blind faith in the algorithm

Putting too much faith in computer programs may cause problems for the unwary.

It’s fairly safe to say Apple’s ditching of Google Maps for their own navigation system has proved not to be company’s smartest move.

The humiliation of Apple was complete when the Victoria Police issued a warning against using the iPhone map application after people became lost in the desert when following faulty directions to the town of Mildura.

Mapping is a complex task and it’s not surpising these mistakes happen, particular given the dynamic nature of road conditions and closures. It’s why GPS and mapping systems incorporate millions of hours of input into the databases underlying these services.

Glitches with GPS navigations and mapping applications aren’t new. Some of the most notorious glitches have been in the UK where huge trucks have been directed down small country lanes only to find themselves stuck in medieval villages far from their intended location.

While those mishaps make for good reading, there are real risks in these misdirections. One of the best publicised tragedies of mis-reading maps was the death of James Kim in 2007.

Kim, a well known US tech journalist, was driving with his family from Portland, Oregan to a hotel on the Pacific Coast in November 2006 when they tried to take a short cut across the mountains.

After several hours driving the family became lost and stuck in snowdrifts and James died while hiking out to find help. His wife and two children were rescued after a week in the wilderness.

Remarkably, despite warnings of the risks, people still get stuck on that road. The local newspaper describes it the annual ritual as find a tourist in the snow season.

Partly this irresponsibility is due to our modern inability to assess risk, but a more deeper problem is blind faith in technology and the algorithms that decide was is good and bad.

A blind faith in algorithms is a risk to businesses as well – Facebook shuts down accounts that might be showing nipples, Google locks people out of their Places accounts while PayPal freeze tens of thousands of dollars of merchants’ funds. All of these because their computers say there is a problem.

Far more sinister is the use of computer algorithms to determine who is a potential terrorist, as many people who’ve inadvertently found themselves on the US government’s No Fly List have discovered.

As massive volumes of information is being gathered on individuals and businesses it’s tempting for all of us to rely on computer programs to tell us what is relevant and to join the dots between various data points.

While the computers often right, it is sometimes wrong as well and that’s why proper supervision and understanding of what the system is telling people is essential.

If we blindly accept what the computer tells us, we risk being stuck in our own deserts or a snowdrift as a result.

Rethinking customer support

As the computer power at our fingertips gets more powerful, the tools to help customers are getting smarter and better.

One of weaknesses in most organisations is getting customer service right, good support takes time which costs money and leads many big and small companies to scrimp on support to save a few costs.

In a conversation with BMC Software’s Suhas Kelkar about customer support – Remedy, one of the biggest helpdesk software packages is a BMC product – the discussion turned to how the process has changed in recent years.

Not too long ago we reached for manuals, but those vanished as CDs and then downloads became common. Then we’d call the manufacturer’s helpline or our unfortunate store who sold us the item.

Today we Google a problem to see if we can find a quick solution and if that fails we reach out to our social networks by posting the question on Twitter or Facebook. We may even post the problem to a support forum to see if anyone has an answer.

Only if can’t find the solution anywhere else do we call the support line, for most of us it is the last resort.

In some ways this is a success for corporate cost cutting as most of us call a “helpline” only in desperation as we’ve trained to expect long waits, confusing menus and poorly trained operators.

That model developed in the 1980s – in order to pay rockstar salaries to executives it was necessary to cut staff wages and training costs with after sales support often being the first business area to suffer cuts.

Eventually this started to backfire and the Dell Hell saga as one of the leading examples where the computer manufacturer’s lousy support became industry legend. It’s fair to argue that Dell has never quite recovered from the damage the period of poorly outsourced support did to their brand.

To repair the damage to their brand, Dell adopted a crowdsourced support model where company forums were available for customers to ask about problems with the hope other customers could answer before expensive staff became involved. Eventually other companies adopted this system.

Social media has created a doubled-edged sword for businesses, it’s easier for people to ask their friends for help but it also increases the risk of brand damage if online posts aren’t monitored and responded to.

All of this is forcing a rethink of how customer support works. For businesses big and small, social media and crowdsourcing tools are changing the way we talk to customers and how they can talk about us.

The big data push is also changing customer support as businesses now have the computing power available to mine knowledge bases, issue registers and call logs to identify market trends and weaknesses in their products or sales teams.

For business owners and managers stuck in the 1980s ways of customer support, they are in for a wretched time over the next few years.

Meeting the solid state Woz

The fast talking Steve Wozniak surfs into town on a wave of enthusiasm.

When the opportunity comes to meet co-founder of Apple computer Steve Wozniak you jump at it, despite being jet lagged from the previous day’s flight.

One of the tough things when writing about Steve Wozniak is that he is a fast talker. You have to be quick to keep up with his ideas and words.

Steve was in town to show off the range of solid state computer memory cards manufactured by Fusion-iO, a company based in Salt Lake City.

Wozniak liked the idea so much he became Fusion-iO’s chief scientist in 2009. “When I first saw the iO drive, it was so beautiful I had to buy one from the company and put it in a frame just to frame it at home.”

What enthused Woz were Fusion-iO‘s range of NAND flash memory cards that speed up servers while reducing their power and cooling requirements.

Those power savings are important for data centres when hundreds of thousands of servers might be in one building, Fusion-iO’s CEO and co-founder David Flynn estimates this could save up the industry a $250 billion a year in operating costs.

Probably the biggest benefits though are in the corporate space, one Flynn’s boasts is how one movie studio used Fusion-iO’s products to reduce transcoding between formats from two hours to 39 seconds.

Another case study they show off is how grocery chain Woolworths were able to reduce the 17 hours to run their weekly trading reports to three hours meaning they were able to capture weekend figures for their weekly Monday morning board meetings.

For smaller businesses, the biggest benefit is these products can turn fairly basic desktop computers into workstations with the $2,500 ioFX card promising some serious post production capabilities for a system although one would expect an entry level box wouldn’t have the data connection, hard drive or – most importantly – power supply to cope with the demands of such a device would put on the typical cheap components in a basic desktop system.

All of these changes though are heralding some pretty big changes for big and small businesses.

Where Steve Wozniak sees the greatest application of moving data faster is in Artificial Intelligence applications like voice recognition. Apple’s Siri is a good example of this.

The barrier to effective voice recognition is the sheer amount of data processing required to effectively understand voice commands. Doing this on cloud services is a far more efficient and effective way of doing this.

As we saw at Dreamforce last week, the sheer amount of data pouring into companies is changing how they manage information. Getting access quickly to relevant information is an important part of managing it.

“I’ve never gotten so excited about or fell in love with a technology like this since Apple.” Says Wozniak.

Having a chance to speak to Steve Wozniak up close shows that fast talking enthusiasm is for real. The Woz is a real geek.

Like all true geeks Wozniak is passionate about what he believes in – whether it’s about NAND flash cards or becoming an Australian resident he bubbles with enthusiasm. Just don’t try writing notes down while he’s in full flight.

Redefining the social business

Salesforce.com announce a range of new products at Dreamforce 2012

Over the last two years Salesforce.com have been one of the more aggressive buyers of cloud computing and social media startups with acquistions of companies like Rypple, Desk.com, Buddy Media and Radian6.

Today, ahead of the company’s annual Dreamforce conference in San Francisco, Salesforce.com announced a revamped product range that brings together the social media and big data tools from these acquisitions along with some in house innovations.

Salesforce expect nearly a hundred million enterprise tablet computer users and smartphones by 2016, so like all web based services, they have to make their platform available as an app. Salesforce’s new Touch iOS App allows users to use Salesforce.com as an app on the iPhone.

Despite Mark Zuckerburg’s disavowal of HTML5 last week, Salesforce remains committed to the standard despite developing an app for the iPhone.

“Initially we’re rolling out Touch in a way we’ve made sure works the way people want it to work on an iPad,” Peter Coffee, Vice President of Platform Research at Salesforce.com, says.

“We are reiterating our commitment to HTML 5 as a device and platform neutral cluster of standards.

“As HTML5 begins to clearly coalesce we’re making a major commitment to that and we’re going to lead the way while the opposition is still trying to work on one browser.”

Salesforce continues their focus on social media with their Chatter service becoming a key part of their Force.com cloud applications platform. Chatter itself is being extended with a new feature to enable companies to create their own branded communities.

That social integration continues as the company rolls out Social Key, an application which, as Andy MacMillan, senior vice president and general manager of Data.com says “will empower companies to derive value from social data for the first time.”

If Social Key does achieve a real measure of value from retweets and Facebook posts it may well mean many social media experts will have to return to multi-level marketing or real estate sales. This in itself is not a bad thing.

The new Salescloud platform uses Chatter to build business intelligence on customers, bringing data across a business to help sales teams target their efforts more effectively.

While sales is by definition the focus of Salesforce they are also launching a similar Chatter service for support teams. This compliments the acquisition of Assist.ly at the beginning of the year.

Marketing too is being targeted by Salesforce with the launch of Marketing Cloud that combines the Buddy Media Facebook marketing service and the Radian6 social media monitoring platform.

While already the leader in business cloud applications, Salesforce are making a strong bid to dominate the sector in a way that Microsoft did in the desktop computer industry twenty years ago.

Browsing through the 400 partner stands at the Dreamforce Expo shows Cloudforce are building a deep ecosystem around their products that will make it hard for competitors to break into the space.

Whether Salesforce achieve this dominance remains to be seen, but they are certainly giving a new set of tools for businesses to understand their customers.

Pricing and Availability
Salesforce Touch is generally available today on iOS devices, and included in all Salesforce editions.

Sales Cloud Partner Communities is currently scheduled to be available in limited pilot in Fall 2012.

Sales Cloud Partner Communities is currently scheduled to be generally available the second half of 2013.

Data.com Social Key is currently scheduled to be generally available the second half of 2013.

Pricing of Sales Cloud Partner Communities and Data.com Social Key will be announced at general availability.

Finding the perfect customer

Combining old techniques with big data technologies and social media monitoring open new opportunities for businesses to learn more about their customers.

With the rise of social media we’ve spoken a lot about customers’ ability to rate businesses and overlooked that companies have been rating their clients for a lot longer. The same technologies that are helping consumers are also assisting companies to find their best prospects.

A business truism is that Pareto’s Rule applies in all organisations – 20% of customers will generate 80% of a company’s profits. Equally a different 20% of clients will create 80% of the hassles. The Holy Grail in customer service is to identify both groups as early as possible in the sales cycle.

Earlier this week The New York Times profiled the new breed of ratings tools known as consumer valuation or buying-power scores. These promise to help businesses find the good customers early.

While rating customers according to their credit worthiness has been common for decades, measuring a client’s likely value to a business hasn’t been so widespread and most companies have relied on the gut feeling of their salespeople or managers. The customer valuation tools change this.

One of the companies the NYT looked at was eBureau, a Minnesota-based company that analyses customers’ likely behaviour. eBureau’s founder Gordy Meyer tells how 30 years ago he worked for Fingerhut, a mailorder catalogue company that used some basic ways of figuring out who would be a good customer.

Some of the indicators Fingerhut used to figure if a client was worthwhile included whether an application form was filled in by pen, if the customer had a working telephone number or if the buyer used their middle initial – apparently the latter indicates someone is a good credit risk.

Many businesses are still using measures like that to decide whether a customer will be a pain or a gain. One reliable signal is those that complain about previous companies they’ve dealt with; it’s a sure-fire indicator they’ll complain about you as well.

What we’re seeing with services like eBureau is the bringing together of Big Data and cloud computing. A generation ago even if we could have collected the data these services collate, there was no way we could process the information to make any sense to our business.

Today we have these services at our fingertips and coupled with lead generators and the insights social media gives us into the likes and dislikes of our customers these tools suddenly become very powerful.

While we’ll never get rid of bad customers – credit rating services didn’t mean the end of bad debts – customer valuation tools are another example of how canny users of technology can get an advantage over their competitors along with saving time in chasing the wrong clients.

Extending the knowledge graph

Big data takes baby steps on our desktops

Google’s latest search changes – introduced by Search Senior Vice President Amit Singhal – are another development, or baby step as Amit would call it, in making data more useful for us.

The flood of data that’s washed over us since the web arrived has left most of us befuddled. Increasingly, a basic keyword search just hasn’t been enough to find the information we’re looking for and we’ve had to trawl through pages of irrelevant information.

One of the aims of Google’s new features is to make data more relevant to a user – so if someone in the US types “Kings” into Google, they will be given details of the Los Angeles Kings hockey team or the TV series “Kings”.

Those details will include snippets of information about the topic. It could be the teams venue or the cast of the TV show. For tourist locations it could be some basic facts or transport information.

Amit is particularly proud of integrating tourist information and flight details into search and Gmail which indicates Google is beginning to leverage its buyout of the ITA travel booking network last year.

Google’s treatment of data reflects what’s happening with other services. At the recent Australian Xero conference and in an interview with MYOB executives, it’s been emphasised how knowledge is being aggregated to give customers and users better, more useful results.

With Google’s knowledge graph we’re seeing the realisation of what big data can do, there’s many baby steps ahead but there’s a lot of potential.

Hacking the hacks

Do journalists have the skills to ride the Big Data wave?

Hacks and Hackers is an informal global network of meetings discussing the intersection of technology and journalism. The inaugural Sydney Hacks and Hackers meetup recently looked at how journalists use data and showed the challenges the news media face in an age where information isn’t scarce.

The panel in Sydney were Sharona Coutts, Investigative Reporter at Global Mail; Edmund Tadros, Data Journalist at Australian Financial Review; and Courtney Hohne, Director of Communications Google Australia.

Courtney looked at some of the big data opportunities for journalists, a topic covered in the Closed Data Doors post. One of the areas she highlighted was emergency services sending out PDFs of updates during crises like bushfires and floods.

Listening to Sharona and Edmund, it was clear they were two overworked but keen young journalists who had neither the resources or the training to deal with the data flowing into their organisations.

Because journalists in modern media organisations don’t have the skills or the resources to properly understand and use raw data the public ends up with relatively trivial stories like league tables of school exam results or council building approvals – both of which are important, but are misread and used to confect outrage against incompetent public servants and duplicitous politicians.

For the public servant, school teacher or even bus driver it’s understandable they don’t want their performance measured if the measure is going to be misused and possibly jeopardize their jobs.

A deeper problem for journalism is the skills of the trade. Both Edmund and Sharona are smart young journos who will go far; but both admitted they had no training in statistic and mathematics.

Even more worrying are the older journalists, when I mentioned the lack of older and more experienced journalists to the organiser she said none would agree to come on the panel. One suspects this is because forty and fifty year old journalists have even fewer data skills than their young colleagues.

This lack of skills or understanding of data is probably one of the biggest challenges facing the media. In a world awash with data, the role of journalists is to filter the feed, interpret and explain it.

Pure reportage is being overwhelmed by the sheer quantity of news and information available; the 1980s model of opinion based journalism is also failing as the audience now realise they have a voice, and better informed opinions, than the experts and columnists.

One of the notable themes that seemed to jump out of the evening was the divide between journalists and the wider community that always seems to appear when the future of journalism is discussed.

Usually this expressed in terms of those employed by major mastheads sneering at “citizen journalists” but at Hacks and Hackers it was about “geeks and journos coming together.”

In reality there is no divide – good analytic and technology skills should be as much a part of journalism as any other field in a modern economy.

The fear from the Sydney Hacks and Hackers night is that the media industry is one of the sectors that’s failing to deal with technological change.

It’s hard not to think that journalists wondering at the power of spreadsheets and pivot tables is like 18th Century blacksmiths trying to figure out how steam engines can make better horseshoes.

For an industry that is so deeply challenged by technological change, it seems the news media is still unprepared for the changes that hit nearly a decade ago.