Tag: big data

Fifty trillion shades of grey

Something that’s missed when we talk about Big Data is the risk of false positives – if you dip into the stream, you can prove anything against person.

If you give me six lines written by the hand of the most honest of men, I will find something in them which will hang him said the 17th Century French politician Cardinal Richelieu.

Today those six lines could be written on a social media site or be six disparate points drawn from a database. Without context those six lines could condemn us.

Something that’s missed when we talk about Big Data is the risk of false positives – if you dip into the stream, you can prove anything against person.

The world isn’t black or white, there are fifty trillion shades of gray and that’s why it’s important to think before posting an image on the web, firing someone or calling the cops.

In an era where we’re quick to judge and condemn people, the stakes are very high.

Similar posts:

Recruiting big data

Software company Evolv is an example of how businesses can use big data

One of the predictions for 2020 is that decade’s business successes will be those who use big data well.

A good example of a big data tool is recruitment software Evolv that helps businesses predict not only the best person to hire but also who is likely to leave the organisation.

For employee retention, Evolv looks at a range of variables which can include anything from gas prices and social media usage to local unemployment rates then pulls these together to predict which staff are most likely to leave.

“It’s hard to understand why it’s radically predictive, but it’s radically predictive,” Venture Beat quotes Jim Meyerle, Evolv’s cofounder.

There are some downsides in such software though – as some of the comments to the VentureBeat story point out – a blind faith in an alogrithm can destroy company morale and much more.

Recruiters as an industry haven’t a good track record in using data well, while they’ve had candidate databases for two decades and stories abound of poor use of keyword searches carried out by lazy or incompetent headhunters. The same is now happening with agencies trawling LinkedIn for candidates.

Using these tools and data correctly going to separate successful recruitment agencies and HR departments from the also-rans.

It’s the same in most businesses – the tools are available and knowing them how to use them properly will be a key skill for this decade.

Job classifieds image courtesy of Markinpool through SXC.HU

Similar posts:

Leadership in a connected world

How do managers and executives lead in a connected world?

Managing a business or government agency as information pours into organisations is one of the great challenges for modern executives.

As part of the Australian Cisco Live event, a panel looked  at Public Sector Leadership in a Connected World, many of the issues discussed apply to private sector executives as they do to public sector managers.

Cisco’s Director of Global Public Sector Practice, Martin Stewart-Weeks, kicked off the panel with the observation that “we now live in a world where information has become completely unmanageable.”

Martin quoted from David Weinberg’s book Too Big To Know, Rethinking Knowledge Now That the Facts Aren’t the Facts, Experts Are Everywhere, and the Smartest Person in the Room Is the Room. The author has a good explanation of his book in this YouTube clip.

Trusting the community seems to be the biggest problem facing politicians and the public service, policy consultant Rod Glover puts the general distrust towards governments on the failure of leaders to consult over changes and decision.

Economist Nick Gruen and Australian Industry Group adviser Kate Pound echoed this problem in that a change of culture is needed among leaders towards the way information is controlled and managed.

Nick sees that culture changing while Rod thinks there will need to be demonstrated successes before risk adverse public service leaders will be prepared to adopt new ways of managing.

Kate’s view is that culture change will require a realignment of incentives which will make managers accountable for the delivery of services. She cites a situation where businesses are obligated to register online but the agency’s website doesn’t work.

So the problem is as much gathering the right data along with processing the information inside an agency. Both are challenges for organisations with rigid hierarchies and  information flows.

Information is no longer power — it’s how you use it. But the structures are still based around access and control of knowledge.

The big culture shift for politicians, public servants and corporate executives is we can no longer hoard information.

For managers in both the public and private sectors, the task is now to share information and trust the right people will use it well.

Paul travelled to Cisco Live courtesy of Cisco Systems

Similar posts:

Dealing with the data explosion

Supply the mobile base stations for data hungry customers is one of the great challenges for telcos. How they resolve this will create some unusual alliances.

“Last year’s mobile data traffic was nearly twelve times the size of the entire global Internet in 2000.”

That little factoid from Cisco’s 2013 Virtual Networking Index illustrates how the business world is evolving as various wireless, fibre and satellite communications technologies are delivering faster access to businesses and households.

Mobile data growth isn’t slowing; Cisco estimate global mobile data traffic was estimated at 885 petabytes a month and Cisco estimate it will grow fourteen fold over the next five years.

Speaking at the Australian Cisco Live Conference, Dr. Robert Pepper, Cisco Vice President of Global Technology Policy and Kevin Bloch, Chief Techincal Officer of  Cisco Australia and New Zealand, walked the local media through some of the Asia-Pacific results of Virtual Networking Index.

Dealing with these sort of data loads is going to challenge Telcos who were hit badly by the introduction of the smartphone and the demands it put on their cellphone networks.

A way to deal with the data load are heterogeneous networks, or HetNets, where phones automatically switch from the telcos’ cellphone systems to local wireless networks without the caller noticing.

The challenge with that is what’s in it for the private property owners whose networks the telcos will need to access for the HetNets to work.

One of the solutions in Dr Pepper’s opinion is to give business owners access to the rich data the telcos will be gathering on the customers using the HetNets.

This Big Data idea ties into PayPal’s view of future commerce and shows just how powerful pulling together disparate strands of information is going to be for businesses in the near future.

But many landlords and wireless network owners are going to want more than just access to the some of the telco data — we can also be sure that the phone companies are going to be careful about what customer data they share with their partners.

It may well be that we’ll see telcos providing free high capacity fibre connections and wireless networks into shopping malls, football stadiums, hotels and other high traffic locations so they can capture high value smartphone users.

One thing is for sure and that’s fibre connections are necessary to carry the data load.

Anyone who thinks the future of broadband lies in wireless networks has to understand that the connections to the base stations doesn’t magically happen — high speed fibre is essential to carry the signals.

Getting both the fibre and the wireless base stations is going to be one of the challenges for telcos and their data hungry customers over the next decade.

Paul travelled to the Cisco Live event in Melbourne courtesy of Cisco Systems.

Similar posts:

Using big data to find the cupboard is bare

Yahoo! Chief Executive Marissa Mayer is an example of how modern managers are diving into big data to figure out what is going on in their company

Last week this blog discussed whether telecommuting was dead in light of Marissa Mayer’s banning of the practice at Yahoo.

While I don’t think telecommuting is dead, Marissa Mayer has a big problem figuring out exactly who is doing what at the company and abolishing remote working is one short term way of addressing the issue.

If Business Insider is to be believed, Yahoo!’s absent staff problem is bad.

After spending months frustrated at how empty Yahoo parking lots were, Mayer consulted Yahoo’s VPN logs to see if remote employees were checking in enough.

Mayer discovered they were not — and her decision was made.

Business Insider’s contention is that Mayer makes her decisions based on data analysis. At Google she drove designers mad by insisting on reviewing user reactions to different layouts and deciding based on the most popular results.
If this is true, then Marissa Mayer is the prototype of tomorrow’s top executives – the leaders in business by the end of this decade will be the ones who manage data well and can sift what matters out of the information deluge.
For all of us this is going to be a challenge with the probably the biggest task of all being able to identify which signals are worth paying attention to and which should be ignored.
Of course, all this assumes the data is good quality in the first place.
An assumption we’ve all made when talking about Big Data is that it’s about marketing – we made the same assumption about social media.
While Big Data is a good marketing tool, it’s just as useful in areas like manufacturing, logistics, credit evaluations and human resources. The latter is what Yahoo!’s staff are finding out.
In age of Big Data it may not pay to a slacker, but it’s going to be handy if you want to know what’s going on your business.

Similar posts:

Smelling digital garbage

Excel spreadsheets lie at the core of business computing, but what happens when they go wrong?

Excel spreadsheets lie at the core of business computing, but what happens when they go wrong?

James Kwak writing in the Baseline Scenario blog describes how Excel spreadsheets have an important role in the banking industry and their key role in one of the industry’s most embarrassing recent scandals.

In the early days of the personal computer spreadsheets; it was company accountants and bookkeeping clerks who bought the early PCs into offices to help them do their jobs in the late 1980s .

From the accounts department, desktop computers spread through the businesses world and the PC industry took off.

Over time, Microsoft Excel displaced competitors like Excel 1-2-3 and the earliest spreadsheet of all, VisiCalc, and became the industry standard.

With the widespread adoption of Excel and millions of people creating spreadsheets to help do their jobs came a new set of unique business risks.

The weakness with Excel isn’t with the program itself, it’s that the formulas in many spreadsheets aren’t properly tested and often incorrect data is put into the wrong fields.

In his story Kwak cites the JP Morgan spreadsheets that miscalculated the firms Value-At-Risk (VAR) calculations for synthetic derivatives. The result was the London Whale debacle where traders were allowed to take positions – some would call them bets – exposing the bank to huge potential losses.

It turns out that faulty spreadsheets had a key role as traders cut and paste data between various spreadsheets and the formulas that made the calculations had basic errors.

That a bank would have such slapdash procedures is surprising but not shocking, almost every organisation has a similar setup and it gets worse as a project becomes more complex and bigger numbers become involved. The construction industry is particularly bad for this.

Often, a spreadsheet will show out a bunch of numbers which simply aren’t correct. Someone made a mistake entering some data or one of the formulas has an error.

The business risk lies in not picking up those errors, JP Morgan fell for this and probably every business has, thankfully to less disastrous results.

My own personal experience was with a major construction project in Thailand. One sheet of calculations had been missed and the entire budget for lights – not a trivial amount in a 35 storey five star hotel – hadn’t been included in the contractor’s price.

This confirmed in my mind that most competitive construction tenders are won by the contractor who made the most costly errors in calculating their price. Little has convinced me otherwise since.

In the computer industry there’s a saying that “garbage in equals garbage out” which is true. However if the computer program itself is flawed, then good data becomes garbage.

Excel’s real flaw is that it can make impressive looking garbage that appears credible if it isn’t checked and treated with suspicion. The responsibility lies with us to notice the smell when the computer spits out bad figures.

Spreadsheet image courtesy of mmagallan through sxc.hu

Similar posts:

Democratising Big Data

Why a not for profit disrupting Google and the Big Data industry is important for business and society

Common Crawl is a not-for-profit web crawler service that makes the data collected open for all to use. A post on the MIT Technology Review blog speculates how the initiative might spawn the next Google.

One of the problems with Big Data is that it’s held mainly by large corporations and government agencies, both of which have the tendency to keep their data private on that basis that information is power and power means money.

We see this in the business models of Facebook, Google and many of Silicon Valley’s startups; the information garnered about users is as, if not more so, valuable as an utility from the product.

Initiatives like Common Crawl tilt the balance somewhat back towards consumers, citizens, and smaller businesses.

How well Common Crawl and other similar initiatives fare remains to be seen – Wikileaks was a good example of how such projects can flare out, collapse under the weight of egos or be harrassed by corporatist interests.

In search, Google are open to disruption as they tweak their results to suit initiatives like Google Plus. During the company’s earnings call earlier this week Larry Page spoke of the challenges of staying focused on the opportunities that matter, it may well be the company is more distracted from its core business than it should be.

Whether Common Crawl disrupts Google is up to history, it could just as well be a couple of kids called Sergei and Larry with a smart idea.

The imperative now though is to try and keep as much public data available for everyone to use and not lock it away for the privileged few. That will let the future Googles develop while making our societies more fairer and open.

Similar posts: