Sep 272016
Kennedy Nixon Presidential Debate 1960

As the 2016 US Presidential race enters its final stages, it’s interesting to see how data is being used by American political candidates and what this means for business.

During last week’s Oracle Open World in San Francisco a panel hosted by the company’s Political Action Committee featured Stephanie Cutter, who worked on Obama’s 2008 and 2012 campaigns, and Mike Murphy, a Republican operative whose most recently worked on Jeb Bush’s primary effort against Donald Trump.

While the discussion mainly focused on the politics – “Crazy times seem to require crazy candidates” says Murphy – it was the technology aspect of modern elections that was notable.

Setting the data standard

The Obama campaign of 2008 set the standard for how modern political campaigns used social media and information, “we revolutionized how data analytics helps predict how people will vote and how they will persuade voters to turn out.” Cutter said.

“We put a big investment into it and Republicans have caught up,” she continued. “The key though was we relied on our own data and nothing that was out in the public domain. We didn’t rely on one piece of data, we had multiple sources. We had an analytics program where we were making 9,000 calls a night where we were predicting the votes.”

Murphy agreed with the political campaigns using data, “the kind of polling you see in the media has kind of vanished in campaigns where they have money to spend on research.” He said, “we don’t do telephone polling any more because we have so much data we can collect.”

Capturing everything

“We capture everything. We have about four hundred data points on the American voter and we’ll have five hundred in the next two years. We’ll be able to build massive data models without phone polling,” Murphy pointed out. “We’re waiting for the tech folk to get ahead on AI so we can predict what voters are going to do in two weeks.”

Despite the amount data collected by US political parties, the real key to success is the candidate’s organisation and management. Cutter made a strong point about the strength of Obama’s campaign team in both the 2008 and 2012 campaigns.

How the US political parties use data points to how businesses will be managing data in the future. Increasingly using information well is going to be the measure of successful organisations in both politics and industry.

Sep 232016

Dealing with the massive wave of data flowing into businesses will be one of the defining management issues of the next decade. One company that is already dealing with this is New Zealand’s Weta Digital.

Wellington based Weta that’s best known for its work on Lord of the Rings and is part owned by director Peter Jackson employs 1400 staff for its movie special effects work and has won five visual effects Academy Awards over its 23 years of operations.

Kathy Gruzas, WETA Digital’s CIO, spoke to Decoding the new Economy at the Oracle OpenWorld forum in San Francisco this week about some of the challenges in dealing with the massive amount of data generated by the movie effects industry.

“We have some very heavy loads.” Kathy states. “We push our systems to the limit.”

Applying powerful systems

One challenge is the sheer computing power required, ‘the render frame processes one frame per server until you have four seconds of footage. Sometimes that takes over night or even longer and for that we use a lot of storage,” Kathy says. “The render farm being six thousand servers will write 60 to 100 terabytes of data a day and read a quarter to half a petabyte each day.”

“We need systems that will be very large to handle the volume of data we generate but also be very quick to handle those read and writes.”

“One render could use a thousand computers, sometimes more, and all of those will be reading and writing against the same block of storage so we have our own software layer that directs those loads but we try to minimise the load on our storage but we have the worst work load you can imagine with lots of servers, lots of small reads and writes and many of them random and concurrent with pockets of hot files.”

Despite the automation, the business is still extremely capital intensive. “In visual effects you probably need at least three hundred artists to work on one film, it’s a very labour intensive process to do the artistry and much like a production line.”

Going mobile

The nature of modern movie production means the effects teams are now part of the shoot which adds another level of complexity for Weta. “Although we are visual effects which is largely post-production we do go out with crews when they’re shooting the movie so we can do reference photography,” says Kathy.

“We do 3D scans so if we need to do something digitally and we do motion and facial capture as well,” she says. “There are 240 muscles that we tweak individually to get the expression. That’s a huge amount of data to capture.”

To do this, Weta created their own ‘road case’ that contains everything they need to grab the shots and store the data they need, “you can’t ask the director retake the shot because we missed something.”

Into the forest

“We have to take the case into the forest and into the rain and everywhere. It’s good having that roadcase that has storage, networking and servers in it.” The case, which was self assembled by Weta’s team is “probably the most travelled Oracle system on the planet,” laughs Kathy with “lots of data capture and sub-rendering.”

Weta’s story illustrates just how managing data is becoming a critical issue for companies. While movie special effects is very much a specialised field that’s far ahead of the curve in its technology use than most businesses, they do show the importance of managing and securing their data.

For other businesses, lessons from Weta is understanding your company’s – including staff and customers’ – needs then investing in the right tools to deliver is essential.

One important difference between technology intensive businesses like Weta and most other organisations is the New Zealand company is doing most of its processing and storage in house. Those without the same needs will almost certainly be shifting these tasks onto the cloud.

Jul 122016
Big data takes our online, shopping and social media use it is the business challenge for our time

“We want to be the Wayze of enterprise software” is the line being repeated by executives at the Inforum2016 conference in New York today.

This is an interesting strategy for Infor, who provides a range of enterprise software tools to help companies track what is going on in their business, as Wayze is built upon aggregating user data to identify traffic problems to improve commuting times. It’s no surprise that Google bought the company a few years ago.

Infor position though is slightly different as it’s aggregating individual clients’ data for them. In a world where organisations are struggling not to be overwhelmed by information, Informa are in a good position, even if their executives do overdo it on the buzzwords.

Which leads us to another buzzphrase – design thinking – which has been drifting in and out of fashion over recent years. During the opening keynotes one of the comments was about the rise of  “network thinking.”

“Eighty percent of what most companies do deals with data from outside of their organisation,” says Kurt Cavano, Infor’s General Manager of their commerce cloud division. “We’ve seen in the power of networks with sites like Facebook, LinkedIn and Wayze.”

“Nobody wants to be on a network but everyone’s on a network. It takes a long time to build but once you have one it’s magical. That’s what we’re thinking for business, they need to evolve.”

In one respect this is another take on the ecosystem idea, that one vital corporate asset in the connected world is an ecosystem of partners, suppliers and users, however the Infor view articulated by Cavano is much more about the flow of data rather than the goodwill of a community.

So we may well be entering a world of ‘networked thinking’ where thinking about the effects of data flows and being able to understand them – if not manage them – becomes a key executive skill.

Paul travelled to New York as a guest of Infor

Mar 232016
Meg Whitman (center), President and Chief Executive Officer of Hewlett Packard Enterprise, rings the opening bell of the New York Stock Exchange on November 2, 2015. CREDIT: Eric Draper

I’m currently at the HP Enterprise Seize the Data roadshow in Singapore where the recently split company is showing off its range of data analytics tools.

Like companies such as IBM and Google, HPE are looking to make money out of data feeds and analytics with a key part being a platform for developers to create applications.

In launching their Haven OnDemand service, HPE are entering a crowded field with IBM, Salesforce, AWS and Splunk – among others – offering similar products. What compelling difference HPE will add to the field will be something I’ll be asking the company’s executive later.

One of the other services, HP Vertica, looks running data analytics against structured and ‘semi-structured’ sources. Again this is a field where other companies are well established and have an advantage in being able to examine unstructured data.

The overwhelming question though is how big, and lucrative, the market is for these data products. It’s not clear exactly how all of these companies are going to monetize these services and, should they be able to, their profitability.

As a company finding its feet less than a year after being split in two with the added problem of seeing its core server hardware business being eroded, HP Enterprise is realigning its business around data analytics and cloud services.

The challenge for the company is differentiating itself and providing competitive products in these markets, this will be a tough challenge.

Mar 212016

What’s your ethnic affinity? Apparently Facebook thinks its algorithm can guess your race based upon the nature of your posts.

This application is an interesting, and dangerous, development although it shouldn’t be expected that it’s any more accurate than the plethora of ‘guess your age/nationality/star sign’ sites that trawl through Facebook pages.

Guessing your race is something clumsy and obvious but its clear that services like Google, LinkedIn and Facebook have a mass of data on each of their millions of users that enables them to crunch some big numbers and come up with all manner of conclusions.

Some of these will be useful to governments, marketers and businesses and in some cases it may lead to unforeseen consequences.

The truth may lie in the data but if we don’t understand the questions we’re asking, we risk creating a whole new range of problems.

Feb 192016

One of the truisms of modern industry is we’re going to need more workers with data skills. Could it be actuaries will be the profession of the information age.

Much of the focus around how companies will deal with an information rich age come down to the need for ‘data scientists’, those with a combination of statistical, analytical and coding skills will be required to coax insights out of complex and rapidly changing data sets.

At a Future of PR meetup in Sydney earlier this week, one of the panellists raised the possibility that tomorrow’s most valued agency employees will be actuaries as data analytics comes to dominate the industry.

That boring old actuaries – one particularly cruel joke is atuaries are accountants who failed the personality test – could be the hottest profession in the sexy PR industry is quite a delicious scenario.

Should that turn out to be the case though, it won’t just be the PR industry chasing actuaries, almost every industry is going to demanding the same set of skills.

In a strange way it could be the staid professions of today that are the exciting jobs of tomorrow, we’ll reserve judgement on the actuaries though.

Jan 122016
censorship on the internet and social media

Forget credit scores, police are now running Threat Scores reports the Washington Post.

This isn’t surprising given the risks involved for officers attending an incident or detaining a suspect and now with treasure troves of data available, police forces and public safety agencies are able to evaluate what threats are present.

However there are real concerns about these databases and tools, particularly in how the algorithm determines what a ‘threat’ is. As the Washington Post explains one package will give a military veteran a greater risk rating as they are more likely than the general population to be suffering post traumatic stress disorder.

In promotional materials, Intrado writes that Beware could reveal that the resident of a particular address was a war veteran suffering from post-traumatic stress disorder, had criminal convictions for assault and had posted worrisome messages about his battle experiences on social media. The “big data” that has transformed marketing and other industries has now come to law enforcement.

The marketing industry’s use of Big Data has, and continues to be, problematic from a privacy and security point of view, that public agencies are using the same tools raises bigger concern.

Over time, we’re going to need rigorous supervision of how these tools are used. The stakes for individual citizens are high.