Jan 122016
censorship on the internet and social media

Forget credit scores, police are now running Threat Scores reports the Washington Post.

This isn’t surprising given the risks involved for officers attending an incident or detaining a suspect and now with treasure troves of data available, police forces and public safety agencies are able to evaluate what threats are present.

However there are real concerns about these databases and tools, particularly in how the algorithm determines what a ‘threat’ is. As the Washington Post explains one package will give a military veteran a greater risk rating as they are more likely than the general population to be suffering post traumatic stress disorder.

In promotional materials, Intrado writes that Beware could reveal that the resident of a particular address was a war veteran suffering from post-traumatic stress disorder, had criminal convictions for assault and had posted worrisome messages about his battle experiences on social media. The “big data” that has transformed marketing and other industries has now come to law enforcement.

The marketing industry’s use of Big Data has, and continues to be, problematic from a privacy and security point of view, that public agencies are using the same tools raises bigger concern.

Over time, we’re going to need rigorous supervision of how these tools are used. The stakes for individual citizens are high.

Jan 062016
Big data takes our online, shopping and social media use it is the business challenge for our time

Are algorithms getting too complex asks Forbes Magazine’s Kalev Leetaru in an examination of how the formulas that are increasingly governing our lives have grown beyond the understanding of their creators.

With computer code now controlling most of the devices and processes we rely on in daily life, understanding the assumptions and limitations of  those programs and formulas becomes essential for designers, managers and users.

Leetaru cites the Apollo 13 malfunction and Volvo’s recent embarrassment where a self driving car nearly ran over a group of journalists however there’s no shortage of more tragic mistakes from the consequences of software design decisions, the crash of Air France 447 over the Atlantic Ocean with the loss of 228 lives where two pilots who stalled their plane due to misunderstanding the characteristics of their cockpit  is one recent sad example.

As business and government becomes more dependent on software, more risks will arise from managers not understanding the limitations of the algorithms they use in their business.

Similarly a range of industries to exploit the quirks of algorithm driven markets are developing, the Search Engine Optimisation business designed to exploit quirks in Google’s search algorithm is an established example but more will come to the fore as people find ways to profit by anticipating price movements.

However algorithms have a way to go before they fully take over, as Salon’s examination of Facebook’s news feed reveals a key part of the social media service’s deciding what appears on users screens are the decisions of around thousand ‘power users’.

The news feed algorithm had blind spots that Facebook’s data scientists couldn’t have identified on their own. It took a different kind of data—qualitative human feedback—to begin to fill them in.

While Facebook falls back on large focus groups to fill in the algorithm’s gaps, Uber has found a different problem in estimating driver arrival times where it’s currently not possible to accurately calculate estimated times of arrival in real time.

“The best way to minimise time differential issue is to communicate statistically expected time, which will result in almost always being different than actual (i.e. wrong), but will be less different/wrong on average,” says Uber CEO Travis Kalanick.

Uber and Facebook’s challenges with their algorithms illustrate there’s some way to go before all critical business functions can be handed over to software but as automation becomes standard in many areas, not least autonomous vehicles, the limitations of programs and the assumptions of programmers will become apparent.

Dec 132015
Big data takes our online, shopping and social media use it is the business challenge for our time

Silicon Valley leaders including Peter Thiel, Elon Musk and Reid Hoffman have pledged a billion dollars towards the OpenAI foundation to open source the development of Artificial Intelligence.

With one of the greatest challenges facing business, political and community leaders in coming being how to deal with the massive amounts of data generated by the Internet of Things and pervasive computers, this is a major step in making the tools available to everyone.

With both Google and Facebook opening their AI platforms in recent weeks, it seems the consensus in the tech industry is that open source is the way to develop these technologies. As a consequence we may see them become commonplace a lot faster than expected.

Oct 012015

“It’s a kid in a candy store opportunity,” says Telstra CTO Vish Nandlall on being asked what excites him about the telecommunications industry.

Nandlall was talking to Decoding the New Economy about the challenges facing telcos in an industry facing massive change as the once immensely profitable voice and text services are being displaced by less lucrative data products.

Previously we’ve spoken to Nandlall about the future of Australia’s incumbent telco in a competitive market and this interview was an opportunity to explore some of the broader opportunities in a radically changing market.

A data business

“While our business sounds complicated, we actually only do three things.” Nandlall observes about telecommunications companies, “we move data, we store data and we compete on data.”

“In the course of my lifetime in telecoms any two of those coming together meant a major shift. Today all three are converging.”

That convergence creates a range of challenges and opportunities, Nandlall believes. “When I look at what we see on the consumer side, I see the Internet of Things which really does promise a golden age of convenience.”

“Underpinning it all is going to be a massive transformation around data, the data insights suddenly become the thing that we’re going to need to differentiate our businesses from competitors in the industry.”

Differentiation through data

The differentiation of telecoms companies is going to lie in the software and data services being offered, Nandlall believes. “I don’t think telcos need to replicate Over The Top services,” he says in reference to services like Facebook or WhatsApp or Skype.

Nandlall sees the value for telcos in providing the next level of services in areas such as API management, content delivery and security. “We need to have new digital delivery systems,” he says, flagging software defined systems as being key to delivering to the new generation of telco services, “we can’t be restricted to fixed lines.”

Facing the skills shortage

The challenge facing telcos and all businesses is finding skilled workers, Nandlall observes. “Because change has been so rapid there has been a pipeline of students or workers being readily available.”

Nandlall sees initiatives like Cloud Foundry and Hadoop offering a means to address the skills shortage by standardising processes, reducing complexity and automating many of the tasks occupying today’s developers and technology workers.

This change also promises to speed up business as well and, combined with cloud services, changing the operating models of entire industries.

A new competitive advantage

For businesses without the scale of Telstra Nandlall has an important message, “I think we’ve hit a point in industry is where the competitive advantage is not just through some sustained differentiation,” he observes. “Today it’s about your ability to rapidly adopt new things.”

That rapid adoption is only going to accelerate, Nandlall believes, as the Internet of Things and wearable devices bring a whole new range of ways to collect and display information. For a kid fascinated with data, that’s a big candy store.

Sep 292015

Management is going to become flatter and organisations more transparent as the physical and the digital start to become start to merge  says Splunk’s CTO Snehal Antani.

Antani, who was appointed the company’s CTO in May was previously CIO for multiple divisions of GE Capital and before that numerous IT strategy and technology roles at IBM. He spoke to Decoding the New Economy at last week’s Splunk.conf in Las Vegas.

“It’s an opportunity to change organisational structure,” Antani says in regards to how data analytics is changing business. “Transparency across managers allows me to see quantitatively and qualitatively”

An age of transparent data

“Everyone has access to the data so the question becomes ‘what decision do we need to make?“ He claims, “transparency really transforms the management style and culture of an organisation. It gets rid of middle managers trying to massage the message and allows me to be the leader.”

While at GE, Antani put this transparency into action with a serious of real time indicators to hold staff and contractors to account. “I was tired, as a CIO, of middle managers showing me status reports with every box was green.”

“For my software development process I’d built a fully instrumented continuous delivery process. When a developer checks in code, I run a fully automated set off steps and a developer would get immediate feedback. In real time I could tell you who were the best developers.”

“I could pit my vendors up against each other,” “the cute thing there was transparency. Everyone had access to that data so we got out of Powerpoint into real time dashboards.”

Moving IT from the back office

That access to technology changes the role of the IT department, Atani believes. “We’ve evolved IT from a being a back office function to being a core part of the value they deliver to their customers,” he says. “In the past, when IT walked into the room people assumed they were there to fix the projector.

This changing role is where he sees opportunities for his current company, “one of the really cool things about Splunk is that it’s a very versatile technology platform. So we were never prescriptive about up front about we were never going to solve a healthcare problem or we were going to solve a financial services problem. Our customers discovered they could apply Splunk to solve these problems”

“We’re equally amazed as we never envisioned how the product would be used. We’re seeing really amazing use cases across health care, financial services and it’s really interesting to see how partners’ uses have evolved over the last few years.”

Data changing management

For companies though this means a change in the way of doing business, which can challenge management, “In order for an organization to move at market speed you have to be able to respond fast and transparency is absolutely critical to management.”

A flatter, more transparent workplace means a radical change to the way many companies manage their organisation. It’s one of the challenges facing the modern business as we enter an age of almost unlimited data.

Paul travelled to Splunk.conf in San Francisco as a guest of Splunk

Sep 252015

How do companies analyse the data coming off wearable devices? At the Las Vegas Splunk.Conf, the developers of wearable communications device Onyx showed off how they use data to enhance their business.

A lightweight push to talk device that can be clipped to a shirt, jacket or bag strap the Onyx is designed for teams to easily communicate. The device has a microphone, speaker and GPS that tethers with a smartphone, which in turn connects to Orion’s cloud network and communicates with groups defined by the user.

“Our goal and mission at Orion is to make this as easy and seamless at possible,” says Dan Phung, the company’s software engineer. “Technology is something you shouldn’t have to deal with.”

Some of the data Orion collects are the battery levels in the devices, time spent on conversations and volume levels that gives the company insights into useage patterns. One of the big benefits they’ve found as a startup is in tracking what operating systems are being used, enabling them to carry out what Phung calls “data driven engineering decisions”

As a startup with a team of 35, they managed to get the Onyx to market in a year, having that ‘operational intelligence’ has allowed the startup to focus its scarce resources in the areas where the device is being used and not waste time developing for systems that are less popular.

The Orion Onyx is a good example of how a business can get valuable information from a limited data set from a relatively simple device, their use of Splunk also shows the value of being able to analyse that data quickly.

Paul travelled to Splunk.conf in Las Vegas as a guest of Splunk

Sep 232015

“We’re early in the marathon but making good progress”, opened Godfrey Sullivan, the CEO of Splunk, as he opened the company’s annual conference in Las Vegas today.

Helping businesses understand their data has proved lucrative for Splunk with the analytics company seeing a 46% increase in year on year revenue to $148 million for the last quarter with the organisation narrowing its losses over the same period.

As with all tech conferences, the focus in the opening keynote is on new product announcements. For Splunk, the main release is its latest enterprise version of Splunk Enterprise 6.3 billed as delivering faster results, better analytics and tying into the masses of machine data being collected from the Internet of Things.

Machine data as a cornerstone

That IoT data is a key part of Sullivan’s strategy of “making machine data more accessible usable and valuable to everyone.” The company also highlights their alliances with IoT data consolidator services such as Xively and Octoblu.

Security is another focus of Splunk with the launch of  Splunk User Behavior Analytics (UBA) that analyses usage patterns on networks to identify risky or suspicious activity and a version upgrade of their their Enterprise Security.

The original business of Splunk was to monitor server log files and that IT focus remains with their new IT Service Intelligence (ITSI), an improved IT monitoring and analytics service.

Sullivan’s key message was that IT departments can be offering ‘operational intelligence’ as they gather and analyse data from all aspects of a business. “IT departments have to earn a seat at the table”, as Splunk’s CTO Snehal Antani says and providing rich data analytics, in his view, enable this.

Surprising a bank

Antani cited one of his previous clients, a bank which would ordinarily would deal with ten million dollars of deposits a day so an alarm had been set for when less than half of that had been received by midday.

One day that alarm sounded, and the IT department assumed there was a problem with the bank’s systems. After checking, they found everything was running normally so flagged deposits were unusually low to senior management.

It turned out to be a competitor had launched a successful campaign to open new accounts which had caught the bank by surprise. “The CMO acted as if he’d been hacked,” Antani recalls.

Antani’s anecdote illustrates how business data is no longer just the concern of the IT department and a small group of geeky business analysts, with real time information every part of an organisation can improve its performance.

For Splunk, using data to improve all aspects of business its key message to the market and one it hopes to drive its business forward although it’s highly unlikely they’ll achieve Antani’s hopes of “making IT sexy again.” That would take much more than a marathon.

Paul travelled to the Splunk.conf in Las Vegas as a guest of Splunk