Paul Wallbank – Brilliant Communications

Tag: big data

  • The limitations of algorithms

    The limitations of algorithms

    Are algorithms getting too complex asks Forbes Magazine’s Kalev Leetaru in an examination of how the formulas that are increasingly governing our lives have grown beyond the understanding of their creators.

    With computer code now controlling most of the devices and processes we rely on in daily life, understanding the assumptions and limitations of  those programs and formulas becomes essential for designers, managers and users.

    Leetaru cites the Apollo 13 malfunction and Volvo’s recent embarrassment where a self driving car nearly ran over a group of journalists however there’s no shortage of more tragic mistakes from the consequences of software design decisions, the crash of Air France 447 over the Atlantic Ocean with the loss of 228 lives where two pilots who stalled their plane due to misunderstanding the characteristics of their cockpit  is one recent sad example.

    As business and government becomes more dependent on software, more risks will arise from managers not understanding the limitations of the algorithms they use in their business.

    Similarly a range of industries to exploit the quirks of algorithm driven markets are developing, the Search Engine Optimisation business designed to exploit quirks in Google’s search algorithm is an established example but more will come to the fore as people find ways to profit by anticipating price movements.

    However algorithms have a way to go before they fully take over, as Salon’s examination of Facebook’s news feed reveals a key part of the social media service’s deciding what appears on users screens are the decisions of around thousand ‘power users’.

    The news feed algorithm had blind spots that Facebook’s data scientists couldn’t have identified on their own. It took a different kind of data—qualitative human feedback—to begin to fill them in.

    While Facebook falls back on large focus groups to fill in the algorithm’s gaps, Uber has found a different problem in estimating driver arrival times where it’s currently not possible to accurately calculate estimated times of arrival in real time.

    “The best way to minimise time differential issue is to communicate statistically expected time, which will result in almost always being different than actual (i.e. wrong), but will be less different/wrong on average,” says Uber CEO Travis Kalanick.

    Uber and Facebook’s challenges with their algorithms illustrate there’s some way to go before all critical business functions can be handed over to software but as automation becomes standard in many areas, not least autonomous vehicles, the limitations of programs and the assumptions of programmers will become apparent.

    Similar posts:

    • No Related Posts
  • Open sourcing artificial intelligence

    Open sourcing artificial intelligence

    Silicon Valley leaders including Peter Thiel, Elon Musk and Reid Hoffman have pledged a billion dollars towards the OpenAI foundation to open source the development of Artificial Intelligence.

    With one of the greatest challenges facing business, political and community leaders in coming being how to deal with the massive amounts of data generated by the Internet of Things and pervasive computers, this is a major step in making the tools available to everyone.

    With both Google and Facebook opening their AI platforms in recent weeks, it seems the consensus in the tech industry is that open source is the way to develop these technologies. As a consequence we may see them become commonplace a lot faster than expected.

    Similar posts:

    • No Related Posts
  • A kid in a telco candy store

    A kid in a telco candy store

    “It’s a kid in a candy store opportunity,” says Telstra CTO Vish Nandlall on being asked what excites him about the telecommunications industry.

    Nandlall was talking to Decoding the New Economy about the challenges facing telcos in an industry facing massive change as the once immensely profitable voice and text services are being displaced by less lucrative data products.

    Previously we’ve spoken to Nandlall about the future of Australia’s incumbent telco in a competitive market and this interview was an opportunity to explore some of the broader opportunities in a radically changing market.

    A data business

    “While our business sounds complicated, we actually only do three things.” Nandlall observes about telecommunications companies, “we move data, we store data and we compete on data.”

    “In the course of my lifetime in telecoms any two of those coming together meant a major shift. Today all three are converging.”

    That convergence creates a range of challenges and opportunities, Nandlall believes. “When I look at what we see on the consumer side, I see the Internet of Things which really does promise a golden age of convenience.”

    “Underpinning it all is going to be a massive transformation around data, the data insights suddenly become the thing that we’re going to need to differentiate our businesses from competitors in the industry.”

    Differentiation through data

    The differentiation of telecoms companies is going to lie in the software and data services being offered, Nandlall believes. “I don’t think telcos need to replicate Over The Top services,” he says in reference to services like Facebook or WhatsApp or Skype.

    Nandlall sees the value for telcos in providing the next level of services in areas such as API management, content delivery and security. “We need to have new digital delivery systems,” he says, flagging software defined systems as being key to delivering to the new generation of telco services, “we can’t be restricted to fixed lines.”

    Facing the skills shortage

    The challenge facing telcos and all businesses is finding skilled workers, Nandlall observes. “Because change has been so rapid there has been a pipeline of students or workers being readily available.”

    Nandlall sees initiatives like Cloud Foundry and Hadoop offering a means to address the skills shortage by standardising processes, reducing complexity and automating many of the tasks occupying today’s developers and technology workers.

    This change also promises to speed up business as well and, combined with cloud services, changing the operating models of entire industries.

    A new competitive advantage

    For businesses without the scale of Telstra Nandlall has an important message, “I think we’ve hit a point in industry is where the competitive advantage is not just through some sustained differentiation,” he observes. “Today it’s about your ability to rapidly adopt new things.”

    That rapid adoption is only going to accelerate, Nandlall believes, as the Internet of Things and wearable devices bring a whole new range of ways to collect and display information. For a kid fascinated with data, that’s a big candy store.

    Similar posts:

    • No Related Posts
  • Management in time of data transparency

    Management in time of data transparency

    Management is going to become flatter and organisations more transparent as the physical and the digital start to become start to merge  says Splunk’s CTO Snehal Antani.

    Antani, who was appointed the company’s CTO in May was previously CIO for multiple divisions of GE Capital and before that numerous IT strategy and technology roles at IBM. He spoke to Decoding the New Economy at last week’s Splunk.conf in Las Vegas.

    “It’s an opportunity to change organisational structure,” Antani says in regards to how data analytics is changing business. “Transparency across managers allows me to see quantitatively and qualitatively”

    An age of transparent data

    “Everyone has access to the data so the question becomes ‘what decision do we need to make?“ He claims, “transparency really transforms the management style and culture of an organisation. It gets rid of middle managers trying to massage the message and allows me to be the leader.”

    While at GE, Antani put this transparency into action with a serious of real time indicators to hold staff and contractors to account. “I was tired, as a CIO, of middle managers showing me status reports with every box was green.”

    “For my software development process I’d built a fully instrumented continuous delivery process. When a developer checks in code, I run a fully automated set off steps and a developer would get immediate feedback. In real time I could tell you who were the best developers.”

    “I could pit my vendors up against each other,” “the cute thing there was transparency. Everyone had access to that data so we got out of Powerpoint into real time dashboards.”

    Moving IT from the back office

    That access to technology changes the role of the IT department, Atani believes. “We’ve evolved IT from a being a back office function to being a core part of the value they deliver to their customers,” he says. “In the past, when IT walked into the room people assumed they were there to fix the projector.

    This changing role is where he sees opportunities for his current company, “one of the really cool things about Splunk is that it’s a very versatile technology platform. So we were never prescriptive about up front about we were never going to solve a healthcare problem or we were going to solve a financial services problem. Our customers discovered they could apply Splunk to solve these problems”

    “We’re equally amazed as we never envisioned how the product would be used. We’re seeing really amazing use cases across health care, financial services and it’s really interesting to see how partners’ uses have evolved over the last few years.”

    Data changing management

    For companies though this means a change in the way of doing business, which can challenge management, “In order for an organization to move at market speed you have to be able to respond fast and transparency is absolutely critical to management.”

    A flatter, more transparent workplace means a radical change to the way many companies manage their organisation. It’s one of the challenges facing the modern business as we enter an age of almost unlimited data.

    Paul travelled to Splunk.conf in San Francisco as a guest of Splunk

    Similar posts:

    • No Related Posts
  • Analysing the value of IoT data

    Analysing the value of IoT data

    How do companies analyse the data coming off wearable devices? At the Las Vegas Splunk.Conf, the developers of wearable communications device Onyx showed off how they use data to enhance their business.

    A lightweight push to talk device that can be clipped to a shirt, jacket or bag strap the Onyx is designed for teams to easily communicate. The device has a microphone, speaker and GPS that tethers with a smartphone, which in turn connects to Orion’s cloud network and communicates with groups defined by the user.

    “Our goal and mission at Orion is to make this as easy and seamless at possible,” says Dan Phung, the company’s software engineer. “Technology is something you shouldn’t have to deal with.”

    Some of the data Orion collects are the battery levels in the devices, time spent on conversations and volume levels that gives the company insights into useage patterns. One of the big benefits they’ve found as a startup is in tracking what operating systems are being used, enabling them to carry out what Phung calls “data driven engineering decisions”

    As a startup with a team of 35, they managed to get the Onyx to market in a year, having that ‘operational intelligence’ has allowed the startup to focus its scarce resources in the areas where the device is being used and not waste time developing for systems that are less popular.

    The Orion Onyx is a good example of how a business can get valuable information from a limited data set from a relatively simple device, their use of Splunk also shows the value of being able to analyse that data quickly.

    Paul travelled to Splunk.conf in Las Vegas as a guest of Splunk

    Similar posts:

    • No Related Posts