Paul Wallbank – Brilliant Communications

Category: Big Data

  • Putting machine learning into wine

    Putting machine learning into wine

    As we gather more data, the opportunities to apply it become wider. A good example of this is Seer Insights, a South Australian company started by pair of university students that calculates the likely grape yields for vineyards.

    Seer Insights’ product Grapebrain is made up of two components, a mobile app that the farmer uses to count the grape clusters on the vines and then a cloud service that analyses the data and produces web based reports for the farmers.

    The current methods are notoriously unreliable with Seer Insights estimating mistakes cost the Australian viticulture industry $200 million a year as harvests are miscalculated resulting in either rotting fruit or wasted contractor fees.

    Born in an elevator

    Seer’s founders, Harry Lucas and Liam Ellul, started the business after a chance meeting on their university campus. “We started off doing this after being stuck in a lift together,” remembers Liam. “Originally we were looking at the hyper-spectrum imaging for broadacre farming but when we started looking at the problems we ended up talking to wine organisations about this.”

    “The technology predicts how many grapes will be coming off the vineyards at the end of the season to enable people to sort out their finances,” Harry says. “The growth process grapes go through is difficult to model so we use machine learning to do that.”

    For both the founders having an off the shelf product, in this case Microsoft’s machine learning tools, to run the data analysis made it relatively easy to launch the product.

    As a winner of Microsoft’s Tech eChallenge, the startup has won a trip to the United States as well as being profiled by the company as a machine learning case study.

    Over time as these tools become more accessible to small companies we’ll see more businesses accessing machine learning services to enhance their operations.

    As companies face the waves of data flowing into their businesses over the next decade, it will be those who manage it well and gather valuable insights from their information that will be the winners.

    Similar posts:

    • No Related Posts
  • Goodbye to the media buyers long lunch

    Goodbye to the media buyers long lunch

    Yesterday Decoding The New Economy posted an interview with Michael Rubenstein of AppNexus about the world of programmatic advertising and being part of a rapidly growing startup.

    The whole concept of programmatic advertising is a good example of a business, and a set of jobs, being disrupted.

    Media buying has been a cushy job for a generation of well fed advertising executives. David Sarnoff’s invention of the broadcast media model in the 1930s meant salespeople and brokers were needed to fill the constant supply of advertising spots.

    Today the rise of the internet has disrupted the once safe world of broadcast media where incumbents were protected by government licenses and now the long lunching media buyers are finding their own jobs are being displaced by algorithms like those of AppNexus.

    A thought worth dwelling on though is that media buyers are part of a wider group of white collar roles being disrupted by technology – the same Big Data algorithms driving AppNexus and other services is also being used to write and select news stories and increasingly we’ll see executive decisions being made by computers.

    It’s highly likely the biggest casualties of the current data analytics driven wave won’t be truck drivers, shelf pickers or baristas but managers. The promise of a flat organisation may be coming sooner than many middle managers – and salespeople – think.

    Similar posts:

    • No Related Posts
  • Dealing with the biggest of data

    Dealing with the biggest of data

    How do you deal with the biggest data sets of all? Bob Jones, a project leader for the European Organization for Nuclear Research – commonly known as CERN – described how the world’s largest particle physics laboratory manages 100 petabytes of data.

    The first step is not to collect everything, ““We can’t keep all the data, the key is knowing what to keep” says Jones. This is understandable given the cameras capturing the collisions have 150 million sensors delivering data at 40 million times per second.

    Jones was speaking at the ADMA Global Conference’s Advancing Analytics stream where he was describing how the project manages and analyses the vast amounts of data generated by the huge projects.

    Adding to Jones’ task and that facing CERN’s boffins is that data has to be preserved and verifiable so scientists can review the results of experiments.

    Discovering the Higgs Boson for instance required finding 400 positive results out of 600,000,000,000,000,000 events. This requires massive processing and storage power.

    Part of the solution is to have a chain of data centres across the world to carry out both the analytics and data storage supplemented by tape archiving, something that creates other issues..

    “Tape is a magnetic medium which means it deteriorates over time.” Jones says, “we have to repack this data every two years.”

    Another advantage with a two year refresh is this allows CERN to apply the latest advances in data storage to pack more data into the medium.

    CERN itself is funded by its 21 member states – Pakistan is its latest member – which contribute its $1.5 billion annual budget and the organisation provides data and processing power to other multinational projects like the European Space Agency and to private sector partners.

    For the private sector, CERNs computing power gives the opportunity to do in depth analytics of large data sets while the unique hardware and software requirements mean the project is a proving ground for high performance equipment.

    Despite the high tech, Jones says the real smarts behind CERN and the large Hadron Collider lie in the people. “All of the people analysing the data are trained physicists with detailed, multi year domain knowledge.”

    “The reason being is the experiment and the technology changes so quickly, it’s not written down. It’s in the heads of those people.”

    In some respects this is comforting for those of us worrying about the machines taking over.

    Similar posts:

    • No Related Posts
  • Literacy in old and new terms

    Literacy in old and new terms

    I’m in Wellington, the capital of New Zealand, for the next few days for the Open Source, Open Society conference.

    During one of the welcome events Lillian Grace of Wiki New Zealand mentioned how today we’re at the same stage with data literacy that we were two hundred years ago with written literacy.

    If anything that’s optimistic. According to a wonderful post on Our World In Data, in 1815 the British literacy rate was 54%.

    world-literacy-rates

    That low rate makes sense as most occupations didn’t need literate workers while a hundred years later industrial economies needed employees who could read and write.

    Another notable point is the Netherlands has led the world in literacy rates for nearly four hundred years. This is consistent with the needs of a mercantile economy.

    Which leads us to today’s economy. In four hundred years time will our descendants  be commenting on the lack of data literacy at the beginning of the Twenty-First Century?

     

    Similar posts:

    • No Related Posts
  • Building the closed internet

    Building the closed internet

    One of the great strengths of the social and cloud business model was the idea of the open API, recent moves by Twitter and LinkedIn show that era might be coming to an end.

    This week Nick Halstead, the founder and CEO of business intelligence service Datasift, bemoaned his company’s failure to negotiate an API access agreement with Twitter that restricts their ability to deliver insights to customers.

    Earlier this year LinkedIn announced they would be restricting API access to all but “partnership integrations that we believe provide the most value to our members, developers and business.”

    Monetizing APIs

    Increasingly social media and web services companies are seeing access to APIs as being a revenue opportunity – something many of them are struggling to find – or as a way of building ‘strategic partnerships’ that will create their own walled gardens on the internet.

    For developers this is irritating and for users it restricts the services and applications available but it may turn out to backfire on companies like LinkedIn and Twitter as closing down APIs opens opportunities for new platforms.

    A few years ago industry pundits, like this blog, proclaimed open APIs will be a competitive advantage for online services. Now we’re about to find out how true that is.

    One thing is for sure; many of the companies proclaiming their support for the ‘open internet’ are less free when it comes to allowing access to their own data.

    Similar posts:

    • No Related Posts