Category: Big Data

Engineering for change – the ethics of the new economy

What are the ethical and societal considerations we should consider with today’s technology?

Technologies like the internet of things, cloud computing, 3D printing and big data are changing our industries and society. At the ACI Connect event today, I gave a presentation on some of the opportunities, risks and ethical issues facing technologists and engineers in the connected economy.

While many of the engineering principles underlying these technologies aren’t new, their scale and the power they give businesses and governments means there are serious ethical, security and societal issues we have to consider.

This presentation explores some of those issues and the technologies and trends driving them.

Entering the Data era

A conceit among technologists is that we’re in an unprecedented era of change. This is not true.

The Twentieth Century saw massive restructuring of our society as the telephone, mains electricity, the motor car and television changed our society. Many of today’s settled industries came out of the huge technological steps forward over the last hundred years.

Just as cheap energy – delivered to us through the motor car and mains electricity – defined the Twentieth Century, this century will be defined by easily accessible and abundant information.

Those changes over the last hundred years give us some hint as to where we are going; the shifts that saw coal carters, newspaper sellers and night soil men eventually become extinct, along with a shift from a largely agricultural workforce to industrialised employment, is going to be repeated this century as information becomes abundant.

Harnessing the Internet of bees

Cheap and small sensors mean it’s easier to put a chip on something. In this case we have a CSIRO project tracking bee activity where Tasmanian scientists have put tracking devices on bees.

Those tracking devices would have weighed several hundred grams and cost hundreds of dollars ten years ago but today they are small and cheap enough to fit onto the backs of bees.

Being able to deploy these sensors means we can fit them to things we couldn’t have imagined a few years ago and the data they generate is going to give us insights into patterns and behaviours we couldn’t have contemplated.

However not all of this data is useful or necessary and some may even be damaging to individuals and groups. One ethical question we have to ask ourselves is whether it is in the community’s interests to collect this information.

Another aspect of connecting devices, or even animals and people, to the Internet or a network is it opens the possibility of hacking, as we’ve seen in the recent Jeep case where engineers showed they could control a vehicle remotely. The security and privacy aspects of the IoT are critical and something designers and product engineers can’t overlook.

Decoding the data

It’s often said that Data is the New Oil. In truth it isn’t, data is increasingly cheap and easy to access. Being able to analyse that information is where the power lies.

Data analytics is probably going to be one of the most important fields in an information rich economy and already we’re seeing companies springing up to help farmers estimate crop yields, truck drivers plan their routes and even organisations like the Royal Flying Doctor Service using cloud services to better plan their operations.

Again these services plan a lot but there’s also downsides as inappropriate data matching risks breaching consumers’ privacy and even drawing false conclusions from confusing correlation with causation. A good example of this is Facebook being used to judge credit worthiness.

Removing the human element

Automation – whether it’s through robotics, machine learning or algorithms – will change many industries and the workforces employed by them.

One understated field is management where many white collar supervisor jobs are at risk from business automation. It may be that the executive suites are the next sector to be decimated by computers and robots.

Similarly, many services industry jobs such as taxi drivers and baristas are at risk from robotics while large scale 3D printing of buildings threatens to put many building trades under pressure.

No more truck drivers

Driverless vehicles have a whole range of applications, in logistics were seeing them put forklift drivers out of work while mining companies are rolling out massive dump trucks in their new mines that don’t require $200,000 a year drivers.

One study estimates that half the police workforce in the United States would become redundant as law abiding driverless cars become common.

Similarly electric cars will have a massive impact on government revenues. Currently Australian governments raise $17bn a year from fuel excise and has ramifications for businesses involved in the supply chain for service stations.

Once driverless vehicles become commonplace we may well see them changing industries like daycare, public transport and couriers as it becomes possible to summon an autonomous vehicle, put the kids or the luggage into it and then send it off to its destination. If you’re worried, you can track the progress on an app.

The effects of the driverless car show how we have to think laterally about the effects of new technologies on our businesses, sometimes the effects of a new way of doing things could indirectly hurt our business or create new opportunities.

Squeezing out inefficiencies

One of the great promises for the IoT, Big Data and business automation is to remove inefficiencies from industry. Cisco believe that up to 14% of the Oil and Gas industry’s costs could be stripped away with today’s technologies. That in itself is worth over a 100 billion dollars a year in cost savings.

GE are deploying their technologies into a diverse range of industrial equipment ranging from jet engines to railway locomotives and wind turbines with spectacular results in reducing costs and improving productivity.

The effect of these improvements means less downtime and maintenance costs which are good news for customers and shareholder of these companies, but bad news if you’re a maintenance business. It also means the speed of change in business is accelerating.

Skilling the future workforce

In summary the skills needed today are very different to those of 1915 and 1965 and those of the next fifty years will be even different.

As a society we have to decide what skills we are going to give not our children but those currently still in the workforce who are going to be working longer and later into their lives as the workforce ages.

We also have to consider what sort of ethical compass we have. While the technology we have today is powerful and capable of great things, it’s also capable of great harm. We need to have an understanding of what the effects and limits are of our actions with the Internet of Things, Big Data and analytics.

Ultimately we need to ask what value we as individuals can add to our communities and society.

Putting machine learning into wine

Two South Australian students are showing how the wine industry can use machine learning and cloud computing

As we gather more data, the opportunities to apply it become wider. A good example of this is Seer Insights, a South Australian company started by pair of university students that calculates the likely grape yields for vineyards.

Seer Insights’ product Grapebrain is made up of two components, a mobile app that the farmer uses to count the grape clusters on the vines and then a cloud service that analyses the data and produces web based reports for the farmers.

The current methods are notoriously unreliable with Seer Insights estimating mistakes cost the Australian viticulture industry $200 million a year as harvests are miscalculated resulting in either rotting fruit or wasted contractor fees.

Born in an elevator

Seer’s founders, Harry Lucas and Liam Ellul, started the business after a chance meeting on their university campus. “We started off doing this after being stuck in a lift together,” remembers Liam. “Originally we were looking at the hyper-spectrum imaging for broadacre farming but when we started looking at the problems we ended up talking to wine organisations about this.”

“The technology predicts how many grapes will be coming off the vineyards at the end of the season to enable people to sort out their finances,” Harry says. “The growth process grapes go through is difficult to model so we use machine learning to do that.”

For both the founders having an off the shelf product, in this case Microsoft’s machine learning tools, to run the data analysis made it relatively easy to launch the product.

As a winner of Microsoft’s Tech eChallenge, the startup has won a trip to the United States as well as being profiled by the company as a machine learning case study.

Over time as these tools become more accessible to small companies we’ll see more businesses accessing machine learning services to enhance their operations.

As companies face the waves of data flowing into their businesses over the next decade, it will be those who manage it well and gather valuable insights from their information that will be the winners.

Goodbye to the media buyers long lunch

Big data and analytics are changing roles in the media industry, managers in other sectors should worry about the changes.

Yesterday Decoding The New Economy posted an interview with Michael Rubenstein of AppNexus about the world of programmatic advertising and being part of a rapidly growing startup.

The whole concept of programmatic advertising is a good example of a business, and a set of jobs, being disrupted.

Media buying has been a cushy job for a generation of well fed advertising executives. David Sarnoff’s invention of the broadcast media model in the 1930s meant salespeople and brokers were needed to fill the constant supply of advertising spots.

Today the rise of the internet has disrupted the once safe world of broadcast media where incumbents were protected by government licenses and now the long lunching media buyers are finding their own jobs are being displaced by algorithms like those of AppNexus.

A thought worth dwelling on though is that media buyers are part of a wider group of white collar roles being disrupted by technology – the same Big Data algorithms driving AppNexus and other services is also being used to write and select news stories and increasingly we’ll see executive decisions being made by computers.

It’s highly likely the biggest casualties of the current data analytics driven wave won’t be truck drivers, shelf pickers or baristas but managers. The promise of a flat organisation may be coming sooner than many middle managers – and salespeople – think.

Dealing with the biggest of data

The CERN research project generates huge amounts of data however the human touch is needed to analyse and manage the information

How do you deal with the biggest data sets of all? Bob Jones, a project leader for the European Organization for Nuclear Research – commonly known as CERN – described how the world’s largest particle physics laboratory manages 100 petabytes of data.

The first step is not to collect everything, ““We can’t keep all the data, the key is knowing what to keep” says Jones. This is understandable given the cameras capturing the collisions have 150 million sensors delivering data at 40 million times per second.

Jones was speaking at the ADMA Global Conference’s Advancing Analytics stream where he was describing how the project manages and analyses the vast amounts of data generated by the huge projects.

Adding to Jones’ task and that facing CERN’s boffins is that data has to be preserved and verifiable so scientists can review the results of experiments.

Discovering the Higgs Boson for instance required finding 400 positive results out of 600,000,000,000,000,000 events. This requires massive processing and storage power.

Part of the solution is to have a chain of data centres across the world to carry out both the analytics and data storage supplemented by tape archiving, something that creates other issues..

“Tape is a magnetic medium which means it deteriorates over time.” Jones says, “we have to repack this data every two years.”

Another advantage with a two year refresh is this allows CERN to apply the latest advances in data storage to pack more data into the medium.

CERN itself is funded by its 21 member states – Pakistan is its latest member – which contribute its $1.5 billion annual budget and the organisation provides data and processing power to other multinational projects like the European Space Agency and to private sector partners.

For the private sector, CERNs computing power gives the opportunity to do in depth analytics of large data sets while the unique hardware and software requirements mean the project is a proving ground for high performance equipment.

Despite the high tech, Jones says the real smarts behind CERN and the large Hadron Collider lie in the people. “All of the people analysing the data are trained physicists with detailed, multi year domain knowledge.”

“The reason being is the experiment and the technology changes so quickly, it’s not written down. It’s in the heads of those people.”

In some respects this is comforting for those of us worrying about the machines taking over.

Literacy in old and new terms

Is data literacy as important today as being able to read and write was a century ago?

I’m in Wellington, the capital of New Zealand, for the next few days for the Open Source, Open Society conference.

During one of the welcome events Lillian Grace of Wiki New Zealand mentioned how today we’re at the same stage with data literacy that we were two hundred years ago with written literacy.

If anything that’s optimistic. According to a wonderful post on Our World In Data, in 1815 the British literacy rate was 54%.

world-literacy-rates

That low rate makes sense as most occupations didn’t need literate workers while a hundred years later industrial economies needed employees who could read and write.

Another notable point is the Netherlands has led the world in literacy rates for nearly four hundred years. This is consistent with the needs of a mercantile economy.

Which leads us to today’s economy. In four hundred years time will our descendants  be commenting on the lack of data literacy at the beginning of the Twenty-First Century?

 

Building the closed internet

Social media services like Facebook and LinkedIn are building API walled gardens that will change how online services work

One of the great strengths of the social and cloud business model was the idea of the open API, recent moves by Twitter and LinkedIn show that era might be coming to an end.

This week Nick Halstead, the founder and CEO of business intelligence service Datasift, bemoaned his company’s failure to negotiate an API access agreement with Twitter that restricts their ability to deliver insights to customers.

Earlier this year LinkedIn announced they would be restricting API access to all but “partnership integrations that we believe provide the most value to our members, developers and business.”

Monetizing APIs

Increasingly social media and web services companies are seeing access to APIs as being a revenue opportunity – something many of them are struggling to find – or as a way of building ‘strategic partnerships’ that will create their own walled gardens on the internet.

For developers this is irritating and for users it restricts the services and applications available but it may turn out to backfire on companies like LinkedIn and Twitter as closing down APIs opens opportunities for new platforms.

A few years ago industry pundits, like this blog, proclaimed open APIs will be a competitive advantage for online services. Now we’re about to find out how true that is.

One thing is for sure; many of the companies proclaiming their support for the ‘open internet’ are less free when it comes to allowing access to their own data.

Who owns a smartcar’s smarts?

The question of software ownership in a smartcar opens a range of difficult questions about the internet of things.

Automakers Say You Don’t Really Own Your Car states the Electronic Frontiers Foundation.

In their campaign to amend the US Digital Millenium Copyright Act to give vehicle owners the right to access and modify their automobiles’ software the EFF raises an important point.

Should the software licensing model be applied to these devices then purchasers don’t really own them but rather have a license to use them until the vendor deems overwise.

Cars, of course, are not the only devices where this problem arises. The core of the entire Internet of Things lies in the software running intelligent equipment, not the hardware. If that software is proprietary and closed then no purchaser of a smart device truly owns it.

Locking down the smarthome

This raises problems in smarthomes, offices and businesses where the devices people come to depend upon are ‘black boxes’ that they aren’t allowed to peer into. It’s not hard to see how in industrial or agricultural applications that arrangement will often be at best unworkable.

Four years ago tech industry leader Marc Andreessen pointed out how software is eating the world; that most of the value in an information rich economy lies in the computer programs that processes the data, not the hardware which collects and distributes it.

That shift was flagged decades ago when the initial fights over software patents occurred in the 1980s and 90s and today we’re facing the consequences of poorly thought out laws, court decisions and patent approvals that now challenge the concepts of ownership as we know it.

Is ownership outdated?

However it may well be that ‘ownership’ itself is an outdated concept. We could be entering a period where most of our possessions are leased rather than owned.

If we are in a period where ownership is an antiquated concept then does it matter that our cars, fitness bands, kettles, smoke alarms and phones are in effect owned by a corporation incorporated in Delaware that pays most of its tax in the Dutch Antilles?

Who owns the smartcar’s data?

The next question of course is if the software in our smart devices is secret and untouchable then who owns the data they generate?

Ownership of a smartcar’s data could well be the biggest issue of all in the internet of things and the collection of Big Data. That promises to be a substantial battle.

In the meantime, it may not be a good idea to tinker too much with your car’s software or the data it generates.

Smartcars and sports tech – ABC Nightlife April 2015

Sports technology, smart cars and the internet of things is are the topics for the April Nightlife

Paul Wallbank joins Tony Delroy on ABC Nightlife nationally from 10pm Australian Eastern time on Thursday, April2 to discuss how technology affects your business and life.

For the April 2015 program Tony and Paul look at Tesla founder Elon Musk’s prediction that driverless cars could be on US roads by the middle of the year.

Another industry that’s currently being disrupted by technology is sports. On the field, in the stadium and at home how games are played and watched is being changed.

Join us

Tune in on your local ABC radio station from 10pm Australian Eastern Summer time or listen online at www.abc.net.au/nightlife.

We’d love to hear your views so join the conversation with your on-air questions, ideas or comments; phone in on 1300 800 222 within Australia or +61 2 8333 1000 from outside Australia.

You can SMS Nightlife’s talkback on 19922702, or through twitter to @paulwallbank using the #abcnightlife hashtag or visit the Nightlife Facebook page.

Panning for digital gold

A new breed of social media analytics is finding gold in online data.

Today social media analytics startup Vintank announced their acquisition by the W2O Group, a network of data driven marketing and communications firms.

W2O’s acquisition shows how data analytics and visualisation is increasingly an important tool for management in a world where businesses are drowning in information.

Last year Vintank co-founder Paul Mabray spoke to Decoding the New Economy about the company and how social media data is a valuable tool for the wine industry.

“The wine industry is last industry to have been changed by internet,” Mabray says. One reason for this in his view is how that the sector hasn’t had a disruptive startup like Yelp or Open Table to drive change and upset incumbents.

Despite the wine industry’s reluctance to adopt digital technologies, social media and the disruption of established media channels is having a profound effect on the sector’s marketing and sales.

“In the old days there was a playbook originating with Robert Mondavi in the 1970s which is create amazing wine, you get amazing reviews and you go find wholesalers who bring this wine to the market,” Mabray told Decoding the New Economy during a visit to Australia in 2014.

Dealing with global proliferation

Mabray also flags the massive growth in the wine industry as being one of Vintank’s driving forces, “the global proliferation of brands the increase of awareness and consumption patterns where people like wine more, those playbooks didn’t work in 2009 when the crisis started.”

A proliferation of new competitors coupled with disrupted communications channels isn’t unique to the wine industry, the attraction Vintank has to the w2O Group’s president Bob Pearson; “VinTank provides us with a way to create agile audience engines for a brand, where we can learn what an audience is doing online, understand what content they like.”

For many businesses social media is a both an opportunity and a mystery; while customers are telling the world what they’re buying through services like Facebook and Twitter capturing, managing and using that information remains a challenge.

Panning for digital gold

As Robyn Lewis of Visit Vineyards whose database holds details on over 30,000 Australian wineries and associated tourism business says, “the gold is in the data.”

Panning for that gold is Emma LoRusso of Sydney social analytics startup Digivizer who told Decoding The New Economy two years ago “the truth is in data”. Services like Vintank, Salesforce’s Radian6, Klout and startups like Digivizer attempt to add context to that data.

Another aspect of Vintank’s technology is the ‘geofencing’ of information, creating a virtual geographic perimeter so only data relevant in that region is flagged. As well as reducing noise, this increases the value to local wineries and tourist operations.

In some respects the geofencing is possibly the most powerful part of services like Vintank as it allows regional operators to focus on visitors and customers to their districts rather than worrying about national or global activity.

W2O’s acquisition gives Vintank access to a broader market outside the wine industry as well as deeper data analytics capabilities. For W20 the purchase adds to the social media tools the company can offer.

Data driven business

The Vintank deal with W2O shows how the marketing and advertising industries are increasingly becoming data driven. For other business functions this is true as well.

For businesses of all types, understanding the data pouring into their companies is going to be the difference between success and failure in an increasingly digital world. Providing those tools to do so is one of the great opportunities in today’s economy.

The risks of government surveillance – how Australia’s data retention laws hurt

The Australian government is about to pass data retention laws which will be expensive and won’t work

This morning I’m speaking on ABC Radio’s Overnights about the risks of the Australian government’s law to force telecommunications companies to retain users’ metadata for two years.

While the act, currently before the Senate having passed the House of Representatives last week after the poorly named ‘opposition’ Labor Party supported it, mandates that telcos and ISPs will have to retain the details of users’ connection times, places and type of device for two years and that government agencies will be able to access this data without a warrant.

The program was broadcast on 26 March 2015 at 4.15am Eastern Time with Trevor Chappell and is can be listened to on the ABC radio website.

Some resources on the data retention bill follow;

Big sports data – how tech is changing the playing field

The internet of things is dramatically changing the world of sports

“When you’re playing, it’s all about the winning but when you retire you realise there’s a lot more to the game,” says former cricketer Adam Gilchrist.

Gilchrist was speaking at an event organised by software giant SAP ahead of a Cricket World Cup quarter final at the Melbourne Cricket Ground yesterday.

SAP were using their sponsorship of the event to demonstrate their big data analytics capabilities and how they are applied to sports and the internet of things.

Like most industries, the sports world is being radically affected by digitalisation as new technologies change everything from coaching and player welfare through to stadium management and fans’ experience.

Enhancing the fan experience

Two days earlier rival Melbourne stadium Etihad in the city’s Docklands district showed off their new connected ground where spectators will get hi-definition video and internet services through a partnership between Telstra and Cisco.

While Etihad’s demonstration was specifically about ‘fan experience’, the use of the internet of things and pervasive wireless access in a stadium can range from paperless ticketing to managing the food and drink franchises.

In the United States, the leader in rolling out connected stadiums, venues are increasingly rolling out beacon technologies allowing spectators to order deliveries to their seats and push special offers during the game.

While neither of the two major Melbourne stadiums offer beacon services at present, the Cisco devices around the Etihad have the facility to add Bluetooth capabilities when the ground managements decide to roll them out.

Looking after players

Probably the greatest impact of technology in sport is with player welfare; while coaches and clubs have been enthusiastic adopters of video and tracking technologies for two decades, the rate of change is accelerating as wearable devices are changing game day tactics and how injuries are managed.

One of the companies leading this has been Melbourne business Catapult Sports which has been placing tracking devices on Australian Rules football players and other codes for a decade.

For coaches this data has been a boon as it’s allowed staff to monitor on field performance and tightly manage players’ health and fitness.

Professional sports in general have been early adopters of new technologies as a small increase in performance can have immediate and lucrative benefits on the field. Over the last thirty years clubs have adopted the latest in video and data technology to help coaches and players.

As the technology develops this adoption is accelerating, administrators are looking at placing tracking devices within the balls, goals and boundary lines to give even more information about what’s happening on the field.

Managing the data flow

The challenge for sports organisations, as with every other industry, is in managing all the data being generated.

In sports managing that data has a number of unique imperatives; gamblers getting access to sensitive data, broadcast rights holders wanting access to game statistics and stadium managers gathering their own data all raise challenges for administrators.

There’s also the question of who owns the data; the players themselves have a claim to their own personal performance data and there could potentially be conflicts when a competitor transfers between clubs.

As the sports industry explores the limits of what they can do with data, the world is changing for players, coaches, administrators and supporters.

Gilchrist’s observation that there’s a lot more to professional sports than just what happens on the field is going to become even more true as data science assumes an even greater role in the management of teams, clubs and stadiums.

Paul travelled to Melbourne as a guest of Cisco and SAP.

The high cost of distrust

A lack of trust in data is going to cost the world’s economy over a trillion dollars forecast a Cisco panel

A lack of trust in technology’s security could be costing the global economy over a trillion dollars a panel at the Australian Cisco Live in Melbourne heard yesterday.

The panel “how do we create trust?” featured some of Cisco’s executives including John Stewart, the company’s Security and Trust lead, along with Mike Burgess, Telstra’s Chief Information Security Officer and Gary Blair, the CEO of the Australian Cyber Security Research Institute.

Blair sees trust in technology being split into two aspects; “do I as an individual trust an organisation to keep my data secure; safe from harm, safe from breaches and so forth?” He asks, “the second is will they be transparent in using my data and will I have control of my data.”

In turn Stewart sees security as being a big data problem rather than rules, patches and security software; “data driven security is the way forward.” He states, “we are constantly studying data to find out what our current risk profile is, what situations are we facing and what hacks we are facing.”

This was the thrust of last year’s Splunk conference where the CISO of NASDAQ, Mark Graff, described how data analytics were now the front line of information security as threats are so diverse and systems so complex that it’s necessary to watch for abnormal activity rather than try to build fortresses.

The stakes are high for both individual businesses and the economy as technology is now embedded in almost every activity.

“If you suddenly lack confidence in going to online sites, what would happen?” Asks Stewart. “You start using the phone, you go into the bank branch to check your account.”

“We have to get many of these things correct, because going backwards takes us to a place where we don’t know how to get back to.”

Gary Blair described how the Boston Consulting Group forecast digital economy would be worth between 1.5 and 2.5 trillion dollars across the G20 economies by 2016.

“The difference between the two numbers was trust. That’s how large a problem is in economic terms.”

As we move into the internet of things, that trust is going to extend to the integrity of the sensors telling us the state of our crops, transport and energy systems.

The stakes are only going to get higher and the issues more complex which in turn is going to demand well designed robust systems to retain the trust of businesses and users.