Adrift in the data lake

We’re awash with data and businesses have to figure out how not to drown in it

Last week Yahoo! closed down their directory pages ending one of the defining services of the 1990s internet and showing how the internet has changed since the first dot com boom.

The Yahoo! Directory was victim of a fundamental change in how we manage data as Google showed it wasn’t necessary to tag and label every piece of information before it could be used.

Yahoo!’s Directory was a classic case of applying old methods to new technologies – in this case carrying out a librarian’s function of cataloguing and categorising every web page.

One problem with that way of saving information is you need to know part of the answer before you can start searching; you need to have some idea of what category your query comes under or the name of the business or person you’re looking for.

That pan was exploited by the Yellow Pages where licensees around the world harvested a healthy cash flow from businesses forced to list under a dozen different categories to make sure prospective customers found them.

With the arrival of Google that way of structuring information came to an end as Sergey Brin and Larry Page’s smart algorithm showed it wasn’t necessary to pigeonhole information into highly structured databases.

Unstructured data

Rather than being structured, data is now becoming ‘unstructured’ and instead of employing an army of clerks to categorise information it’s now the job of computers to analyse that raw information and pick out what we need for our businesses and lives.

As information pours into companies from increasingly diverse sources, a flood that’s becoming so great it’s being referred to as the ‘data lake’, it’s become clear the battle to structure data is lost.

At the Splunk Conference in Las Vegas this week, the term ‘data lake’ is being used a lot as the company explains its technology for analysing business information.

Splunk, along with services like IBM’s Watson and Tableau Software, is one the companies capitalising on businesses’ need to manage unstructured data by giving customers the tools to analyse their information without having first to shoehorn it into a database.

“Thanks to Google we got to look at data a different way,” says Splunk’s CEO and Chairman Godfrey Sullivan. “You don’t have to know the question before you start the search.”

Diving into the data lake

It’s always dangerous applying simple labels to computing technologies but some terms, like ‘Cloud Computing’, don’t do a bad job of describing the principles involved and so it is with the ‘data lake’.

Rather than a nice, orderly world where everything can be pigeonholed, we know have a fluid environment where it wouldn’t be possible to label everything even if we wanted to. A lake is a good description of the mass of data pouring into our lives.

The web was an early example of having to manage that data lake and Google showed how it could be done. Now it’s the turn of other companies to apply the principles to business.

Google fatally damaged both Yahoo! and the Yellow Pages, other companies that are stuck in the age of structured data are going to find the future equally dismal. Don’t drown in that data lake.

Paul travelled to Las Vegas as a guest of Splunk

Building community knowledge

Google’s Waze is a good example of shared intelligence

One of the promises of big data and the internet of things is that local governments will be able to gather information about the state of their infrastructure.

A good working example of this is Google’s Waze, the Israeli traffic monitoring startup bought by the search engine giant two years ago.

Waze gathers information about traffic delays and transit times from users then aggregates them to give a picture of commuting times. It has always been a good example of how collaborative data can work.

This week Google announced the service will share its information with a handful of transit agencies and councils to improve their knowledge of the traffic choke points in their cities.

In return the agencies will give their transit information to Waze.

Waze’s story is a good example of how sensors and people, in this case smartphones and their users, are going to gather information on infrastructure and cities. The key is going to be in making sure that data isn’t locked into proprietory databases.

ABC Nightlife with Tony Delroy October 2014

Connected cars, smart watches, Microsoft and social media will be the topics of the next Nightlife with Tony Delroy technology segment.

Paul Wallbank joins Tony Delroy on ABC Nightlife across Australia from 10pm Australian Eastern time on Thursday, October 2 to discuss how technology affects your business and life.

Update: If you missed the program you can listen to the podcast at the ABC site or stream it below.


For this month’s spot we’re looking at smartcars, smartwatches, the next version of Microsoft Windows and whether the new social media platform Ello can displace Facebook.

Some of the questions we’ll cover include;

  • What’s happening with connected car technologies?
  • Isn’t all this talk about smart cars another way ?
  • So how far are we off the driverless car?
  • Are our mobile phone choices going to dictate what brand car we buy?
  • How does the smart watch fit into how companies are trying to lock us into their software?
  • A new social media platform called Ello is taking off,  what is it?
  • Do we really need another social media platform?
  • Microsoft have announced Windows 10, aren’t we only up to eight?
  • What’s different in Windows 10?
  • Has Windows 8 been a success?
  • When will Windows 10 be released on the market?

 

Join us

We’d love to hear your views so join the conversation with your on-air questions, ideas or comments; phone in on the night on 1300 800 222 within Australia or +61 2 8333 1000 from outside Australia.

Tune in on your local ABC radio station from 10pm Eastern Summer time or listen online at www.abc.net.au/nightlife.

You can SMS Nightlife’s talkback on 19922702, or through twitter to @paulwallbank using the #abcnightlife hashtag or visit the Nightlife Facebook page.

Building inclusive cities

Barcelona’s success in the 2014 Bloomberg Mayors’ Challenge show the human side of smart cities.

Yesterday Barcelona won the 2014 Bloomberg Mayors’ Challenge — a ideas competition for European cities.

Barcelona’s winning idea was collaborative care networks for older citizens. In Barcelona’s case one in five residents is over 65 and by 2o40 seniors will make up a quarter of the city’s population.

The approach Barcelona’s council has proposed is a combination of high tech and the community working together.

Barcelona will use digital and low-tech strategies to create a network of family members, friends, neighbors, social workers, and volunteers who together make up a “trust network” for each at-risk elderly resident.

Last year I had the opportunity to interview the Deputy Mayor of Barcelona, Antoni Vives, on how the city was using the internet of things to improve citizens’ lives.

In that interview Vives spoke on how important was that these technologies improved the lives of all citizens, not just the young and the rich. Today’s prize illustrates how the city is applying that philosophy.

For technologists, one of the tasks ahead is to show how today’s inventions are more than the toys of rich men, but are things that genuinely improve society’s well being.

 

Apple and the long game

Apple’s real game is in controlling a large part of the payments industry and the internet of things. The iPhone6 and watch are key steps in that strategy.

As expected, Apple announced their new range of iPhones and a smart watch today with many digital trees being felled as the tech media falls over to describe all the shiny features of the new devices.

Buried in Apple’s announcements though are the company’s real long game in payments and the Internet of Things.

For the IoT, the various ‘kits’ Apple have announced in the last year — HomeKit, HealthKit and now CloudKit — are the serious plays in this space as they bundle together programs, devices and data streams across health and smarthome applications.

CloudKit moves Apple onto another level as it makes it easier for developers to build back end applications that tie into smart devices; even if someone isn’t using Apple equipment they still may find themselves firmly in the walled garden of Cuptertino.

The long awaiting release of Apple Pay leverages iTunes’ strength as a payment platform, bundling a secure chip into the new iPhone adds to the company’s pitch of being a trusted partner to merchants and payments processors.

What today’s announcements of new hardware, software and APIs indicate is Apple’s shoring up the perimeters of its walled garden.

For it’s competitors, this raises the ante; Google Wallet has nothing like the market penetration or customer acceptance that iTunes has and earlier this week Amazon effectively admitted the Fire smartphone has been a failure by slashing prices. Facebook has made promising noises about payments but still remains locked in an advertising driven business model.

While there’s no doubt the new iPhone will be a success, although the jury is out on the smart watch, Apple’s real game is in controlling a large part of the payments industry and the internet of things. Today’s announcements are a key step in that strategy.

Skipping the trough of disillusionment

Will the IoT have a smooth transition from the top of Gartner’s hype cycle to general acceptance?

When consulting group Gartner placed the Internet of Things at the peak of their hype cycle last month it raised concerns that the technologies might be about to take a tumble.

Speaking to Networked Globe this week in San Jose; Maciej Kranz, VP and GM of Cisco’s technology group described how he believes the IoT’s evolution from the top of the hype cycle to the plateau of acceptance will be quick.

“We’re happy that Gartner put IoT on top as it means there’s awareness,” said Kranz. “We hope to prove Gartner wrong, that in IoT we don’t go through the classic hype cycle we go from hype into reality.”

Kranz’s reasoning while the IoT will suffer a short spell, if any at all, in Gartner’s ‘trough of disillusionment’ is because the major industry players are working closely together to build the sector and its standards.

“Where we think it’s a little bit different from some of the other hype cycles than some of the other hype cycles is that we continue to work very close at the industry,” Kranz explained.

“Because we’re all working as an industry to make it real it will go through the disillusionment and quickly into a reality.”

This may well turn out to be true if the big players like Cisco and GE in the industrial space along with companies such as Google and Apple in the consumer sector stay committed to the concept. If the major vendors stay the course, then it’s likely IoT technologies won’t suffer much at all.

Another aspect in the IoT’s favour is that it isn’t really a specific technology or product at all, instead being more of a concept bought about by various technologies such as home automation, industrial controls and cloud computing all reaching maturity.

Rather than one separate item on the Gartner hype cycle, the IoT is really made of dozens of different technologies that are mostly on the ‘plateau of acceptance’ themselves.

Kranz sees Gartner’s listing of the company as being on the top of the hype cycle as being a vindication for how the IoT has been adopted by industry and the community, “it is remarkable how we’ve gone in the last nine months from people saying it’s a vision to n

Apple’s security challenge

As Apple move into the internet of things, they are going to have to take cloud security more seriously.

This week’s news about celebrities’ personal photos being stolen from their iCloud accounts would be irritating Apple ahead of their September 9 media event.

Unfortunately for Apple they seemed to have walked into this by making things convenient for users rather than enforcing strong security measures.

As Arik Hesseldahl in Re/Code describes, this breach was probably due to Apple not encouraging two factor authentication and not limiting the number of password guesses.

The latter is particularly irritating as it shouldn’t be hard for a system to pick when a brute force attack — a computer guessing a password millions of times a second — is being staged against a user.

It’s also trivial to limit the number of guesses as most other services do.

For users, the best protection is to have complex passwords which reduces the effectiveness of brute force attacks. It’s also worthwhile being careful with your personal nudie photos.

The consequences of having your iCloud account compromised are more than just losing your embarrassing photos, Wired’s Mat Honan had his entire digital life hijacked through this method two years ago.

With Apple aspiring to control the smarthome and smartcar markets, the consequences of accounts being breached becomes exponentially greater. These are issues Apple and the rest of the internet of things industry need to take seriously.

Hopefully at Apple’s big media event next week, some brave journalist will stand out of the assembled masses of sycophant hacks and ask CEO Tim Cook some hard questions about security on the shiny new iDevices.

Can the Internet of Things survive a tumble?

Can the Internet of Things survive a fall into the trough of disillusionment?

That the Internet of Things is posed to fall into the depths of the trough of disillusionment according to Gartner’s latest Hype Cycle should come as no surprise to those following the industry.

For the industry, such a fall might not be a bad thing. During the upswing to the Peak of Heightened Expectations technologies attract the hot, dumb money along with the motley collection of shysters and opportunists a gold rush always lured in by the prospect of easy returns.

When a product, technology or industry falls into what Gartner calls the trough of disillusionment it’s usually the time when its real value is discovered. Without the distractions of hype or dumb money distorting the market, the industry finds a way of using a product that’s become somewhat passe.

For the Internet of Things, it won’t be a bad thing if the sector tumbles into the abyss. The sooner it happens, the faster industry will figure out where the real value and benefits lie.

The only damage might be to some of the more prominent boosters’ egos and the hip pockets of some of the more over eager investors.

Stages of hype – the Gartner Hype Cycle turns twenty

The Gartner Hype Cycle turns twenty

Gartner’s Hype Cycle has been a favourite of this blog as it’s been pretty accurate at describing where various technologies are in the tech media’s eye.

This year is the twentieth edition and the most notable aspect is the Internet of Things is shown as being right on the peak of industry hype.

Other sectors struggling on the cycle are cloud computing, big data and machine-to-machine technologies; all of them are tumbling into the trough of disillusionment.

gartner-hype-cycle-2014

In itself this isn’t a bad thing for these technologies as the ‘trough of disillusionment’ is where the true business cases are found, certainly for the Internet of Things this will not be bad for a sector that’s clearly overhyped.

There’s also the thought that not all troughs of disillusionment are the same as some concepts – such as Big Data – are actually trends which means they aren’t subject to the whims of corporate marketing departments.

How the hype cycle will look in five years will be fascinating as things like brain-computing interfaces and the quantified self start to take form. When they reach the peak of the hype cycle we can expect many of today’s disillusioned technologies will be on the plateau of productivity.

Reaching Peak Tablet as iPad and android sales begin to plateau

An electrical retailer’s financial results might mark turning points in two different economies.

Today Australian electronics retailer JB Hi Fi released its annual results. They confirm what’s been becoming apparent over the last year that tablet computer sales seem to have peaked.

A plateauing of tablet sales is bad news for retailers like JB whose stock price fell by 8% on the news.

It’s not surprising that tablet computer sales have peaked as the growth had been spectacular and, unlike PCs of a decade ago, there isn’t an obvious five year replacement cycle.

That the old PC industry business model doesn’t apply to tablets is why Apple is focusing on other revenue sources like the App Store and internet of things plays such as HomeKit and HealthKit.

Once again, the industry leaders are finding they have to pivot to stay up with a rapidly evolving market.

The other notable point from JB’s management was that Australian consumer confidence is tanking, which might indicate the economy is entering its first recession in twenty years.

If it is true that the Aussie economy is entering a recession, then it might be time for the adults to take charge in a very immature government. Some of the Liberal Party’s pampered princelings may have to start earning their salaries soon.

Hacks on a plane

That avionic systems could be vulnerable to hacking is a wake up call for the internet of things industry.

One of the great concerns about the internet of things is what happens when older computer technology that was never designed to be connected to the net is exposed to the online world.

A presentation to the Black Hat Conference in Las Vegas this Thursday by researcher Ruben Santamarta promises to show some of the vulnerabilities in aircraft avionic systems.

Today’s aircraft are extremely smart devices with the downsides shown in the tragedy of AF447 where an Air France jet plunged into the Atlantic Ocean when two undertrained pilots didn’t understand what their plane was doing as it encountered severe ice conditions in a storm.

With aircrew increasingly dependent upon computers to help them fly planes, the risks of bugs or security weaknesses in aircraft systems is a serious issue and with the continued mystery of MH370’s fate adds an element of speculation that a glitch of some form was responsible for its disappearance.

It wouldn’t be the first time a passenger plane came to grief because of a computer error; most notably Air New Zealand flight 901 crashed into Antarctica’s Mount Erebus during a 1979 sightseeing trip due to wrong information being loaded into the navigation system.

The internet adds numerous risk factors to aircraft – Santamarta’s hack allegedly works through in plane WiFi systems – particularly given these avionics systems haven’t been designed to deal with unauthorised access into their networks.

Should Santamarta’s demonstration prove feasible, it will be an important warning to the aviation industry and the broader Internet of Things community that security is a pressing issue in a world where critical equipment is connected.

Blackberry’s quest for its future

BlackBerry stakes its future on increased enterprise security concerns and the internet of things

This is the unedited, submitted version of ‘is BlackBerry ripe for a comeback‘ that appeared in Technology Spectator on 30 July, 2014.

“What do we well?” is the question Blackberry CEO John Chen asked when he took the reigns of the Canadian communication company last November.

Chen was speaking on Tuesday at Blackberry’s Security Summit in New York where he and his executive team laid out the company’s roadmap back to profitability.

Since the arrival of the iPhone and Android smartphones, times have been tough for the once iconic business phone vendor as enterprise users deserted Blackberry’s handsets and the company struggled to find a new direction under former CEO Thorsten Heins.

Back to BlackBerry’s secure roots

In Chen’s view, the company’s future lies in its roots of providing secure communications for large organisations, “It became obvious to us that security, productivity and collaboration have to be it.”

“This is not to say we are not interested in the consumer, but we have to anchor ourselves around the enterprise.” Chen said in a clear move distancing himself from his predecessor and products like the ill fated Blackberry Playbook

An early step in this process of focusing on enterprise security concerns is the acquisition of German voice security company Secusmart which was the cornerstone of Chen’s New York keynote.

Blackberry’s acquisition of the company is a logical move says the CEO of Secusmart, Dr Hans-Christoph Quelle, who points out the two organisations have been working closely together for several years.

“It fits perfectly,” says Quelle. “We are not strangers having worked together since 2009,” in describing how Secusmart technology has been increasingly incorporated into Blackberry’s devices.

Secusmart’s key selling point has been its adoption by NATO and European government agencies; the Snowden revelations on the US bugging of Angela Merkel coupled with the Russian FSB leaking intercepted US state department conversations along with the release of Ukrainian separatist conversations after the shooting down of MH17 has focused the European view on the security of voice communications.

Launching new services

Along with the acquisition of Secusmart, Blackberry will also be launching an new enterprise service in November, the new Passport handset in December along with a range of security applications including BlackBerry Guardian, a new service that will scan Android apps for malicious software.

Blackberry’s executives were at pains to emphasise their products aren’t focused on any single smartphone operating system and not dependent on customers buying their smartphones although to get the maximum security benefits.

“We will provide the best level of security possible to as many target devices out there as possible,” said Dan Dodge who heads Blackberry’s QNX embedded devices division.

Longer term plans

In the longer term, Blackberry sees QNX division as being one of the major drivers of future revenues as the Internet of Things is rolled out across industries.

QNX was acquired by Blackberry in 2010 to broadband the communication company’s product range, now it is one of the pillars of the organisation’s future as Chen and his team see that connected devices will need secure and reliable software.

Dodge says: “With the internet of things, you can have devices that can change your world.”

While QNX is best known for its smartcar operating system – it underpins Apple’s CarPlay system being rolled out for BMW as well as its own system deployed in Audis – the company’s products are used for industrial applications ranging from wind turbines to manufacturing plants.

Despite Blackberry’s announcements in New York, the company still facing challenges in the marketplace with the Ford Motor Company announcing earlier this week it will drop the Blackberry for its employees by the end of the year and replace them with iPhones.

Chen’s though is dismissive about Apple’s and IBM’s moves into Blackberry’s enterprise markets, “what we do and what they do is completely different.”

Focusing BlackBerry

The focus for Chen is to differentiate Blackberry and play on its strengths, particularly the four markets it calls ‘regulated industries’ – government, health care, financial and energy that the company claims makes up half of enterprise IT spending.

Whether this is enough to bring Blackberry back on track remains to be seen but Chen says this is where he sees the company’s future, “This is why we are so focused on enterprise and so focused on these pillars.”

For Blackberry, the emphasis on enterprise communications is a step back to the profitable past. It may well be successful as businesses become more security conscious in a post-Snowden world.

Paul travelled to the Blackberry Security Summit in New York as a guest of the company.