Author: Paul Wallbank

  • Democratising Big Data

    Democratising Big Data

    Common Crawl is a not-for-profit web crawler service that makes the data collected open for all to use. A post on the MIT Technology Review blog speculates how the initiative might spawn the next Google.

    One of the problems with Big Data is that it’s held mainly by large corporations and government agencies, both of which have the tendency to keep their data private on that basis that information is power and power means money.

    We see this in the business models of Facebook, Google and many of Silicon Valley’s startups; the information garnered about users is as, if not more so, valuable as an utility from the product.

    Initiatives like Common Crawl tilt the balance somewhat back towards consumers, citizens, and smaller businesses.

    How well Common Crawl and other similar initiatives fare remains to be seen – Wikileaks was a good example of how such projects can flare out, collapse under the weight of egos or be harrassed by corporatist interests.

    In search, Google are open to disruption as they tweak their results to suit initiatives like Google Plus. During the company’s earnings call earlier this week Larry Page spoke of the challenges of staying focused on the opportunities that matter, it may well be the company is more distracted from its core business than it should be.

    Whether Common Crawl disrupts Google is up to history, it could just as well be a couple of kids called Sergei and Larry with a smart idea.

    The imperative now though is to try and keep as much public data available for everyone to use and not lock it away for the privileged few. That will let the future Googles develop while making our societies more fairer and open.

    Similar posts:

  • Tracking the knowledge graph

    Tracking the knowledge graph

    “Married Men Who Like Prostitutes” is juicy search term and the results can wreck marriages, careers and lives.

    This is one of the Facebook Graph searches UK tech commentator Tom Scott posted on his Actual Searches on Facebook Tumblr site which lists, mercifully anonymised, the results.

    What should worry anybody who uses Facebook is that this data has been in the system all along, advertisers for instance have been able to target their marketing based on exactly this information, Graph Search just makes it quicker and easier to access. This is why you should be careful of what you like and who you friend online.

    Tom Scott has a terrific Ignite London presentation which looks at just how vulnerable an individual is by over sharing online. In I know what you did five minutes ago, Tom finds an individual, discovers his mother’s maiden name and phone number all within two minutes.

    Facebook isn’t the only service we should be careful of, it just happens to be the one we overshare data with the most. When you start stitching together social media services with government and corporate databases then a pretty comprehensive picture can be made of a person’s likes and preferences.

    The best we can hope for in such a society is that picture is accurate, fair and doesn’t cast us in too unfavourable a light.

    In same cases though that data can be dangerous, if not fatal.

    As potential employers, spouses and the media can easily access this information, it might be worthwhile unliking obnoxious, racist and downright stupid stuff. There’s a very good chance you’ll be asked about them.

    Similar posts:

  • PayPal struggles with the Soviet customer service model

    PayPal struggles with the Soviet customer service model

    CNN reports that internet payment giant PayPal is looking at an “aggressive changes” to its fraud detection systems which see thousands of customers accounts frozen every year.

    PayPal’s announcement follows last year’s promise by CEO David Marcus to institute a “culture change” at the company,

    Our intention has always been to protect our customers. Not to mess around with our merchants.
    I want to share two things with all of you:

    #1 — there’s a massive culture change happening at PayPal right now. If we suck at something, we now face it, and we do something about it.

    #2 — you have my commitment to make this company GREAT again. We’re reinventing how we work, our products, our platforms, our APIs, and our policies. This WILL change, and we won’t rest until you all see it. The first installments are due very soon. So stay tuned…

    Screwing around merchants and buyers has become synonymous with PayPal and their parent company eBay who together are the poster children for the Silicon Valley Soviet Customer Service Model.

    Reader comments to the CNN article cited at the beginning of this post give a taste of just how bad the problem is at PayPal.

    Once your business attracts the attention of PayPal’s algorithms, you’re locked into a Kafkaesque maze of dead ends and arbitrary, made up rules.

    To be fair to PayPal and eBay this problem isn’t just theirs, it’s shared by Google, Amazon and almost every major online company. Their view of customer service is to shoot first and ask no questions, they certainly won’t answer anything from their victim beyond a trite passive-aggressive corporate statement.

    Part of the current Silicon Valley mania around web and app based services is that, along with providing free content, users will provide support for each other and that customer service is an unnecessary overhead which should be kept to a minimum.

    In this respect, many of these new businesses are little different from the legacy airlines, telcos and declining department stores who have spent the last thirty years stripping away customer service with the result of locking them into shrinking commodity markets.

    That failure to value customer service is the biggest weakness for companies like eBay, Amazon and Google. The very forces that favour them, the reduction of the entry barriers, also makes it easier for more customer orientated businesses to grab market share.

    Just as Silicon Valley’s new businesses has challenged a whole range of incumbent operators, they too are at risk from upstarts who value their customers. This is something PayPal’s management can’t afford to forget.

    Similar posts:

  • Why you won’t retire

    Why you won’t retire

    Outliving Our Super is the headline of an Australian Financial Review story on the problems of an aging population.

    Jacqui Hayes cites a billboard in San Francisco declaring that life expectancy will soon be 150 and we have to plan for longer retirements.

    The flaw in this discussion is the idea of retiring in our 60s. When the age pension was introduced in 1910, a new-born boy could expect to live 55 years and a girl, 59 years. The odds were against the average person every receiving the pension which was an effective, if ruthless, way of ensuring the solvency of social security programs.

    A hundred years later, a new born can expect to live well into their eighties. Meaning the average person will spend two decades in retirement.

    Making matters worse is the nature of that Millennial’s work pattern – when great, great grandpa entered the workforce in the 1920s,  he was almost certainly in his early teens and worked a solid fifty years paying his taxes before prospect of retirement arrived.

    Today, that child won’t enter the workforce until at least their late teens and more likely until their early twenties. A modern child is also going to have a much more fragmented work career and will likely have periods of unemployment or low earnings as a casual or contract worker.

    For today’s child to retire at 65 it would mean he or she will have had to saved enough over a forty year working life to sustain them for fifteen years of retirement, those numbers are tough and to achieve it most won’t be living the millionaire lifestyle during their golden years.

    With a life expectancy of 150, the early twentieth century model of retiring at 60 or 65 means today’s child would spend less than 30% of their lives in the workforce. Put simply, the numbers don’t add up.

    The reality is most of us won’t be retiring at 65, the baby boomers reaching retirement age now are learning this and it’s a lesson that’s going to get harder for the Gen X’s and Y’s following them.

    As a society, or an electorate, we can pretend there’s no problem and policy makers and politicians will pander to our refusal to face the truth by keeping structures that reflect early Twentieth Century aspirations rather than Twenty-First Century realities.

    We have to face the reality that the retiring at 65 is unaffordable dream for most of us. Once we accept this, we can get on with building longer lasting careers.

    Picture of pensioners courtesy of andreyutzu on SXC.HU

    Similar posts:

  • 2013 – the year of the incumbents

    2013 – the year of the incumbents

    Bigger, quicker and more congested are the predictions from consulting firm Deloitte’s 2013 Technology, Media and Telecommunications survey.

    In Sydney last Friday, the Australian aspects of the report were discussed by Clare Harding and Stuart Johnston, both partners in Deloitte’s Technology, Media and Telecommunications practice.

    Most of the predictions tie into global trends, with the main exception being the National Broadband network which Stuart sees as addressing some of the bandwidth problems that telecommunication companies are going to struggle with in 2013.

    Technology predictions

    For the technology industry, Deloitte sees 2013 as being a consolidation of existing trends with the trend away from passwords continuing, crowdfunding  growing, conflict over BYOD policies and enterprise social networks finding their niches.

    Some technologies are not dead; Deloitte sees the the PC retaining its place in the home and office, with over 80% of internet traffic and 70% of time still being consumed on desktop and laptop computers.

    Deloitte also sees gesture based interfaces struggling as users stick with the mouse, keyboard and touchscreen.

    Media predictions

    Like 3D TV two years ago, the push from vendors is now onto smart TVs and high definition 4K televisions. As with 3DTV, much of the market share of smart and hard definition TVs is going to be because television manufacturers will include these features in base models.

    Deloitte’s consultants see 2013 as one where “over the top” services (OTT) like Fetch TV and those provided by incumbents delivered start to get traction on smart TVs with 2% of industry revenues coming from these platforms.

    Catch up TV is the main driver of the over the top services with 75% of traffic being around viewers watching previously broadcast content. This will see OTT services firmly become part of the incumbent broadcasters’ suite of services.

    The bad news for some incumbents is the increase in ‘cord cutters’ as consumers move from pay-TV services to internet based content.

    Smartphone and tablet computer adoption which is expected to treble will be a driver of OTT adoption as viewers move to ‘dual screen’ consumption, the connections required to deliver these services will put further load on already strained telco infrastructure which is going to see prices rise as providers respond to shortages.

    Telecommunications predictions

    The telecommunications industry is probably seeing the greatest disruption in 2013. With smartphones dominating the market world wide as price points collapse.

    One of the big product lines pushed at this year’s CES was the “phablet” – while the Deloitte consultants find it interesting hey don’t seem convinced that the bigger form factors will displace the standard 5″ screen size during 2013.

    As a consequence of the smartphone explosion is that apps will become more pervasive and telcos will try and build in their own walled gardens with All You Can App to lock customers onto their services.

    With smartphones moving down market, largely because of the cost benefits for manufacturers, Deloitte also predicts many new users won’t access data plans given they’ll use the devices as sophisticated ‘feature phones’.

    Data usage will continue to grow, particularly with the adoption of LTE/4G networks, although much of the growth will still be on the older 2 and 3G networks as lower income users choose plans which don’t require high speed data.

    The looming data crunch

    There is a cost to booming data usage and that’s the looming shortage of bandwidth, Deloitte sees this as getting far worse before it gets better.

    With bandwidth becoming crowded, prices are expected to rise. In the United States, the “all you can eat” nature of internet plans is being replaced with “pay as you go” while in Australia data plans are becoming stingier and per unit costs are rising.

    The London Olympics were cited as an example of how the shortages are appearing – while the Olympic site itself was fine, outside events like the long distance cycle races strained infrastructure along the route. We can expect this to become common as smartphones push base station capacity.

    Where to in 2013

    Deloitte’s view of where the telecom, technology and media industries are heading in 2013 is that incumbents will take advantage of their market positions as technology runs ahead of available bandwidth.

    In Australia, governments might be disappointed as telcos internationally aren’t interested in bidding huge amounts for bandwidth. As Stuart Johnston says “globally what we’re seeing is that carriers are not as willing to spend. It’s not the cash cow that governments are expecting.”

    For government and consumers, we’re going to get squeezed a little bit harder.

    While things do look slightly better for telcos, broadcasters and other incumbents there’s always the unexpected which eludes all but the most outrageous pundits, it’s hard to see what the disruptive technologies of 2013 will be but we can be sure they are there.

    The main takeaway from the 2013 Deloitte report is that smart TVs, 4K broadcasting, tablet computers and smartphones are going to be the biggest drivers for the technology, media and telecommunications industry for this year. There’s some opportunities for some canny entrepreneurs.

    Similar posts: