Tag: research

Dealing with the biggest of data

The CERN research project generates huge amounts of data however the human touch is needed to analyse and manage the information

How do you deal with the biggest data sets of all? Bob Jones, a project leader for the European Organization for Nuclear Research – commonly known as CERN – described how the world’s largest particle physics laboratory manages 100 petabytes of data.

The first step is not to collect everything, ““We can’t keep all the data, the key is knowing what to keep” says Jones. This is understandable given the cameras capturing the collisions have 150 million sensors delivering data at 40 million times per second.

Jones was speaking at the ADMA Global Conference’s Advancing Analytics stream where he was describing how the project manages and analyses the vast amounts of data generated by the huge projects.

Adding to Jones’ task and that facing CERN’s boffins is that data has to be preserved and verifiable so scientists can review the results of experiments.

Discovering the Higgs Boson for instance required finding 400 positive results out of 600,000,000,000,000,000 events. This requires massive processing and storage power.

Part of the solution is to have a chain of data centres across the world to carry out both the analytics and data storage supplemented by tape archiving, something that creates other issues..

“Tape is a magnetic medium which means it deteriorates over time.” Jones says, “we have to repack this data every two years.”

Another advantage with a two year refresh is this allows CERN to apply the latest advances in data storage to pack more data into the medium.

CERN itself is funded by its 21 member states – Pakistan is its latest member – which contribute its $1.5 billion annual budget and the organisation provides data and processing power to other multinational projects like the European Space Agency and to private sector partners.

For the private sector, CERNs computing power gives the opportunity to do in depth analytics of large data sets while the unique hardware and software requirements mean the project is a proving ground for high performance equipment.

Despite the high tech, Jones says the real smarts behind CERN and the large Hadron Collider lie in the people. “All of the people analysing the data are trained physicists with detailed, multi year domain knowledge.”

“The reason being is the experiment and the technology changes so quickly, it’s not written down. It’s in the heads of those people.”

In some respects this is comforting for those of us worrying about the machines taking over.

Similar posts:

  • No Related Posts

Daily links – the future of Google, Silicon Valley’s name and how startups die

The future of Goodle,,how the name ‘Silicon Valley’ came about, why solar power is getting cheaper and how some startups die.

On many measures Google are in trouble, but one analyst thinks we’re panicking and his view is the lead of today’s links of the day. We also look at how the name ‘Silicon Valley’ came about, why solar power is getting cheaper and how some startups die.

Does Google’s future lie in R&D?

“Google is down but it’s not out” is the warning of this analyst’s report on the company’s earnings and strategy. Interestingly Google outspends Apple by $4bn a year on research and development, but both of them are dwarfed by Microsoft’s spending, which indicates R&D investment doesn’t guarantee success.

The origins of the name ‘Silicon Valley’

Last Sunday marked the 44th anniversary of the first time the label ‘Silicon Valley’ appeared in print. The US Computer History Museum looks at how the name came about and no-one will be surprised it was a marketing person who coined it.

Why does solar power keep getting cheaper

A few years ago putting solar cells on a building was expensive, now in many parts of the world the price of PV panels is becoming competitive with mains power. Vox Magazine looks at the factors driving the price drops and finds that economies of scale are now the main factor affecting the falling cost of installed solar power systems.

RIP Urbanspoon

One of the earliest food review platforms was Urbanspoon which was founded on the basis it would only grow as a bootstrapped company. In 2009 the founders sold out to a larger company who have now sold it onto an Indian business who is going to shut the name down.

Startups who’ve fallen off the map

Business Insider lists 17 formerly hot businesses who’ve fallen out of the public view this year, while some of them haven’t disappeared, it’s a list that reminds us that most new businesses, particularly tech startups, fail.

Similar posts:

  • No Related Posts

Links of the day – redesigning the car and South China Mall.

Interesting links include Mercedes’ vision of a driverless car, an analysis of the ill fated South China Mall’s flaws and how Amazon is reorganising its R&D efforts after the failure of the Amazon Fire.

The CES extravaganza continues in Las Vegas with a wave of announcement, most of which I’m ignoring, however the motor industry continues to show off new developments with Mercedes displaying their vision of how a driverless car will look.

Other interesting links today include an analysis of the ill fated South China Mall’s flaws and how Amazon is reorganising its R&D efforts after the failure of the Amazon Fire.

Mercedes redesigns the car

A little while back I suggested that we could do better in redesigning the driverless car – Mercedes have gone ahead and done it.

Mercedes’ redesign of the driverless car indicates just what can be done when we rethink what passengers will need in the vehicles of the future.

Ford recalls a vehicle for a UI upgrade

Ford has recalled its Lincoln MKC SUV models for a software upgrade after discovering drivers were shutting down the cars by accident.

What’s notable with this story is how software changes are now one of the main reasons for recalling vehicles and how design flaws in an automobile’s computer programs are relatively quickly discovered and resolved.

We will probably find in the near future car manufacturers will carry out the upgrades remotely rather than ask owners to bring their vehicles into dealerships.

A long running security flaw is exposed

In August 2013 a security researcher warned UK online greeting card vendor Moonpig that its system exposed up to six million users’ account and financial details. Until Monday the company had ignored him. This is a tale of classic management disregard for customer security and one area where business culture needs to dramatically change.

Rumours of an AOL – Verizon merger

It’s a speculative story but if a merger between US telco Verizon and former internet giant AOL goes ahead it may mark another wave of telcos moving into content services, although it’s hard not to think that Verizon could spend its money more wisely.

After a flop, Amazon restructures its R&D

The Amazon Fire was by all measures a miserable flop as a smartphone however it seems the company learned some important lessons from the device’s market failures. Instead of abandoning its research efforts, the online behemoth is increasing it’s R&D budget and reorganising its development division.

Design fails of the South China Mall

South China Mall just south of Guangzhou has been the poster child of Chinese malinvestment during the nation’s current boom. In a blog post from 2011, a shopping mall expert visits the development and points out the major design faults in the complex which may well have doomed the project from the beginning.

Similar posts:

  • No Related Posts

David Cameron and the Internet of Things

Britain’s Prime Minister backs the nation’s move into the Internet of Things

Last year I interviewed the CEO of London and Partners, Gordon Innes, on how Britain’s capital is making a bid to become Europe’s Silicon Valley.

At the opening of CeBIT last night, UK Prime Minister David Cameron increased the country’s bid with a plan on building Britain’s capability in the digital industries.

Cameron portrayed the moves as being a partnership with Germany. This may be partly because he was being gracious towards his host and also because the Brits might not see Germany as being a competitor in these fields.

The fields that Cameron highlighted are deploying 5G networks, more efficient use of spectrum and increasing research into the Internet of Things.

A research boost is a notable as it may give the Brits a foothold in an area that’s evolving rapidly as the Internet of Things raises a whole range of security, privacy and governance issues.

While there’s still a sniff of Harold Wilson’s 1963 White Heat of Technology speech in the Cameron government’s policies, at least the British government is articulating policies for the 21st Century.

It may well be that Cameron’s digital revolution will be no more successful than Wilson’s technological revolution fifty years ago, but at least it will be a brave attempt.

Similar posts:

  • No Related Posts

“He looks like a geek”

The media scrum around alleged Bitcoin founder Dorian Nakamoto is based on some flimsy thinking

The unseemly media scrum around alleged Bitcoin inventor Dorian Nakamoto has not been the press’ finest hour.

What’s more worrying though is a Business Insider interview with Sharon Sargent a ‘forensics analyst’ who was part of the Newsweek investigative team.

A systems engineer by training with experience in computing security, military protocol analysis, and artificial intelligence, Sergeant said everything she found converged on an individual with a background apparently similar to hers — and who ended up sharing a name with Bitcoin’s creator.

“I said, ‘I think I know this guy — he wears a pocket protector, he has a slide rule, he comes from that genre,’ which was very different from other characterizations,” she told BI by phone Friday.

He wears a pocket protector and uses a slide rule? Hell yeah, not only did he create Bitcoin but he’s probably a witch as well.

One hopes Newsweek have found the right man.

Picture courtesy of forwardcom through sxc.hu

Similar posts:

  • No Related Posts

Mortein and The Queen

The story of Queen Elizabeth II and fly spray tells us much about modern scientific research.

A great little story from the Australian government’s research arm, the CSIRO, tells the story of how the Queen Elizabeth II lead the commercial insect repellent industry and how intellectual property has changed.

The story tells how the original experiments were carried out in 1940 to see what substances were best in repelling mosquitos as part of the preparation for a tropical war against Japan.

After the war, research continued and during the 1963 Royal Tour of Australia, the Queen was sprayed with the government repellant to keep flies off her while she played golf.

Journalists following the Queen noted the absence of flies around the official party, and word about CSIRO’s new fly-repellent spread. A few days later representatives from the company making Mortein insecticides called Doug Waterhouse for his formula, which he passed on freely, as was CSIRO’s policy at the time and the rest, as they say, is history.

It’s unthinkable today that any research organisation would give intellectual property away and a modern agreement would include hansom royalties for the formula.

There’s an argument that giving away the intellectual property helped innovation and public health, but in these stingy and cash strapped days it’s hard to see how government scientific organisations could survive without royalty payments.

It certainly is true that the past is a different country.

Fly spray can courtesy of Wikipedia

Similar posts:

  • No Related Posts

Finding the perfect customer

Combining old techniques with big data technologies and social media monitoring open new opportunities for businesses to learn more about their customers.

With the rise of social media we’ve spoken a lot about customers’ ability to rate businesses and overlooked that companies have been rating their clients for a lot longer. The same technologies that are helping consumers are also assisting companies to find their best prospects.

A business truism is that Pareto’s Rule applies in all organisations – 20% of customers will generate 80% of a company’s profits. Equally a different 20% of clients will create 80% of the hassles. The Holy Grail in customer service is to identify both groups as early as possible in the sales cycle.

Earlier this week The New York Times profiled the new breed of ratings tools known as consumer valuation or buying-power scores. These promise to help businesses find the good customers early.

While rating customers according to their credit worthiness has been common for decades, measuring a client’s likely value to a business hasn’t been so widespread and most companies have relied on the gut feeling of their salespeople or managers. The customer valuation tools change this.

One of the companies the NYT looked at was eBureau, a Minnesota-based company that analyses customers’ likely behaviour. eBureau’s founder Gordy Meyer tells how 30 years ago he worked for Fingerhut, a mailorder catalogue company that used some basic ways of figuring out who would be a good customer.

Some of the indicators Fingerhut used to figure if a client was worthwhile included whether an application form was filled in by pen, if the customer had a working telephone number or if the buyer used their middle initial – apparently the latter indicates someone is a good credit risk.

Many businesses are still using measures like that to decide whether a customer will be a pain or a gain. One reliable signal is those that complain about previous companies they’ve dealt with; it’s a sure-fire indicator they’ll complain about you as well.

What we’re seeing with services like eBureau is the bringing together of Big Data and cloud computing. A generation ago even if we could have collected the data these services collate, there was no way we could process the information to make any sense to our business.

Today we have these services at our fingertips and coupled with lead generators and the insights social media gives us into the likes and dislikes of our customers these tools suddenly become very powerful.

While we’ll never get rid of bad customers – credit rating services didn’t mean the end of bad debts – customer valuation tools are another example of how canny users of technology can get an advantage over their competitors along with saving time in chasing the wrong clients.

Similar posts: