Too much data, we’re only human after all

• More data does not equal more insights
• Natural clusters and groupings the next BDA trend

 
MOST people think that the more data you have, the more and better insights you get. And that means the more data you gather, the more valuable it is, right?
 
Unfortunately, having too much data will only decrease its value, according to Mike Potter, senior vice president of global research and development at Radnor, Pennsylvania-based Qlik Technologies.
 
“The amount of data that’s growing is far exceeding our ability to understand it, and importantly, the value of that data is decreasing,” he says in a recent conversation with Digital News Asia (DNA) in Singapore.
 
This is contrary to what most companies have been hearing from vendors and analysts alike, where it seems like there is no such thing as too much data.
 
But there’s only so much a poor human being can take.
 
“It all has to do with the ability to process data – not just the technological ability to process data, but the human ability to process data,” says Potter.
 
“What happens is that there is a higher degree of noise, there is a lot of duplication, and the value of data gets lost in the granularity of it all,” he adds.
 
Mining for insights
 
Even systems that generate value out of data often take it from derivative sets and not the raw data itself, according to Potter (pic).
 
“Most systems that are successful in generating value are interpreting the data and then using the interpreted data for analytics,” he says.
 
“It is not just simply traditional forms of aggregation, it is also rules, it is predictive, it is being able to run it through machine learning technologies.
 
“The idea is that the base data is not what you analyse, but the derivative data,” he adds.
 
Another issue is that the cost of storage has come down so much that it has led to data hoarding.
 
“We are in the hoarder stage – we know how to store data, but we don’t know what to do with it,” says Potter.
 
“CIOs (chief information officers) often tell me their data collection strategy, and when I ask what they plan to do with it, too often I’m told, ‘I don’t know yet, but when somebody figures it out we’ll have it’,” he adds.
 
But the reality is that organisations need to start understanding what questions need to be asked, and making sure they have the data to answer them.
 
“Most data strategies are bottom-up – collect the data then look for the insight,” says Potter.
 
“More advanced companies are able to understand what they are looking for, then go out and make sure their data strategy matches it,” he adds.
 
From silos to clusters
 
While companies work out their data strategies, analytics is starting to extend beyond silos – into connected devices, and into the network.
 
“What you are seeing more and more with the Internet of Things (IoT) is that everything is going to be connected, and it’s not just individual devices connected, but they are going to connect in natural clusters and groupings,” says Potter.
 
“It’s happening today with the watch and the phone, and I think that trend is going to continue.
 
“What’s going to happen is that automation is going to connect across the supply chain, and connect across operations,” he adds.
 
These natural clusters and self-forming networks are the next trends to watch out for, Potter argues, using crowd-sourced navigation app Waze as an example.
 
“Waze is a great integration of geo-spatial and mapping data, but it is also a great integration of social data and the ability to understand from your peer drivers what is going on, on the roads.
 
“I think that trend is the one to watch – self-forming networks of information that are beginning to integrate and collaborate into a larger purpose, such as traffic management or navigation,” he adds.
 
Related Stories:
 
The IoT, and how not to hoard your data
 
It’s about data literacy, says Qlik exec
 
Making Asia’s embedded analytics and BI space ‘Qlik’
 
 
For more technology news and the latest updates, follow us on Twitter, LinkedIn or Like us on Facebook.