Big data blues? It's all about the 'so what' question

• Not enough focus on asking the  ‘So what?’ question in big data
• Develop a workable business model before focusing on tech

FOR some time now, the technology world has been expounding the virtues of big data.

When the term was first used, many hailed it as the next big (no pun intended) thing in the evolution of the structured database – the convenient classification of data into rows and columns – which has been dominating the business world since the personal computer was invented.

Companies had been using structured databases for decades in the form of mainframe computers. Later behemoths like Oracle Corp, whose founder Larry Ellison, further exploited structured data by building his company around relational databases that are aimed at serving large organisations such as banks, governments and the like by the late 1970s.

Fast forward three decades and unstructured data now reigns. Thanks to the rise of user-generated content and social media in the form of video, text, and pictures, information comes from disparate sources that cannot be neatly fitted into rows and columns – hence the term unstructured data.

Vendors and analysts speak of the five ‘Vs’ of big data: Volume, variety, veracity, velocity, and value. Indeed the industry in general has begun expounding at length what these five Vs mean to them and how enterprises not only need to understand these terms but master them too.

The goal? To gain first mover advantage they need to outfox the competition and stay ahead of the game.

For clarity, big data refers to the massive volume of both structured and unstructured data (non row- or column-based data sets) that is so large and moves so fast that it’s difficult to process using traditional database and software techniques as well as current processing capacity.

In 2013, Digital News Asia (DNA) reported that while big data was  still relatively nascent at the time, there was confusion over what it was and what it actually meant to companies grappling with it, according to a study conducted by Forrester Research.

The report noted that most Asia Pacific organisations were still trying to come to grips with what big data was – and whether it was truly different from traditional business intelligence (BI), analytics and data warehousing approaches. 

Another pertinent point raised by the  study was that organisations have been approaching big data from a tactical point of view rather than from a strategic perspective.

So where are we after three years? How fares the industry with regards to big data? Are we any wiser in dealing with big data? Is the industry approaching the issue of big data properly? What are the impediments that slow its adoption among enterprises?

These are but some of the questions that DNA put to Lutz Finger (pic, above), data scientist at Cornell University, who visited Malaysia recently during Malaysia’s Big Data Week Asia 2016.

Get first principles right first

According to Finger, men have always used data for actionable intelligence. For example, men such as the Incas were already using a rudimentary form of data observation from the sun to help them determine when it was time to plan their crops, he explains.

The right question to ask in big data isn’t whether men can derive better information for better intelligence but rather if men are asking the right questions about what they want to achieve with the data they collect, Finger contends.

“I think what has changed in recent years, is we digitise more, so it’s easier to get to data, but that doesn’t mean that it’s right or useful data.

“We have the technology to work and analyse the data but the technology and data is not an end in themselves,” he argues. “Analytics hasn’t changed that much. Linear regression is 200-years-old. So what we want to do [with the data] has not changed."

Finger, also the director of data science and data engineering for enterprise social media giant LinkedIn Corp, firmly believes that before any organisation gets into big data, they must have a clear idea of what they want to do with the data.

Finger says what has changed is the depth of data because men have more data and can look at more things. Men have more technology so it’s not as burdensome as before to look at data, he added.

“But so what? There are a lot of people who try to sell technology which you don’t really need. For example, if you need to commute, you may need a car. But to commute on the road, you don’t need a Lamborghini. You may only need a bike,” he argues.

“[Basically] you need tools that are commensurate to the task. If a company does not know what it needs, it can’t possibly know what tools it needs to achieve its tasks.”

These are some questions that Finger challenges the industry as whole to think about before embarking on big data, and are the core issues that he explores in the book he co-authored and published in 2014 called Ask, Learn, Measure. 

“The crux for any company or organisation is to ask the ‘so what?’ question,” he argues. “Companies may not need big data. What they want is to make money, a better business model, to improve processes, reduce costs, be faster than the competition.

“[But] as long as you can’t answer the ‘so what?’ question, you don’t need big data. You need to understand the ‘so what?’ first,” he declares.

In an hour-long candid conversation with DNA, Finger also fielded some other important questions with regards to big data. Below are excerpts from the interview:

DNA: In your opinion, what is the biggest pain in big data today?

Finger: The biggest pain is actually human beings. One can know this from our own experience. For example, you can have LinkedIn, Twitter and Facebook contacts, as well as your own contact repository and even a collection of physical cards. But if you want to search for someone, you will not find it. This is your own personal big data problem. Data can help us to steer, it can help us to drive change in society but the biggest threat is humans misunderstanding how to use data. That’s the reason why I teach at the university.

DNA: We’ve always had data. What has changed between then and now? How are men using data differently?

Finger: We have always had data and the earliest civilisation that had data was the Incas. I think what has changed in recent years is that we digitise more, so it’s easier to get to data, but that doesn’t mean that it’s right or useful data. The other thing to note is that we have the technology to work and analyse the data but the technology and data are not an end in themselves. Analytics haven’t changed that much and what we want to do [with the data] has not changed. What has changed is the depth of data because when we have more data, we can look at more things.

DNA: But what about the early business intelligence systems? How has that worked out and where have there been failures? What lessons can we learn from the past?

Finger: We have more technology so it’s not as burdensome as before to look at data. In the 1980s, a company could invest millions on a business intelligence system and consultants to analyse things in different departments and ask them what questions they wanted to answer. But by the time these questions are answered, it’s too late as the answers are no longer needed. This is why business intelligence systems of the past failed. There was a disconnect between when the questions were asked and when the answers were needed [provided].

Today however, we have the power and tools to shorten the time needed to understand data. We can use technology to give an answer and the more data we have, the better we can solve problems. But still, you must begin by asking the right questions.

Next page: Hype vs. Reality