It may be everywhere now, but big data matters more than ever.
I was having lunch recently with the editor of a leading business publication, when I was asked, “Don’t you see a backlash against ‘big data’? Aren’t people growing tired of hearing about it?” This seemed a fair question, given the sheer volume of articles and media coverage on the topic of big data during the past three years. But my response was a resounding, “No!”.
In my conversations with customers whilst I sense that some people are becoming a little tired of the phrase ‘Big Data’, I believe that most businesses are aware that the management of increasing data volumes is now a critical business issue. I still use the term ‘Big Data’, but I also expand of this to describe the challenges faced by business with rapidly growing data volumes and an increasing variety of data sources and formats to capture, manage and exploit. Those people that refuse to acknowledge there is a Big Data issue out there are probably fooling themselves and are unaware of the threats and opportunities it presents. My advice is to put the jargon aside and talk business issues
The volume of data that has to be analysed is definitely increasing as we collect more and more data, however we are not into a situation today that we didn’t anticipate earlier. Telecom and banking companies have been processing billions or records on a daily basis more than a decade back, as much as at the start of this century. If that is not big data, what is? This phrase has caught the attention because the challenge of dealing with larger data sets is becoming widespread and a relatively more fundamental requirement for even the mid sized and smaller companies to stay in the market. The problem is made more acute from the fact that while there is acknowledgement of bigger data sets and an urgency to deal with them, the proper products that can handle large data easily are still out of reach as they are costly. So you have a proliferation of open source and apparently low TCO tools that have rushed in to fill the gap and therefore quite suddenly you have a number of people talking about big data today than there was earlier. There is definitely a market of big data, but a lot of it is hype..