Many Companies Unprepared for Big Data Boom

6943970366_4e296d2414_m

In a recent press release Gartner Group identified Big Data as one of the top 10 strategic trends in 2012[1]

  1. [1] Pettey, C (2011 October 18). Gartner Identifies the Top 10 Strategic Technologies for 2012. Gartner.com Retrieved January 27 2012 from http://www.gartner.com/it/page.jsp?id=1826214
.

Big data has rushed on to the information technology scene over the past two years and it seems nearly everyone is looking to cash in on the promises of newly generated revenue streams teeming with profit. However, as more and more organizations make the decision to take the big data plunge, many of them are coming to the realization that they are unprepared.

Wrestling with Big Data is going to be the single biggest IT challenge businesses face over the next two years. By the end of that period they will either have it right or they will be very seriously adrift from their own business and the threats and opportunities posed by Big Data.

– Luigi Freguia, senior vice president, Systems, Oracle EMEA

Big Data, Big Demand

Like any technology solution, the return on implementing a big data solution does not come without some investment. Since big data solutions are predicated on processing very large data sets of structured and unstructured data which is very rapidly changing, one of the most critical components to the investment is hardware, lots of hardware. To give some perspective on how much data can be involved in a big data solution, Wal-Mart handles more than 1 million customer transactions every hour which they subsequently import into databases estimated at more than 2.5 petabytes (the equivalent of 167 times the books in America’s Library of Congress). This would not be possible without a significant investment in hardware.

Failure to plan is planning to fail

As I stated earlier in the article, big data has rushed on to the information technology scene and in that rush many organizations have not been able to sufficiently plan for this surge in new hardware demands. As a consequence, there has been a trend in leveraging external data centers to accommodate the demand. In a recent press release by Oracle[2]

  1. [2] Walker, S (2012 January 11). Oracle Next Generation Data Centre Index Raises Questions as to the Preparedness of Businesses for the ‘Big Data’ Boom. Oracle.com Retrieved January 27 2012 from https://emeapressoffice.oracle.com/Press-Releases/Oracle-Next-Generation-Data-Centre-Index-Raises-Questions-as-to-the-Preparedness-of-Businesses-for-the-Big-Data-Boom-26db.aspx
, a study of approximately 1,000 managers in large organizations revealed that there was a 16% increase in organizations using an external data center. 92% of those who responded felt they would need a new data center within the next two years.

Conclusion

Big data has come on fast and furious with demands that far exceed the resources of most companies. Without an implementation plan which includes a significant investment in hardware, many organizations will find themselves unprepared for the big data boom.