Big Data Big Challange Big Oppportunity

Real-time access to multi-sourced complex information creates competative advantages

We are all aware of the speed exponential way information is multiplying. The fastest increasing quantity on this planet is the amount of information we are generating. It is (and has been) expanding faster than anything else we create or can measure over decades. Information is accumulating faster than any material or artifact in this world, faster than any by-product of our activities. The rate of growth in information may even be faster than any biological growth at the same scale.


We on the research end struggle to manage, filter and aggregate this treasure trove of complex data into something consumable, actionable and meaningful. Unfiltered data have minimal utility no matter how accessible. The wonder of Google was that it gave everyone the capacity to search a very small portion of the web in a way that was meaningful to the searcher. Google’s mission statement when it incorporated in 1998 was “to organize the world's information and make it universally accessible and useful". In less than 5 years we learned to turn to the web for answers to most of our questions. Google created the system changed the way individuals seek, consume and believe information.


In this century we have generated “Big data”. Big data is a collection of data sets from multiple sources so large and complex that it becomes difficult to process using existing database management tools or traditional data processing applications. The primary goal of big data analytics is to help organizations make better decisions by enabling data scientists and other researchers to analyze huge volumes of transaction data as well as multiple data sources that previously had been inaccessible. The problem is that Big data is very hard to work with on multiple levels. It requires capabilities beyond traditional data base management software and personnel to capture, store, curate, analyze and communicate the meaning.


Big data is in the start-up phase that the internet was in pre-Google era. Several large data companies (including IBM, SAS, SAP, CRM, Oracle, Microsoft Strategy, etc.) are trying to establish themselves as the turnkey integrator of big data analytics for the business community. To date none has demonstrated the paradigm shift in analytics that Google did for the web, hence none has established itself as the industry leader.


There are several real consequences of Big data accessibility. There has yet to be a democratization of Big data so that the power of that information is available to small and mid-sized organizations. While the younger generations of business, science, non-profits and public policy communities are better champions of data-based collaborative engagement, there is little access to this information. The cultural change necessary to utilize data at all levels of decision making in preference to “gut” feelings or “in my experience” of senior management has yet to occur. So for the moment Big data is simply unavailable to any but the largest organizations for real-time decision making. For the US to maintain market leadership a solution has to be developed. The divide between organizational manager experience and rapidly evolving information has to be bridged if entrepreneurial success is going to thrive. Victory will go to the small, agile companies which make Big data information available to growing organizations which use it to accelerate their positions in the marketplace. The gauntlet is down. The need is profound.


Go back

Add a comment