It was only 25 years ago that relational databases were the perfect fit for enterprise computing. But then again some of the biggest shops were bursting at data capacity much less than 1TB.
Today, things are much different. We have to deal with data growth that is out of control and there is much more unstructured data such as streams and feeds as well as data being external to the enterprise and floating out on the Net somewhere. So how do we deal with such drastic change? Bill Mccoll states in a recent post, “From 1985 to 2004, SQL was essentially the only game in town. Around 2004, a number of companies, led by Google, and including Ebay, Yahoo and later Facebook, realized that they required levels of scalability, parallelism, performance and data flexibility that went way beyond what relational databases and SQL could provide.” And the new solution is to “adopt a simple parallel programming framework, MapReduce, in place of SQL”.
What will be next after we are finished cutting our teeth on this latest generation of tools?
