Era of Big Data

The New York Times has a great article discussing the era of big data. This might have a Kurzweil-esque ring to it, but due to technology change big data is becoming increasingly available and ready for analysis: in fact, there are more data sets out there than brains to analyze them, especially when one notes the incredible number of combinations of analyses that could be conducted even on a single data set with 100 variables (in what is known as the curse of dimensionality). However, one problem with big data is that, since so much of the data are collected by private entities, much of it may not be available to academics and independent researchers.