« My Hadoop Summit Keynote | Main | The education part of my keynote... »



In a word, yes. ;-)
No amount of pretending complex analytics are "easy" will make them so. The skills are not newly needed - they always have been. We've pursued "easy" for decades and gotten better and better at it for things everyday business users and analysts understand. But the hard stuff is still hard - because choosing the right statistical analysis, predictive model technique, or what have you requires depth. Tools don't fix that.


I agree with you. I also don't like the term data scientist. This will just make for more #hadoop black magic priest & less wide adoption by today data analysts.

Anshu Sharma

Yet another great post. As with past high end systems, eventually big data analysis will be done by regular business analysts and so I think your observations are right. And great to see you blogging regularly.

Alexander Lang

The key ingredient for building predictive models is domain expertise, because that's the only thing that helps you to find meaningful predictive variables. I fully agree that we don't need more "scientists" as intermediaries between business and its problems. @itmarketstrategy: with the advent of ensemble modeling techniques, built on "best of breed" data models, tools can indeed help finding "the right" modeling approach...


A rose by any other name still smells as sweet.

I think most organizations, in the near term, will be scared off more by the pricetag of a data scientist than the name itself.

Working with Hadoop and other schemaless data formats isn't cheap, and is still outside the reach of a lot of organizations that could make use of the technology.

Still, we saw this as late as the late '90s, where it took four full-time DBAs getting paid north of $80-100k to manage a single e-commerce website (running TCL and an early SQL variant) for Nokia during my time there.

Account Deleted

great post, very informative


The comments to this entry are closed.