Big Data is watching you

Or, as some of my colleagues would prefer “Big Data are watching you”.  In Stuff.   The story is about the potential disadvantages of your life being predictable by sophisticated analysis, and it’s pretty good.

I will comment on one example:

The Corrections Department first developed a computer system, RocRol, in 1995 that calculates the chances of prisoners being reconvicted within five years of their release…

Corrections analyst Arul Nadesu told a conference at Te Papa in February that new software developed by business analytics firm SAS that incorporates neural networking technology – a technique for processing data that mimics the way signals are passed between neurons in the brain – could reduce the risk of RocRol “misclassifying” an offender to just one in seven.

The software may be new, but neural networks for prediction have been around since the 1960s, when people did really believe that they mimicked the way the brain works. Neuroscience has come a long way since then.  Neural networks were very popular in the 1980s, but by the time I learned about them in the early 1990s they were no longer anything special or distinctive.

Also “reduce the risk … to just one in seven” suggests that it’s substantially worse than one in seven at the moment. While things may change in the future, that’s exactly the current problem with Big Data: the predictions aren’t all that good