DIY statistics
From a Herald editorial
There is much intolerance of any use of this “ropey” information. A high priesthood of data analysis bemoans news media interest, however hedged with caveats, as betraying the apple in favour of the orange. Yet the combined “wisdom of the crowd” of thousands of schools and teachers, warts and all, does suggest, for example, fewer children meet standards in writing nationally than reading or mathematics.
I, like many people, was against the use of the data for league tables, though I thought it was probably inevitable. But if the high priesthood of data analysis has issued any edicts on analysis of the data, they forgot to copy me on the email. Perhaps it’s because I wasn’t wearing the high priestly hat.
At StatsChat we’re in favour of more people doing DIY statistics, which is why we keep linking to data sources when newspapers don’t provide them. As with any form of DIY, though, the results will be better if you have the right materials for the job at hand.
For any given set of data there are some questions that obviously can be answered (do fewer kids meet the writing standards?), and some that obviously can’t (are the writing standards just harder?). There are also many questions where the results will be unclear because it’s not possible to reliably separate out the huge socioeconomic effects. For example, it looks as though Maori children perform worse than non-minority children even within the same decile, but ‘within the same decile’ is a pretty broad range of schools, and the conclusion has to be pretty weak.
Thomas Lumley (@tslumley) is Professor of Biostatistics at the University of Auckland. His research interests include semiparametric models, survey sampling, statistical computing, foundations of statistics, and whatever methodological problems his medical collaborators come up with. He also blogs at Biased and Inefficient See all posts by Thomas Lumley »
This so called “dopey” data is pretty much reflective of all other research on achievement in NZ schools, in terms of gender, decile, ethnicity differences etc. where is the big surprise ?
12 years ago
As a DIY dilettante I concur with Thomas and have been advocating for the democratization of statistics for a while.
Many of us that have spent some time going over the data do not bemoan media interest, but worry about their simplistic headline-grabbing approach towards their analyses. As pointed out by Keith Ng, one doesn’t need to belong to a priesthood to get the right idea, but only NCEA-level statistics. In contrast, the Herald’s position sounds self-serving and uncritical. As a reader I would like to get not simply an Excel file, but some critical evaluation of what is going on.
I would also like to see the government providing direct access to this data in a manner useful for analysis. The idea of releasing independent parts of results (one school at the time) may delay putting together the data but not avoid it.
It would be excellent if we could connect many levels of government information to learn from the whole exercise. Many of us would be happy to contribute to this effort, in the process better understanding what makes some schools better than others.
12 years ago