Measuring what you care about
The Herald is reporting on Auckland Transport’s monthly report (you can find the reports here). One of the recurring surprises in these reports is how high the punctuality figures are, and the Herald comments on these for some of the train services.
The bus punctuality statistics are even shinier
As a regular bus commuter it’s hard to imagine how these could be correct — and if they were, there would be no need for the real-time bus predictions, since the timetables would be more accurate than the predictions.
The solution is in the fine print: “Service punctuality for July 2012 was 99.24%, measured by the percentage of services which commence the journey within 5 minutes of the timetabled start time and reach their destination“.
Or, to quote Lewis Carroll’s Humpty Dumpty “When I use a word it means just what I choose it to mean — neither more nor less”
Thomas Lumley (@tslumley) is Professor of Biostatistics at the University of Auckland. His research interests include semiparametric models, survey sampling, statistical computing, foundations of statistics, and whatever methodological problems his medical collaborators come up with. He also blogs at Biased and Inefficient See all posts by Thomas Lumley »
Hmm. The statistic I REALLY want to know is how often NZ Bus in Auckland cancels scheduled services – every regular bus user can tell of times when a scheduled bus just doesn’t turn up.
12 years ago
Then there’s not really much difference between the reliability and punctuality stats is there?
reliability: service left and reached it’s destination.
punctuality: service left within 5 minutes of when it was supposed to and reached it’s destination.
There’s not much you can actually say without, as Julie suggests, knowing how many of the services never actually left in the first place.
12 years ago
I think the issue of punctuality which isn’t revealed (and Thomas might be alluding to) is that a bus can be 15 minutes ahead at some stops and 15 minutes behind at others but so long as it gets to the final destination within 5 mintus that counts as 100%.
12 years ago
Steve,
It’s worse than that. If the bus starts off less than 5 minutes late, it is ‘punctual’ no matter how late it is further down the route. Even if it gets to the destination an hour late, it still counts. It just has to not have a mechanical breakdown or get abducted by aliens en route.
The real threshold is that if a bus is more than about 20 minutes late at the destination, it’s likely to make the next trip start more than 5 minutes late, and that will show up on the punctuality statistics.
12 years ago
Even worse is that these statistics are gathered by the bus companies themselves. They have no obvious basis in fact.
12 years ago
I’m less worried about these being self-reported (they are auditable, at least in principle) than about the fact that they are meaningless.
I’d much rather have a self-reported measure of actual punctuality than a external measure of something useless.
Still, we should be able to get accurate and relevant data — the bus prediction system may not be great at actual prediction, but it’s pretty good at knowing where the buses are.
12 years ago