Prediction is hard, especially about buses?

The bus prediction system is potentially a very good idea — research in other cities has shown that it reduces actual waiting time, and reduces perceived waiting time even more.

Unfortunately, the illusory sense of control over one’s fate that the prediction system gives is easy to shatter.  This morning it was a stealth bus: arriving when it was allegedly still ten minutes away.   On other occasions the waiting time counts down steadily to DUE and then disappears, indicating that a ghost bus has gone by.

Bus prediction involves some hard engineering problems: the buses need to know where they are, they need to be able to tell the central office, and the central office needs to be able to get the information to transit users.  Fortunately, the first problem is solved by GPS (and odometers), the second by packet radio, and the third by the internet (and text-message gateways).

What remains is mostly a statistical problem, and partly a problem in applied psychology.    The data are fairly straightforward: the system knows approximately where the bus was every couple of minutes into the past, and this needs to be projected forward.   The Seattle MyBus project did a good job of implementing a simple version about ten years ago:  the prediction is a weighted average between where the bus would arrive at its current speed and when it should arrive by schedule, and the weights come from a large collection of actual bus trips.   There’s actually a lot more information available — for example, it’s hard to predict from a bus’s performance along Manukau Rd how long it will take to get through Newmarket, but since there’s a bus along Broadway every few minutes the system potentially has up-to-date information on congestion and crowding.

My guess, based on the relatively high frequency of buses that apparently go backwards, is that the Auckland system is a bit over-optimistic about how fast a late bus can return to schedule, and isn’t using the congestion information.  It also doesn’t seem to know which incoming bus will be running each route out of the city, so the city-center predictions can be a bit useless.  The real problem, though, is how to incorporate this uncertainty when presenting the results.   Even when you’re using all the available information there’s always prediction error, and sometimes a bus will stop transmitting information either for a few minutes or for its whole route.   In that case, the system has to fall back on the timetable, but the Auckland system doesn’t tell you it’s done that.

OneBusAway, in Seattle, distinguishes clearly between real predictions and timetable predictions. It also does helpful things like indicating which buses have just left, and it doesn’t seem to have ghost buses or stealth buses.  It’s also based on an open data stream that anyone can use — both the real-time bus location data and the predictions are accessible to anyone who wants to set up an improved system or just write a better app.