Thresholds and tolerances

The post on road deaths sparked off a bit of discussion in comments about whether there should be a `tolerance’ for prosecution for speeding.  Part of this is a statistical issue that’s even more important when it comes to setting environmental standards, but speeding is a familiar place to start.

A speed limit of 100km/h seems like a simple concept, but there are actually three numbers involved: the speed the car is actually going, the car’s speedometer reading, and a doppler radar reading in a speed camera or radar gun.  If these numbers were all the same there would be no problem, but they aren’t.   Worse still, the motorist knows the second number, the police know the third number, and no-one knows the actual speed.

So, what basis should the police use to prosecute a driver:

• the radar reading was above 100km/h, ignoring all the sources of uncertainty?
• their true speed was definitely above 100km/h, accounting for uncertainty in the radar?
• their true speed might have been above 100km/h, accounting for uncertainty in the radar?
• we can be reasonably sure their speedometer registered above 100km/h, accounting for both uncertainties?
• their true speed was definitely above 100km/h, accounting for uncertainty in the radar and it’s likely that their speedometer registered above 100km/h, accounting for both uncertainties?

You can’t make the problem go away: if the law ignores all the uncertainties, then a driver who honestly and scrupulously kept the speedometer needle below 100km/h will sometimes be convicted, which is unjust and tends to bring the police and the law into disrepute.

A better approach is to separate the intended true speed limit and the regulatory threshold.  The intended true speed limit is 100km/h, and the regulatory threshold for a radar reading is chosen in light of the uncertainties.  If the goal is to give the driver the benefit of the doubt, which makes sense under our legal traditions, the regulatory threshold might be 105km/h.  This doesn’t mean that driving at 105km/h is legal, it just means that a radar reading of 105 is what it takes to be sufficiently sure that the driver was actually speeding.

When setting environmental thresholds the problem is more severe, because the uncertainties are much larger.  Suppose you want to set a limit for fine particulate air pollution, and that you want to base the limit on 24-hr average levels.  You might decide that the concentration should be above 30μg/m3 on no more than 1% of days.  That is the intended true threshold.  The next step is to come up with a regulatory threshold based on, say, a year of daily measurements.  If the uncertainty was ignored you would say that a city was in compliance with 3 or fewer days above 30, and out of compliance with 4 or more days.  But we don’t want to ignore uncertainty here any more than in the setting of speed limits.

In the case of pollution it would probably be inappropriate to have the benefit of the doubt in favor of polluters, so you need to work out what results from a year of data would demonstrate convincingly that the 99th %ile was below 30.  One approach is to still use counts of days, but to require a stricter regulatory threshold. If a city was in compliance with 2 or fewer days over 30μg/m3, and out of compliance with 3 or more, then a city with a true exceedance rate of 1.5% would be fairly sure to be ruled out of compliance, but a city with a true exceedance rate of 0.5% would still have nearly a 1 in 3 chance of being rule out of compliance.

Counts are a fairly crude summary in this context, and it should be possible to do better by considering the actual values on the highest days.  This will require statistical models for the pollution distribution, and has the disadvantage of potentially leading to different regulatory thresholds in different cities, but that is inevitable if you want to control a true environment variable which is measured only imperfectly. For more detail, see Peter Guttorp’s paper “Setting environmental standards: a statisticians perspective”.