Algorithmic bias in justice
There’s a pretty good piece on Stuff about bias in the justice system that might be attributable to biased algorithms. You should read it.
The story talks about two specific people, one who had a low predicted risk and did re-offend, and one who had a high predicted risk and, well, we don’t know yet. That’s evidence that the model isn’t perfect; it doesn’t tell us much about how good or bad it is: if you have a well-calibrated model and it says someone has a 0.06 chance of re-offending, then out of every sixteen people like that you’d expect one to re-offend. Individual cases aren’t very helpful in assessing how good or bad the system is; you need statistics.
As the story makes clear, though, if you want a system that gives Māori and Pākehā the same sentences, simply leaving out the ethnicity variable from your model isn’t going to do that. Differences by ethnicity are all over the data. A statistical model is going to see who is in prison, and send along more people like that.
Part of the problem (as the story says) is the data: we don’t actually have data on re-offending, only on re-conviction, and the difference between the two involves the justice system and its biases, and there’s a potentially very nasty feedback loop there. But that’s only part of the problem. The other part is that basing imprisonment on the likelihood of re-offending is going to result in longer terms in prison for people from groups that re-offend more often. And that will include Māori: the over-representation of Māori in the prison population is not just because the justice system is racist, but also because society is racist.
There’s not just a problem with the answer that the model gives; I think there’s a problem with the question, too. The intuition behind predictive sentencing is that if you have two people convicted for the same crime, and they are otherwise similar, and one of them is more likely to commit future crimes, you want to keep that one out of the community for longer. For me, at least, the intuition relies quite strongly on the ‘otherwise similar’ qualification. If you came along and said “young people are more likely to commit future crimes than old people, so we should lock them up for longer”, I wouldn’t be at all persuaded. The same for poor vs rich. Or men vs women. Or Māori and Pākehā. These don’t seem like the sort of relevantly-similar-but-different-risk distinctions that are intuitively a good idea to base imprisonment on.
That is, I think one of the reasons many people don’t like the outputs of predictive sentencing models is that we don’t actually believe in sentencing based on risk of re-offending; at most, we believe in something much more complicated that the models don’t try to do.
I have to admit a distinction here between initial sentencing and parole. Parole decisions, according to the Parole Act must consider both the likelihood of further offending; and the nature and seriousness of any likely subsequent offending. Parole fundamentally does involve the risk of re-offending. Initial sentencing has a lot of purposes, and risk of re-offending is much less tightly linked with it. According to the story, though, the predictive model is an input to both processes, and similar models are certainly an input to sentencing in the USA.
And finally, it’s important to remember that one of the original reasons people built statistical models to help with sentencing and parole decisions was that it was previously being done by the humans who are the source of the biased data we’re complaining about. Getting rid of statistical models and just relying on the fairness and objectivity of people in the justice system isn’t a panacea either.
Thomas Lumley (@tslumley) is Professor of Biostatistics at the University of Auckland. His research interests include semiparametric models, survey sampling, statistical computing, foundations of statistics, and whatever methodological problems his medical collaborators come up with. He also blogs at Biased and Inefficient See all posts by Thomas Lumley »
A couple of points in the article you link to where I thought “wow!”
Corrections research and analysis manager Peter Johnston said he had “high confidence in its accuracy, given repeated exercises, which show a high degree of correlation between predicted rates of reimprisonment, and actual rates of reimprisonment.”
As you and the journalist point out, predictive accuracy will be contaminated by feedback. It is amazing that the research and analysis manager would not realise this.
And later on in the article…
“Are there any other variables that are highly correlated with ethnicity that could be used instead?” The curators of the algorithm were clearly wanting to include ethnicity but hide it.
5 years ago
Where does the story say this
” one who had a low predicted risk and ‘did re-offend, and one who had a high predicted risk and, well, we don’t know yet.”
All they say is the low score offender was released early at first parole offending and the other man wasnt.
No one has said anything in this story about subsequent re-offending after these sentences.
oh.. they did mention about the high risk person , who was sentenced for cannabis dealing
“They committed very different offences and had different backgrounds. Wallace also had several previous offences, including some for violent assault, which counted against his score. ”
So the serial offender, who may well had previous jail time , is seen as ‘high risk’
It seems bleeding obvious and algorithms dont change that.
5 years ago
<>
Even if the justice system weren’t racist, even if society weren’t racing, even if the weakest group weren’t overrepresented, there will always be an etnic group more likely to (re)offend than the other ones. Let it be Maori, or black, or white, or Wasp, or non-Community citizens, or whatever you think. That’s the essence of statistics.
<>
So, justice algorithms based on “sensible” variables are not ethical. Have I understood you correctly?
But almost all other variables are related to the first ones: where a person is born, where he lives, where he works, where he goes shopping, which school he has attended, and so on. Are these ones to be discarded as well?
On the other hand, there could be some variables uncorrelated to ethnic membership and other sensible variables. But there are essentially noise: the third plate digit is seven, the preferred planet in the solar system, and so on. Would be ethic a justice algorithm based on these variables?
I think that the problem is essentially without statistical solution. If the system has to take decisions on individual basis, then only previously crime committed are valuable criteria; if, on the other hand, the system could take decisions on statistical basis, then it should rely on differential reoffend likelihood between groups. And every subdivision of population in groups is discriminatory: one could be convicted or negate to parole because other persons similar to him behaved badly.
Am I missing something?
5 years ago
In my previous message the quoted passages have been deleted (by a not-very-clever algorithm :) )
I repeat them for the sake of clarity:
** The other part is that basing imprisonment on the likelihood of re-offending is going to result in longer terms in prison for people from groups that re-offend more often. And that will include Māori: the over-representation of Māori in the prison population is not just because the justice system is racist, but also because society is racist.
Even if the justice system weren’t racist, even if society weren’t racing, even if the weakest group weren’t overrepresented, there will always be an etnic group more likely to (re)offend than the other ones. Let it be Maori, or black, or white, or Wasp, or non-Community citizens, or whatever you think. That’s the essence of statistics.
** If you came along and said “young people are more likely to commit future crimes than old people, so we should lock them up for longer”, I wouldn’t be at all persuaded. The same for poor vs rich. Or men vs women. Or Māori and Pākehā. These don’t seem like the sort of relevantly-similar-but-different-risk distinctions that are intuitively a good idea to base imprisonment on.
So, justice algorithms based on “sensible” variables are not ethical. Have I understood you correctly?
But almost all other variables are related to the first ones: where a person is born, where he lives, where he works, where he goes shopping, which school he has attended, and so on. Are these ones to be discarded as well?
On the other hand, there could be some variables uncorrelated to ethnic membership and other sensible variables. But there are essentially noise: the third plate digit is seven, the preferred planet in the solar system, and so on. Would be ethic a justice algorithm based on these variables?
I think that the problem is essentially without statistical solution. If the system has to take decisions on individual basis, then only previously crime committed are valuable criteria; if, on the other hand, the system could take decisions on statistical basis, then it should rely on differential reoffend likelihood between groups. And every subdivision of population in groups is discriminatory: one could be convicted or negate to parole because other persons similar to him behaved badly.
Am I missing something?
5 years ago
Yes. You are missing something.
Its not that the differences in sentencing between racial groups are ‘ detectable’ statistically, but for Maori are way off the chart. That , rightly is seen as a bad outcome for Maori and all of us.
The newspaper example was a very poor example , different crimes and different previous conviction history but if an algorithm is used for Parole decisions ( the Roc- Roi number?) it doesnt serve a wider public interest it it merely continues a gross sentencing imbalance. Theres bound to be a grey area in between ‘must release’ and ‘unlikely to be released’ which maybe could have a deliberate nudge on the Roc-Roi number towards release based on ethnicity.
I think Universities work in a similar way for student entry for some courses with capable students who arent in the top rankings based only on exam scores but are in the grey area. The difference is students have to work to ‘stay in’ while those on parole have to work to ‘stay out’
5 years ago
I’m not saying the bias in the assessments is ok for parole decisions.
I’m saying that the *goal* of trying to assess risk of reoffending is ok for parole decisions.
5 years ago
(way off che chart means too high?)
So, which is your solution? Reduce or cancel the grey area, matching previous one with decisions in favour of directly involved people?
5 years ago