Posts filed under Law (39)

You may have heard some under-rehearsed radio wittering from me on this topic, so I thought I should write something more coherent.

Last August, researchers from the Dunedin Cohort Study published a paper showing that people who had been heavy cannabis users as teenagers performed worse on cognitive function tests later in life, where this wasn’t true of people who started using cannabis as adults. One natural interpretation of these associations is that cannabis has toxic effects during brain development.  As I pointed out at the time, the evidence isn’t overwhelming (since it’s a relatively small study and we know environmental factors can lead to differences as big as those observed) but was somewhat persuasive and is probably better than other studies on this topic.

Now, a Norwegian economist has argued that the results could be explained by purely sociological factors: that people from low-socioeconomic status backgrounds are more likely to use cannabis, and to perform worse on cognitive function tests, and that the difference in cognitive function tests tends to increase over time after school.  He is correct; this could explain the published results.  However, the Dunedin Cohort Study researchers have done further analyses in response, and while the socioeconomic explanation was reasonable, it seems to not be true.  Both the relationship between socioeconomic status and cannabis use, and the relationship between low socioeconomic status and change over time in cognitive function test results were weak in this particular data set.

Even if the association is real and causal there could still be explanations that don’t involve brain toxicity.  For example, imagine that people who enjoy being stoned are less likely to choose jobs and recreational activities that are cognitively demanding.  They would then to some extent tend to end up scoring lower on cognitive function tests in later life.  This, if it were true, would be an explanation that does depend on the properties of cannabis, but not on toxic effects.

Over all, this result doesn’t have huge implications for drug policy.  It doesn’t change the basic fact that cannabis is far from innocuous but is also much safer than alcohol or tobacco.  It doesn’t affect the relevant international treaties and probably won’t shift domestic public opinion.  Differences of opinion on cannabis policy questions depend mostly on different preferences, and partly on other uncertainties. For example, would legal cannabis lead to more or less alcohol consumption?

I’d recommend the book “Marijuana legalization: what everyone needs to know”.  This is written by a group of public-policy academics, who have varying policy preferences, but looked at what evidence they could agree on. It gives a series of questions and what is known about their answers. Unfortunately it’s not (yet?) in the Auckland city library.

Links: Stuff, Science Media Centre

Stuff has a story about genetic testing today, which leads off

Those sending their DNA to be analysed cheaply overseas are obliged to share the results with life insurers and risk losing control of their most sensitive information.

If you read further, it turns out that all the information about insurance and law is Australian.

In October, they had a similar story, with information on the relevant US law.

Perhaps next time there will be some NZ context.

Any binary decision requires us to consider both the probability of getting it right and the consequences of getting it wrong.  Many legal systems have traditionally felt that wrongful convictions are worse than wrongful acquittals, and this forms part of the support for the presumption of innocence.

In other areas of the law, the incentives are different.  In automated detection of unauthorized copying, and resulting ‘takedown’ notices under laws such as the US DMCA, there is effectively no risk to the copyright holder from false positives, so there is not much incentive to avoid them.

An interesting example (via the far-from-unbiased BoingBoing) is this takedown notice, one of the stream routinely posted by Google at ChillingEffects.  The first few pages just show torrent sites that posted unauthorised copies of MS Office and deserve what’s coming to them, but if you scroll down to Copyright Claim #2, it starts to look different: (more…)

Stuff is reporting on new drug tests being pushed by NZDDA

Hardy said hair testing was more accurate and effective method of detecting drug use, and it gave a history of drug or alcohol use over the previous 90 days….With urine tests more drugs were undetectable if urinalysis was carried out more than three days after use.

Since the advertised purpose of employee drug testing is to catch people who are impaired on the job, expanding the history from 3 days to 90 days surely makes the test less accurate, not more accurate.  It’s more accurate only if you don’t distinguish between on-the-job and off-the-job drug and alcohol use — like the drunk looking for his keys under the lamppost because he could see better there.

One of the key contributions of statistics to evidence-based medicine has been in forcing medical researchers to measure what they really want to affect, not what is convenient and plausibly related to it.  Drug use in the past 90 days is not the same as on-the-job impairment, and it’s probably a pretty lousy surrogate.

The Assistant Privacy Commissioner is quoted in the article as saying

“Employers should only use it where there is a genuine business need. For example, drug testing has been allowed where there are safety issues with operating machinery.

An interesting approach used by some US companies is drug testing after accidents.  A study from Princeton (PDF) found that this did reduce accidents by about 10%, though some of the reduction may have been due to under-reporting of accidents — an important tradeoff to consider.

There isn’t going to be a quick technological fix, however, and we do need some sort of regulation. As Stanford’s Keith Humphreys puts it

…we use public policy to pick the particular sort of drug problem society will have. For example, different policy environments can make it a human rights problem, an addiction problem, a crime problem, an AIDS problem, a public disorder problem etc., but no policy will produce a true ending of all of society’s problems with drugs. There are some policies that ameliorate multiple aspects of the problem, but in most cases we are faced with hard choices about what sort of problem we will have rather than a problem-free alternative.

On 3 News last night, we got a tour of South Auckland, just missing various drunken disturbances.   The video starts “3News has uncovered some shocking figures about the amount of alcohol some offenders are drinking” , and the webpage story says  “new research has revealed the alarming impact alcohol is having on crime“.

That’s approximately truish.  The `new research’ is a report dated December 2010 by the New Zealand Drug Use Monitoring (NZ‐ADUM) research programme at Massey University.  They surveyed 800 arrestees in Whangarei, Auckland Central, Wellington Central, and Christchurch Central, asking them about their drug and alchol use.   And there was a lot of drug and alcohol use — evidently, police arrests are selecting for people who are insufficiently risk-averse in all sorts of ways.  The figures possibly qualify as ‘shocking’, with arrestees reporting an average of 12 standard drinks both on typical days when they drink, and on the day they were arrested.  Some people reported much higher consumption, and some reported none.

3News compares this to “The average New Zealander drinks around two standard drinks a day”, which looks like this StatsNZ figure on total alcohol sales.  In addition to the problems of comparing self-reported drinking with sales data, the national average is over all days and the arrestee average is over days when they drank.  Since the arrestees reported drinking an average of about 1 day in 3, the correct comparison to 2 drinks/day is 12/3=4 drinks per day, about twice the NZ average.    The arrestees also consumed more tobacco, more cannabis, more P, more opioids, more ecstasy, and more of everything else than the average Kiwi; in fact, as far as I can tell alcohol showed a smaller difference than any of the other drugs — mostly because the average for all New Zealanders is fairly high.

The other notable statistic about drinking is that the average drinks per day that they drank was the same as the average drinks on the day they were arrested.  That is, although many of the arrestees were drunk when they were arrested, they were no drunker than they usually are on about one day in three.   The relative risk of arrest given their heavy drinking for these arrestees is thus about 3, which is a lot smaller than I would expect even without the headlines.  If you read the report, while alcohol is unquestionably a problem, it is far from the only problem that the arrestees have.

Stuff is reporting on a new paper in the BMJ (or as Stuff calls it, only 24 years after the name change, the British Medical Journal) saying that smoking dope doubles your chance of  a serious or fatal car accident.  Regular readers of StatsChat posts on drug journalism will be pleasantly surprised to hear that this is actually what the research paper says, and that the researchers had moderately good evidence for this conclusion.

The paper compiled the results of previous studies, restricting their attention to studies that had a control group, and that measured  actual THC concentration in the blood, rather than the inactive metabolites that can be detected days later in blood or urine.   Some of the studies compared drivers in crashes to drivers not in crashes, others compared the driver responsible for the crash to other drivers involved in the crash.

The summary of the findings looking at various ways of breaking up the data is:

That’s a forest plot, with the dots showing the estimated risk ratio [technically odds ratio, but it doesn’t matter in this context] and the lines showing the 95% margin of uncertainty.   Though some of the lines include the null value 1.0 and others don’t, there’s pretty clear agreement across the analyses that cannabis increases crash risk, by a factor of two or so.

After getting it right so far, Stuff unfortunately went further and said “The research found cannabis significantly impaired the psychomotor response, or muscle activity linked to mental processes.”  That isn’t actually what they found. That’s in the Introduction section of the paper, because it’s telling us what was already known — dope makes you slower and clumsier.  The BMJ isn’t publishing research telling us that, any more than they publish conclusions on the spatial distribution of ursine excretory activities, or the religious affiliation of the Pope.

They finish up by saying “The Land Transport Amendment Act 2009 will be reviewed this year and police are investigating the possibility of introducing more effective roadside drug tests.” This paper doesn’t really lend much support to changes in current roadside drug tests, which have the advantage of testing for actual impairment and not being specific to a single drug.

The NALP has released data on starting salaries for US lawyers in 2011, and the distribution is really weird.

Usually we expect salary distributions to be skewed, with a long upper tail, but in this case there are two modes: a large group earning around $45k and a smaller group earning about $160k. The mean income is about $80k, the median is about $60k, and neither is a good summary of what someone is likely to make.

The distribution didn’t always look like this. Twenty years ago, starting salaries for lawyers had a more familiar skewed distribution, with a single mode around $30k.

Over the twenty-year period, the income at the lower mode has rised by about 50%, but US median household income has roughly doubled, and the CPI has increased by about 65%.  Some law graduates are raking it in; most are not, and they nearly all have to pay off huge sums in student loans.

In reality the figures are probably worse than this for the majority: there’s a lot of missing data.  As Paul Campos puts it “People without salaries are reluctant to report their salaries”

A number of my colleagues have sent me this link from British newspaper The Guardian, and asked me to comment. In some sense I have done this. I am a signatory to an editorial published in the journal Science and Justice which protests the law lords’ ruling.

The Guardian article refers to a Court of Appeal ruling in the United Kingdom referred to as R v T. The original charge against Mr. T. is that of murder and, given the successful appeal, his name is suppressed. The nature of the appeal relates to whether an expert is permitted to use likelihood ratios in provision of evaluative opinion, whether an evaluative opinion based on an expert’s experience is permissible, and whether it is necessary for an expert to set out in a report the factors on which evaluative opinion based.

It is worthwhile noting before we proceed that to judge a case solely on one aspect of the whole trial is dangerous. Most trials are complex affairs with many pieces of evidence, and much more testimony that the small aspects we concentrate on here.

The issue of concern to members of the forensic community is the following part of the ruling:

In the light of the strong criticism by this court in the 1990s of using Bayes theorem before the jury in cases where there was no reliable statistical evidence, the practice of using a Bayesian approach and likelihood ratios to formulate opinions placed before a jury without that process being disclosed and debated in court is contrary to principles of open justice.

The practice of using likelihood ratios was justified as producing “balance, logic, robustness and transparency”, as we have set out at [54]. In our view, their use in this case was plainly not transparent. Although it was Mr Ryder’s evidence (which we accept), that he arrived at his opinion through experience, it would be difficult to see how an opinion of footwear marks arrived at through the application of a formula could be described as “logical”, or “balanced” or “robust”, when the data are as uncertain as we have set out and could produce such different results.

A Bayesian, or likelihood ratio (LR) approach to evidence interpretation, is a mathematical embodiment of three principles of evidence interpretation given by Ian Evett and Bruce Weir in their book Interpreting DNA Evidence: Statistical Genetics for Forensic Scientist. Sinauer, Sunderland, MA 1998. These principles are

1. To evaluate the uncertainty of any given proposition it is necessary to consider at least one alternative proposition
2. Scientific interpretation is based on questions of the kind “What is the probability of the evidence given the proposition?”
3. Scientific interpretation is conditioned not only by the competing propositions, but also by the framework of circumstances within which they are to be evaluated

The likelihood ratio is the central part of the odds form of Bayes’ Theorem. That is

The likelihood ratio gives the ratio of the probability of the evidence given the prosecution hypothesis to the probability of the evidence given the defense hypothesis. It is favoured by members of my community because it allows the expert to comment solely on the evidence, which is all the court has asked her or him to do.

The basis for the appeal in R v T was that the forensic scientist, Mr Ryder, in the first instance computed a likelihood ratio, but did not explicitly tell the court he had done so. In the second instance, there was also criticism that the data needed to evaluate the LR was not available.

Mr Ryder considered four factors in his evaluation of the evidence. These were the pattern, the size, the wear and the damage.

The sole pattern is usually the most obvious feature of a shoe mark or impression. Patterns are generally distinct between manufacturers and to a lesser extent between different shoes that a manufacturer makes. Mr Ryder considered the probability of the evidence (the fact that the shoe impression “matches” the impression left by the defendant’s shoe) if it indeed was his shoe that left it. It is reasonable to assume that this probability is one or close to one. If the defendant’s shoe did not leave the mark, then we need a way of evaluating the probability of a “adventitious” match. That is, what’s the chance that the defendant’s shoe just happened to match by sheer bad luck alone? A reasonable estimate of this probability is the frequency of the pattern in the relevant population. Mr Ryder used a database of shoe pattern impressions found at crime scenes. Given that this mark was found at a crime scene this seems a reasonable population to consider. In this database the pattern was very common with a frequency of 0.2. The defense made much stock of the fact that the database represented only a tiny fraction of the shoes produced in the UK in a year (0.00006 per cent), and therefore it was not comprehensive enough to make the evaluation. In fact, the defense had done its own calculation which was much more damning for their client. Using the 0.2 frequency gives a LR of 5. That is, the evidence is 5 times more likely if Mr T.’s shoe left the mark rather than a shoe of a random member of the population.

The shoe size is also a commonly used feature in footwear examination. The shoe impression was judged to be size 11. Again the probability of the evidence if Mr T.’s shoe left the mark was judged to be one. It is hard to work out exactly what Mr Ryder did from the ruling, because a ruling is the judges’ recollection of proceedings, which is not actually an accurate record of what may, or may not, have been said. According to the ruling, Mr Ryder used a different database to assess the frequency of size. He estimated this to be 3%. The judges incorrectly equate this to 0.333, instead of 0.03 which would lead to an LR of 33.3. Mr Ryder used a “more conservative” figure to reflect to some uncertainty in size determination to 0.1, giving an LR of 10.

Wear on shoes can be different between different people. Take a look at the soles of your shoes and those of a friend. They will probably be different. To evaluate the LR, Mr Ryder considered that the wear on the trainers. He felt could exclude half of the trainers of this pattern type and approximate size/configuration. He therefore calculated the likelihood ratio for wear as 1/0.5 or 2. Note here that Mr Ryder appears to have calculated the probability of wear given pattern and size.

Finally, Mr Ryder considered the damage to the shoes. Little nicks and cuts accumulate on shoes over time and can be quite distinctive. Mr Ryder felt he could exclude very few pairs of shoes that could not previously have been excluded by the other factors. That is the defendant’s shoes were no more, or less, likely to have left the mark than any other pair in the database that had the same pattern, size and wear features. Therefore therefore calculated the likelihood ratio for damage as 1.

The overall LR was calculated by multiplying the four LRs together. This is acceptable if either the features were independent, or the appropriate conditional probabilities were considered. This multiplication gave an LR of 100, and that figure was converted using a “verbal scale” into the statement “the evidence provides moderate support for the proposition that the defendant’s shoe left the mark.” Verbal scales are used by many forensic agencies who employ an LR approach because they are “more easily understood” by the jury and the court.

The appeal judges ruled that this statement, without the explicit inclusion of information explaining that it was based on an LR, was misleading. Furthermore, they ruled that the data used to calculate the LR was insufficient. I, and many of my colleagues, disagree with this conclusion.

So what are the consequences of this ruling? It remains to be seen. In the first instance I think it will be an opening shot for many defense cases in the same way that they try to take down the LR because it is “based on biased Bayesian reasoning.” I do think that it will force forensic agencies to be more open about their calculations, but I might add that Mr Ryder didn’t seek to conceal anything from the court. He was simply following the guidelines set out by the Association of Footwear, Tool marks, and Firearms Examiners guidelines.

It would be very foolish of the courts to dismiss the Bayesian approach. After all, Bayes’ Theorem simply says (in mathematical notation) that you should update your belief about the hypotheses based on the evidence. No judge would argue that against that.

Today’s NZ Herald reports that Māori and Pacific Islanders are highly represented in the statistics for the use of tasers by the NZ police. Green MP Keith Locke is quoted as saying “Certainly they’re being fired disproportionately at Māori.” Mana Party spokeswoman Annette Sykes is quoted as saying “there has been this disproportionate outcome for Māori and Polynesian individuals, which is a sad indictment on us.”

Looking at the numbers, a taser has been used 35 times out of a total of 88 on Māori, or just under 40%. This is certainly much higher than 15% reported for the percentage of Māori from the 2006 census. However, this is not the relevant population. Rather, we should consider the proportion of Māori involved in the criminal justice system.

Figure 1, on page 17 of the report on Identifying and Responding to Bias in the Criminal Justice System: A Review of International and New Zealand Research (Bronwyn Morrison, Ministry of Justice 2009: p17) shows that approximately 40% of individuals involved New Zealand criminal justice system (in 2006) were Māori. These figures support the statement by Police Minister, Judith Collins that “the figures merely reflect the “sad fact” that Māori are over-represented in crime statistics.”

For those of you who like the statistics, then assuming a binomial model, the probability of observing 35 or more out of 88 incidents with p = 0.4, is approximately 0.47.

What do we take from this? Māori are not over-represented in the taser statistics. They occur in almost exactly the same proportion as they do in all other aspects of the criminal justice system.