Criteria for criteria for mānuka honey

There’s a new proposed definition of NZ Mānuka Honey, as you may have seen. The MPI page on the topic is here; no-one is linking it, which is sad because it’s interesting if you’re enough of a nerd.

I’m not going to comment on the biochemistry or botany, but there are two statistically-interesting parts of the proposal.  First, how the statistical method for classifying honey was constructed. The document says:

A classification modelling approach (CART – classification and regression tree) was the most suitable method of analysis for determining the identification criteria for mānuka honey because:

• test results for several different attributes were available and needed to be assessed in combination;
• the identification criteria needed to be related to the attributes tested;
• the identification criteria needed to be straightforward, transparent and easily interpreted
• the outputs would enable an unknown honey sample to be authenticated as monofloral or multifloral mānuka honey.

CART is a relatively old classification method, developed in the early 1980s by adding statistical ‘pruning’ to automated methods for building decision trees. It hasn’t been the most accurate method in head-to-head prediction competitions for a long time now, but it remains very useful for basically the reasons the MPI scientists gave.  CART tends to end up with simple rules based whether a small selection of variables all or mostly exceed some thresholds, and while building a good CART prediction rule takes experience and statistical knowledge, using it doesn’t.

Using a collection of honey samples from known origins, and other information about chemical composition of the plants, a rule was developed for distinguishing mānuka honey from other NZ honeys such as kānuka or pōhutukawa, and from Leptospermum species other than mānuka. The resulting rule for monofloral (`pure’) mānuka honey is a threshold that four chemicals have to exceed, plus the presence of mānuka DNA.  For multifloral mānuka honey, the threshold for one of the four chemicals is lowered.

The second interesting aspect of the criteria is that none of the four chemicals have anything to do with real or imagined medical benefits of mānuka honey.  Methylglyoxal, the leading candidate for a somewhat mānuka-specific antimicrobial, isn’t in there.  The rule attempts to identify honey produced by bees foraging on mānuka flowers — scientists know what a mānuka flower is. It doesn’t try to identify honey that prevents miscellaneous diseases when you eat it, because no-one one knows what characteristics that honey would have, or even if it exists.

As I’ve noted before, the largest controlled trial of eating mānuka honey to prevent minor illness was conducted by a London primary school. On the other hand, people are willing to pay a lot of money for honey from NZ mānuka, and as long as MPI isn’t officially supporting the health arguments I’m definitely in favour of that money going to NZ apiarists rather than counterfeiters.