Posts from November 2011 (34)

As you may remember, there was a flurry of headlines recently in the Kiwi media about the new anti-clotting drug dabigatran(Pradaxa) — it’s not often than Pharmac is accused of pushing too hard for an expensive new substitute instead of  an old, cheap drug.  Now, in Australia, the reverse is happening: the Goverment has refused to list dabigatran for subsidy (going against its own experts), and the media is reporting this decision as a danger to Australian patients (with help from the manufacturer’s lobbyists).

Warfarin (yes, rat poison) is a very safe and effective anti-clotting drug if it’s given at exactly the right dose. Unfortunately, the right dose varies between people, and changes depending on almost anything else (food, medications, herbals, alcohol) you might consume.  Keeping the dose right requires frequent blood tests, which are inconvenient and expensive, and even then the dosing is often imperfect unless you and your physician are truly obsessive.  But there hasn’t been anything else that came close to the safety and efficacy of warfarin.

Now there is a rush of new medications coming out that selectively disrupt a very late stage in clot formation: as well as dabigatran there are apixaban, rivaroxaban, and betrixaban.  These don’t have the variable dosage of warfarin, so it should be possible to just prescribe them and stop worrying: a big improvement in convenience and saving in cost.  On the other hand, these drugs are new (and so less well understood), and unlike warfarin there is no antidote and no convenient lab test for overdose.  And they are quite  expensive, even with Pharmac’s well-known haggling powers.

In this case I think the evil multinational drug company is right and the Australian Health Minister is wrong: these new drugs really are safer and more effective — they were better than warfarin in a large randomized trial, where warfarin dosing is likely to be better than in free-range patients — and the reduced need for blood tests will pay for quite a bit of the increased cost.  Still, it’s hard to avoid the suspicion that the main urgency in this debate is to get dabigatran into widespread use before its competitors make it on to the market…

Stuff is reporting on this year’s “Global Gender Gap Report” from the World Economic Forum.  New Zealand has improved very slightly over last year in its gender equality score, but has lost 5th place in the rankings to Ireland, which had greater improvement. The biggest improver was Yemen, whose situation went from dire to merely awful, though it is still last. Colombia, Nepal, and Sri Lanka had the biggest falls.

One thing Stuff did get wrong: the rankings are based on a complicated “gender gap score”, not just on pay equity.  New Zealand still does slightly better on pay equity than Ireland, but Ireland does better on having women in senior positions.

TV3 on Tuesday reported that “early results in a poll suggest Labour’s Phil Goff won the debate with Prime Minister John Key last night.” The poll was by RadioLIVE and Horizon.

The TV piece concluded by lambasting a recent One News text poll, saying: “A One News text poll giving the debate to Mr Key 61-39 has been widely discredited, since it cost 75c to vote.”

This text poll should be lambasted if it is used to make inference about the opinions of the population of eligible willing voters. Self-selection is the major problem here: those who can be bothered have selected themselves.

The problem: there is no way of ascertaining that this sample of people is a representative sample of willing voters. Only the interested and motivated, who text message, have answered and, clearly, the pollsters do not have information on the not-so-interested and motivated non-texters.

The industry standard is to randomly select from all eligible willing voters and to adjust for non-response.  The initial selection is random and non-response is reduced as much as possible. This is to ensure the sample is as representative of the population as possible. The sample is collected via a sampling frame which, hopefully, is a comprehensive a list of the population you wish to talk to.  For CATI polls the sample frame is domestic landline (traditional) phone numbers.

With election polls, as landlineless voters have been in lower socio-economic groups which tend to have lower voter participation, this has not been so much of a problem.  The people we wish to talk to are eligible willing voters – so the polls have not been unduly biased by not including these landlineless people.

However, as people move away from landlines to mobile phones, CATI interviewing has been increasingly criticised. Hence alternatives have been developed, such as panel-polls and market prediction polls like IPredict – and the latter will be the subject of a post for another day.

But let’s go back to the Horizon panel poll mentioned above. It claims that it’s to be trusted as it has sampled from a large population of potential panellists who have been recruited and can win prizes for participation. The Horizon poll adjusts for any biases by reweighting the sample so that it’s more like the underlying New Zealand adult population – which is good practice in general.

However, the trouble is this large sampling frame of potential panellists have been self-selected. So who do they represent?

To illustrate, it’s hard to imagine people from more affluent areas feeling the need to get rewards for being on a panel. Also, you enrol via the internet and clearly this is biased towards IT-savvy people. Here the sampling frame is biased, with little or no known way to adjust for any biases bought about from this self-selection problem. They may be weighted to look like the population but they may be fundamentally different in their political outlook.

Panel polls are being increasingly used by market researchers and polling companies. With online panel polls it’s easier to obtain samples, collect information and transfer it, without all the bother involved in traditional polling techniques like CATI.

I believe the industry has been seduced by these features at the expense of representativeness – the bedrock of all inference. Until such time as we can ensure representativeness, I remain sceptical about any claims from panel polls.

I believe the much-maligned telephone (CATI) interviewing, which is by no means perfect, still remains the best of a bad lot.

There are many ways that sophisticated pollsters can tilt the result of a survey: biased samples, carefully framed sets of questions, creative subsets of the data.

Based on a survey of protesters at Occupy Wall Street, pollster Doug Schoen wrote ” it comprises an unrepresentative segment of the electorate that believes in radical redistribution of wealth, civil disobedience and, in some instances, violence. . . .”.

The actual data from that very survey:

What would you like to see the Occupy Wall Street movement achieve? {Open Ended}
35% Influence the Democratic Party the way the Tea Party has influenced the GOP
4% Radical redistribution of wealth
5% Overhaul of tax system: replace income tax with flat tax
7% Direct Democracy
9% Engage & mobilize Progressives
9% Promote a national conversation
11% Break the two-party duopoly
4% Dissolution of our representative democracy/capitalist system
4% Single payer health care
4% Pull out of Afghanistan immediately
8% Not sure

Andrew Gelman says about this poll “as I like to remind students, the simplest way to lie with statistics is to just lie!”.