Posts filed under Surveys (188)

Stuff is using a survey from Weight Watchers as a hook for a story

Four out of five Generation Z youths – those born from 1990 – don’t use any fresh ingredients in their daily evening meals and 39 per cent are unable to correctly identify staple veggies such as leek or zucchini.
 
The new Weight Watchers survey of 1000 people, studying Kiwis’ eating attitudes and food knowledge, has been released today as part of an initiative aimed at tackling obesity.
 
It found New Zealanders were largely sedentary, have lost touch with the importance of savouring mealtimes, and younger generations’ cooking skills were at risk.

As usual, there’s no indication of whether this is a real survey or how it was done, and Weight Watchers itself doesn’t have anything on its website.  It’s not even clear whether this was a survey just of ‘Generation Z’ or of the country as a whole. If the formerlatter, it can’t be very reliable about younger generations, if the latterformer it can’t tell us whether New Zealanders are ‘largely sedentary’.  Of course, if you don’t really care whether the numbers mean anything, it doesn’t matter much.

Also (as the story partially addresses), it’s not clear that there was ever a Golden Age when 18 year olds all cooked nutritious meals from raw ingredients, and while it may be true that fewer Kiwis can identify leeks than in the 1950s, I bet more can identify bok choy. Not that you actually need to be able to identify a leek: when you go to the shop, they’re the things labelled “leeks”, or perhaps “leek’s”.

[Update: Rachel found a presentation of the results. It shows that the 34% of Generation Z who eat vegetables with their evening meal compares to a massive 40% for ‘Silent Generation’, presumably the largest difference between generations they could find.  Not impressed.]

From Frances Woolley at the economics blog ‘Worthwhile Canadian Initiative’

Canada’s 2005 National Graduates Survey asked respondents the following question: “Compared to the rest of your graduating class in your field(s) of study, did you rank academically in the top 10? Below the top 10% but in the top 25%…”

Usually when I write about a survey story that gives no details, it’s because the survey is bogus.  Today, Stuff has a story about a genuine survey that says nice things about Kiwis. The World Giving Index has NZ in second place, with a score of 57%, and Stuff says this means “At least 57 per cent of Kiwis give either their money or time to some sort of charity.” 

In fact, the World Giving Index is an average over three questions from the Gallup World Poll, a survey that does face-to-face interviews with a random sample of people from every country in the world (with a few exceptions where it’s not feasible).  The questions are

Have you done any of the following in the past month? 

• „ Donated money to a charity? 
• „ Volunteered your time to an organisation? 
• „ Helped a stranger, or someone you didn’t know who needed help? 

NZ scores 63%, 39%, 69% respectively, so the 57% giving money or time to a charity reported by Stuff is an underestimate.

The Herald tells us

Having a purple bedroom could enhance your sex life, according to a new study.

with red and sky blue coming in next in line.  Beige/white/grey colours are apparently at the bottom of the list. We aren’t told anything about how the survey was done, but we are told it was sponsored by UK retailer Littlewoods, who would be happy to help you redecorate.   If the association were true, it would almost certainly be in the opposite direction, but there isn’t much reason for us to believe it’s true.

In fact, it doesn’t seem as if even Littlewoods really believe their survey. If you look at their decorating suggestions pages, the bedrooms are mostly pale and beigey, with the colours suggested for kids’ bedrooms.

Stuff has a story about another sex survey, with the lead

It appears almost a given that men’s sexual fantasies will differ from those of women and now there’s a survey to prove it.

The link is to the Huffington Post. There’s another link further down the story to the Armenian Medical Network, which, to be fair, is running almost exactly the original press release, including the typos (the respondents are described as 49.6% mend and 0.4% women).

So what was this survey?  It appears to have been a serious attempt to find stuff out, conducted by psychology researcher at the University of Granada.  It hasn’t been published, which is a bad sign and makes it hard to get details, but the press release implies that it has been accepted for publication.

The survey was conducted by questionnaire, and “A number of provincial ongoing training centers, adult education centers, Granada local library and several University of Granada and Universidad Complutense schools collaborated in this study”.  That suggests it wasn’t a random sample. The extremely equal representation of men and women  suggests perhaps a quota sample — interviewers choosing people to ask, from a set of categories. That’s not ideal, but it’s better than just putting a link out on the internet, as was done by the last sex survey we examined, and as the Spanish researchers are doing for their next study according to the press release.

Anyway, what really attracted my attention when I saw the press release was the title:

Republicans and Democrats are hearing very different news about the economy:

In this example there are more possible explanations than last time:

• They really are hearing different news, because local conditions vary.  This one can’t really be true, because the geographical polarisation of voters isn’t strong enough
• They really are hearing different news, because they get it from different sources. In some ways that’s the most worrying possibility — a massive breakdown in the effectiveness of journalism.
• They are hearing the same news, but it has different implications.  For example, perhaps Republicans think the prospect of higher tax rates on income above $250,000 is bad economic news and Democrats think it is good economic news.  I don’t think this can explain such a big and recent difference.
• They are exposed to the same news, but only really hear the bits that confirm their beliefs.  Quite likely, and worrying.
• They don’t really believe what they are saying. The most positive interpretation, except if you’re in the survey business.

Stuff’s story starts out

Half of all Kiwis like to change jobs regularly, with 51 per cent of people surveyed by online recruiter Seek starting their current role less than two years ago.

I don’t see why “like to change jobs regularly” is remotely the same as “have changed jobs recently”, and I’m sure people who were laid off in the recession or lost jobs to the ChCh quake would agree.  But, more importantly, Stats New Zealand collects real data on changes of employment, so why not use that rather than a non-random sample from what Seek has in previous years described as “a broad online audience”. 

In fact, Seek doesn’t do too badly in estimating: the true figure is 54%, with the difference being only about twice the margin of error for a random  sample of the size of their previous years’ surveys.

As usual, I’m having to rely on previous years’ press releases for any methodology information, since Seek hasn’t posted this year’s one and Stuff isn’t giving any details.

A respectable survey in the US state of Ohio reports that Obama is still polling well. But it’s one of the secondary questions that is especially interesting: from Ezra Klein’s blog at the Washington Post

PPP asked voters who they thought deserved more credit for the killing of Osama bin Laden: Obama or Romney. 63 percent said Obama, 31 percent weren’t sure, and 6 percent said Romney.

The results for Republican voters were even more astonishing. 38 percent said Obama, 47 percent weren’t sure, and 15 percent said Romney. What the heck is going on?

As they go on to explain, this is an example of one of the big problems with opinion polls in situations where the respondents know what you are doing.  The tendency is for people to answer in the way that represents their political affiliations rather than their actual opinions.

The Herald’s clicky poll accompanying the story on Kiwi self-satisfaction is a nice illustration of the unreliability of bogus polls (they’re only using it for entertainment, so that’s ok).

If the clicky poll was a real random sample, its margin of error would be about 1%. It isn’t, and it’s not.

The new-look Herald has a good survey-based story on where people want to live: they did both in-depth interviews with a moderate number of people, and a reasonably good quantitative survey (online panel-based DigiPoll).

The results say that most people in New Zealand want to live here.  In a sense that’s not surprising — they are living here — and that’s especially true for Australia as an alternative, since it’s not that difficult to move.

A similar US survey a couple of years ago, by the Pew Research Center, didn’t even bother to ask about other countries, just about other places to live in the US.  It’s still a reasonable comparison: different parts of the US are more different than many countries, and some don’t even seem to be in the same universe as each other.  In the Pew survey, nearly half of respondents said they wanted to live in a different kind of community from where they currently live.  Several major cities had nearly 40% approval rating nationwide as somewhere people would like to live: Denver topped the list at 43%, and my previous home in Seattle was at 38%.

So, even accounting for the natural bias towards saying you have chosen the right place to live, Kiwis do seem happy with their home.