Hard, soft, and real

In clinical trials we make two important distinctions between measurements. There are ‘hard’ and ‘soft’ outcomes — ‘hard’ ones are objectively and reproducibly measurable, ‘soft’ ones have some subjectivity and observer bias.  There are also ‘surrogate’ and ‘real’ or ‘patient-centered’ outcomes. ‘Real’ outcomes are what we care about; ‘surrogate’ outcomes are things we measure because we can measure them well and we expect them to correlate with real outcomes. Hard and soft outcomes are valuable; real and surrogate outcomes are valuable; you don’t want to confuse them.

There’s a story on NewsHub headlined Bisexual men are real, study finds. One of the researchers had previously doubted this, but has now been convinced, and has paper in PNAS about an analysis combining data from many previous studies.  These studies involve wiring someone’s penis up to detect arousal and then showing him erotic images.  The claim is that these measurements are objective

If men who self-report Kinsey scores in the bisexual range indeed have relatively bisexual arousal patterns, then both Minimum Arousal and the Bisexual Arousal Composite should show an inverted U-shaped distribution across the Kinsey range (i.e., men who self-identify as 0 [exclusively heterosexual] and 6 [exclusively homosexual] should have the lowest scores for these variables; men in intermediate groups should have greater values, with the peak resting at a Kinsey score of 3); the Absolute Arousal Difference should show a U-shaped distribution (i.e., exclusively heterosexual and exclusively homosexual men should have lower values than bisexual-identified men).

The reason for emphasising these measurements is that they doesn’t completely trust self-report (while agreeing it is valuable)

However, because the scale relied on self-reports, results could not provide definitive evidence for bisexual orientation. For example, surveys have shown that a large proportion of men who identify as gay or homosexual had gone through a previous and transient phase of bisexual identification 

I don’t think anyone (whatever their opinion on bisexuality) would deny that men who lie about sex are real. The problem is treating the physical arousal measurements as basically definitive of bisexuality.  In the clinical trials terminology, the arousal measurement is a relatively hard outcome, but it is a surrogate outcome.

With modern data science (and sufficiently dodgy ethics) there would be other surrogate outcomes that someone has probably explored.  Are there a significant number of men on Tinder who swipe right for both male and female profiles?  Are there many PornHub accounts of men who watch both straight and gay porn? Are there men who have shared a one-bedroom home with both men and women over time?  All of these are clearly reductive: they would give you one-dimensional information about bisexuality, but they are measuring different things and there’s no reason to expect they would agree on how common it is.  The same is true for physiological arousal.  Measuring it can be valuable; the demographics of physiological arousal can be a valid area of study; but it can’t answer the yes/no question.

Some men claim to be attracted to both men and women, and behave as if their claims are true. It turns out, according to this paper, that for some of these men the physiological measurements of arousal show the relationships that you’d expect.  If there weren’t any men whose physiological measurements of arousal show those relationships, that would be an interesting fact, but the real question would be why the measurements don’t fit with the phenomenon of bisexuality.  If you think of this paper as just trying to answer a question about physiological arousal then, ok, that’s the question it tries to answer. And in fact one of the researchers is quoted further down in the NewsHub story saying

“It has always been clear that bisexual men exist in terms of self-identity and behaviour, but many, including myself, were sceptical about their ability to be sexually aroused to both men and women.” 

Contrast that, though, with the paper’s “Significance” section, which starts out

“There has long been skepticism among both scientists and laypersons that male bisexual orientation exists.”

Or with the second sentence of the press release:

“The existence of male bisexuality is contested, with skeptics claiming that men who self-identify as bisexual are actually either homosexual or heterosexual.”.

Or with the title of the research paper itself

Robust evidence for bisexual orientation among men

The stretching of the study findings to the headline “Bisexual men are real, study finds” can’t just be blamed on the media.

When we talk about whether Alexander the Great or Shakespeare was bisexual, there are difficulties in even agreeing on the concept over centuries or millennia of social distance.  But I think most people would agree there’s more to the question than what would have happened if you wired them up to a machine and showed them porn.