Freestyle Cyclists and More Misinformation about Helmets

When someone comments or cites any of my research, I take it on faith the person has actually read and understands my work. I am often reminded how naive I am as evidenced by a recent interview on FIVEaa radio in Adelaide.

Due to the ongoing Velo-City conference in Adelaide, there is keen interest in cycling issues and attention has turned to bicycle helmet legislation (as it seems every conversation about cycling devolves to this topic). In conjunction with the conference events, the Freestyle Cyclist’s Group have organized a “unhelmeted cycling protest” ride from the Adelaide CBD to the beach. In the interview, spokesman Alan Todd gave his views on helmet legislation – much of it is tired criticisms without much of an evidence base. When asked to comment about a 2011 article[1] I co-authored with former student Scott Walter, Todd said

If Tim (Churches) had stretched out his study period another six months either side he would’ve found a story. What you find in Australia was is the head injury rate went down sharply when helmets were mandated but then the level of cycling went down even more.

I found that troubling because we actually extended the time after the NSW helmet law as part of a sensitivity analysis (it is impossible to go the other direction because usable data does not exist). We discuss this in our paper where we state (emphasis added)

Both models using arm injury rates as the comparison showed  approximately parallel trends in the post-law period while the models using leg injury rates as a comparison exhibited contrasting trends. With the inclusion of three or five years of post-law data these trends tended to approach stability. With 18 months of post-law data, trends ranged from −7.5% to 21.2% per year, whereas with five years of data the range of trends was −0.6 to 9.2. For all four models, a test of the Pearson’s chi-square statistic was nonsignificant at the 0.05 level indicating a reasonable fit.

A few years ago, I became concerned that people like Todd and anti-helmet advocates were distorting the data and analyses discussed in our paper (and the work of others in the broader scientific community that find any evidence positive of helmets). In particular, as it relates to my work, it’s the belief cycling head injuries increased after the NSW helmet law. As I note above, we actually directly addressed that issue in our paper, but that analysis seems (back then and now) to be routinely ignored.

Besides this being a clear case of cherry-picking data to support a cause, it troubles me when someone dismisses research because the researcher didn’t do the analysis that person wanted (whether coming from a biased position or not). This is not sufficient grounds to discredit someone’s research (keep in mind we actually performed the analysis Todd criticized us for). It just means we don’t know the outcome until that analysis is performed using relevant data (this does assume the criticism is legitimate).

Even though we knew cycling head injuries were relatively flat 5 years after the helmet law after an initial, abrupt drop with law, we petitioned the NSW government for all cycling injury hospitalizations since the helmet law in 1991 up to the most recently available data (which was 2010 at the time). This allowed us to estimate the trends in cycling head injuries after the law and to test whether the benefits of the law were maintained in the long term.

The results were staggering. Not only did head injuries remain low over the next 20 years, but they diverged from limb injuries during that time. We then compared that with estimates of cycling participants (available from 2001-2010), and found the increase in limb injuries coincided with increases in cycling, while head injuries steadily declined. This evidence is completely contrary to Todd’s comment.

Walter-Olivier Plot

The same oversight was committed by Prof Chris Rissel of The University of Sydney in his rejoinder to our paper[2] – a paper in which he was allowed to cite his own retracted paper as evidence against our own – where he states a longer post-law period would “significantly reduce any impact of helmet legislation in the regression analysis.” Note that Rissel is the NSW spokesperson for Freestyle Cyclists.

We even pointed this out in our response to Rissel’s rejoinder published last year[3], yet people like Todd seem to ignore research that doesn’t align with their advocacy position. I think this is troubling as someone like Todd is actively trying to shape public health policy.

Todd also commented about other research stating

If you have a prang, a helmet may offer you some protection. it probably won’t be as much as you think. The latest findings from international journal Accident Analysis and Prevention says that it’s between 5 and 15% improvement. It’s not very big.

I believe this is in reference to the re-re-re-analysis of a meta-analysis originally published in 2011. By my count, this paper was corrected twice and a third version was published as a corrigendum last year. The history of this paper is quite interesting and would make a good article just by itself. But, with regards to Todd’s comment, the paper estimates odds ratios from random effects models adjusting for possible publication bias as 0.50 (95% CI: 0.39-0.65) for head injury and 0.67 (95% CI: 0.56-0.82) when head, face and neck injuries are combined. So, the paper estimates statistically significant reductions in these injuries by 50% or 33% depending on the model. I find the latter analysis confusing as helmets are designed to protect the head and not the face or neck. None of these values are anywhere near the values quoted by Todd and he seems to have ignored the paper’s discussion which states:

With respect to head injury, the answer is clearly yes, and the re-analysis of the meta-analysis reported by Attewell et al. (2001) in this paper has not changed this answer.

Rissel has also misquoted this paper on at least two occasions (see here and here). He also claims helmets cause diffuse axonal injury which is not backed by any available evidence.

Bicycle helmet or helmet law effectiveness is certainly a controversial topic for some and, perhaps, this is a reason to have an active debate. However, the evidence presented should be factual and be the result of rigorous analysis using relevant data. Alan Todd and the Freedom Cyclists have presented neither. If you’re interested, they seem to have a new website that’s big on flash and small on substance.

Later during the interview, neurosurgeon John Close, when asked to comment about the issues Todd raises, called him an “idiot”. Although I believe name-calling is counterproductive in polite discussion, I agree with the sentiment.

  1. Walter, S.R., Olivier, J., Churches, T., & Grzebeita, R. (2011). The impact of compulsory cycle helmet legislation on cyclist head injuries in New South Wales,  Australia. Accident Analysis and Prevention, 43, 2064–2071.
  2. Rissel, C. (2012). The impact of compulsory cycle helmet legislation on cyclist head injuries in New South Wales, Australia: A rejoinder. Accident Analysis and Prevention, 45, 107-109.
  3. Walter, S.R., Olivier, J., Churches, T. & Grzebieta, R. (2013). The impact of compulsory helmet legislation on cyclist head injuries in New South Wales, Australia: A response. Accident Analysis and Prevention, 52, 204-209.
Advertisements

New Zealand Cycling Fatalities and Bicycle Helmets

A colleague sent me an assessment of cycling fatalities in New Zealand. The report’s author is Dr Glen Koorey of the University of Canterbury. He’ll be one of the keynote speakers at the upcoming Velo-City Conference in Adelaide. In particular, I was tasked to comment about his section regarding bicycle helmets as they, in part, now form the basis of the Wikipedia page on Bicycle Helmets in New Zealand.

In the report, Koorey states

Only nine victims were noted as not wearing a helmet, similar to current national helmet-wearing rates (92%). This highlights the fact that helmets are generally no protection to the serious forces involved in a major vehicle crash; they are only designed for falls. In fact, in only one case did the Police speculate that a helmet may have saved the victim’s life. There is a suspicion that some people (children in particular) have been “oversold” on the safety of their helmet and have been less cautious in their riding style as a result.

On the surface, he has a point based on independence for probabilities. In mathematical terms, Koorey is stating

P(helmet | fatality) \approx P(helmet)

which is, by definition, independence (if they are equal). So, if the helmet wearing proportion among fatalities is equal to that in population, then helmet wearing is independent of fatality.

As I see it, the problem is in the interpretation as it is not a pure measure of helmet effectiveness. Helmets are a directed safety intervention, so they won’t protect body parts other than the head and you can certainly die from other injuries. It could very well be that helmet wearing is independent of fatalities, but the the sheer force of the collision makes other serious (and possibly fatal) injuries more likely negating any benefit to helmet wearing.

I searched through the publicly available data (found here) and asked around about what’s available in the complete data. In the end, there’s not enough information to identify location or severity of injuries. If we had all the data, a more appropriate probability to investigate would be

P(helmet | \hbox{fatality due to head injury}) = P(helmet)

When looking at the reported data, however, Koorey’s claim the proportion of fatalities wearing a helmet is “similar to current national helmet‐wearing rates (92%)” doesn’t appear justified.

First, he states there were 84 cycling fatalities between 2006-2012 in New Zealand. Of these, about 10% did not have information about helmet wearing. So, there is information on 76 fatalities and 9 of those were not wearing helmets. This gives us the proportion of non-helmet wearers among fatalities of 11.84% (9/76). This is not an estimate since this figure comes from all cycling fatalities in New Zealand.

Koorey wants to compare this to estimates of helmet wearing in New Zealand. Over this time frame, I compute a yearly average helmet wearing rate of 92.57%. So, the proportion of cyclists not wearing helmets is 7.43% during that time. This data could then be summarized by a 2 \times 2 table as

Helmet
Yes No
Death Yes a b
No c d

From the data available, we do know a=67, b=9, \frac{c}{c+d}=0.9257 and \frac{d}{c+d}=0.0743. We would like to compute the risk of death for those wearing helmets versus those that do not; however, this is not possible using this summary data as we don’t really know how many cyclists there are.

Instead, we can compute the odds ratio (OR) which is a good estimate of relative risk for rare events (cycling deaths are certainly rare). The odds ratio is

OR=\dfrac{ad}{bc}=\dfrac{a\frac{d}{c+d}}{b\frac{c}{c+d}}=0.598

If helmet wearing were identical among fatalities and the general population, as Koorey has suggested, the odds ratio would be 1. Instead of being similar, the risk of death is 40% less among helmeted NZ cyclists versus those without a helmet. This figure is consistent with the latest re-re-analysis of a meta-analysis from case-control studies, although this is likely a conservative figure since head (or any other) injuries were not identified.

Statistical significance would be hard to come by here considering we don’t have the exact counts of cyclists from those surveys (or from the general population). However, the asymptotic variance of the log(OR) is

\widehat{var}(log(OR)) \approx 1/a + 1/b + 1/c + 1/d

The last available helmet use survey came from over 4600 cyclists (that is 7*4600 over the study period). Since this is such a sizable number, the last two terms of the variance formula do not contribute much.

Using only the fatalities in the variance formula gives us an asymptotic confidence interval for the odds ratio of

OR\times e^{\pm 1.96 \times s.e.} = (0.298, 1.198)

where the s.e. = \sqrt{1/a + 1/b} (this assumes both 1/c and 1/d are small). Note this result is not statistically significant; however, this is due to having relatively few cycling fatalities (which is good and having less would be better).

There’s also the issue regarding the effect of missing data. One method is to recompute the odds ratio assuming all missings did not wear helmets and repeat assuming all missings did wear helmets giving a range of possible values. The odds ratios are 0.316 and 0.669 respectively. So, at worst, there is an estimated 33% decrease in the risk of death when wearing a helmet versus not.

Koorey’s claims are therefore not justified as the risk of death was much less among helmeted cyclists.This is even without specific information about cause of death and properly assessing helmet effectiveness to lower the risk of a fatality.

I also take issue with Koorey’s statement “This highlights the fact that helmets are generally no protection to the serious forces involved in a major vehicle crash; they are only designed for falls.” A recently published article in Accident Analysis and Prevention states

Considering a realistic bicycle accident scenario documented in the literature (Fahlstedt et al., 2012) where a cyclist was thrown at 20 km/h (i.e. 5.6 m/s which corresponds to a drop height of approximately 1.5 m), our analysis indicates that a helmeted cyclist in this situation would have a 9% chance of sustaining the severe brain and skull injuries noted above whereas an unhelmeted cyclist would have sustained these injuries with 99.9% certainty. In other words, a helmet would have reduced the probability of skull fracture or life threatening brain injury from very likely to highly unlikely.

I also published a paper last year where we found helmets reduced the odds of severe head injury by up to 74% (these were NSW cyclists hospitalised after a motor vehicle crash and reported to the police from 2001-2009). Severe injuries included “Open wound of head with intracranial injury” (S01.83), “Multiple fractures involving skull and facial bones” (S02.7), “Fracture of skull and facial bones, part unspecified” (S02.9), “Loss of consciousness [30 mins-24hrs]” (S06.03), “Loss of consciousness prolonged without return of consciousness ” (S06.05), “Traumatic cerebral oedema” (S06.1), “Diffuse brain injury” (S06.2), “Other diffuse cerebral & cerebellar injury” (S06.28), “Traumatic subdural haemorrhage” (S06.5), “Traumatic subarachnoid haemorrhage” (S06.6), “Other intracranial injuries” (S06.8), and “Intracranial injury, unspecified” (S06.9). None of these are minor injuries.

Using available data, the evidence does suggest helmet wearing mitigates cycling fatalities and serious injury. It does not appear as though the public have been oversold on the benefits of bicycle helmets.

Update: The original version focused on the relative risk of helmet wearing among fatalities and helmet wearing surveys in New Zealand. This made the wording quite strange and difficult to interpret. However, the odds ratio isn’t as problematic and is a good estimate of relative risk of death in this instance.