More Misinformation from Freestylecyclists.org

I recently came across another excellent cycling article from Alan Davies at The Urbanist. In the article, Davies discusses claims that cycling accidents are on the rise. This is due to an increase in the cycling road toll in 2013. This may be a concern, but it’s impossible to establish a trend from one data point.

Davies briefly mentions helmet legislation, but notes it isn’t relevant to the current discussion (and I agree). However, in the comments, I found a few troubling responses regarding that topic. Strewth states

But we also know from analysis done in the 1990s that among cyclists, the decline in non-head injuries over this period was as great or greater than the decline in head injuries.

No citation or link is given to support this claim. This comment is strange since a previous study of mine estimates a 35% drop in cycling head injury hospitalizations with the NSW helmet law while arm and leg injuries dropped by only 11% and 6% respectively. A more comprehensive response was left by Linda Ward.

Another comment by Nik Dow states

A fact-based explanation is linked (see “detailed explanation”) http://www.freestylecyclists.org/do-helmet-laws-make-much-difference/ and covers the introduction of demerit points and ramping up of speed and red-light cameras.

The link takes you to Freestylecyclists.org, an anti-helmet organization I’ve discussed previously. My previous post pointed to misinformation presented by national spokesperson Alan Todd and the given link is more of the same.

The following plot of cycling and pedestrian fatalities from 1980-2006 in Australia is given and the anonymous author concludes declines in cycling deaths were mostly “due to massive ramping up of speed and red-light cameras, together with the introduction of demerit points.” I’ve assumed this is due to the pedestrian and cycling time series being placed on top of each other.

FSC-Chart-Post-Re-BuildsFA_Pedestrians

What is problematic here is this is not an accurate representation of the fatality data (much of it can be found here). The author has apparently rescaled the pedestrian time series to get them to overlay. As I’ve discussed before, this a strategy too often used to mislead actual temporal relationships of data. In the comments, Davies also questions the accuracy of this figure.

Plotting both time series on the same graph is problematic here since pedestrian fatalities have historically dwarfed those for cyclists (in 1989 they were 501 and 98 respectively). One method to address this problem is to index the time series to a starting value. An advantage of this approach is you’re able to plot trends without distorting scales; however, a disadvantage is the actual data is not being presented and each data point is a comparison with some starting value.

Below is such a plot for the period 1971 to 2013 for cycling and pedestrian fatalities in Australia (the vertical red lines represent the first and last helmet law dates in Australia).

Aust_Cyc_Ped_Fatal

This looks virtually nothing like the Freestylecyclist.org plot. Relative to 1971, pedestrian fatalities have steadily declined over the next 40 years, while cycling fatalities were flat up to the 1990, followed by a substantial decline by 1992 and flat thereafter. This does not suggest declines in cycling fatalities are associated with general road safety improvements such as demerit points or speed cameras. Further, the lack of temporal agreement between cycling and pedestrian fatalities prior to 1990 raises questions regarding pedestrians as a suitable comparator to cyclists.

It is not appropriate to make decisions about trends from eye-balling a figure, so I fit an interrupted time series model to this data. The Poisson model I used was

log(\mu)=\beta_0+\beta_1TIME+\beta_2LAW+\beta_3TIME*LAW

where TIME is centered at 1991 and LAW is an indicator that takes on the value 1 for years 1991 onwards and 0 everywhere else. My results suggest no pre-1990 trend for cycling fatalities (p=0.84) and a 42% decline in cycling fatalities at 1991 (p<0.001). Residual plots indicate good overall fit, although the 2013 observation may exhibit high leverage.0000

Aust_Fatal_Fit_Plots

Something profound happened for cycling fatalities in Australia between 1990-1992. It is often argued helmet legislation deters cycling; however, this is an argument I largely reject due to conflicting evidence from data of low quality.[1] Still, this does not necessarily indicate helmet legislation is a causal factor in lowering cycling fatalities. Yet, this analysis does rule out general road safety interventions as a causal influence proposed by Freedomcyclists.org.

  1. Olivier, J., Grzebieta, R., Wang, J.J.J. & Walter, S. (2013). Statistical Errors in Anti-Helmet Arguments. Australasian College of Road Safety Conference.
Advertisements

Freestyle Cyclists and More Misinformation about Helmets

When someone comments or cites any of my research, I take it on faith the person has actually read and understands my work. I am often reminded how naive I am as evidenced by a recent interview on FIVEaa radio in Adelaide.

Due to the ongoing Velo-City conference in Adelaide, there is keen interest in cycling issues and attention has turned to bicycle helmet legislation (as it seems every conversation about cycling devolves to this topic). In conjunction with the conference events, the Freestyle Cyclist’s Group have organized a “unhelmeted cycling protest” ride from the Adelaide CBD to the beach. In the interview, spokesman Alan Todd gave his views on helmet legislation – much of it is tired criticisms without much of an evidence base. When asked to comment about a 2011 article[1] I co-authored with former student Scott Walter, Todd said

If Tim (Churches) had stretched out his study period another six months either side he would’ve found a story. What you find in Australia was is the head injury rate went down sharply when helmets were mandated but then the level of cycling went down even more.

I found that troubling because we actually extended the time after the NSW helmet law as part of a sensitivity analysis (it is impossible to go the other direction because usable data does not exist). We discuss this in our paper where we state (emphasis added)

Both models using arm injury rates as the comparison showed  approximately parallel trends in the post-law period while the models using leg injury rates as a comparison exhibited contrasting trends. With the inclusion of three or five years of post-law data these trends tended to approach stability. With 18 months of post-law data, trends ranged from −7.5% to 21.2% per year, whereas with five years of data the range of trends was −0.6 to 9.2. For all four models, a test of the Pearson’s chi-square statistic was nonsignificant at the 0.05 level indicating a reasonable fit.

A few years ago, I became concerned that people like Todd and anti-helmet advocates were distorting the data and analyses discussed in our paper (and the work of others in the broader scientific community that find any evidence positive of helmets). In particular, as it relates to my work, it’s the belief cycling head injuries increased after the NSW helmet law. As I note above, we actually directly addressed that issue in our paper, but that analysis seems (back then and now) to be routinely ignored.

Besides this being a clear case of cherry-picking data to support a cause, it troubles me when someone dismisses research because the researcher didn’t do the analysis that person wanted (whether coming from a biased position or not). This is not sufficient grounds to discredit someone’s research (keep in mind we actually performed the analysis Todd criticized us for). It just means we don’t know the outcome until that analysis is performed using relevant data (this does assume the criticism is legitimate).

Even though we knew cycling head injuries were relatively flat 5 years after the helmet law after an initial, abrupt drop with law, we petitioned the NSW government for all cycling injury hospitalizations since the helmet law in 1991 up to the most recently available data (which was 2010 at the time). This allowed us to estimate the trends in cycling head injuries after the law and to test whether the benefits of the law were maintained in the long term.

The results were staggering. Not only did head injuries remain low over the next 20 years, but they diverged from limb injuries during that time. We then compared that with estimates of cycling participants (available from 2001-2010), and found the increase in limb injuries coincided with increases in cycling, while head injuries steadily declined. This evidence is completely contrary to Todd’s comment.

Walter-Olivier Plot

The same oversight was committed by Prof Chris Rissel of The University of Sydney in his rejoinder to our paper[2] – a paper in which he was allowed to cite his own retracted paper as evidence against our own – where he states a longer post-law period would “significantly reduce any impact of helmet legislation in the regression analysis.” Note that Rissel is the NSW spokesperson for Freestyle Cyclists.

We even pointed this out in our response to Rissel’s rejoinder published last year[3], yet people like Todd seem to ignore research that doesn’t align with their advocacy position. I think this is troubling as someone like Todd is actively trying to shape public health policy.

Todd also commented about other research stating

If you have a prang, a helmet may offer you some protection. it probably won’t be as much as you think. The latest findings from international journal Accident Analysis and Prevention says that it’s between 5 and 15% improvement. It’s not very big.

I believe this is in reference to the re-re-re-analysis of a meta-analysis originally published in 2011. By my count, this paper was corrected twice and a third version was published as a corrigendum last year. The history of this paper is quite interesting and would make a good article just by itself. But, with regards to Todd’s comment, the paper estimates odds ratios from random effects models adjusting for possible publication bias as 0.50 (95% CI: 0.39-0.65) for head injury and 0.67 (95% CI: 0.56-0.82) when head, face and neck injuries are combined. So, the paper estimates statistically significant reductions in these injuries by 50% or 33% depending on the model. I find the latter analysis confusing as helmets are designed to protect the head and not the face or neck. None of these values are anywhere near the values quoted by Todd and he seems to have ignored the paper’s discussion which states:

With respect to head injury, the answer is clearly yes, and the re-analysis of the meta-analysis reported by Attewell et al. (2001) in this paper has not changed this answer.

Rissel has also misquoted this paper on at least two occasions (see here and here). He also claims helmets cause diffuse axonal injury which is not backed by any available evidence.

Bicycle helmet or helmet law effectiveness is certainly a controversial topic for some and, perhaps, this is a reason to have an active debate. However, the evidence presented should be factual and be the result of rigorous analysis using relevant data. Alan Todd and the Freedom Cyclists have presented neither. If you’re interested, they seem to have a new website that’s big on flash and small on substance.

Later during the interview, neurosurgeon John Close, when asked to comment about the issues Todd raises, called him an “idiot”. Although I believe name-calling is counterproductive in polite discussion, I agree with the sentiment.

  1. Walter, S.R., Olivier, J., Churches, T., & Grzebeita, R. (2011). The impact of compulsory cycle helmet legislation on cyclist head injuries in New South Wales,  Australia. Accident Analysis and Prevention, 43, 2064–2071.
  2. Rissel, C. (2012). The impact of compulsory cycle helmet legislation on cyclist head injuries in New South Wales, Australia: A rejoinder. Accident Analysis and Prevention, 45, 107-109.
  3. Walter, S.R., Olivier, J., Churches, T. & Grzebieta, R. (2013). The impact of compulsory helmet legislation on cyclist head injuries in New South Wales, Australia: A response. Accident Analysis and Prevention, 52, 204-209.