New Zealand Cycling Fatalities and Bicycle Helmets

A colleague sent me an assessment of cycling fatalities in New Zealand. The report’s author is Dr Glen Koorey of the University of Canterbury. He’ll be one of the keynote speakers at the upcoming Velo-City Conference in Adelaide. In particular, I was tasked to comment about his section regarding bicycle helmets as they, in part, now form the basis of the Wikipedia page on Bicycle Helmets in New Zealand.

In the report, Koorey states

Only nine victims were noted as not wearing a helmet, similar to current national helmet-wearing rates (92%). This highlights the fact that helmets are generally no protection to the serious forces involved in a major vehicle crash; they are only designed for falls. In fact, in only one case did the Police speculate that a helmet may have saved the victim’s life. There is a suspicion that some people (children in particular) have been “oversold” on the safety of their helmet and have been less cautious in their riding style as a result.

On the surface, he has a point based on independence for probabilities. In mathematical terms, Koorey is stating

P(helmet | fatality) \approx P(helmet)

which is, by definition, independence (if they are equal). So, if the helmet wearing proportion among fatalities is equal to that in population, then helmet wearing is independent of fatality.

As I see it, the problem is in the interpretation as it is not a pure measure of helmet effectiveness. Helmets are a directed safety intervention, so they won’t protect body parts other than the head and you can certainly die from other injuries. It could very well be that helmet wearing is independent of fatalities, but the the sheer force of the collision makes other serious (and possibly fatal) injuries more likely negating any benefit to helmet wearing.

I searched through the publicly available data (found here) and asked around about what’s available in the complete data. In the end, there’s not enough information to identify location or severity of injuries. If we had all the data, a more appropriate probability to investigate would be

P(helmet | \hbox{fatality due to head injury}) = P(helmet)

When looking at the reported data, however, Koorey’s claim the proportion of fatalities wearing a helmet is “similar to current national helmet‐wearing rates (92%)” doesn’t appear justified.

First, he states there were 84 cycling fatalities between 2006-2012 in New Zealand. Of these, about 10% did not have information about helmet wearing. So, there is information on 76 fatalities and 9 of those were not wearing helmets. This gives us the proportion of non-helmet wearers among fatalities of 11.84% (9/76). This is not an estimate since this figure comes from all cycling fatalities in New Zealand.

Koorey wants to compare this to estimates of helmet wearing in New Zealand. Over this time frame, I compute a yearly average helmet wearing rate of 92.57%. So, the proportion of cyclists not wearing helmets is 7.43% during that time. This data could then be summarized by a 2 \times 2 table as

Helmet
Yes No
Death Yes a b
No c d

From the data available, we do know a=67, b=9, \frac{c}{c+d}=0.9257 and \frac{d}{c+d}=0.0743. We would like to compute the risk of death for those wearing helmets versus those that do not; however, this is not possible using this summary data as we don’t really know how many cyclists there are.

Instead, we can compute the odds ratio (OR) which is a good estimate of relative risk for rare events (cycling deaths are certainly rare). The odds ratio is

OR=\dfrac{ad}{bc}=\dfrac{a\frac{d}{c+d}}{b\frac{c}{c+d}}=0.598

If helmet wearing were identical among fatalities and the general population, as Koorey has suggested, the odds ratio would be 1. Instead of being similar, the risk of death is 40% less among helmeted NZ cyclists versus those without a helmet. This figure is consistent with the latest re-re-analysis of a meta-analysis from case-control studies, although this is likely a conservative figure since head (or any other) injuries were not identified.

Statistical significance would be hard to come by here considering we don’t have the exact counts of cyclists from those surveys (or from the general population). However, the asymptotic variance of the log(OR) is

\widehat{var}(log(OR)) \approx 1/a + 1/b + 1/c + 1/d

The last available helmet use survey came from over 4600 cyclists (that is 7*4600 over the study period). Since this is such a sizable number, the last two terms of the variance formula do not contribute much.

Using only the fatalities in the variance formula gives us an asymptotic confidence interval for the odds ratio of

OR\times e^{\pm 1.96 \times s.e.} = (0.298, 1.198)

where the s.e. = \sqrt{1/a + 1/b} (this assumes both 1/c and 1/d are small). Note this result is not statistically significant; however, this is due to having relatively few cycling fatalities (which is good and having less would be better).

There’s also the issue regarding the effect of missing data. One method is to recompute the odds ratio assuming all missings did not wear helmets and repeat assuming all missings did wear helmets giving a range of possible values. The odds ratios are 0.316 and 0.669 respectively. So, at worst, there is an estimated 33% decrease in the risk of death when wearing a helmet versus not.

Koorey’s claims are therefore not justified as the risk of death was much less among helmeted cyclists.This is even without specific information about cause of death and properly assessing helmet effectiveness to lower the risk of a fatality.

I also take issue with Koorey’s statement “This highlights the fact that helmets are generally no protection to the serious forces involved in a major vehicle crash; they are only designed for falls.” A recently published article in Accident Analysis and Prevention states

Considering a realistic bicycle accident scenario documented in the literature (Fahlstedt et al., 2012) where a cyclist was thrown at 20 km/h (i.e. 5.6 m/s which corresponds to a drop height of approximately 1.5 m), our analysis indicates that a helmeted cyclist in this situation would have a 9% chance of sustaining the severe brain and skull injuries noted above whereas an unhelmeted cyclist would have sustained these injuries with 99.9% certainty. In other words, a helmet would have reduced the probability of skull fracture or life threatening brain injury from very likely to highly unlikely.

I also published a paper last year where we found helmets reduced the odds of severe head injury by up to 74% (these were NSW cyclists hospitalised after a motor vehicle crash and reported to the police from 2001-2009). Severe injuries included “Open wound of head with intracranial injury” (S01.83), “Multiple fractures involving skull and facial bones” (S02.7), “Fracture of skull and facial bones, part unspecified” (S02.9), “Loss of consciousness [30 mins-24hrs]” (S06.03), “Loss of consciousness prolonged without return of consciousness ” (S06.05), “Traumatic cerebral oedema” (S06.1), “Diffuse brain injury” (S06.2), “Other diffuse cerebral & cerebellar injury” (S06.28), “Traumatic subdural haemorrhage” (S06.5), “Traumatic subarachnoid haemorrhage” (S06.6), “Other intracranial injuries” (S06.8), and “Intracranial injury, unspecified” (S06.9). None of these are minor injuries.

Using available data, the evidence does suggest helmet wearing mitigates cycling fatalities and serious injury. It does not appear as though the public have been oversold on the benefits of bicycle helmets.

Update: The original version focused on the relative risk of helmet wearing among fatalities and helmet wearing surveys in New Zealand. This made the wording quite strange and difficult to interpret. However, the odds ratio isn’t as problematic and is a good estimate of relative risk of death in this instance.

Advertisements

3 thoughts on “New Zealand Cycling Fatalities and Bicycle Helmets

  1. Public health research students are taught that a literature review is an important part of any (credible) reasearch project. They are also taught how to calculate odds ratios, and the conditions under which an odds ratio is a good estimate of relative risk. In claiming that “Only nine victims were noted as not wearing a helmet, similar to current national helmet-wearing rates (92%). This highlights the fact that helmets are generally no protection to the serious forces involved in a major vehicle crash; they are only designed for falls.”, it seems that Dr Koorey did not conduct a literature review, and is not aware that it is not valid to claim “no protection” on the basis of his crude comparison of (NZ) population vs fatality helmet wearing rates.

    If Dr Koorey had conducted a literature review, he should have encountered not just the the Elvik, Fahlstedt, and Bambach studies cited by A/Prof Olivier, but also the (1995) Rowe (http://www.ncbi.nlm.nih.gov/pubmed/7804921) and Carr (http://www.monash.edu.au/miri/research/reports/muarc076.pdf) studies, the (2001) Attewell study (http://www.ncbi.nlm.nih.gov/pubmed/11235796), the (2010) Tin Tin study (http://www.ncbi.nlm.nih.gov/pubmed/23071369), and the (2012) Persaud study (http://www.ncbi.nlm.nih.gov/pubmed/?term=persaud+helmet).

    Persaud reviewed 129 cyclist fatalities in Ontario, and found that unhelmeted cyclists were 3.1 times more likely to have died of a head injury than unhelmeted cyclists.

    Rowe found that prior to the helmet legislation in Ontario, head injuries were responsible for over 75% of cyclist fatalities.

    Re-analysing Attewell’s data, Elvik obtained the same results as Attewell for fatal head and brain injury: a risk reduction of about 70% for fatal head injury, and a reduction of about 55% for brain injury.

    The data in the Carr study shows that after taking changes in exposure into account, the Victorian helmet law was associated with an almost 50% reduction the number of AIS3/4 head/brain injuries with motor vehicle involvement.

    It is surprising that Dr Koorey (a New Zealander whose “wide-ranging experience includes considerable work in road safety”, and who was “involved in the development and delivery of national guidelines and training on Planning & Design for Cycling and Walking”) is apparently unaware of the Tin Tin study: “Injuries to pedal cyclists on New Zealand roads, 1988-2007”. Tin Tin et al. found that in 1996-99 (post- NZ helmet law), compared to 1988-81 (pre-law), the number of cyclist hospital admissions for AIS3+ brain injuries with motor vehicle involvement dropped by about 50%, yet that there were no reductions in the admissions for other cyclist injuries with (or without) motor vehicle involvement.

    ATSB data (http://www.infrastructure.gov.au/roads/safety/publications/2006/pdf/death_cyclists_road.pdf) shows that in 1996-2000, 92 cyclists who were fatally injured had been wearing a helmet, and 71 had been unhelmeted. The NSW helmet-wearing data (Smith and Milthorpe, 1993) indicates that in 1993, helmet wearing was 70-80%.

    If the helmet-wearing rate was 70% in 1996-2000, the corresponding odds ratio would be (0.3*92)/(0.7*71) = 0.55, which is (‘amazingly’ close to the NZ OR calculated by A/Prof Olivier and) translates into a reduction in the risk death (from any injury) of 45%. As noted by A/Prof Olivier, such an measure would under-estimate the true helmet effect, as the OR is based on deaths from all injuries, not just deaths from head/brain injuries (the Attewell and Elvik results indicate that helmets reduce the risk of a fatal head injury by about 75%).

    According to http://www.velo-city2014.com/pages/speakers.php#GlenKoorey, Dr Koorey “investigated cycling fatalities as part of a national Inquiry. It is of considerable concern that people attending Dr Koorey’s presentation may assume that there is credible evidence to support his assertions that the benefits of helmets have been “oversold” and that “helmets are generally no protection to the serious forces involved in a major motor vehicle crash”, and then stop wearing a helmet.

    (It is unethical for research to be conducted by individuals who lack appropriate training, http://www.nhmrc.gov.au/_files_nhmrc/publications/attachments/r39.pdf.)

  2. A/Prof Olivier,

    The material citing the results/conclusions of Koorey’s ‘helmet analysis’ was added to http://en.wikipedia.org/wiki/Bicycle_helmets_in_New_Zealand by BHRF “editorial board” member (and “senior statistician”) Robinson, to replace Elvik’s fatal head and brain injury results. Robinson argued that citing Elvik’s fatal and brain injury results was “inconsistent with the intention of the source”, and that “Results for fatalities are contained in one line of one table in the paper. Elvik does not attempt to draw any conclusions from this small sample.”.)

    Robinson also did not like some other material material I had added, which cited the ACRS paper you co-authored with Dr Wang et al. (http://www.tars.unsw.edu.au/news/2013/Various_files/28_Wang_PR.pdf). The material I had added stated that your analysis showed that Povey’s results and interpretation were valid.

    Robinson described this as an “obviously incorrect comment”. (BHRF “editorial board” member) Keatinge suggested removing the material. (BHRF “editorial board” member) Perry commented that “time trends alone actually explain the changes in injury rates better, i.e. Robinson is in fact right, or at least a better fit than Povey… Or something like that, as I said it all gets rather tortuous and hard to disentangle.”, removed the material, and subsequently commented “I am left wondering why anybody, regardless of POV, would wish to reference Wang”.

    Robinson also claimed that “rather than identifying flaws, Wang’s results appear to confirm the validity of Robinson’s finding that after accounting for time tends, there is no estimated benefit of helmets . . . It’s as if they have fallen into the trap that correlation implies causation . . . Wang et al. also shoot themselves in the foot in the last paragraph of that section by saying: “it seems that the model assumptions for Model (4) are satisfied and hence the results and conclusions in Povey et al.’s analysis are valid.” I believe this is a careless typo and they meant to say Model (3), the model Povey actually fitted. It doesn’t say very much for the paper (or the peer-review process) that such a key issue that affects the meaning of the entire paper wasn’t corrected.”.

    It would be helpful if you or Dr Wang could add a sentence or so to the article, stating whether your analysis did in fact show that Povey’s results and interpretation were valid; or whether, as claimed by Robinson, your analysis confirmed Robinson’s finding of no benefit of helmets, and that “in the normal sense of the word, Povey’s analysis would be considered invalid”.

  3. Pingback: New Zealand Helmet Law and Validity of a Regression Model | Injury Stats

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s