Critique of "Footwear Style and Risk of Falls in Older Adults"
by Alan Crowe
The study is case controlled. When some-one has a fall, they report what they were wearing to the doctors. The doctors want to know what is typical footwear for that activity, so in addition to the report from the person who had the fall (the case) they telephone another member of the study (the control). The control is the same sex as the case, and about the same age. They are asked what they were wearing when they last performed the activity that lead to the other person's fall.
So far so good. From what I've written, you would imagine that if the case slips and falls walking from the shower cubicle to the towel rail, the control will be asked what they were wearing last time they walked from the shower cubicle to the towel rail.
But no. The study divides activities into 5 categories.
Walking indoors
Walking outdoors
Climbing stairs
Moderate activity (such as vacuuming floors)
Vigorous activity (such as playing tennis)
So the control is asked what they were wearing last time they were walking indoors. Presumably the answers reflect what the controls usually wear, and under-represents or completely ignores temporary or transitional states such as barefoot in the bathroom or stocking feet while changing shoes.
Table 5 is revealing. For walking indoors and climbing stairs, being shoeless appears more dangerous than wearing shoes. For walking outdoors, vigorous activity, and moderate activity, there is no data at all, neither from cases nor controls. This shows that there were no barefooters in the study, and suggests that being barefoot was only ever a transitional state for the cases, and thus unlikely to be reported by the controls when asked about one of the five broad categories.
The study starts with 12 categories of footwear, including both stocking feet and barefoot. They later pool their data into five and then into three categories as they focus on Athletic shoes and canvas shoes versus other shoe styles versus shoeless.
Rewriting Table 2
Cases Controls Athetic/canvas 110 144 Other shoe style 190 177 Stocking feet 12 3 Barefoot 12 1 You can see why they want to pool Stocking feet and Barefoot. They have hardly any data. Look more closely though. Those figures cannot possibly be right.
It is notorious that socks slip and going stocking-footed on tiles or polished wood is just asking for a slip and fall. The controls are elderly persons recruited to a study of slips and falls. They can hardly be unaware of the risks of stocking feet. Yet three were in stocking feet, compared to only one barefoot. Clearing the reporting method is under-recording time spent barefoot.
Another calculation is to divide up the sixteen hours a day not sleeping into the ratio 144+177+3 to 1. Phoning up the controls and asking what were you wearing last time you did X is capturing less than 3 minutes a day of being barefoot by the controls. That seems on the low side.
There is a much worse problem with the paper, that would not go away even if the problems with data collection could be fixed: Simpson's Paradox.
See Simpson's Paradox, Simpson's Paradox: An Example From a Longitudinal Study in South Africa, and Simpson's Paradox.
It is possible for observational data, such as this study, to indicate that a treatment is beneficial to men and women, and also that the treatment is harmful to men and harmful to women.
How is such a thing possible? I invent some numbers that I make up purely to illustrate how this kind of thing can happen.
Consider a study that looks at falls and footwear. It is based on taking 230 photographs, to record whether subjects were shod or barefoot and whether they suffered falls.
Fall OK Shod 100 100 Barefoot 20 10 This made-up data makes bare feet look bad: a 66% chance of a fall while barefoot as compared to a 50% chance of a fall while shod.
Imagine that subjects are having falls when barefoot in slippery places such as shower cubicles and changing rooms. Dividing the data in Slippery and Grippy:
Slippery Grippy Fall OK Fall OK Shod 10 0 90 100 Barefoot 20 5 0 5 I hope you can see how the arithmetic works. Shoes are worse that barefeet in both the Slippery and the Grippy places, but subjects are usually barefoot in the dangerous, slippery places, so the barefeet look worse over all.
What can statisticians do about Simpson's Paradox? One answer is randomised controled trials. A subject is recruited to a trial, a coin is tossed, and he either gets the treatment or a placebo. You cannot really do that with lifestyle issues, but randomised controled trials are about the only way of avoiding Simpson's paradox. Standard scientific practise is to cross one's fingers and hope. See the preface to Compuation, Causation, and Discovery.
Some hopes are more realistic than others. How realistic would it be for researchers to try to assess the safety of athletic shoes versus lace-up oxfords? If the athletic shoes were sometimes being used for athletics, then that is a warning sign. The two groups of persons, the athletic shoe wearers and the lace-up oxford wearers are not homogenous, and Simpson's paradox may well occurr. If on the other hand, the athletic shoes are just a fashion statement, and the lace-up oxfords are just a fashion statement, there is a chance that the two groups are leading similar lives and a different in accident rates might actually reflect one shoe being safer than the other.
The problem for the paper "Footwear Style and Risk of Falls in Older Adults" lies in Table 5. No-one is undertaking Walking Outdoors, Moderate Activity, or Vigorous Activity without shoes. The older adults are doing different activities barefoot than shod. So it is unrealistic to expect to infer a causal connection between being barefoot and having a fall in this kind of study.
I've also noticed a really scary problem with the paper. They are looking at 327 falls. Most fall-related injuries were contusions or lacerations; 25% resulted in a medical care visit, but there were 15 fractures, (4 hip, 5 hand/wrist, 6 other), and 15 head injuries. So quite possibly, nearly all the human misery is concentrated in the 30 falls that caused fractures and head injuries. The authors argue for athletic shoes as being slightly safer than their "other shoe styles" category, but both contain enough falls that all 30 serious falls could be athletic shoe falls, or all 30 serious falls could be "other shoe styles" falls.
One of the attractions of barefoot living is always knowing what kind of grip on has. Even if one slips and falls one is not going to be taken by surprise, go down with a bang, and break something. The paper completely ignores that aspect of footwear, even though it appears from the authors' figures that 10% of the falls have much more serious consequences than the rest, and the factors that determine seriousness must therefore be of comparable importance to the factors that determine whether a fall happens at all.