When is observational data "enough"?

An article by Glasziou et al. in today's British Journal of Medicine, "When are randomised trials necessary? Picking signal from noise," considers when observational data is sufficient in establishing the effectiveness of a treatment.

It gives a number of examples to illustrate situations in which RCTs are probably unnecessary, when the signal to noise ratio is unlikely to be explained by other biases (i.e. inferences about a direct relationship between the treatment and improvement in the condition are large, in the right sequence, and not likely to be due to other influences) - the examples range from laser treatment of portwine stains to tracheostomy for tracheal obstruction to blood transfusion for severe hemorrhage-related shock.

For a more tongue-in-cheek look at this issue, try this article: Smith GC, Pell JP. Parachute use to prevent death and major trauma related to gravitational challenge: systematic review of randomised controlled trials. BMJ. 2003 Dec 20;327(7429):1459-61. PubMed abstract

CONCLUSIONS: As with many interventions intended to prevent ill health, the effectiveness of parachutes has not been subjected to rigorous evaluation by using randomised controlled trials. Advocates of evidence based medicine have criticised the adoption of interventions evaluated by using only observational data. We think that everyone might benefit if the most radical protagonists of evidence based medicine organised and participated in a double blind, randomised, placebo controlled, crossover trial of the parachute.