Evidence Matters

because science reporting and decision-making should be evidence-based

British Chiropractic Association and The Plethora of Evidence for Paediatric Asthma

with 13 comments

British Chiropractic Association (BCA) has issued a third update in the matter of BCA v. Simon Singh. Leaving aside the BCA discussion of the legal aspects of the case, the BCA has risen to the challenge so famously issued by Professor Frank Frizelle when he was entrammelled in a dispute with chiropractors in New Zealand: “let’s hear your evidence not your legal muscle”. The BCA is to be commended for releasing the details of the research papers they rely upon for their stance that “chiropractic can help various childhood conditions” such as asthma.

In the spirit of a wider scientific debate…the BCA has decided that free speech would be best facilitated by releasing the details of research that exists to support the claims which Dr. Singh stated were bogus. This proves that far from there being “no a jot of evidence” to support the BCA’s position, there is actually a significant amount.

It has never been the BCA’s case that the evidence is overwhelmingly conclusive. It is the BCA’s case that there is good evidence…

The BCA welcomes full, frank and open scientific debate.

Evidence Matters accepts the BCA’s invitation to a “full, frank and open scientific debate” and presents a review of the BCA’s evidence for the benefit of chiropractic interventions in children with asthma.

We would like to see a comprehensive selection of randomised, placebo-controlled, double-blind, multi-centre clinical trials that report significant results indicating that chiropractic interventions elicit a clinical significant effect beyond that of a placebo response. The results should reflect parameters that are widely accepted to have clinical significance such as improvements in spirometry, an age-appropriate range of lung function tests or the standardised, validated, robust assessment scales for quality of life such as the Juniper Questionnaire. Above all, we expect to see that these studies report chiropractic interventions with children who have clinically-verified asthma.

It would also be useful if the relevant papers were to lay out a framework for a possible mechanism of action for chiropractic interventions for paediatric asthma. Balon et al (see later discussion) provide an excellent summary of the notional theoretical underpinnings as to why chiropractic might have a useful therapeutic impact.

Spinal manipulation has been shown in randomized, controlled trials to relieve back pain and other musculoskeletal conditions, but many chiropractors and osteopaths report benefit in patients with nonmusculoskeletal conditions, including asthma. The theoretical basis for expecting benefit from chiropractic manipulation in persons with asthma rests on two assumptions. First, reflex irritation of somatic and autonomic nerves at the spinal and nerve-root levels is caused by vertebral subluxation, defined as a palpable restriction of a spinal joint as evidenced by the loss of joint play with surrounding muscle tightness, pain, and tenderness. Second, this mechanical and neurologic disturbance affects chest-wall function or alters airway tone or responsiveness directly or by means of neurogenic inflammation, resulting in a predisposition to or inducement of asthma. Chiropractic theory states that the correction of subluxation by manipulation, with restoration of normal mechanical and nerve function, should improve airway function and aid in the resolution of asthma. [References omitted.]

It might even have been helpful if the BCA had made such a clear summary in their statement. It is clear that the dependence upon the existence and correction of subluxations means that there can be no simple equivalence between chiropractic and osteopathy or other forms of spinal manipulation therapy. It would be unhelpful and possibly misleading to rely upon any argument that implies an equivalence of the therapies when examining the evidence.

Bronfort G, Evans RL, Kubic P, Filkin P. Chronic pediatric asthma and chiropractic spinal manipulation: a prospective clinical series and randomized clinical pilot study. J Manipulative Physiol Ther. 2001 Jul-Aug;24(6):369-77.

BCA summarises Bronfort et al as follows.

After 3 months of combining chiropractic spinal manipulation with optimal medical management for paediatric asthma, children rated their quality of life substantially higher and their asthma severity substantially lower. These improvements were maintained at the 1-year follow-up assessment. The observed improvements are unlikely as a result of the specific effects of chiropractic spinal manipulation alone, but other aspects of the clinical encounter that should not be dismissed readily.

The paper is an admixture of a randomised pilot study and a largely unreported clinical series: the study involved 2 centres, both in Minnesota.

It is unfortunate that the authors do not mention the school of chiropractic to which either the chiropractic physician who conducted the baseline evaluation or the chiropractor who was responsible for the interventions belonged. There are differences of philosophy amongst the schools and there may be differences in technique or assessment.

Although randomisation is discussed, there is no elaboration of the criteria for assessing “spinal dysfunction”:

Patients were also required to have the presence of a spinal dysfunction as determined by a chiropractic physician.

In passing, without an explanation, it seems a little odd that of the 46 people who were screened to have a baseline evaluation, none was excluded for lack of “spinal dysfunction”. It is unknown whether there was a difference in the degree of severity of “spinal dysfunction” between the active intervention group (n=22) and the sham intervention group (n=12) and, if so, whether this might be an artifact of the degree of severity of asthma or an ephiphenomenon.

It is unknown whether the chiropractic physician involved in screening the volunteers/referrals for the study is the same “licensed, experienced chiropractor” who performed both the active and sham chiropractic interventions. The authors do not describe how the dysfunction was assessed, through X-rays, physical examination, or from the clinical history.

It is unknown whether the dysfunction was categorised and recorded in patient notes that were available to the chiropractor who administered both interventions. We raise this point because it is not infeasible that over the course of 20 appointments, each of an unspecified duration that was reported to be constant across the groups, the chiropractor might comment on perceived improvements in freedom of movement, improved stability, ease of breathing and ask the patient and/or the parents/guardians whether he/she had noticed any improvement.

If so, there may be a involuntary difference between how the chiropractor handled this for the active and sham intervention groups: possibly, it was easier to discuss such matters in the active intervention group but they were not mentioned for the patients in the sham intervention group because the chiropractor did not expect there to have been any improvement and might have felt uncomfortable making enquiries that were implicitly grounded in the attitude that there had been an intervention that hadn’t actually occurred. Not everybody is comfortable with the ethics of a ‘benign deception’ and the unblinded chiropractor may have contributed to dissimilar therapeutic encounters.

There may, perhaps, have been some subtle encouragement towards greater physical activity when the chiropractor was engaged with a child who received the active rather than sham intervention. It is unknown if the patients or their parents/guardians were free to ask the chiropractor any questions about their observations or the child’s fitness to participate in physical activities. As above, it is possible that a chiropractor’s response and the nature of the interaction might have been (involuntarily) shaded by the knowledge of the status of the intervention.

It would have been helpful if the authors had discussed these issues because it may be germane to the subjective improvements reported by the children in the active intervention group that were not matched by either objective clinical data or the reports from parents/guardians or supervising clinicians. Such detail might have contributed something useful to the understanding of the placebo response in children. As it is, it is possible that there were substantial differences in the clinical encounter that elicited different placebo response that relate to the conduct of the encounter rather than the chiropractic intervention.

The authors did rely upon a good range of lung function tests for the objective assessment of the children and their verified asthma status. The authors used the Jupiter questionnaire for the quality of life assessments. The authors report that the children kept a peak flow meter diary and were encouraged to journal their symptoms.

It is particularly unfortunate the the authors did not report the results from the sham intervention group. Although the authors report that, “[at] no time were comparisons made between the active and sham treatment groups because of the high risk of committing type I and type II errors” it does mean that there is no ready means of grasping if the rate of clinical improvement in subjective factors was similarly significant (in which case the therapeutic encounters might have been similar) or was effectively unchanged.

It is difficult to perceive this study as supportive of chiropractic intervention and the authors make no such claim. Unfortunately, because of the criticisms outlined above and the lack of reporting from the sham intervention group, it is not a useful exploration of the power of the placebo response that emerges from a therapeutic encounter that is ostensibly based on chiropractic.

There is no evidence from this paper to support chiropractic intervention for children with asthma. The authors seem to recognise this in their conclusion:

[Referring to other trials that reported no important difference between the active and placebo arms]. However, in both trials a clinically important improvement in asthma-related quality of life and a reduction in patient-reported asthma severity appeared to result in both active and sham groups. These improvements are unlikely to occur solely as a result of the natural history or regression to the mean. On that basis, it may not be appropriate to deem the addition of chiropractic care to medical management worthless and to proscribe its use. [Emphasis added.]

Unfortunately, this study is one of many that fail to report whether or not they collected reports of adverse events relating to the treatment intervention. In the absence of such knowledge it may be inappropriate to advise the use of an intervention for which the risk-benefit ratio is unknown. Even if the risk were to be small, in the absence of any positive advantages for the inclusion of chiropractic as part of a therapeutic intervention, there is no clinical evidence to support it in this study.

The BCA offers the following letter into evidence but we do not, as yet, have access to it. It seems to be a comment on the Bronfort et al study (as above).

Kukurin GW. Chronic pediatric asthma and chiropractic spinal manipulation: a prospective clinical series and randomized clinical pilot study. J Manipulative Physiol Ther. 2002 Oct;25(8):540-1.

The BCA summarises the letter:

[The results] add to a curious trend reported in the literature, namely that patients report improvement in their asthma after a course of chiropractic manipulative therapy.

We would argue that publication bias is a plausible explanation of this “curious trend”. Dr Ben Goldacre of Bad Science describes publication bias.

But how can you tell if the research literature on a given subject has been rigged? It’s a tricky problem, because you’re chasing evidence for the existence of trials you cannot see. One option is to use mathematical tools, and something called a funnel plot, one of the cleverest ideas of the last century. It’s so clever that you might need to concentrate for the next bit.
Let’s imagine that there are 30 trials on a given drug. Some are big, and more accurate. Some are small and less accurate, with more random noise. You’d expect that the big, accurate trials should all cluster together around the true finding, all giving similar results for the efficacy of a drug. Meanwhile the smaller, rubbish trials – because they are less accurate measures of the drugs efficacy – will be scattered about randomly, some showing the treatment to be better than the good big trials indicate, some showing that it is worse.
You could then plot all your trials on a graph, one dot for each trial. On the x-axis, left to right, is “how good the drug was shown to be by this trial” and on the y-axis, “how methodologically sound and large the trial was”. If there is no publication bias, you should get a triangle shape: at the top of your graph, you will see all your good-quality, accurate trials, clustered together around the true answer. At the bottom of the graph, you will see a broad smear of results, the poor quality trials showing random variation.
But if there is publication bias, you will see a distorted triangle: the small, poor-quality trials at the bottom will be smeared over to the right, because small trials with unwelcome results are much more likely to be overlooked, and dumped in desk drawers, than huge multicentre collaborative studies involving dozens of academics and tens of thousands of participants, which are almost definitely going to get published. If you get a distorted triangle, you know there are some interesting negative trials missing.

Bockenhauer SE, Julliard KN, Lo KS, Huang E, Sheth AM Quantifiable effects of osteopathic manipulative techniques on patients with chronic asthma. J Am Osteopath Assoc. 2002 Jul;102(7):371-5; discussion 375.

The BCA summarises Bockenhauer et al. as follows:

Measurements of both upper thoracic and lower thoracic forced respiratory excursion statistically increased after osteopathic manipulative procedures compared with sham procedures.

They neglect to mention that the participants in this study were adult women rather than children.

Because this study describes osteopathic rather than chiropractic manipulative techniques it is difficult to accept the argument that they should be regarded as equivalent for the purposes of this exercise.

Bockenhauer et al. reports a single intervention of osteopathic manipulation on 10 women with a mean age of 47 and verified asthma. There are few points of comparison with children with asthma.

The same osteopathic physician administered both the active and sham interventions in this cross-over study. Although the study participants were blinded as to the nature of their interventions, the therapist was not. The examiners who carried out the pre and post-treatment evaluations are reported to have been blinded as to which intervention the subjects were to receive or had received for a particular session. The authors report that:

The primary investigator (S.E.B.) trained two examiners to execute these measurements in a correct, consistent manner.

However, we do not have any indication as to how many assessments and under what conditions these inexperienced examiners had been evaluated on before their performance was judged ‘correct and consistent’ in establishing both peak flow measurements (and instruction in the correct technique to achieve and adequate PEF) thoracic excursion measurements. The measurements reported in the results are so low that it must raise some questions about whether the training had covered such an unusual number of relative outliers.

The authors chose changes in peak expiratory flow (PEF) as a measure of clinical assessment although this is mostly unhelpful except for a sub-group of patients who maintain a PEF diary as part of a range of readings that is kept over time. It is difficult to assess these changes because the authors fail to state the model of the peak flow meter or give the scale measurements, referring only to “a handheld peak flow meter”. The values returned by PEF meter can be remarkably varied and can differ even on the same meter, depending on the nature of the technique and instructions that are given to people who are unfamiliar with their use (as per this study). The considerations may influence the results which report that the PEF declined (slightly deteriorated) in both groups following the intervention but more so in the active intervention group although this was not statistically significant.

The patients indicated the subjective assessment of their symptoms by drawing a line on an ease of breathing visual analogue scale: the authors give no indication as to whether or not this is in general use or a validated assessment instrument for this purpose. The authors do not present the data but report an improvement in subjective assessment after both interventions with no significant difference between the groups.

Bockenhauer et al. report that there was a statistically significant difference between the active and sham interventions for the measurement of both upper and lower thoracic excursion on forced respiration. However, they acknowledge that absolute improvements were minor because the baseline readings were remarkably low and indicate an unusual degree of restriction. There is insufficient detail to assess whether or not the women were familiar with this manoeuvre or if there was another problem that might account for these results.

The changes in thoracic excursion measurements were all minute relative to thoracic circumference; the largest increase was only 1.2 cm, less than 2% of an adult’s thoracic circumference.

It does not follow that although there was a statistically significant increase in thoracic excursion on forced respiration that this is of any clinical relevance for the experience of asthma or its successful management.

Thoracic excursion is not commonly assessed in trials for asthma interventions. Although the results were statistically significant in this study they were not necessarily clinically relevant. The BCA has not made a case that argues why even the limited outcome from these osteopathic techniques, used in 10 adult women, should be extrapolated to substantiate benefit from chiropractic interventions for children with asthma.

Guiney PA, Chou R, Vianna A, Lovenheim J. Effects of osteopathic manipulative treatment on pediatric patients with asthma: a randomized controlled trial. J Am Osteopath Assoc. 2005 Jan;105(1):7-12.

BCA summarises Guiney et al. as follows:

With a confidence level of 95%, results for the manipulation group showed a statistically significant improvement of 7 L per minute to 9 L per minute for peak expiratory flow rates. These results suggest that spinal manipulation has a therapeutic effect among this patient population.

The summary merely reproduces part of the abstract for the paper, the BCA offers no interpretation.

As above, this trial used osteopathic techniques for the active intervention and in the absence of a comparison with standard chiropractic practice, it may be inappropriate to extrapolate the relevance of this study to different styles of chiropractic. However, unlike Bockenhauer et al., this single-centre study does involve a paediatric population (ages 5-17) with verified asthma (n=140) who were attending a hospital asthma clinic: again, unlike the earlier studies, the authors provide a reference for the description of the osteopathic techniques and manoeuvres that the physicians used in the active intervention.

We do not know whether the “allopathic physicians” who delivered sham interventions had been especially trained in delivering these interventions with an air of confidence that would reinforce the image of them as an osteopathic physician who was delivering an ‘authentic’ intervention. We don’t know the extent of the measures that were taken to blind the participants. E.g., if the physicians wore name badges, did these also state the qualification (eg., DO or MD) that might have identified their credentials, similarly for name plaques on office doors or treatment rooms? If the patients and their parents knew the name of their doctor, is it feasible that they might have consulted a staff list that revealed qualifications in osteopathy? The authors do not report whether or not they asked the patients to predict whether they were in the active or sham intervention group.

We do not know how the children were randomised to their treatment allocation groups: of 140 children, 90 were in the active treatment group and 50 in the control (sham intervention) group. It is unclear whether this trial is describing the outcome of a single therapeutic intervention (either sham or active) or several sessions. It is not known whether the children in the different groups received a different number of treatment sessions: ditto, the length of the treatment sessions is not known. It is unclear whether the children received stand alone interventions that involved an extra appointment at the asthma clinic or whether the intervention was offered as part of their standard clinic appointments. If the children had received different numbers of appointments and treatments of varying duration then this increased personal contact might influence the degree of placebo response in addition to any Hawthorn effect of improving merely from participation in a study.

The authors do not mention whether they collected any reports of adverse events following the interventions.

Guiney et al. recorded peak expiratory flow measurements as part of the clinical outcomes. The age range of the children is 5-17 and there are concerns about the reliability and relevance of PEF for younger children both because it is difficult for them to comply with the technique (Guiney et al report that this would have been a criterion for exclusion) and their comparatively low respiratory flow. Guiney et al later acknowledge that spirometry would be a better choice, offering a greater range of lung function tests, and less prone to both participant and measurement error.

It is difficult to interpret the measurement differences that the authors report because PEF must be interpreted in line with the age, height, gender and body mass of the child and those data are not available. The measurement scale is not explicitly stated: this is relevant because there was a substantial change in 2004 when the new european standards were adopted. Because this study ran from 1997-1999, the physicians will have used the earlier scale that is substantially less reliable than the contemporary version.

Although it is clear that the physicians who assessed the patients were not blind to the treatment allocation, it is not entirely clear who took the PEF readings and when.

Peak expiratory flow measurements were recorded according to the guidelines of the 1989 Asthma Management Plan detailed by Woolcock et al11 before and after osteopathic physicians delivered OMT….

Patients’ PEF measurements were taken before and after OMT or the sham procedure…

[A]lthough group assignment was masked (ie, single blind), physician assessment was not. The physicians who were responsible for measuring and recording patient PEFs were not blinded as to the study group to which patients were assigned (ie, OMT or sham procedure). Thus, physicians might have unconsciously affected patient PEFs with subtle acts of encouragement or discouragement. The use of blinded physicians (or a blinded respiratory technician) in measuring and documenting PEFs may correct for this confounding variable.

However, if the PEF readings were only taken before and after the intervention then there is no indication that any improvements were sustained outside the clinic or that they had a clinically significant impact on the children’s experience of their asthma.

Although the mean age for both groups was around 11 years of age, there is no clear description of the age distribution in those groups and so it is difficult to assess whether the reported improvements in PEF of from 7-9 litres per minute is clinically relevant although it may well have been statistically significant. Balon et al (see later note) similarly report what they characterise as “small increases (7 to 12 liters per minute) in peak expiratory flow” but report no statistical significance between their groups and do not ascribe any clinical significance to this improvement.

Despite the optimism of the BCA that this forms part of their case for support for the value of chiropractic interventions for children with asthma, this trial can not sustain the burden of that case. Guiney et al conclude:

It can be said that the various techniques available for osteopathic manipulation may make patients “feel better,” but they have not been proven to be of significant specific benefit for patients with asthma

Certainly, the demonstration of a solid relationship between osteopathic principles and practice and a positive therapeutic effect in this large and increasing patient population could have far-reaching implications for the promotion and integration of OMT by osteopathic physicians. [Emphasis added.]

We should note here that osteopathic physicians in the US may be medically qualified, there may be as substantial a difference between these physicians and their lay counterparts as there is between the medically qualified homeopaths and lay homeopaths. Although it is not stated, it is not implausible that an osteopathic physician would supervise an asthma clinic and be in charge of monitoring asthma management and the prescription of appropriate drugs. It might have been useful if this paper had discussed national differences in qualifications in osteopathy.

Again, we should emphasise that there is not necessarily a simple equivalence between osteopathic and chiropractic treatments.

Hawk C, Khorsan R, Lisi AJ, Ferrance RJ, Evans MW. Chiropractic care for nonmusculoskeletal conditions: a systematic review with implications for whole systems research. J Altern Complement Med. 2007 Jun;13(5):491-512. Review.

The BCA summary of this review is as follows:

Evidence from controlled studies and usual practice supports chiropractic care (the entire clinical encounter) as providing benefit to patients with asthma, cervicogenic vertigo, and infantile colic.

Although there is some discussion of chiropractic interventions for paediatric asthma (a specific point of contention in the dispute between Simon Singh and the BCA), several of the studies that meet the criteria for inclusion as part of the asthma section of this review are in adults, employ various methods of chiropractic (including instrument assisted) or are a single case study or case series rather than a trial. Not all of the studies are controlled. The studies are of heterogeneous duration. For most of the studies, the authors do not specify the school of chiropractic involved or whether the therapist is a lay chiropractor or chiropractic physician. Not all of the studies are double-blinded. The studies are very uneven in their methodological quality and this taints the reliability and relevance of their results.

Hawk et al. make the following conclusion based on all of the papers that they review that include adults (as above) and single case reports and retrospective case reviews:

Physiological measures did not improve in any of the experimental studies except one (Guiney),14 in which peak expiratory volume improved in the treatment but not control group; however, between-groups difference was not analyzed for statistical significance. In all studies, symptoms were reported to improve and in most, medication use decreased.

As discussed above for Kukurin et al, publication bias might account for the number of ‘successful’ rather than unsuccessful case reports that are published and this may well skew the outcome of the reports. It might have been useful if Hawk et al. had attempted to provide funnel plots for these studies, even if their conclusion had been that there are insufficient high quality trials to allow them to provide a usable funnel plot.

Table 5 lists the evidence in this review for chiropractic interventions for asthma: the evidence consists of 3 RCTs (1 involved only adults), 1 systematic review, 1 case series and 4 case reports. Several other studies are included as part of the evidence base for ‘spinal manipulation’ that is not necessarily chiropractic in practice or philosophy.

It is not surprising that single case reports are successful. It would have been useful if the authors had included a discussion of regression to the mean and the likely impact of that phenomenon on the reported outcomes. People are more likely to consult a doctor or therapist when their symptoms are at their worst. The natural history of the illness means that symptoms are likely to improve, particularly if the patient is also taking prescribed medication for that condition. So, it is not implausible that a patient attended a chiropractor at the height of their symptoms, experienced regression to the mean, and both patient and chiropractor attributed any improvement or positive change in symptoms, subjective wellbeing or change in medication to the chiropractic intervention.

There is no indication that even the case series from one chiropractor assessed the entire case load to identify all of the patients who had been treated and who had asthma: it is inappropriate to comment further without seeing the paper and reading the author’s explanation as to how he selected which cases to write up.

There is no clear indication that all of the children and adults in the case reports or series have a diagnosis of asthma that was made or validated by a physician. There is no clear indication that differential diagnosis has been attempted in order to differentiate between varieties of asthma that may be seasonal, situational or occupational. Eg, if a client consulted a chiropractor when experiencing hayfever-related asthma, it is likely that those symptoms might ease off or disappear as the season changes along with the relevant pollen count. The change of season may well have more influence on the disappearance of the symptoms than any other intervention.

Hawk et al make some useful comments that are a sidelight on the current confidence that can be reposed in case reports and series.

Case series and case reports could increase their utility in several ways:

  • a. Report patient-based outcomes using validated instruments (rather than focusing on clinician-based outcomes);
  • b. Specifically address occurrence of adverse effects;
  • c. Describe patient characteristics in greater detail;
  • d. Routinely include measures of expectation, satisfaction, and other attitudinal assessments.

However, overall, it is doubtful that the evidence in this review can appropriately be described as “good evidence” as the the benefits of chiropractic practice as a useful therapeutic intervention for children with verified asthma. The results for paediatric asthma indicate that the results from the better quality trials do not report clinically significant findings of well validated and familiar outcomes for assessing improvement in the management or experience of asthma.

Unity examines the BCA’s evidence for infant colic. Unity notes that the BCA omitted a high quality, comparatively large-scale trial that reported no significant improvement in parents’ proxy symptom scores between spinal manipulation and placebo.

Similarly, the Hawk et al. review includes the Balon et al RCT that was adjudged to be a high quality trial (as assessed by Jadad scale , the SIGN and modified CONSORT checklists). Balon et al has 80 participants, aged between 7-16. and the researchers assessed active and simulated chiropractic manipulation as adjunctive treatment for childhood asthma. The authors implemented a successful double blinding protocol.

The authors provided a clear schedule of the children’s regular appointments for this protocol. The authors described not only the criteria for accepting an asthma diagnosis and history but also the chiropractic-oriented screening criteria:

evidence of vertebral subluxation on palpation as determined by chiropractic screening.

Such a description might have been helpful in the Bronfort et al study (as above).

Balon et al report their results and conclude:

Among the 80 subjects enrolled in this study of the efficacy of chiropractic manipulation as adjunct treatment for childhood asthma, there was a substantial improvement in symptoms and quality of life and a reduction in -agonist use. However, these changes did not differ significantly between the active-treatment and simulated-treatment groups. There were no significant changes in objective measurements of airway function. Hence, the addition of chiropractic spinal manipulation to usual medical care for four months had no effect on the control of childhood asthma

The improvements in symptom scores and quality-of-life scores and the reduced requirement for inhaled -agonists for relief of symptoms irrespective of treatment group were consistent with anecdotal observations and uncontrolled studies of alternative approaches to the management of asthma. Improvement might have resulted from frequent professional attention (visits to a chiropractor three times weekly provide more contact with a care giver than the usual procedures for the management of asthma), increased compliance with medications under trial conditions, and the subjects’ “growing out of” childhood asthma.45 However, airway responsiveness did not change, suggesting that the last two possibilities are unlikely and that the effect is more likely to have been a placebo effect or study (Hawthorne) effect. [Emphasis added.]

Despite the high quality of this trial that met the rigorous publication standards of the New England Journal of Medicine, the BCA does not include Balon et al in its review of the evidence for chiropractic interventions in the treatment of childhood asthma.

Balon J, Aker PD, Crowther ER, Danielson C, Cox PG, O’Shaughnessy D, Walker C, Goldsmith CH, Duku E, Sears MR. A comparison of active and simulated chiropractic manipulation as adjunctive treatment for childhood asthma. N Engl J Med 1998;339: 1013–1020.

The BCA included a trial that involved adults in their review but omitted to include a review by Erst and Harkness that selected trials of good methodological quality for further scrutiny. Ernst and Harkness included two RCTs that investigated chiropractic interventions and asthma. Although one of these involved adults, the BCA’s evidence list includes a similar adult study albeit this one does not report results that support the role for chiropractic interventions in the management of asthma. Ernst and Harkness summarise the outcomes of that study:

The authors conclude that their results do not support the hypothesis that chiropractic treatment is superior to sham as a treatment for chronic asthma. This study is well designed and seems to have been executed with care. However, because of its relatively small sample size, the study had a high chance of missing a true difference between the treatment and placebo groups.

Ernst and Harkness discuss the Balon trial (as above).

Simon Singh directed the BCA’s attention to various relevant studies concerning chiropractic interventions and paediatric asthma in his defence to the BCA action. It is unfortunate that the BCA does not seem to have addressed the outcomes of these studies.

Simon Singh lists a number of trials where Chiropractic has been shown to not be effective: Balon/Aaker (1998), Balon/Mior (2004), Hondras (2005) and Ernst/Canter (2006).

We have discussed Balon et al (1998).

Balon and Mior (2004) is a fine overview of the “current state of evidence for chiropractic care, specifically in the management of asthma”.

Many of the claims of chiropractic success in asthma have been primarily based on anecdotal evidence or uncontrolled case studies. Three recently reported randomized controlled studies showed benefit in subjective measures, such as quality of life, symptoms, and bronchodilator use; however, the differences were not statistically significant between controls and treated groups. There were no significant changes in any objective lung function measures…There is currently no evidence to support the use of chiropractic SMT as a primary treatment for asthma or allergy.

Balon JW, Mior SA. Chiropractic care in asthma and allergy. Ann Allergy Asthma Immunol. 2004 Aug;93(2 Suppl 1):S55-60.

The plain, unvarnished conclusion from Hondras et al (2005) is that this rigorous Cochrane review finds that there was not enough evidence from trials to show that chiropractic interventions (or any manual therapies) are of use in the management of asthma.

The methodological quality of one of two trials examining chiropractic manipulation was good and neither trial found significant differences between chiropractic spinal manipulation and a sham manoeuvre on any of the outcomes measured…

There is no evidence from two trials, one in adults and one in children, to support the use of spinal manipulative therapy for patients with asthma. Although results of these trials demonstrated improvements in outcomes for all patients who received hands-on manual therapy, these improvements were not clinically important, and no statistical differences were found between treatment groups.

Hondras MA, Linde K, Jones AP. Manual therapy for asthma. Cochrane Database of Systematic Reviews 2005, Issue 2. Art. No.: CD001002. DOI: 10.1002/14651858.CD001002.pub2.

Although the BCA listed what they characterise as a serious challenge to Ernst and Canter’s systematic review that assessed the evidence for the use of spinal manipulation as an intervention for asthma, they did not cite the original Ernst and Canter review.

The evidence from the other systematic reviews of [spinal manipulation] for…asthma…is uniformly negative…

[We] have found no convincing evidence from systematic reviews to suggest that SM is a recommendable treatment option for any medical condition.

Ernst E, Canter PH. A systematic review of systematic reviews of spinal manipulation. J R Soc Med. 2006 Apr;99(4):192-6.


The BCA has consented to share the research upon which it relies in its dispute with Simon Singh.

For those wishing to learn more about some of the available research about the effectiveness and safety of chiropractic treatment for children with the symptoms referred to by Dr Singh in his Guardian article they can begin by looking at the following [list].

Evidence Matters accepted that invitation and scrutinised the citations for paediatric asthma.

The BCA did not claim to offer a comprehensive or systematic review of the literature so there are some notable omissions of high quality trials that did not report outcomes that would undermine the BCA’s implicit claim that there is “good evidence” for childhood asthma and therefore a role for chiropractic as therapeutic in itself or as more powerful than other (perhaps cheaper) placebo adjuncts to other effective interventions.

The BCA offered 5 papers into evidence on the topic of asthma. One is the Hawk systematic review discussed above, 1 is a letter to an editor that comments on a pilot study, 1 is that poor-quality pilot study, 2 report studies of osteopathic manipulation – 1 of which is in adult women. The BCA appears to rest the case for its evidence on one pilot trial. It has not acknowledged those reviews and studies that report results that do not support their claims: reviews and studies that are uniformly of a much higher standard than most of the evidence that they do cite.

The BCA’s citations for paediatric asthma list studies that are variable in quality. Although it is accurate to say that there is some evidence of scrutiny and therefore more than a jot of (unspecified quality) evidence, it seems that it is aspiration rather than good scientific evidence that supports the BCA’s claim for a role for the chiropractic element of interventions to manage childhood asthma.

Other blogs that are examining the BCA’s plethora of evidence

Professor David Colquhoun: British Chiropractic Association produces its plethora of evidence

Gimpy assesses the studies and concludes: The BCA’s Proferred Evidence That Chiropractic Can Help with Ear Infections

Unity of Ministry of Truth: Examining the BCA’S ‘Plethora’ of Evidence

Martin of The Lay Scientist: A Review of The BCA’s Evidence for Chiropractic

Holfordwatch: British Chiropractic Association (BCA) demonstrate what evidence-based medicine isn’t

AP Gaylard of A canna’ change the laws of physics offers a thoughtful analysis of the studies dealing with bed wetting. They aren’t quite as persuasive as the BCA seems to argue that they are.

Phil Plait of Bad Astronomy has some wry advice for the BCA: Careful, BCA, you might slip a disk!

Chris Kavanagh has the most complete round-up of the history and coverage of the BCA v Simon Singh and the blogosphere reaction.


Written by oatmealts

June 18, 2009 at 2:14 pm

13 Responses

Subscribe to comments with RSS.

  1. […] “British Chiropractic Association and The Plethora of Evidence for Paediatric Asthma”. On Evidence Matters […]

  2. […] Evidence Matters […]

  3. […] the reference to the BCA’s asthma evidence, here is Evidence Matters with their take on the British Chiropractic Association and the plethora of evidence for paediatric asthma. Possibly related posts: (automatically generated)We’ve Moved!EASY MONEYthis will be my […]

  4. […] Evidence matters reviews and (you’ve probably guessed it…) demolishes the BCA’s ‘evidence&#82…. […]

  5. […] Evidence Matters: British Chiropractic Association and The Plethora of Evidence for Paediatric Asthma […]

  6. […] not going to lie to you, I haven’t read this post on Evidence Matters, because it goes on for pages and pages, and just looking at that right now hurts my brain. But […]

  7. […] General review focusing on three of the colic papers. Andy – Comment on the BCA statement. Evidence Matters – Review of the paediatric asthma papers. Layscience – a review of the flaws in all the […]

  8. […] manipulations and the one remaining study is a seriously flawed pilot study. All of these have been dealt with in detail over at Evidence Matters but the point worth emphasising is that the BCA only managed to produce ONE study relating to […]

  9. […] Evidence Matter’s British Chiropractic Association and the Plethora of Evidence for Paediatric Asthma […]

  10. […] further information on chiropractic for these conditions, see for example Evidence Matters on asthma. Colic has been discussed by Unity and David Colquhoun, among others. We do hope that the GCC has […]

  11. […] selectively quoting a Cochrane review. Every stone was turned by Quackometer, APGaylard, Gimpyblog, EvidenceMatters, Dr Petra Boynton, MinistryofTruth, Holfordwatch, legal blogger Jack of Kent, and many more. At […]

  12. this is a bit late now but anyway noticed

    Also looking at the PDF of the article there wasn’t actually any spinal manipulations performed.

    osteopathic manipulative (OM) techniques, as appropriate: rib raising, muscle energy for ribs, and myofascial release. These techniques are described in the benchmark osteopathic textbook Foundations for Osteopathic Medicine.

    bit naughty of the BCA to state:

    With a confidence level of 95%, results for the manipulation group showed a statistically significant improvement of 7 L per minute to 9 L per minute for peak expiratory flow rates. These results suggest that spinal manipulation has a therapeutic effect among this patient population.


    August 11, 2009 at 6:13 pm

  13. […] Matters, meanwhile, examined the evidence for chiropractic treatment of paediatric asthma. Here, we discover that “The BCA’s citations for paediatric asthma list studies that are variable […]

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: