(From the ISHN Member information service) We have often questioned the over-reliance on RCT studies and systematic reviews of these studies in these ISHN Commentaries. One of our concerns has been the fact that the control group in many studies often has an existing intervention (policy, program or practice) that is quite similar to the new intervention being tested. Often, the authors of the study conclude that the new program works or not and then the systematic review of these types of studies concludes whether this type of intervention is effective. The assumption is that the new program can be treated as a medication (dose, intensity, duration) and be compared with situations where o such program exists. In fact, these comparisons are being made to potentially very similar conditions affecting the control groups. These studies are really only able to conclude that the new intervention being tried is better, worse or similar to the existing situation. An article in the March 2015 Issue of Addiction discusses this weakness in RCT/Systematic Review methods and asks the question: "Compared with what? An analysis of control-group types in Cochrane and Campbell reviews of psychosocial treatment efficacy with substance use disorders" They then go on to make this argument: "A crucial, but under-appreciated, aspect in experimental research on psychosocial treatments of substance use disorders concerns what kinds of control groups are used. This paper examines how the distinction between different control-group designs have been handled by the Cochrane and the Campbell Collaborations in their systematic reviews of psychosocial treatments of substance abuse disorders. Methods We assessed Cochrane and Campbell reviews (n = 8) that were devoted to psychosocial treatments of substance use disorders. We noted what control groups were considered and analysed the extent to which the reviews provided a rationale for chosen comparison conditions. We also analysed whether type of control group in the primary studies influenced how the reviews framed the effects discussed and whether this was related to conclusions drawn. Results The reviews covered studies involving widely different control conditions. Overall, little attention was paid to the use of different control groups (e.g. head-to-head comparisons versus untreated controls) and what this implies when interpreting effect sizes. Seven of eight reviews did not provide a rationale for the choice of comparison conditions. Conclusions Cochrane and Campbell reviews of the efficacy of psychosocial interventions with substance use disorders seem to underappreciate that the use of different control-group types yields different effect estimates. Most reviews have not distinguished between different control-group designs and therefore have provided a confused picture regarding absolute and relative treatment efficacy. A systematic approach to treating different control-group designs in research reviews is necessary for meaningful estimates of treatment efficacy." Read more>>
RSS Feed