(From the ISHN Member information service) An editorial in the January 2015 issue Cochrane Database Systematic Reviews discusses the challenges of reviews of complex interventions such as school health promotion. Although the editorial is discussing coordinated case management of dementia patient care, the comments will likely apply to the complexity of reviewing the variable, multiple, coordinated interventions required in school health promotion.
The authors suggest that "Guidelines have recommended the use of case management but are cautious about the evidence, judged as at least partially inconclusive.There is also uncertainty about the most suitable components of case management interventions.This is no surprise as case management is a prototypical example of a complex intervention. There is complexity in the intervention components as well as in the theoretical background of the intervention, the implementation context, and the targeted outcomes. As with many complex interventions, case management also targets more than one recipient: people with dementia and/or their carers. The challenges of synthesising the evidence for complex interventions have been acknowledged by Cochrane, with a recent series of articles forming the basis for an upcoming new chapter in the Cochrane Handbook for Systematic Reviews of Interventions."
The authors laud the particular review of dementia with comments that could be applied to the variations in school health promotion; " Comprehensive tables allow readers to compare the goals of case management interventions, components of case management and control interventions, methods of intervention implementation, tasks and components of case management, and outcome measures used. Interventions are also categorised into three different approaches to case management. Still, for many studies there is not enough information to clearly describe what has been done. Also, case management interventions were often implemented as a part of wider health system changes, making it more difficult to attribute observations to case management, let alone to distinct components of case management interventions.".
The authors also make suggestions for reviews of complex interventions that also apply to school health and other school-related strategies; "Guidance on conducting systematic reviews of complex interventions often demands the inclusion of further studies to allow for in-depth descriptions of study components and the context and process of implementing the intervention. This frequently requires the inclusion of mixed-method or qualitative studies that could help to disentangle the intervention components and their distinct roles. While this undoubtedly adds to Cochrane authors' already demanding workload, it seems essential if the most meaningful use is to be made of the data. Reporting is a problem, and information is often difficult or even impossible to acquire. Recent reporting guidelines may help authors look for important aspects concerning the intervention (TIDieR guideline) or the whole process of complex intervention development and evaluation (CReDECI guideline)"
They also mention other problems; "Apart from the problems described above, the present review suffers from the fact that most studies are fairly small, with fewer than 100 participants per group in all but one study". We would add that the time period for assessing school health approaches is also problematic. A truly comprehensive, ecological and systems-based approach to SH does more than examine a few schools or some selected interventions. it is an approach that is developed over several years at a national or sate level, with the delivery of multiple policies, funding, personnel and programs from several ministries, local agencies/school boards and then local professionals as well as the people working in the school building. Indeed, reviews of school health promotion and social development are actually far more complex than the one discussed in this editorial, which examines coordinated case management of a single health problem.
It is in the light of this January 2015 Cochrane editorial that we can turn to two major recent and previous reviews of school health promotion (Langford et al, 2014; Stewart-Brown, 2006) and understand better why both of these reviews as well as others conclude that SH promotion is promising but there is insufficient evidence. For further discussion, readers might want to listen to our recent October 23, 2014 ISHN webinar with the authors of the most recent review, as it discusses the limits of RCT studies and the ensuing systematic reviews even further. We hereby challenge researchers and research funding organizations to address this challenge, perhaps beginning with the impending Cochrane Handbook Chapter on complex interventions.
The authors suggest that "Guidelines have recommended the use of case management but are cautious about the evidence, judged as at least partially inconclusive.There is also uncertainty about the most suitable components of case management interventions.This is no surprise as case management is a prototypical example of a complex intervention. There is complexity in the intervention components as well as in the theoretical background of the intervention, the implementation context, and the targeted outcomes. As with many complex interventions, case management also targets more than one recipient: people with dementia and/or their carers. The challenges of synthesising the evidence for complex interventions have been acknowledged by Cochrane, with a recent series of articles forming the basis for an upcoming new chapter in the Cochrane Handbook for Systematic Reviews of Interventions."
The authors laud the particular review of dementia with comments that could be applied to the variations in school health promotion; " Comprehensive tables allow readers to compare the goals of case management interventions, components of case management and control interventions, methods of intervention implementation, tasks and components of case management, and outcome measures used. Interventions are also categorised into three different approaches to case management. Still, for many studies there is not enough information to clearly describe what has been done. Also, case management interventions were often implemented as a part of wider health system changes, making it more difficult to attribute observations to case management, let alone to distinct components of case management interventions.".
The authors also make suggestions for reviews of complex interventions that also apply to school health and other school-related strategies; "Guidance on conducting systematic reviews of complex interventions often demands the inclusion of further studies to allow for in-depth descriptions of study components and the context and process of implementing the intervention. This frequently requires the inclusion of mixed-method or qualitative studies that could help to disentangle the intervention components and their distinct roles. While this undoubtedly adds to Cochrane authors' already demanding workload, it seems essential if the most meaningful use is to be made of the data. Reporting is a problem, and information is often difficult or even impossible to acquire. Recent reporting guidelines may help authors look for important aspects concerning the intervention (TIDieR guideline) or the whole process of complex intervention development and evaluation (CReDECI guideline)"
They also mention other problems; "Apart from the problems described above, the present review suffers from the fact that most studies are fairly small, with fewer than 100 participants per group in all but one study". We would add that the time period for assessing school health approaches is also problematic. A truly comprehensive, ecological and systems-based approach to SH does more than examine a few schools or some selected interventions. it is an approach that is developed over several years at a national or sate level, with the delivery of multiple policies, funding, personnel and programs from several ministries, local agencies/school boards and then local professionals as well as the people working in the school building. Indeed, reviews of school health promotion and social development are actually far more complex than the one discussed in this editorial, which examines coordinated case management of a single health problem.
It is in the light of this January 2015 Cochrane editorial that we can turn to two major recent and previous reviews of school health promotion (Langford et al, 2014; Stewart-Brown, 2006) and understand better why both of these reviews as well as others conclude that SH promotion is promising but there is insufficient evidence. For further discussion, readers might want to listen to our recent October 23, 2014 ISHN webinar with the authors of the most recent review, as it discusses the limits of RCT studies and the ensuing systematic reviews even further. We hereby challenge researchers and research funding organizations to address this challenge, perhaps beginning with the impending Cochrane Handbook Chapter on complex interventions.
RSS Feed