An article in Volume 233, 2019 of Social Science & Medicine discusses the potential weaknesses of systematic reviewes, even when they are following the PRISMA Guidelines for such reviews. The authors "advocate that SR teams consider potential moderators (M) when defining their research problem, along with Time, Outcomes, Population, Intervention, Context, and Study design (i.e., TOPICS + M). We also show that, because the PRISMA reporting standards only partially overlap dimensions of methodological quality, it is possible for SRs to satisfy PRISMA standards yet still have poor methodological quality. As well, we discuss limitations of such standards and instruments in the face of the assumptions of the SR process, including meta-analysis spanning the other SR steps, which are highly synergistic: Study search and selection, coding of study characteristics and effects, analysis, interpretation, reporting, and finally, re-analysis and criticism". Read more...
0 Comments
A scoping review of the research on school health and development programs in low resource countries was published in Issue #5, 2016 of Health Education Research. "We included 30 studies meeting specific criteria: (i) studies mainly targeted school going children or adolescents; (ii) admissible designs were randomized controlled trials, controlled before-after studies or interrupted time series; (iii) studies included at least one measure of impact and (iv) were primary studies or systematic reviews. We found that school-based interventions can be classified in two main categories: those targeting individual determinants of health such as knowledge, skills and health behaviors and those targeting environmental determinants such as the social and physical environment at the school, family and community level. Findings suggest that a comprehensive approach addressing both individual and environmental determinants can induce long-term behavior change and significantly improve health and educational outcomes. We highlight the need for further study of the long-term impact of school-based interventions on health outcomes in developing countries." ISHN has been collecting a number of similar reviews that examine different applications of the "School Health & Nutrition" programs/approach to this context. Our recent ISHN report to Members-Subscribers includes similar conclusions to those reached in this review; that there are several "multi-intervention programs" addressing issues such as water-sanitation-hygiene (WASH), school feeding and the prevention of infectious diseases that produce changes both to the conditions in schools as well as student health & behaviours. Read more >> (This item is among the 5-10 highlights posted for ISHN members each week from the ISHN Member information service. Click on the web link to join this service and to support ISHN)
This blog has been identifying the limits of RCT's & systematic reviews. RCT's are problematic when seeking to assess complex, multiple interventions that develop over several years across several systems, agencies and professionals. An editorial in the October 2015 Issue of Journal of Epidemiology & Community Health continues this discussion by arguing for more use of "pragmatic, formative evaluations" when implementing a complex intervention. The editorial begins: "Recently published guidance on process evaluations by the Medical Research Council's (MRC's) Population Health Sciences Research Network (PHSRN) marks a significant advance in the evaluation of complex public health interventions. In presenting programmes as not just a set of mechanisms of change across multiple socioecological domains, but as an interaction of theory, context and implementation, the guidance extends the remit of evaluation. Process evaluations have emerged as vital instruments in these changing needs through modelling causal mechanisms; identification of contextual influences and monitoring fidelity & adaptations." They go on to say: "One particular conceptual space that needs to be carved out is pragmatic formative process evaluation, defined as the application of formative process evaluation criteria to interventions that have ostensibly been formulated, and are likely in routine practice, but have not been subjected to rigorous evaluation. Moreover, even where some understanding of the theory of change is present, it is unlikely that the unintended consequence of interventions will have been sufficiently explored. For example, our recent evaluation of a school-based social & emotional learning intervention, which had been recommended by the Welsh school inspectorate indicated a number of potential iatrogenic effects due to a stigmatising process". Read more>> (An item from the ISHN Member information service)
Since about 2005, our attention in school health and social development has included a focus on the country, community and neighbourhood contexts as a key factor in selecting the issues to be addressed, the programs to be used and the capacities to be strengthened. ISHN has worked with others to develop frameworks for indigenous communities, disadvantaged communities in high resource countries and more recently, in low resource countries. But those efforts are ahead of the good research required to guide practice. So we were pleased to note the article in the September 2015 Issue of Implementation Science describing a project to Identify the domains of context important to implementation science. "This research program will result in a framework that identifies the domains of context and their features that can facilitate or hinder: (1) healthcare professionals’ use of evidence in clinical practice and (2) the effectiveness of implementation interventions. The framework will increase the conceptual clarity of the term “context” for advancing implementation science, improving healthcare professionals’ use of evidence in clinical practice, and providing greater understanding of what interventions are likely to be effective in which contexts." Read more>> (An item from the ISHN Member information service)
Several articles in Issue #5, 2015 of Administration and Policy in Mental Health and Mental Health Services Research provide insights and methods for significantly changing the research in school health promotion and social development. Most studies, often replicated again and again, measure the linear impact of a selected intervention (instructional program, policy or service) delivered at the school level only on a behaviour or problem. Sometimes, combinations of interventions delivered at the school level, usually aimed at one or two behaviours or combination of problems, are evaluated for a short period of time. Rarely, we see attempts to group these singular or limited intervention studies into a health promoting schools model and assess whether the HPS model works (Again, the assessments almost never extend beyond the school to include clinics or other agencies or upwards into the health, education and other systems. Implementation research, a new type of work in health promotion and social development, has also been limited to this narrow, singular and front-line scope. This is because of the costs and complexity of multi-level research in large systems. As noted in the introduction to this set of articles in this journal "Implementing evidence-based and innovative practices, treatments, and services in large systems is highly complex, and has not, until recently, been guided by empirical or theoretical knowledge. Mixed method designs and approaches have been proposed to promote a more complete and efficient way of understanding the full range of factors that influence the dissemination and implementation of evidence-based innovations in large systems.This special issue provides both an overview of mixed methods designs and approaches, as well as applications and integration of sophisticated sampling, statistical methods and models (borrowed from various fields such as anthropology, statistics, engineering and computer science) to increase the range of solutions for handling the unique challenges of design, sampling, measurement, and analysis common in implementation research. In the six papers in this special issue, we describe conceptual issues and specific strategies for sampling, designing, and analyzing complex data using mixed methods. The papers provide both theoretically-informed frameworks, but also practical and grounded strategies that can be used to answer questions related to scaling-up new practices or services in large systems." Read more>> (An item from the ISHN Member information service)
Several articles in Issue #8-9, 2015 of Substance Use & Misuse are part of a special Issue describing progress (and not) in substance abuse. The title of the issue says it well "Fifty Years Later: Ongoing Flaws and Unfinished Business". The articles include some great insights into the debates, many ongoing, within this filed. They include a discussion of the politics in dropping the term "addiction" and turning to problematic or misuse; the failure to address discrimination as a cause of substance abuse; the clever marketing that underlies "evidence-based practices" and how the concept of "gateway drug" needs to be retired. Another article likes drug use by young people in Hong Kong to the profound economic and social changes that have occurred there in the past five decades. One not to be missed is an article discussing how negative (focus on risk and problems) and positive (focus on assets and youth development) models in adolescent substance abuse prevention are found wanting. The over reliance on RCT studies as the "gold standard" in knowledge development is also debated. Another article describes the ongoing challenges associated with implementation and maintenance of effective programs. Other deal with context, the challenges of integrated programming, evidence-based vs local programs, and community-school cooperation. n brief, the special issues captures much of the recent history and challenges in substance abuse prevention but it also offers insight into prevention and promotion more generally. Read More>> (An item from the ISHN Member information service)
(An item from the ISHN Member information service) Two articles in Volume 183, 2015 of Journal of Affective Disorders report on the use of the Delphi method (used to develop consensus) to list and assess the value of interventions to reduce the risk of adolescent depression and on parenting strategies to reduce childhood depression and anxiety. The first article used a literature search to identify 194 potential interventions. "These were presented over three questionnaire rounds to panels of 32 international research and practice experts and 49 consumer advocates, who rated the preventive importance of each recommendation and the feasibility of their implementation by adolescents. 145 strategies were endorsed as likely to be helpful in reducing adolescents׳ risk of developing depression by ≥80% of both panels. Endorsed strategies included messages on mental fitness, personal identity, life skills, healthy relationships, healthy lifestyles, and recreation and leisure. " The second article used a literature search identified 289 recommendations for parents which were then presented to a panel of 44 international experts over three survey rounds, who rated their preventive importance. "171 strategies were endorsed as important or essential for preventing childhood depression or anxiety disorders by ≥90% of the panel." Some researchers contributed to both articles. The ISHN web site (www.schools-for-all.org) uses a similar combination of research-based evidence and the views of experienced practitioners, descition-makers and advocates and publishes the content in a Wikipedia style web site. Read more>>
(An item from the ISHN Member information service) ISHN has been discussing the limits systematic review in this blog. An article in Issue #3, 2015 of Evidence & Policy extends our skepticism by examining how various "evidence tools (health impact assessments, systematic reviews and cost-benefit studies) are used, misused and misinterpreted in the real world of public decision-making. The authors conclude that "Each has been promoted as a means of synthesising evidence for policy makers but little is known about policy actors' experiences of them. Employing a literature review and 69 interviews, we offer a critical analysis of their role in policy debates, arguing that their utility lies primarily in their symbolic value as markers of good decision making." Not having access to the full text of that article, we did find a very similar version online that includes this statement "‘evidence tools’ can be important means of supporting policy decisions because they appear to be objective and credible (or at least more objective and credible than single studies may be). In addition, the interviewee suggests that ‘evidence tools’ employing quantitative data and providing clear and simple ‘answers’ to policy questions represent ‘gold dust’ to policymakers, who are often desperate for some sense of certainty within complex (and often contested) debates. " They go on to say that " All this leads us to conclude that ‘evidence tools’ represent means of drawing policymakers’ attention to particular kinds of evidence (and, implicitly, away from other kinds). As such, they might best be understood as ‘research-informed advocacy tools’ constructed and employed by actors (researchers using their black magic) working to inform the policy process. Read more>>
(An item from the ISHN Member information service) An article in Issue #6, 2015 of American Journal of Public Health contends that the 200+ health awareness days, weeks and months do little to promote health or well-being. Schools are often a big part of these awareness activities, often viewing the participation in these days to be akin to addressing the problem. The authors "contend that health awareness days are not held to appropriate scrutiny given the scale at which they have been embraced and are misaligned with research on the social determinants of health and the tenets of ecological models of health promotion. We examined health awareness days from a critical public health perspective and offer empirically supported recommendations to advance the intervention strategy. If left unchecked, health awareness days may do little more than reinforce ideologies of individual responsibility and the false notion that adverse health outcomes are simply the product of misinformed behaviors. Read more>>
(An item from the ISHN Member information service) ISHN has been critical of the "training then hoping" strategies that are often the default option for improving practice and introducing new health and social programs in schools. This blog article pulls together three recent articles on how health professionals use knowledge (or not) when modifying their practice. The first article, published in the May 2015 Issue of the International Journal for Equity & Health, "was to identify existing knowledge to action models or frameworks and critically examine their utility for promoting or supporting health equity. Forty-eight knowledge to action models or frameworks were identified. All of the models were then assessed across six characteristics relevant for supporting health equity. While no models scored full marks, the highest scoring models were found to have features relevant to advancing health equity. In the assessment, we propose six characteristics that could be important markers: 1) an explicit mention of equity, justice or similar concept; 2) the involvement of various stakeholders; 3) an explicit focus on engagement across multiple sectors or disciplines; 4) the use of an inclusive conceptualization of knowledge; 5) the recognition of the importance of contextual factors; and, 6) a proactive or problem-solving focus. Specific populations, topics and solutions are marginalized, ignored, or not acted upon when, for example, only certain knowledge is considered valuable, when we don’t have a specific focus on equity or justice, and when we don’t work across sectors or consider contextual determinants of health." The authors concluded that "Each could be strengthened in some way to make them more useful in supporting health equity by considering the six characteristics used in this review. Of particular interest is knowledge brokering as well as the use of holistic and cross-sector models of knowledge to action that consider environmental and contextual determinants. These are specific future avenues identified in this project." In other words, using "knowledge to action" frameworks, even if they are adapted to suit equity purposes, was not sufficient to improve efforts related to equity.
The second article, published in Issue #1, 2015 of Health Technology Assessment, was to identify the impacts and likely impacts on health care, patient outcomes and value for money of Cochrane Reviews published by 20 NIHR-funded CRGs during the years 2007–11. The authors note that "we found 40 examples where reviews appeared to have influenced primary research and reviews had contributed to the creation of new knowledge and stimulated debate. Twenty-seven of the 60 reviews had 100 or more citations in Google Scholar™ (Google, CA, USA). Overall, 483 systematic reviews had been cited in 247 sets of guidance. This included 62 sets of international guidance, 175 sets of national guidance (87 from the UK) and 10 examples of local guidance. Evidence from the interviews suggested that Cochrane Reviews often play an instrumental role in informing guidance, although reviews being a poor fit with guideline scope or methods, reviews being out of date and a lack of communication between CRGs and GDs were barriers to their use. Cochrane Reviews appeared to have led to a number of benefits to the health service including safer or more appropriate use of medication or other health technologies or the identification of new effective drugs or treatments. However, whether or not these changes were directly as a result of the Cochrane Review and not the result of subsequent clinical guidance was difficult to judge." The authors of this second article concluded that " The clearest impacts of Cochrane Reviews are on research targeting and health-care policy, with less evidence of a direct impact on clinical practice and the organisation and delivery of NHS services". In other words, systematic reviews and possibly even the practice guidelines that try to use such Cochrane Reviews as their basis, may or may not affect practice. The third article, published in the April 2015 Issue of the Cochrane Database of Systematic Reviews, examined "whether tailored intervention strategies are effective in improving professional practice and healthcare outcomes. We compared interventions tailored to address the identified determinants of practice with either no intervention or interventions not tailored to the determinants...Attempts to change the behaviour of health professionals may be impeded by a variety of different barriers, obstacles, or factors (which we collectively refer to as determinants of practice). Change may be more likely if implementation strategies are specifically chosen to address these determinants." The authors conclude that "The findings continue to indicate that tailored interventions can change professional practice, although they are not always effective and, when they are, the effect is small to moderate. There is insufficient evidence on the most effective approaches to tailoring, including how determinants should be identified, how decisions should be made on which determinants are most important to address, and how interventions should be selected to account for the important determinants. In addition, there is no evidence about the cost-effectiveness of tailored interventions compared to other interventions to change professional practice." Our take away from these three and other studies we have been reading is that knowledge about better practices or better programs is insufficient to implement or sustain improvements in professional or organizational practices. The answers lie within the organizational or community context, likely based on their current core mandates, perceived and real constraints, traditions and routines and current system-level and adopter concerns at various levels. In other words, we may need to make a significant shift away from "evidence-based practice" towards auch better understanding of "practice-based experience". |
Welcome to our
|