ISHN Commentary: The Limits & Myths of RCT's and Systematic Reviews
Started: 10/11/2011 Completed:
This series of commentaries is being collated from the ISHN/Partners shared School Health Blog and other sources by the International School Health Network and its partners in knowledge exchange and development. These commentaries offer a listing of previously published items with key points noted at the top of the list. Comments on the individual items as well as the overall commentary are welcome. Readers may also be interested in our other ISHN/Partner Commentaries on different topics.
Key Points to Consider:
Background
Started: 10/11/2011 Completed:
This series of commentaries is being collated from the ISHN/Partners shared School Health Blog and other sources by the International School Health Network and its partners in knowledge exchange and development. These commentaries offer a listing of previously published items with key points noted at the top of the list. Comments on the individual items as well as the overall commentary are welcome. Readers may also be interested in our other ISHN/Partner Commentaries on different topics.
Key Points to Consider:
Background
- The work that is done in health promotion is often dominated by its stronger sibling, health care. Consequently it is not surprising that the "gold standard" that is suitable and practical in medical studies, the random, controlled trial (RCT) which precisely measures the dose, duration and response to a particular medicine for specific types of patients as compared to patients receiving no medicine, has been transplanted to the infinitely more complex, more variable real world of human behaviour and various its myriad social influences and determinants. The protocols for systematic reviews, building on these artificially controlled trials, have their own rigid rules, also better suited to the study of individual patients and individual medicines/treatments. This set of blog commentaries suggests that there are significant limits that should be established when using RCT's and systematic reviews when considering the research evidence and professional experience that has developed in the arena of school-based and school-linked health promotion and social development.
- Despite the rhetoric and lip service from senior levels in government and various systems, the managers and professionals within the systems often do not rely on systematic reviews or RCT's to guide their decision-making. There are several other, more practical and circumstantial factors that are often at play.
- The "control" group in RCT's is often not well described nor studied in behavioural studies. Rather than being a group that does not have any intervention being implemented, the control groups often have similar existing interventions or programs that already exist. In essence, the trial is often comparing two similar interventions and not whether that type of intervention is better than no intervention.
- As recent research reveals the complexity of ecological, systems-based thinking and practice in behavioral research, some are now questioning whether RCT's and systematic reviews can ever truly measure and explain the multiple levels of influence.
- We all know that context really matters in the delivery and sustainability of any program or approach, but context is often ignored in many studies.
- We also know that local people, relationships, histories, incidents and other "drivers and barriers" will affect the adoption, implementation and sustainability of programs. However, these are rarely described in RCT's and systematic reviews.
- Longitudinal studies are expensive and therefore are rare. Consequently, most school health and development research is done over a short time period of 2-3 years, usually at the school or even classroom level. We know that comprehensive school health or similar multiple intervention approaches will take several years to develop within individual schools, longer within a school board/health authority and even longer at a ministry/government level. As a result, there are almost no RCT's or systematic reviews that have studied school health promotion effects in a system over a period of ten or so years.
- Most RCT studies are focused on single health behaviors or conditions. It is therefore almost impossible to determine if multiple health and social behaviors are affected by comprehensive, multi-intervention school health promotion or social development programs. As well, since the approach is really just a planning framework or set of potential interventions to respond to a particular set of problems, it is difficult to make any conclusions.
The evidence valued & actually used by health promotion practitioners 27/4/2015
In a series of ISHN commentaries in this blog, we have discussed the limits of RCT-based studies and the mistaken notion that they represent the "gold standard" when considering modifications to practice. An article in Issue #2, 2015 of Health Education Research continues the discussion with a small scale study examining the evidence that is valued most and used by health promotion practitioners. The authors report that "Although there is a general consensus that adopting an evidence-based approach is necessary for practice, disagreement remains about what types of evidence practitioners should use to guide their work. An empirical understanding of how practitioners conceptualize and use evidence has been lacking in the literature. In this article, we explore (i) practitioners’ purposes for using evidence, (ii) types of evidence they valued, and (iii) qualities that made evidence useful for practice. 58 semi-structured interviews and 250 h of participant and non-participant observation were conducted with 54 health promotion practitioners working across New South Wales, Australia. Interviews were recorded and transcribed, and field notes were written during the observations; these were analysed using Grounded Theory methods. Practitioners used evidence for practical and strategic purposes, and valued four different types of evidence according to their relevance and usefulness for these purposes. Practitioners’ ideal evidence was generated within their practice settings, and met both substantive and procedural evaluation criteria. We argue that due to the complex nature of their work, practitioners rely on a diverse range of evidence and require organizational structures that will support them in doing so." Read more>> 0 Comments
In a series of ISHN commentaries in this blog, we have discussed the limits of RCT-based studies and the mistaken notion that they represent the "gold standard" when considering modifications to practice. An article in Issue #2, 2015 of Health Education Research continues the discussion with a small scale study examining the evidence that is valued most and used by health promotion practitioners. The authors report that "Although there is a general consensus that adopting an evidence-based approach is necessary for practice, disagreement remains about what types of evidence practitioners should use to guide their work. An empirical understanding of how practitioners conceptualize and use evidence has been lacking in the literature. In this article, we explore (i) practitioners’ purposes for using evidence, (ii) types of evidence they valued, and (iii) qualities that made evidence useful for practice. 58 semi-structured interviews and 250 h of participant and non-participant observation were conducted with 54 health promotion practitioners working across New South Wales, Australia. Interviews were recorded and transcribed, and field notes were written during the observations; these were analysed using Grounded Theory methods. Practitioners used evidence for practical and strategic purposes, and valued four different types of evidence according to their relevance and usefulness for these purposes. Practitioners’ ideal evidence was generated within their practice settings, and met both substantive and procedural evaluation criteria. We argue that due to the complex nature of their work, practitioners rely on a diverse range of evidence and require organizational structures that will support them in doing so." Read more>> 0 Comments
Systematic Reviews of RCT Studies: A Fatal Flaw when Comparing to Control Group 25/3/2015
We have often questioned the over-reliance on RCT studies and systematic reviews of these studies in these ISHN Commentaries. One of our concerns has been the fact that the control group in many studies often has an existing intervention (policy, program or practice) that is quite similar to the new intervention being tested. Often, the authors of the study conclude that the new program works or not and then the systematic review of these types of studies concludes whether this type of intervention is effective. The assumption is that the new program can be treated as a medication (dose, intensity, duration) and be compared with situations where o such program exists. In fact, these comparisons are being made to potentially very similar conditions affecting the control groups. These studies are really only able to conclude that the new intervention being tried is better, worse or similar to the existing situation. An article in the March 2015 Issue of Addiction discusses this weakness in RCT/Systematic Review methods and asks the question: "Compared with what? An analysis of control-group types in Cochrane and Campbell reviews of psychosocial treatment efficacy with substance use disorders" They then go on to make this argument: "A crucial, but under-appreciated, aspect in experimental research on psychosocial treatments of substance use disorders concerns what kinds of control groups are used. This paper examines how the distinction between different control-group designs have been handled by the Cochrane and the Campbell Collaborations in their systematic reviews of psychosocial treatments of substance abuse disorders. Methods We assessed Cochrane and Campbell reviews (n = 8) that were devoted to psychosocial treatments of substance use disorders. We noted what control groups were considered and analysed the extent to which the reviews provided a rationale for chosen comparison conditions. We also analysed whether type of control group in the primary studies influenced how the reviews framed the effects discussed and whether this was related to conclusions drawn. Results The reviews covered studies involving widely different control conditions. Overall, little attention was paid to the use of different control groups (e.g. head-to-head comparisons versus untreated controls) and what this implies when interpreting effect sizes. Seven of eight reviews did not provide a rationale for the choice of comparison conditions. Conclusions Cochrane and Campbell reviews of the efficacy of psychosocial interventions with substance use disorders seem to underappreciate that the use of different control-group types yields different effect estimates. Most reviews have not distinguished between different control-group designs and therefore have provided a confused picture regarding absolute and relative treatment efficacy. A systematic approach to treating different control-group designs in research reviews is necessary for meaningful estimates of treatment efficacy." Read more>> 0 Comments
We have often questioned the over-reliance on RCT studies and systematic reviews of these studies in these ISHN Commentaries. One of our concerns has been the fact that the control group in many studies often has an existing intervention (policy, program or practice) that is quite similar to the new intervention being tested. Often, the authors of the study conclude that the new program works or not and then the systematic review of these types of studies concludes whether this type of intervention is effective. The assumption is that the new program can be treated as a medication (dose, intensity, duration) and be compared with situations where o such program exists. In fact, these comparisons are being made to potentially very similar conditions affecting the control groups. These studies are really only able to conclude that the new intervention being tried is better, worse or similar to the existing situation. An article in the March 2015 Issue of Addiction discusses this weakness in RCT/Systematic Review methods and asks the question: "Compared with what? An analysis of control-group types in Cochrane and Campbell reviews of psychosocial treatment efficacy with substance use disorders" They then go on to make this argument: "A crucial, but under-appreciated, aspect in experimental research on psychosocial treatments of substance use disorders concerns what kinds of control groups are used. This paper examines how the distinction between different control-group designs have been handled by the Cochrane and the Campbell Collaborations in their systematic reviews of psychosocial treatments of substance abuse disorders. Methods We assessed Cochrane and Campbell reviews (n = 8) that were devoted to psychosocial treatments of substance use disorders. We noted what control groups were considered and analysed the extent to which the reviews provided a rationale for chosen comparison conditions. We also analysed whether type of control group in the primary studies influenced how the reviews framed the effects discussed and whether this was related to conclusions drawn. Results The reviews covered studies involving widely different control conditions. Overall, little attention was paid to the use of different control groups (e.g. head-to-head comparisons versus untreated controls) and what this implies when interpreting effect sizes. Seven of eight reviews did not provide a rationale for the choice of comparison conditions. Conclusions Cochrane and Campbell reviews of the efficacy of psychosocial interventions with substance use disorders seem to underappreciate that the use of different control-group types yields different effect estimates. Most reviews have not distinguished between different control-group designs and therefore have provided a confused picture regarding absolute and relative treatment efficacy. A systematic approach to treating different control-group designs in research reviews is necessary for meaningful estimates of treatment efficacy." Read more>> 0 Comments
Systematic Reviews of School Health: Too Complex to Measure? 20/2/2015
An editorial in the January 2015 issue Cochrane Database Systematic Reviews discusses the challenges of reviews of complex interventions such as school health promotion. Although the editorial is discussing coordinated case management of dementia patient care, the comments will likely apply to the complexity of reviewing the variable, multiple, coordinated interventions required in school health promotion.
The authors suggest that "Guidelines have recommended the use of case management but are cautious about the evidence, judged as at least partially inconclusive.There is also uncertainty about the most suitable components of case management interventions.This is no surprise as case management is a prototypical example of a complex intervention. There is complexity in the intervention components as well as in the theoretical background of the intervention, the implementation context, and the targeted outcomes. As with many complex interventions, case management also targets more than one recipient: people with dementia and/or their carers. The challenges of synthesising the evidence for complex interventions have been acknowledged by Cochrane, with a recent series of articles forming the basis for an upcoming new chapter in the Cochrane Handbook for Systematic Reviews of Interventions."
The authors laud the particular review of dementia with comments that could be applied to the variations in school health promotion; " Comprehensive tables allow readers to compare the goals of case management interventions, components of case management and control interventions, methods of intervention implementation, tasks and components of case management, and outcome measures used. Interventions are also categorised into three different approaches to case management. Still, for many studies there is not enough information to clearly describe what has been done. Also, case management interventions were often implemented as a part of wider health system changes, making it more difficult to attribute observations to case management, let alone to distinct components of case management interventions.".
The authors also make suggestions for reviews of complex interventions that also apply to school health and other school-related strategies; "Guidance on conducting systematic reviews of complex interventions often demands the inclusion of further studies to allow for in-depth descriptions of study components and the context and process of implementing the intervention. This frequently requires the inclusion of mixed-method or qualitative studies that could help to disentangle the intervention components and their distinct roles. While this undoubtedly adds to Cochrane authors' already demanding workload, it seems essential if the most meaningful use is to be made of the data. Reporting is a problem, and information is often difficult or even impossible to acquire. Recent reporting guidelines may help authors look for important aspects concerning the intervention (TIDieR guideline) or the whole process of complex intervention development and evaluation (CReDECI guideline)" They also mention other problems; "Apart from the problems described above, the present review suffers from the fact that most studies are fairly small, with fewer than 100 participants per group in all but one study". We would add that the time period for assessing school health approaches is also problematic. A truly comprehensive, ecological and systems-based approach to SH does more than examine a few schools or some selected interventions. it is an approach that is developed over several years at a national or sate level, with the delivery of multiple policies, funding, personnel and programs from several ministries, local agencies/school boards and then local professionals as well as the people working in the school building. Indeed, reviews of school health promotion and social development are actually far more complex than the one discussed in this editorial, which examines coordinated case management of a single health problem. It is in the light of this January 2015 Cochrane editorial that we can turn to two major recent and previous reviews of school health promotion (Langford et al, 2014; Stewart-Brown, 2006) and understand better why both of these reviews as well as others conclude that SH promotion is promising but there is insufficient evidence. For further discussion, readers might want to listen to our recent October 23, 2014 ISHN webinar with the authors of the most recent review, as it discusses the limits of RCT studies and the ensuing systematic reviews even further. We hereby challenge researchers and research funding organizations to address this challenge, perhaps beginning with the impending Cochrane Handbook Chapter on complex interventions. 0 Comments
An editorial in the January 2015 issue Cochrane Database Systematic Reviews discusses the challenges of reviews of complex interventions such as school health promotion. Although the editorial is discussing coordinated case management of dementia patient care, the comments will likely apply to the complexity of reviewing the variable, multiple, coordinated interventions required in school health promotion.
The authors suggest that "Guidelines have recommended the use of case management but are cautious about the evidence, judged as at least partially inconclusive.There is also uncertainty about the most suitable components of case management interventions.This is no surprise as case management is a prototypical example of a complex intervention. There is complexity in the intervention components as well as in the theoretical background of the intervention, the implementation context, and the targeted outcomes. As with many complex interventions, case management also targets more than one recipient: people with dementia and/or their carers. The challenges of synthesising the evidence for complex interventions have been acknowledged by Cochrane, with a recent series of articles forming the basis for an upcoming new chapter in the Cochrane Handbook for Systematic Reviews of Interventions."
The authors laud the particular review of dementia with comments that could be applied to the variations in school health promotion; " Comprehensive tables allow readers to compare the goals of case management interventions, components of case management and control interventions, methods of intervention implementation, tasks and components of case management, and outcome measures used. Interventions are also categorised into three different approaches to case management. Still, for many studies there is not enough information to clearly describe what has been done. Also, case management interventions were often implemented as a part of wider health system changes, making it more difficult to attribute observations to case management, let alone to distinct components of case management interventions.".
The authors also make suggestions for reviews of complex interventions that also apply to school health and other school-related strategies; "Guidance on conducting systematic reviews of complex interventions often demands the inclusion of further studies to allow for in-depth descriptions of study components and the context and process of implementing the intervention. This frequently requires the inclusion of mixed-method or qualitative studies that could help to disentangle the intervention components and their distinct roles. While this undoubtedly adds to Cochrane authors' already demanding workload, it seems essential if the most meaningful use is to be made of the data. Reporting is a problem, and information is often difficult or even impossible to acquire. Recent reporting guidelines may help authors look for important aspects concerning the intervention (TIDieR guideline) or the whole process of complex intervention development and evaluation (CReDECI guideline)" They also mention other problems; "Apart from the problems described above, the present review suffers from the fact that most studies are fairly small, with fewer than 100 participants per group in all but one study". We would add that the time period for assessing school health approaches is also problematic. A truly comprehensive, ecological and systems-based approach to SH does more than examine a few schools or some selected interventions. it is an approach that is developed over several years at a national or sate level, with the delivery of multiple policies, funding, personnel and programs from several ministries, local agencies/school boards and then local professionals as well as the people working in the school building. Indeed, reviews of school health promotion and social development are actually far more complex than the one discussed in this editorial, which examines coordinated case management of a single health problem. It is in the light of this January 2015 Cochrane editorial that we can turn to two major recent and previous reviews of school health promotion (Langford et al, 2014; Stewart-Brown, 2006) and understand better why both of these reviews as well as others conclude that SH promotion is promising but there is insufficient evidence. For further discussion, readers might want to listen to our recent October 23, 2014 ISHN webinar with the authors of the most recent review, as it discusses the limits of RCT studies and the ensuing systematic reviews even further. We hereby challenge researchers and research funding organizations to address this challenge, perhaps beginning with the impending Cochrane Handbook Chapter on complex interventions. 0 Comments
Bayesian Statistical Analysis: Stepping out of the RCT Box 18/9/2014
A constant refrain in practitioner and policy-maker commentary on random control trial based research (usually leading to systematic reviews and other conclusions that favour artificially "controlled" conditions over the real world). An article in Issue #3, 2014 of Child Development explains how the statistical methodology used in these studies (Frequentist methods) often dictates the nature of the investigation. Although the "gentle introduction" to Bayesian methods provided in the article is hardly such, the different methodology may help us all to get out of the RCT box. The authors note that "Conventional approaches to developmental research derive from the frequentist paradigm of statistics. This paradigm associates probability with long-run frequency. The canonical example of long-run frequency is the notion of an infinite coin toss. A sample space of possible outcomes (heads and tails) is enumerated, and probability is the proportion of the outcome (say heads) over the number of coin tosses. The Bayesian paradigm, in contrast, interprets probability as the subjective experience of uncertainty (De Finetti, 1974b). Bayes’ theorem is a model for learning from data. In this paradigm, the classic example of the subjective experience of uncertainty is the notion of placing a bet. Here, unlike with the frequentist paradigm, there is no notion of infinitely repeating an event of interest. Rather, placing a bet—for example, on a baseball game or horse race—involves using as much prior information as possible as well as personal judgment. Once the outcome is revealed, then prior information is updated. This is the model of learning from experience (data) that is the essence of the Bayersian method." The authors go on to explain that " the Bayesian paradigm offers a very different view of hypothesis testing (e.g., Kaplan & Depaoli, 2012, 2013; Walker, Gustafson, & Frimer, 2007; Zhang, Hamagami, Wang, Grimm, & Nesselroade, 2007). Specifically, Bayesian approaches allow researchers to incorporate background knowledge into their analyses instead of testing essentially the same null hypothesis over and over again, ignoring the lessons of previous studies. In contrast, statistical methods based on the frequentist (classical) paradigm (i.e., the default approach in most software) often involve testing the null hypothesis. In plain terms, the null hypothesis states that “nothing is going on.” This hypothesis might be a bad starting point because, based on previous research, it is almost always expected that “something is going on." It is this faulty assumption of "nothing going on" that may force RCT type studies to compare a new program/intervention to a controlled one (which is assumed to be the null hypothesis (nothing going on) but which actually may have a lot going on. The researchers using "frequentist" statistics then conclude that the new program works (or not) when in fact, they are really comparing the new program to others in which very similar programs, or similar but disorganized activities, are actually taking place. We leave it to others more schooled in statistics to respond, but from our vantage point, the increased use of Bayersian statistical methods deserves our consideration. (Full text of the article can be accessed) Read more>> 0 Comments
A constant refrain in practitioner and policy-maker commentary on random control trial based research (usually leading to systematic reviews and other conclusions that favour artificially "controlled" conditions over the real world). An article in Issue #3, 2014 of Child Development explains how the statistical methodology used in these studies (Frequentist methods) often dictates the nature of the investigation. Although the "gentle introduction" to Bayesian methods provided in the article is hardly such, the different methodology may help us all to get out of the RCT box. The authors note that "Conventional approaches to developmental research derive from the frequentist paradigm of statistics. This paradigm associates probability with long-run frequency. The canonical example of long-run frequency is the notion of an infinite coin toss. A sample space of possible outcomes (heads and tails) is enumerated, and probability is the proportion of the outcome (say heads) over the number of coin tosses. The Bayesian paradigm, in contrast, interprets probability as the subjective experience of uncertainty (De Finetti, 1974b). Bayes’ theorem is a model for learning from data. In this paradigm, the classic example of the subjective experience of uncertainty is the notion of placing a bet. Here, unlike with the frequentist paradigm, there is no notion of infinitely repeating an event of interest. Rather, placing a bet—for example, on a baseball game or horse race—involves using as much prior information as possible as well as personal judgment. Once the outcome is revealed, then prior information is updated. This is the model of learning from experience (data) that is the essence of the Bayersian method." The authors go on to explain that " the Bayesian paradigm offers a very different view of hypothesis testing (e.g., Kaplan & Depaoli, 2012, 2013; Walker, Gustafson, & Frimer, 2007; Zhang, Hamagami, Wang, Grimm, & Nesselroade, 2007). Specifically, Bayesian approaches allow researchers to incorporate background knowledge into their analyses instead of testing essentially the same null hypothesis over and over again, ignoring the lessons of previous studies. In contrast, statistical methods based on the frequentist (classical) paradigm (i.e., the default approach in most software) often involve testing the null hypothesis. In plain terms, the null hypothesis states that “nothing is going on.” This hypothesis might be a bad starting point because, based on previous research, it is almost always expected that “something is going on." It is this faulty assumption of "nothing going on" that may force RCT type studies to compare a new program/intervention to a controlled one (which is assumed to be the null hypothesis (nothing going on) but which actually may have a lot going on. The researchers using "frequentist" statistics then conclude that the new program works (or not) when in fact, they are really comparing the new program to others in which very similar programs, or similar but disorganized activities, are actually taking place. We leave it to others more schooled in statistics to respond, but from our vantage point, the increased use of Bayersian statistical methods deserves our consideration. (Full text of the article can be accessed) Read more>> 0 Comments
Less than half of US State Health Officials use Journals for Evidence 8/9/2014
A survey of 900 officials in all US states in the August 2014 issue pf Preventing Chronic Disease reports that "On average, 45.7% of staff per state health department use journals. Common barriers to use included lack of time, lack of access, and expense. The 904 respondents were from each of the 50 state health departments and DC. There were 6 to 45 participants per state health department (mean = 31; median = 30). Response rates from state health departments varied from 58.6% to 96.0%. Participants self-identified as program managers or coordinators (57.3%), health educators (12.1%), epidemiologists (8.6%), bureau or division chiefs or directors of chronic disease units (4.5%), and 17.5% other (eg, program evaluators). An average of 45.7% of staff per state health department reported using journals as a top method for finding evidence. State health departments where at least 50% of staff identified journals as a top source participated in more research activities compared with state health departments where fewer than 50% of staff used journals (6.5 activities vs 5.0). There were no other notable differences in journal use for staffing levels, mean employee age, mean years of service, number of people served, or revenue." In the discussion section, the authors note that "Relevance of journal content for state health department practice may also influence use, although relevance was not a top barrier selected. Previous studies of public health agencies found staff wanted access to journals and gray literature for evidence directly related to public health practice (12). Unfortunately, there is limited literature focused on practice; instead, scientific evidence in journals focuses heavily on discovery research (3), which identifies existence of and relationships between health risks and health conditions (eg, smoking and lung cancer) (6,10)." Read more>> 0 Comments
A survey of 900 officials in all US states in the August 2014 issue pf Preventing Chronic Disease reports that "On average, 45.7% of staff per state health department use journals. Common barriers to use included lack of time, lack of access, and expense. The 904 respondents were from each of the 50 state health departments and DC. There were 6 to 45 participants per state health department (mean = 31; median = 30). Response rates from state health departments varied from 58.6% to 96.0%. Participants self-identified as program managers or coordinators (57.3%), health educators (12.1%), epidemiologists (8.6%), bureau or division chiefs or directors of chronic disease units (4.5%), and 17.5% other (eg, program evaluators). An average of 45.7% of staff per state health department reported using journals as a top method for finding evidence. State health departments where at least 50% of staff identified journals as a top source participated in more research activities compared with state health departments where fewer than 50% of staff used journals (6.5 activities vs 5.0). There were no other notable differences in journal use for staffing levels, mean employee age, mean years of service, number of people served, or revenue." In the discussion section, the authors note that "Relevance of journal content for state health department practice may also influence use, although relevance was not a top barrier selected. Previous studies of public health agencies found staff wanted access to journals and gray literature for evidence directly related to public health practice (12). Unfortunately, there is limited literature focused on practice; instead, scientific evidence in journals focuses heavily on discovery research (3), which identifies existence of and relationships between health risks and health conditions (eg, smoking and lung cancer) (6,10)." Read more>> 0 Comments
A Tool to Assess Organizational Context Prior to Introducing an Innovation 20/8/2014
A planning tool developed in Alberta Canada for health care organizations could be adapted for use in school health and development promotion. A 2009 journal article presents the development and initial psychometric validation of the Alberta Context Tool (ACT), an eight dimension measure of organizational context for healthcare settings. Three principles guided the development of the ACT: substantive theory, brevity, and modifiability. The Promoting Action on Research Implementation in Health Services (PARiHS) framework and related literature were used to guide selection of items in the ACT. The ACT was required to be brief enough to be tolerated in busy and resource stretched work settings and to assess concepts of organizational context that were potentially modifiable. The English version of the ACT was completed by 764 nurses (752 valid responses) working in seven Canadian pediatric care hospitals as part of its initial validation. Cronbach's alpha, exploratory factor analysis, analysis of variance, and tests of association were used to assess instrument reliability and validity. The concepts that underlie the tool are very relevant to school health promotion and development. These concepts were converted to several definitions that were generally confirmed in the research study. These definitions were developed into research questions for the study and include:
A planning tool developed in Alberta Canada for health care organizations could be adapted for use in school health and development promotion. A 2009 journal article presents the development and initial psychometric validation of the Alberta Context Tool (ACT), an eight dimension measure of organizational context for healthcare settings. Three principles guided the development of the ACT: substantive theory, brevity, and modifiability. The Promoting Action on Research Implementation in Health Services (PARiHS) framework and related literature were used to guide selection of items in the ACT. The ACT was required to be brief enough to be tolerated in busy and resource stretched work settings and to assess concepts of organizational context that were potentially modifiable. The English version of the ACT was completed by 764 nurses (752 valid responses) working in seven Canadian pediatric care hospitals as part of its initial validation. Cronbach's alpha, exploratory factor analysis, analysis of variance, and tests of association were used to assess instrument reliability and validity. The concepts that underlie the tool are very relevant to school health promotion and development. These concepts were converted to several definitions that were generally confirmed in the research study. These definitions were developed into research questions for the study and include:
- Leadership: The actions of formal leaders in an organization (unit) to influence change and excellence in practice, items generally reflect emotionally intelligent leadership
- Culture: The way that 'we do things' in our organizations and work units, items generally reflect a supportive work culture
- Evaluation: The process of using data to assess group/team performance and to achieve outcomes in organizations or units
- Social Capital: The stock of active connections among people. These connections are of three types: bonding, bridging, and linking
- Formal Interactions: Formal exchanges that occur between individuals working within an organization (unit) through scheduled activities that can promote the transfer of knowledge
- Informal Interactions: Informal exchanges that occur between individuals working within an organization (unit) that can promote the transfer of knowledge
- Structure/Electronic Resources: The structural and electronic elements of an organization (unit) that facilitate the ability to assess and use knowledge
- Organizational Slack: The cushion of actual or potential resources which allows an organization (unit) to adapt successfully to internal pressures for adjustments or to external pressures for changes
- Human Resources (staffing): Perceived and actual levels of staffing are sufficient
- Time: There is sufficient time available for staff to complete knowledge related work
- Space: There is sufficient work space in the organization for staff to complete knowledge related work.
Community Context Interacts with Intervention Characteristics to Affect Outcomes 4/7/2014
We all know that context matters in almost everything but an article in Issue #1, 2014 of the International Journal of Public Health helps to explain how context influences the process and outcomes. This narrative systematic review explored how neighbourhood interventions promote positive youth development (PYD) and the role played by the local context for these interventions. The authors analyzed 19 articles using a framework integrating standards of health promotion evaluation and elements of the ecological systems perspective. They report that "First, results highlight the key characteristics of interventions that promote PYD. An intervention’s atmosphere encouraging supportive relationships and an intervention’s activities aiming to build skills and that are real and challenging promoted PYD elements including cognitive competences, confidence, connection, leadership, civic engagement, and feelings of empowerment. Secondly, this review identified facilitators (e.g. partnerships and understanding of the community) and constraints (e.g. funding and conflicts) to an intervention’s integration within its context. They conclude that their review confirmed other reviews that suggested that interventions’ characteristics affected outcomes. But their findings indicate that context is an important element of effective interventions because context interacts with the characteristics of the intervention to create a good or poor fit with those particular circumstances. Read more>> 0 Comments
We all know that context matters in almost everything but an article in Issue #1, 2014 of the International Journal of Public Health helps to explain how context influences the process and outcomes. This narrative systematic review explored how neighbourhood interventions promote positive youth development (PYD) and the role played by the local context for these interventions. The authors analyzed 19 articles using a framework integrating standards of health promotion evaluation and elements of the ecological systems perspective. They report that "First, results highlight the key characteristics of interventions that promote PYD. An intervention’s atmosphere encouraging supportive relationships and an intervention’s activities aiming to build skills and that are real and challenging promoted PYD elements including cognitive competences, confidence, connection, leadership, civic engagement, and feelings of empowerment. Secondly, this review identified facilitators (e.g. partnerships and understanding of the community) and constraints (e.g. funding and conflicts) to an intervention’s integration within its context. They conclude that their review confirmed other reviews that suggested that interventions’ characteristics affected outcomes. But their findings indicate that context is an important element of effective interventions because context interacts with the characteristics of the intervention to create a good or poor fit with those particular circumstances. Read more>> 0 Comments
Local Health Authorities Use of Research Evidence on CSH 16/1/2014
The research supporting multi-intervention approaches and programs in school health promotion is abundant and long standing. Recent research is underlining the need for long-term, systemic planning and investments in capacity-building in areas such as coordination, work force development, inter-agency agreements, coordinated policy development and joint issue management. However, experience in the real world suggests that local health authorities (LHA) do not follow that research and instead, constantly try to implement short-term, project-style interventions based on a "training, then hoping" strategy. Four articles in Issue #6, 2013 of American Journal of Preventive Medicine examine evidence-based decision-making in LHA's and provide clues for this cognitive dissonance. One article examines the factors that affect evidence-based decision-making (EBDM) and found that "Although most people understood the concept, a relatively small number had substantial expertise and experience with its practice. Many indicated that they applied EBDM unevenly. Factors associated with use of EBDM included strong leadership; workforce capacity (number and skills); resources; funding and program mandates; political support; and access to data and program models suitable to community conditions." A second article found that "Local decision-making authority was perceived as greatly restricted by what public health activities were legally mandated and the categoric nature of funding sources, even as some leaders exercised deliberate strategic approaches. One’s workforce and board of health were also influential in making decisions regarding resource allocations. Programmatic mandates, funding restrictions, local stakeholders, and workforce capacity appear to trump factors such as research evidence and perceived community need in public health resource allocation." A third article suggested that LHA participation in a "Practice-based Research Network" will help in the implementation of research. A fourth article introduced the idea that LHA's could make use of local taxation revenue. Read More>> 0 Comments
The research supporting multi-intervention approaches and programs in school health promotion is abundant and long standing. Recent research is underlining the need for long-term, systemic planning and investments in capacity-building in areas such as coordination, work force development, inter-agency agreements, coordinated policy development and joint issue management. However, experience in the real world suggests that local health authorities (LHA) do not follow that research and instead, constantly try to implement short-term, project-style interventions based on a "training, then hoping" strategy. Four articles in Issue #6, 2013 of American Journal of Preventive Medicine examine evidence-based decision-making in LHA's and provide clues for this cognitive dissonance. One article examines the factors that affect evidence-based decision-making (EBDM) and found that "Although most people understood the concept, a relatively small number had substantial expertise and experience with its practice. Many indicated that they applied EBDM unevenly. Factors associated with use of EBDM included strong leadership; workforce capacity (number and skills); resources; funding and program mandates; political support; and access to data and program models suitable to community conditions." A second article found that "Local decision-making authority was perceived as greatly restricted by what public health activities were legally mandated and the categoric nature of funding sources, even as some leaders exercised deliberate strategic approaches. One’s workforce and board of health were also influential in making decisions regarding resource allocations. Programmatic mandates, funding restrictions, local stakeholders, and workforce capacity appear to trump factors such as research evidence and perceived community need in public health resource allocation." A third article suggested that LHA participation in a "Practice-based Research Network" will help in the implementation of research. A fourth article introduced the idea that LHA's could make use of local taxation revenue. Read More>> 0 Comments
Sustainability of Programs Doubtful without Continued External Support 8/1/2014
Two articles in the November 2013 Issue of Preventing Chronic Disease cast doubt on whether public health and school health programs are sustainable without continuing external funding and staff support. The first article examines the sustainability of state obesity prevention programs after the discontinuation of a US federal granting program. Noting that program evaluation rarely occurs after such funding periods, the authors report "Many of the programs that continued to run after the grant expired operated at reduced capacity, either reaching fewer people or conducting fewer activities. Many states were able to find funding from other sources, this shift often resulted in priorities changing. Evaluation capacity suffered in all states. Nearly all states reported losing infrastructure to communicate widely with partners. All states reported a severe or complete loss of their ability to provide training and technical assistance to partners. Despite these reduced capacities, states reported several key resources that facilitated continued work on the state plan". In the second article, the study examined the impact of a US requirement for local wellness policies on activity and nutrition in rural Colardo elementary schools after five years. The researchers reported "modest changes", with "Minutes for PE and recess did not increase, nor did offerings of fresh fruits and vegetables. More schools adopted policies prohibiting teachers from taking recess away as punishment (9.7% in 2005 vs 38.5% in 2011, P = .02) or for making up missed instructional time, class work, or tests in other subjects (3.2% in 2005 vs 28.2% in 2011, P = .03). More schools scheduled recess before lunch (22.6% in 2005 vs 46.2% in 2011, P = .04) and developed policies for vending machines (42.9% in 2005 vs 85.7% in 2011, P = .01) and parties (21.4% in 2005 vs 57.9% in 2011, P = .004). Read more>> 0 Comments
Two articles in the November 2013 Issue of Preventing Chronic Disease cast doubt on whether public health and school health programs are sustainable without continuing external funding and staff support. The first article examines the sustainability of state obesity prevention programs after the discontinuation of a US federal granting program. Noting that program evaluation rarely occurs after such funding periods, the authors report "Many of the programs that continued to run after the grant expired operated at reduced capacity, either reaching fewer people or conducting fewer activities. Many states were able to find funding from other sources, this shift often resulted in priorities changing. Evaluation capacity suffered in all states. Nearly all states reported losing infrastructure to communicate widely with partners. All states reported a severe or complete loss of their ability to provide training and technical assistance to partners. Despite these reduced capacities, states reported several key resources that facilitated continued work on the state plan". In the second article, the study examined the impact of a US requirement for local wellness policies on activity and nutrition in rural Colardo elementary schools after five years. The researchers reported "modest changes", with "Minutes for PE and recess did not increase, nor did offerings of fresh fruits and vegetables. More schools adopted policies prohibiting teachers from taking recess away as punishment (9.7% in 2005 vs 38.5% in 2011, P = .02) or for making up missed instructional time, class work, or tests in other subjects (3.2% in 2005 vs 28.2% in 2011, P = .03). More schools scheduled recess before lunch (22.6% in 2005 vs 46.2% in 2011, P = .04) and developed policies for vending machines (42.9% in 2005 vs 85.7% in 2011, P = .01) and parties (21.4% in 2005 vs 57.9% in 2011, P = .004). Read more>> 0 Comments
Studying how health ministries use evidence-based prevention practices 3/1/2014
The protocol for a cross-border study of health ministry use of evidence-based practices in chronic disease prevention is presented in the December 2013 issue of Implementation Science. The authors state "Evidence-based public health approaches to prevent chronic diseases have been identified in recent decades and have the potential for high impact. Yet, barriers to implement prevention approaches persist as a result of multiple factors including lack of organizational support, limited resources, competing emerging priorities and crises, and limited skill among the public health workforce. The purpose of this study is to learn how best to promote the adoption of evidence based public health practice related to chronic disease prevention. This study has the potential to be innovative in several ways. This study will be among the first to provide the public health field with information about the facilitators and strategies that state level practitioners use in evidence based chronic disease prevention. Measures of dissemination among practitioners working in prevention of cancer and other chronic diseases are lacking [79-82]. This study will be among the first to develop, test, and utilize such measures. This study is among the first to apply Institutional Theory with frameworks used in public health, specifically Diffusion of Innovations and a knowledge transfer and utilization framework. The study has the potential for future large scale impact as it may identify effective ways to disseminate public health knowledge needed for EBDM processes in different contexts and help shorten the time between research evidence discovery and program application delivery." To this list of innovative aspects, we add one more. This is one of the first times that the subjects of the study are officials in health ministries, identifying their concerns, rather than focusing on front-line practitioners. At the same time, it should be noted that the specific focus of the study appears to be focused on whether the ministry officials are aware of and are using knowledge about better practices. Since knowledge exchange and transfer is only one of several system capacities required to implement and maintain quality improvements (others include coordinated policy, assignment of coordinators, formal and informal mechanisms for cooperation, ongoing work force development, regular monitoring/reporting, joint strategic issue management across systems and explicit sustainability planning), the study may or may not determine or describe the real world roles of ministry officials in promoting better practices and system change. Read more>> 0 Comments
The protocol for a cross-border study of health ministry use of evidence-based practices in chronic disease prevention is presented in the December 2013 issue of Implementation Science. The authors state "Evidence-based public health approaches to prevent chronic diseases have been identified in recent decades and have the potential for high impact. Yet, barriers to implement prevention approaches persist as a result of multiple factors including lack of organizational support, limited resources, competing emerging priorities and crises, and limited skill among the public health workforce. The purpose of this study is to learn how best to promote the adoption of evidence based public health practice related to chronic disease prevention. This study has the potential to be innovative in several ways. This study will be among the first to provide the public health field with information about the facilitators and strategies that state level practitioners use in evidence based chronic disease prevention. Measures of dissemination among practitioners working in prevention of cancer and other chronic diseases are lacking [79-82]. This study will be among the first to develop, test, and utilize such measures. This study is among the first to apply Institutional Theory with frameworks used in public health, specifically Diffusion of Innovations and a knowledge transfer and utilization framework. The study has the potential for future large scale impact as it may identify effective ways to disseminate public health knowledge needed for EBDM processes in different contexts and help shorten the time between research evidence discovery and program application delivery." To this list of innovative aspects, we add one more. This is one of the first times that the subjects of the study are officials in health ministries, identifying their concerns, rather than focusing on front-line practitioners. At the same time, it should be noted that the specific focus of the study appears to be focused on whether the ministry officials are aware of and are using knowledge about better practices. Since knowledge exchange and transfer is only one of several system capacities required to implement and maintain quality improvements (others include coordinated policy, assignment of coordinators, formal and informal mechanisms for cooperation, ongoing work force development, regular monitoring/reporting, joint strategic issue management across systems and explicit sustainability planning), the study may or may not determine or describe the real world roles of ministry officials in promoting better practices and system change. Read more>> 0 Comments
Finally, Common Sense about Community Contexts 13/5/2013
An article in the May 2013 issue of the Journal of School Health confirms a common sense understanding about the impact of the local community context on health and behaviours such as physical activity. Tge authors report that: "This study examined differences in students' time spent in physical activity (PA) across secondary schools in rural, suburban, and urban environments and identified the environment-level factors associated with these between school differences in students' PA. Multilevel linear regression analyses were used to examine the environment- and student-level characteristics associated with time spent in PA among grades 9 to 12 students attending 76 secondary schools in Ontario, Canada, as part of the SHAPES-Ontario study. This approach was first conducted with the full data set testing for interactions between environment-level factors and school location. Then, school-location specific regression models were run separately. Statistically significant between-school variation was identified among students attending urban, suburban, and rural schools, where school-level differences accounted for 4.0%, 2.0%, and 2.1% of the variability in students' time spent in PA, respectively. Students attending an urban or suburban school that provided another room for PA or was located within close proximity to a shopping mall or fast food outlet spent more time in PA. Developing a better understanding of the environment-level characteristics associated with students' time spent in PA by school location may help public health and planning experts to tailor school programs and policies to the needs of students in different locations. Read More>> 0 Comments
An article in the May 2013 issue of the Journal of School Health confirms a common sense understanding about the impact of the local community context on health and behaviours such as physical activity. Tge authors report that: "This study examined differences in students' time spent in physical activity (PA) across secondary schools in rural, suburban, and urban environments and identified the environment-level factors associated with these between school differences in students' PA. Multilevel linear regression analyses were used to examine the environment- and student-level characteristics associated with time spent in PA among grades 9 to 12 students attending 76 secondary schools in Ontario, Canada, as part of the SHAPES-Ontario study. This approach was first conducted with the full data set testing for interactions between environment-level factors and school location. Then, school-location specific regression models were run separately. Statistically significant between-school variation was identified among students attending urban, suburban, and rural schools, where school-level differences accounted for 4.0%, 2.0%, and 2.1% of the variability in students' time spent in PA, respectively. Students attending an urban or suburban school that provided another room for PA or was located within close proximity to a shopping mall or fast food outlet spent more time in PA. Developing a better understanding of the environment-level characteristics associated with students' time spent in PA by school location may help public health and planning experts to tailor school programs and policies to the needs of students in different locations. Read More>> 0 Comments
A Framework for Understanding Complexity 11/2/2013
An article in Issue #1, 2012 of Health Promotion International presents a framework to better understand complexity in health promotion research and practice. The authors note that "Health promotion addresses issues from the simple (with well-known cause/effect links) to the highly complex (webs and loops of cause/effect with unpredictable, emergent properties). Yet there is no conceptual framework within its theory base to help identify approaches appropriate to the level of complexity. The default approach favours reductionism—the assumption that reducing a system to its parts will inform whole system behaviour. Such an approach can yield useful knowledge, yet is inadequate where issues have multiple interacting causes, such as social determinants of health. To address complex issues, there is a need for a conceptual framework that helps choose action that is appropriate to context. This paper presents the Cynefin Framework, informed by complexity science—the study of Complex Adaptive Systems (CAS). It introduces key CAS concepts and reviews the emergence and implications of ‘complex’ approaches within health promotion. It explains the framework and its use with examples from contemporary practice, and sets it within the context of related bodies of health promotion theory. The Cynefin Framework, especially when used as a sense-making tool, can help practitioners understand the complexity of issues, identify appropriate strategies and avoid the pitfalls of applying reductionist approaches to complex situations." Read more. 0 Comments
An article in Issue #1, 2012 of Health Promotion International presents a framework to better understand complexity in health promotion research and practice. The authors note that "Health promotion addresses issues from the simple (with well-known cause/effect links) to the highly complex (webs and loops of cause/effect with unpredictable, emergent properties). Yet there is no conceptual framework within its theory base to help identify approaches appropriate to the level of complexity. The default approach favours reductionism—the assumption that reducing a system to its parts will inform whole system behaviour. Such an approach can yield useful knowledge, yet is inadequate where issues have multiple interacting causes, such as social determinants of health. To address complex issues, there is a need for a conceptual framework that helps choose action that is appropriate to context. This paper presents the Cynefin Framework, informed by complexity science—the study of Complex Adaptive Systems (CAS). It introduces key CAS concepts and reviews the emergence and implications of ‘complex’ approaches within health promotion. It explains the framework and its use with examples from contemporary practice, and sets it within the context of related bodies of health promotion theory. The Cynefin Framework, especially when used as a sense-making tool, can help practitioners understand the complexity of issues, identify appropriate strategies and avoid the pitfalls of applying reductionist approaches to complex situations." Read more. 0 Comments
A Fidelity Implementation Model includes Assessing Participants & Practitioner Responses 18/1/2013
An article from Issue #6, 2012 of the Journal of Child & Family Development presents and applies a fidelity implementation model that nudges out of the traditional rigid box of implementation science by actually including a major component on participant responses to the intervention as well as program differentiation. The authors note that " Fidelity, or the degree to which interventions are implemented as originally designed, is generally conceptualized as a system of unique and interrelated components including adherence, dosage and quality of delivery". To accept the notions that practitioners will inevitably modify the program and that different types of program participants may react differently to the intervention is a step forward. Read more. 0 Comments
An article from Issue #6, 2012 of the Journal of Child & Family Development presents and applies a fidelity implementation model that nudges out of the traditional rigid box of implementation science by actually including a major component on participant responses to the intervention as well as program differentiation. The authors note that " Fidelity, or the degree to which interventions are implemented as originally designed, is generally conceptualized as a system of unique and interrelated components including adherence, dosage and quality of delivery". To accept the notions that practitioners will inevitably modify the program and that different types of program participants may react differently to the intervention is a step forward. Read more. 0 Comments
Realist Controlled Trials: A New Research Method? 13/12/2012
An article in the December 2012 Issue of Social Science & Medicine suggests that realist perspectives should be integrated within random controlled trials in order to better understand the complexity of interventions and how their components and their characteristics interact with the local context. The authors suggest that `Randomized trials of complex public health interventions generally aim to identify what works, accrediting specific intervention ‘products’ as effective. This approach often fails to give sufficient consideration to how intervention components interact with each other and with local context. ‘Realists’ argue that trials misunderstand the scientific method, offer only a ‘successionist’ approach to causation, which brackets out the complexity of social causation, and fail to ask which interventions work, for whom and under what circumstances. We counter-argue that trials are useful in evaluating social interventions because randomized control groups actually take proper account of rather than bracket out the complexity of social causation. Nonetheless, realists are right to stress understanding of ‘what works, for whom and under what circumstances’ and to argue for the importance of theorizing and empirically examining underlying mechanisms.`The authors also propose that ‘realist’ trials should aim to: examine the effects of intervention components separately and in combination, explore mechanisms of change, analysing how pathway variables mediate intervention effects; use multiple trials across contexts; draw on qualitative & quantitative data; and be oriented towards building theories setting out how interventions interact with context. This last suggestion resonates with recent suggestions that, in delivering truly ‘complex’ interventions, fidelity is important not so much in terms of precise activities but, rather, key intervention ‘processes’ and ‘functions’. Read more
0 Comments
An article in the December 2012 Issue of Social Science & Medicine suggests that realist perspectives should be integrated within random controlled trials in order to better understand the complexity of interventions and how their components and their characteristics interact with the local context. The authors suggest that `Randomized trials of complex public health interventions generally aim to identify what works, accrediting specific intervention ‘products’ as effective. This approach often fails to give sufficient consideration to how intervention components interact with each other and with local context. ‘Realists’ argue that trials misunderstand the scientific method, offer only a ‘successionist’ approach to causation, which brackets out the complexity of social causation, and fail to ask which interventions work, for whom and under what circumstances. We counter-argue that trials are useful in evaluating social interventions because randomized control groups actually take proper account of rather than bracket out the complexity of social causation. Nonetheless, realists are right to stress understanding of ‘what works, for whom and under what circumstances’ and to argue for the importance of theorizing and empirically examining underlying mechanisms.`The authors also propose that ‘realist’ trials should aim to: examine the effects of intervention components separately and in combination, explore mechanisms of change, analysing how pathway variables mediate intervention effects; use multiple trials across contexts; draw on qualitative & quantitative data; and be oriented towards building theories setting out how interventions interact with context. This last suggestion resonates with recent suggestions that, in delivering truly ‘complex’ interventions, fidelity is important not so much in terms of precise activities but, rather, key intervention ‘processes’ and ‘functions’. Read more
0 Comments
Fallacy of Using Medical Terms/Structures in Public Health: Example of "Dose-Response" 4/10/2012
An article in Issue #4, 2012 of Critical Public Health opens the door to a discussion of how public health systems need to divest themselves of terms, practices, structures and underlying assumptions imported from the health care system. The authiors note that "Public Health specialists have increasingly deployed the concept of ‘dose–response’ in areas such as diet (‘five-a-day’), alcohol (‘21 weekly units’) and physical activity (‘150 minutes of weekly activity’). Using these examples and a case study that sought to establish an optimal dose of physical activity for mental health gain, this article offers a critical assessment of the nature, robustness and function of ‘dose’ in public health. Drawing on a ‘sociology of knowledge’, the article argues that dose–response can best be considered an analogy that does not necessarily translate favourably from its original expression in toxicology to some public health domains – an over-extended analogy. Rather than having technical robustness, its attractiveness and utility is seen to lie in it possessing ‘cultural capital’ (ie sounding medical). Here, the ability to link behavioural concerns to clinical practice, to simplify complex ideas and to act as a regulatory form of behavioural governance. The article is skeptical of further empirical pursuits in identifying optimal doses and offers an alternative course for public health framing. Read more.. 0 Comments
An article in Issue #4, 2012 of Critical Public Health opens the door to a discussion of how public health systems need to divest themselves of terms, practices, structures and underlying assumptions imported from the health care system. The authiors note that "Public Health specialists have increasingly deployed the concept of ‘dose–response’ in areas such as diet (‘five-a-day’), alcohol (‘21 weekly units’) and physical activity (‘150 minutes of weekly activity’). Using these examples and a case study that sought to establish an optimal dose of physical activity for mental health gain, this article offers a critical assessment of the nature, robustness and function of ‘dose’ in public health. Drawing on a ‘sociology of knowledge’, the article argues that dose–response can best be considered an analogy that does not necessarily translate favourably from its original expression in toxicology to some public health domains – an over-extended analogy. Rather than having technical robustness, its attractiveness and utility is seen to lie in it possessing ‘cultural capital’ (ie sounding medical). Here, the ability to link behavioural concerns to clinical practice, to simplify complex ideas and to act as a regulatory form of behavioural governance. The article is skeptical of further empirical pursuits in identifying optimal doses and offers an alternative course for public health framing. Read more.. 0 Comments
Local Drivers of Change In Cincinnati & St. Louis School Systems 4/10/2012
Local drivers of change and continuity in each community and organization will affect the processes of adoption, implementation, operation and sustainability of school programs and multi-intervention approaches. The local history, existing inter-personal relationships, incidents, external directives and demands and many other circumstances, which are often ignored in controlled trial studies, all eventually become significant in one way or another. A case study in Issue #5, 2012 of Education and Urban Society gives us an idea of how these local drivers play out in on-going narratives. This study investigated the recent histories of the urban public school districts of St. Louis, Missouri and Cincinnati, Ohio in the United States of America. The purpose was to identify the similar challenges that each had recently faced in regard to the maintenance and development of its public school systems, as well to gauge the level of community support for the numerous reforms proposed by local education officials in each city over the past decade. Each city was also examined in its particular historical context, with the accompanying educational, political, and social issues which have manifested themselves to the present state of affairs. After a critical review of the research and applicable theory, the authors offer suggestions for solidifying the future success of each system with examples of positive experimentation occurring in these cities and others. Read more.. 0 Comments
Local drivers of change and continuity in each community and organization will affect the processes of adoption, implementation, operation and sustainability of school programs and multi-intervention approaches. The local history, existing inter-personal relationships, incidents, external directives and demands and many other circumstances, which are often ignored in controlled trial studies, all eventually become significant in one way or another. A case study in Issue #5, 2012 of Education and Urban Society gives us an idea of how these local drivers play out in on-going narratives. This study investigated the recent histories of the urban public school districts of St. Louis, Missouri and Cincinnati, Ohio in the United States of America. The purpose was to identify the similar challenges that each had recently faced in regard to the maintenance and development of its public school systems, as well to gauge the level of community support for the numerous reforms proposed by local education officials in each city over the past decade. Each city was also examined in its particular historical context, with the accompanying educational, political, and social issues which have manifested themselves to the present state of affairs. After a critical review of the research and applicable theory, the authors offer suggestions for solidifying the future success of each system with examples of positive experimentation occurring in these cities and others. Read more.. 0 Comments
Knowledge Management to Coordinate Health & Social Systems 4/10/2012
One of the characteristics of large "professional bureaucracies" such as education, health and social service is that knowledge is a source of power and influence within those layered and loosely-coupled structures. Consequently, it makes sense that knowledge management (KM) strategies can be helpful in coordinating these systems and enabling the local agencies and professionals to work together. An article in Issue #5, 2012 of Health & Social Care in the Community reports on the use of KM strategies to integrate the efforts of two systems. The authors completed a critical review of the literature to identify theoretical insights and models in this field. The findings were then used to explore the approach to KM. This case study involved an interrogation of relevant documentary material, together with 25 in-depth interviews with managers and professionals. The authors no planned KM strategies for learning and KM, but rather, interventions and mechanisms at different levels to support integration processes. These included formal activities, training and appraisal, but also informal ones within communities of practice and networking. Although structural enablers such as a co-location of facilities and joint appointments were important, the value of trust and inter-personal relationships was highlighted especially for tacit knowledge exchange. The conclusion to the article uses Nonaka’s knowledge conversation model to reflect on the research findings, to comment on the absence of an explicit approach to learning and KM, and to develop a template to assist policy-makers with the design of planned strategies" Read more.. 0 Comments
One of the characteristics of large "professional bureaucracies" such as education, health and social service is that knowledge is a source of power and influence within those layered and loosely-coupled structures. Consequently, it makes sense that knowledge management (KM) strategies can be helpful in coordinating these systems and enabling the local agencies and professionals to work together. An article in Issue #5, 2012 of Health & Social Care in the Community reports on the use of KM strategies to integrate the efforts of two systems. The authors completed a critical review of the literature to identify theoretical insights and models in this field. The findings were then used to explore the approach to KM. This case study involved an interrogation of relevant documentary material, together with 25 in-depth interviews with managers and professionals. The authors no planned KM strategies for learning and KM, but rather, interventions and mechanisms at different levels to support integration processes. These included formal activities, training and appraisal, but also informal ones within communities of practice and networking. Although structural enablers such as a co-location of facilities and joint appointments were important, the value of trust and inter-personal relationships was highlighted especially for tacit knowledge exchange. The conclusion to the article uses Nonaka’s knowledge conversation model to reflect on the research findings, to comment on the absence of an explicit approach to learning and KM, and to develop a template to assist policy-makers with the design of planned strategies" Read more.. 0 Comments
Two revolutionary thoughts about the use of research-based knowledge 27/9/2012
Earlier this month, we noted a debate about the usefulness of RCT's in real world conditions that was initiated by a leading authors of systematic reviews, Sarah Stewart-Brown. Two articles in Issue #6, 2012 of the American Psychologist continue these revolutionary thoughts. One article questions the wisdom of basing scholarship and knowledge development on an ever-increasing number of research reviews, that examine different interventions in different contexts and often clumped together in inappropriate ways. The second article suggests that rather than trying to reframe systems in the light of accumulated evidence from research, we seek to identify "disruptive innovations" such as micro-clinics in retail chain drug stores, $2 generic eyeglasses and automatic teller machines that fit into real world situations and offer practical convenience to the intended users. Read more.. 0 Comments
Earlier this month, we noted a debate about the usefulness of RCT's in real world conditions that was initiated by a leading authors of systematic reviews, Sarah Stewart-Brown. Two articles in Issue #6, 2012 of the American Psychologist continue these revolutionary thoughts. One article questions the wisdom of basing scholarship and knowledge development on an ever-increasing number of research reviews, that examine different interventions in different contexts and often clumped together in inappropriate ways. The second article suggests that rather than trying to reframe systems in the light of accumulated evidence from research, we seek to identify "disruptive innovations" such as micro-clinics in retail chain drug stores, $2 generic eyeglasses and automatic teller machines that fit into real world situations and offer practical convenience to the intended users. Read more.. 0 Comments
Linear Logic Models, Competing Needs and Complexity 24/8/2012
Farm to school nutrition programs are emerging in many countries as the next innovative approach to healthy eating. Three articles in Issue #4, 2012 of Childhood Obesity provide excellent analyses, along with others on school salad bars, school gardens and the elimination of processed foods by cooking from scratch in schools. All of these are good but in this commentary, we want to illustrate how narrow our focus can become, even when we use evidence-based logic models and behaviour theory. One article in this journal issue presents an excellent linear logic model that correctly depicts the multiple inputs, processes, outputs and outcomes associated with FtoS programs. As well, the model correctly seeks to take local context and feedback loops within the social system of the school and community into account. In summary, the paper presents one of the better linear logic models that we have seen. But here is the rub: what about the complexity of the school-neighbourhood-family setting and the competition from the many, many other school programs also entering that same context? If we do consider these things, then program innovations like this can become pretty academic pretty quickly. Read more 0 Comments
Farm to school nutrition programs are emerging in many countries as the next innovative approach to healthy eating. Three articles in Issue #4, 2012 of Childhood Obesity provide excellent analyses, along with others on school salad bars, school gardens and the elimination of processed foods by cooking from scratch in schools. All of these are good but in this commentary, we want to illustrate how narrow our focus can become, even when we use evidence-based logic models and behaviour theory. One article in this journal issue presents an excellent linear logic model that correctly depicts the multiple inputs, processes, outputs and outcomes associated with FtoS programs. As well, the model correctly seeks to take local context and feedback loops within the social system of the school and community into account. In summary, the paper presents one of the better linear logic models that we have seen. But here is the rub: what about the complexity of the school-neighbourhood-family setting and the competition from the many, many other school programs also entering that same context? If we do consider these things, then program innovations like this can become pretty academic pretty quickly. Read more 0 Comments
A System & Scale to Measure & Develop Sustainable School Strategies 17/8/2012
An article in Issue #5, 2012 of the Journal of Community Psychology reports on a framework to guide and monitor the development of sustainable school-based prevention programs and approaches. School-based prevention and promotion interventions (SBPPI) improve desirable outcomes (e.g., commitment to school and attendance) and reduce undesirable outcomes (e.g., suspensions and violence). Unfortunately, our understanding of how to effectively implement and sustain SBPPI outside of well-controlled conditions is lacking. To bridge this science/“real world” practice gap, a system of sustainable implementation, which merges implementation strategies and sustainability strategies, is proposed in this article. Ecological levels and phases are part of this new concept. This conceptualization is supported by analyses from a diverse sample of 157 schools implementing Social-Emotional Character Development. The authors used the system of sustainable implementation that was measured using the Schools Implementing Towards Sustainability (SITS) scale, which was designed to be “user-friendly” in field settings by being viable and scalable. The SITS demonstrated strong reliability as well as promising concurrent and construct validity. Read more.. 0Comments
An article in Issue #5, 2012 of the Journal of Community Psychology reports on a framework to guide and monitor the development of sustainable school-based prevention programs and approaches. School-based prevention and promotion interventions (SBPPI) improve desirable outcomes (e.g., commitment to school and attendance) and reduce undesirable outcomes (e.g., suspensions and violence). Unfortunately, our understanding of how to effectively implement and sustain SBPPI outside of well-controlled conditions is lacking. To bridge this science/“real world” practice gap, a system of sustainable implementation, which merges implementation strategies and sustainability strategies, is proposed in this article. Ecological levels and phases are part of this new concept. This conceptualization is supported by analyses from a diverse sample of 157 schools implementing Social-Emotional Character Development. The authors used the system of sustainable implementation that was measured using the Schools Implementing Towards Sustainability (SITS) scale, which was designed to be “user-friendly” in field settings by being viable and scalable. The SITS demonstrated strong reliability as well as promising concurrent and construct validity. Read more.. 0Comments
Local drivers (opportunities/barriers) in adopting and implementing prevention approaches and programs 10/11/2011
The nitty-gritty, practical details of introducing, adopting and implementing prevention programs raises issues and questions that are almost always ignored or controlled by formal research studies. The discussion will facilitate the development of questions and perspectives from teachers, nurses, social workers, addiction workers, educational administrators and police officers. This “worm’s eye” view is essential if we are to truly understand how various prevention programs or educational resources can be developed, disseminated and diffused effectively.This discussion of the practical details of implementation is part of an international series of webinars, web meetings and online wiki-based discussions about Implementation, Capacity, Sustainability and Systems Change. Join the call for contributors to this discussion. Read More 2 Comments
The nitty-gritty, practical details of introducing, adopting and implementing prevention programs raises issues and questions that are almost always ignored or controlled by formal research studies. The discussion will facilitate the development of questions and perspectives from teachers, nurses, social workers, addiction workers, educational administrators and police officers. This “worm’s eye” view is essential if we are to truly understand how various prevention programs or educational resources can be developed, disseminated and diffused effectively.This discussion of the practical details of implementation is part of an international series of webinars, web meetings and online wiki-based discussions about Implementation, Capacity, Sustainability and Systems Change. Join the call for contributors to this discussion. Read More 2 Comments