The second article, published in Issue #1, 2015 of Health Technology Assessment, was to identify the impacts and likely impacts on health care, patient outcomes and value for money of Cochrane Reviews published by 20 NIHR-funded CRGs during the years 2007–11. The authors note that "we found 40 examples where reviews appeared to have influenced primary research and reviews had contributed to the creation of new knowledge and stimulated debate. Twenty-seven of the 60 reviews had 100 or more citations in Google Scholar™ (Google, CA, USA). Overall, 483 systematic reviews had been cited in 247 sets of guidance. This included 62 sets of international guidance, 175 sets of national guidance (87 from the UK) and 10 examples of local guidance. Evidence from the interviews suggested that Cochrane Reviews often play an instrumental role in informing guidance, although reviews being a poor fit with guideline scope or methods, reviews being out of date and a lack of communication between CRGs and GDs were barriers to their use. Cochrane Reviews appeared to have led to a number of benefits to the health service including safer or more appropriate use of medication or other health technologies or the identification of new effective drugs or treatments. However, whether or not these changes were directly as a result of the Cochrane Review and not the result of subsequent clinical guidance was difficult to judge." The authors of this second article concluded that " The clearest impacts of Cochrane Reviews are on research targeting and health-care policy, with less evidence of a direct impact on clinical practice and the organisation and delivery of NHS services". In other words, systematic reviews and possibly even the practice guidelines that try to use such Cochrane Reviews as their basis, may or may not affect practice.
The third article, published in the April 2015 Issue of the Cochrane Database of Systematic Reviews, examined "whether tailored intervention strategies are effective in improving professional practice and healthcare outcomes. We compared interventions tailored to address the identified determinants of practice with either no intervention or interventions not tailored to the determinants...Attempts to change the behaviour of health professionals may be impeded by a variety of different barriers, obstacles, or factors (which we collectively refer to as determinants of practice). Change may be more likely if implementation strategies are specifically chosen to address these determinants." The authors conclude that "The findings continue to indicate that tailored interventions can change professional practice, although they are not always effective and, when they are, the effect is small to moderate. There is insufficient evidence on the most effective approaches to tailoring, including how determinants should be identified, how decisions should be made on which determinants are most important to address, and how interventions should be selected to account for the important determinants. In addition, there is no evidence about the cost-effectiveness of tailored interventions compared to other interventions to change professional practice."
Our take away from these three and other studies we have been reading is that knowledge about better practices or better programs is insufficient to implement or sustain improvements in professional or organizational practices. The answers lie within the organizational or community context, likely based on their current core mandates, perceived and real constraints, traditions and routines and current system-level and adopter concerns at various levels. In other words, we may need to make a significant shift away from "evidence-based practice" towards auch better understanding of "practice-based experience".