Increasing the Generalisability of Improvement Research with an Improvement Replication Programme

Generalization of quality improvements (QI) is different (and more difficult) than for randomized controlled trials. To draw a parallel, the subject of the QI study is a provider or organizational unit and the treatment is an intervention to change behavior or organization. Findings in QI have less strong internal and external validity than randomized controlled trials. Further complicating the matter, many QI interventions cannot be easily described in a fixed protocol as practitioners adapt them during implementation.

The authors present a design for a program of research that deals with a specific problem across a variety of settings. The program has four parts or steps:

  1. Generalizable knowledge. The program design starts with previous knowledge about the intervention to be investigated and assumptions about how the intervention will produce the desired results.
  2. Site implementation. Implementers consider how to accommodate differences in the population being treated, staff and other aspects of the setting and resources available.
  3. Test and refine. The Plan–Do–Study–Act cycle formalizes the iterative process and amendments to the original change.
  4. Outcomes improved. Information from each part of the program design provides feedback to inform local improvements.

Individually funded studies can contribute knowledge that is useful to decision-makers.