IMPLEMENT AN EVIDENCE-BASED PROGRAM

SECTION

5.4

QUALITY ASSURANCE

ADDITIONAL RESOURCE

Quality assurance (QA) is a collection of planned, systematic activities applied to ensure that program components are being implemented well. Initiatives to improve program activities and prevent mistakes or defects also constitute QA. Despite the initial appearance of similarity, QA differs from evaluation in that QA focuses on ongoing work instead of final outcomes at a particular point in time. In an era where funding opportunities are often competitive, QA is critical because it demonstrates to funders and stakeholders that EBPs are credible, worthwhile investments.

Continuous Quality Improvement (CQI), an ongoing quality improvement process that is closely related to QA, involves four steps: (1) planning (e.g., selecting implementation objectives and methods of monitoring program delivery), (2) monitoring (e.g., collecting process and outcome data and obtaining feedback from stakeholders), (3) evaluating (e.g., analyzing data collected during monitoring to identify challenges and develop solutions for them), and (4) making corrective changes (e.g., implementing solutions to identified challenges in order to improve overall program performance and participant satisfaction).

A QA program that integrates CQI will include the following components.

  • A specified timeline for QA activities and designated roles and responsibilities for personnel.

  • Personnel and stakeholder orientation to the QA plan and procedures.

  • Designated performance indicators including measures of participant reach, organizational capacity, and program delivery.

  • Mechanisms for personnel to conduct periodic reviews of fidelity and assess performance using indicators.

  • Protocols for taking corrective action when fidelity is jeopardized or performance requires improvement.


The concept of QA can initially seem abstract and difficult to apply, but frameworks such as RE-AIM (described in the subsequent section) can help organizations implement QA in logical, manageable pieces.

RE-AIM framework


The RE-AIM framework was developed as a model for the planning, evaluation, reporting, and review of translational research and practice. (Translation is the process of taking a program originally implemented in a controlled “laboratory-like” setting and making it suitable for implementation in the community.) The framework has five components from which the framework name is drawn: Reach, Effectiveness, Adoption, Implementation, and Maintenance. Each component and its significance are described in Table 5. Information about the development and application of RE-AIM can be found at www.re-aim.org.

TABLE 5 - RE-AIM COMPONENTS

COMPONENT

DEFINITION

SIGNIFICANCE

Reach

The extent to which the program reaches the intended audience

Assesses the adequacy of program marketing, participant recruitment, and participant retention strategies.

Effectiveness

The impacts and outcomes of an intervention in terms of measures such as disease rates, behaviors, knowledge, quality of life, and healthcare expenses

Evaluates whether a program is producing positive changes in audience members’ well-being.

Adoption

The extent to which organizations and implementation sites deliver and embed the program into their ongoing activities and the level of organizational support provided for it.    

Investigates the adequacy of program partners, resources, and the implementation process (e.g., session locations, frequency, skill of personnel).

Implementation

Fidelity monitoring.    

Determines if the program is being implemented as designed, regardless of the setting or which staff member is delivering the program.

Maintenance

The extent to which a program becomes institutionalized or a routine part of an organization’s activities.    

Assesses the extent to which the program is accessible to new participants and the sustainability infrastructure.