Table 2 Methods of economic-focused data collection and analysis by phase of implementation
From: Extending analytic methods for economic evaluation in implementation science
Exploration | Preparation | Implementation | Sustainment | |
|---|---|---|---|---|
Economic Evaluation Methods | Selection of interventions and consideration of what implementation requires using prior literature and examples | Selection and tailoring of implementation strategies and the creation of tools to support planned intervention implementation and to guide subsequent data collection | Data collection and ongoing evaluation of intervention implementation | Evaluation of implementation and effectiveness outcomes and planning for local sustainment and/or scale-up/spread of the intervention |
Stakeholder interviews, surveys, and periodic reflections | • Identify key stakeholders and assess their readiness for an intervention under consideration • Review existing studies on personnel time and costs associated with intervention implementation • Solicit feedback on potential barriers to implementation of the intervention | • Assess capacity and stakeholders’ goals and preferences • Determine which activities and types of costs are associated with usual care • Design guides to evaluate costs and time from the perspective of those implementing the intervention • Identify and prepare for potential challenges in intervention implementation | • Obtain feedback on the successes and challenges of implementation • Understand unexpected challenges (e.g., turnover) in intervention implementation • Identify opportunities to better support agents involved in implementing the intervention • Assess how usual care and associated costs may change over time as a result of intervention implementation | • Assess stakeholder buy-in for sustainment of the intervention and necessary adaptations • Understand unexpected challenges (e.g., turnover) in intervention implementation |
Patient and caregiver interviews, surveys, and focus groups | • Review existing studies on patient/caregiver preferences, costs, etc. when designing the intervention and identifying relevant costs | • Design guides to evaluate costs and time from the patient/caregiver perspective | • Conduct interviews/surveys to collect patient/caregiver costs and time spent • Refine implementation strategies to better address the needs of patients/caregivers and reduce burden | • Include patient/caregiver-level costs in societal perspective for cost-effectiveness analyses |
Process flow diagramming | • Review process flow diagrams identified in the literature and compare documented processes to local context • Identify potential resource points • Consider which personnel are needed and what types of training they need • Identify potential adaptations to implementation given local challenges, processes in place, and available resources | • Assess key processes involved that affect costs before doing microcosting, cost-effectiveness analysis, etc. through developing a local process flow diagram • Identify all resource points • Consider which personnel are needed and what types of training they need • Obtain feedback on the diagram from diverse implementation agents to ensure the included steps are comprehensive and reflective of their activities • Map cost data collection activities, including type, source, and frequency, onto the steps to guide methods for estimating costs • Identify points to assess fidelity using the steps outlined in the diagram | • Track costs and other resources associated with each step involved in implementing the intervention • Identify and measure potential voltage drops (i.e., retention, missed opportunities, etc.) in implementation reach and desired outcomes • Revise diagram as needed to reflect changes and adaptations to the intervention and how it is implemented | • Include costs from all steps documented in the diagram in cost-effectiveness analyses • Adapt processes to address voltage drops and increase effectiveness in the local context • Identify resource-intensive process steps and consider opportunities for change • Track and evaluate costs associated with sustainment of the intervention |
Time-and-motion analysis | • Obtain preliminary list of cost components from existing studies and consider feasibility in the local context | • Harmonize cost measures • Design time-and-motion study, including the creation of toolkits for tracking costs, using process flow diagrams • Pilot time-and-motion toolkits and solicit feedback from agents involved in implementation | • Observe processes and record activities • Assess at multiple time points to consider improved efficiencies over time, differences across staff, etc. | • Use time-and-motion data in cost-effectiveness and simulation studies • Determine how different inputs affect overall cost-effectiveness • Optimize how processes are performed • Estimate budget for those interested in program implementation in other settings |
System support mapping | • Review existing studies on resources and staffing required for intervention implementation • Identify the individuals responsible for implementing the intervention in the local context | • Identify the individuals responsible for implementing the intervention in the local context • Develop guide for facilitating system support mapping sessions | • Identify roles and responsibilities of staff implementing the intervention and the resource requirements associated with those responsibilities • Consider contextual factors that may affect resource adequacy and delineation of roles • Adapt intervention as needed to address issues related to resource adequacy in the local context | • Adapt intervention as needed to address issues related to resource adequacy in the local context • Determine the essential activities and resources needed to implement the intervention to support scale-up and spread |
Simulation | • Review prior simulation studies to motivate the implementation of an intervention • Develop and continue to update a list of the intervention costs and effectiveness estimates from literature review • Identify potential gaps in the types of costs associated with implementation of the intervention reported in prior studies | • Identify short-term and long-term outcomes of interest • Document all decisions made about how the intervention is implemented • Estimate the costs and other resources required to implement the intervention using tools developed (e.g., process flow diagrams, time-and-motion toolkits, etc.) | • Estimate the costs and other resources required to implement the intervention using tools developed (e.g., process flow diagrams, time-and-motion toolkits, etc.) • Develop and refine model that accounts for all aspects of intervention implementation in the local context • Estimate population-level projections of the impact of the intervention for the local context • Compare expected outcomes in the local context for the overall population and by subgroups to explore potential equity issues | • Develop and refine model that accounts for all aspects of intervention implementation in the local context • Estimate population-level projections of the impact of the intervention for the local context • Compare expected outcomes in the local context for the overall population and by subgroups to explore potential equity issues • Estimate the potential cost-effectiveness of the intervention if sustained or scaled up • Determine optimal strategies for intervention sustainment/scale-up of the intervention |
Sensitivity/Uncertainty analysis | • Identify important parameters in prior estimates of intervention effectiveness and costs • Identify areas of uncertainty related to the intervention and implementation of the intervention in prior simulation studies | • Identify important parameters in prior estimates of intervention effectiveness and costs • Identify areas of uncertainty related to the intervention and implementation of the intervention in prior simulation studies • Identify areas of uncertainty while planning implementation of the intervention in the local context | • Identify areas of uncertainty while planning implementation of the intervention in the local context • Track variations in the time spent and costs associated with intervention implementation to inform best and worst case scenario estimates • Determine the key parameters and assumptions that have the largest influence on the results • Evaluate the impact of uncertainty on outcomes of interest in the local context • Estimate the sensitivity of the simulation results to changes in intervention implementation in the local context • Graphically display the results of sensitivity and uncertainty analyses to inform decision-making | • Determine the key parameters and assumptions that have the largest influence on the results • Evaluate the impact of uncertainty on outcomes of interest in the local context • Estimate the sensitivity of the simulation results to changes in intervention implementation in the local context • Estimate and report plausible ranges of outcomes of interest if the intervention is sustained or scaled up • Graphically display the results of sensitivity and uncertainty analyses to inform decision-making |