Monitoring Impact
Health care organizations should view person-centered engagement as a way to drive measurable improvements in operations that ultimately drive improvements in the quality of care they provide, in the health of their patients and in stabilizing or reducing costs. Quite frankly, it is also simply “the right thing to do.”
Pro Tip: Do Not Start With a Blank Slate
In order to help organizational staff prioritize monitoring impact from the start, the Planning Workgroup can create a checklist of potential measures. This is not meant to be limiting, but rather as a means of stimulating conversation about the expected outcomes. The checklist will be especially helpful to patient and family advisors who may be less familiar with quality measurement concepts.
Monitoring impact should begin during the Planning phase, when the Planning Workgroup should establish goals and a clear framework for how the organization will define success.
There are a variety of qualitative and quantitative approaches that health care organizations can use to evaluate the performance of person-centered engagement efforts. Evaluation approaches should focus on both process measures and outcome measures, which can be both internal and external. Examples follow:
- Number of advisory meetings
- Advisor participation rate
- Hours provided by advisors
- Utilization rates
- Workflow changes
- Change in visitation policies
- Redesign of staff roles
- New staff training policies
- Staff burnout rates
- Patient satisfaction scores
- Infection rates
- Length of hospital stay
- Decreased health disparities
- Improved function
- Health confidence
- Feelings of autonomy
- Quality of decision-making
Health care organizations should focus on implementing ongoing evaluation strategies, such as Plan-Do-Study-Act (PDSA) cycles, that continuously inform improvements to the design and operation of patient engagement strategies.
Additional Considerations for Evaluation
Notes from the Field
The three organizations featured in the case studies realized important impacts and successes that directly emerged from its patient engagement strategies and structures. Examples include:
- One hospital in the Trinity Health system saw an increase in their patient experience ratings on standard surveys such as HCAHPS and their net promoter score, which they attributed to improved staff interaction with patients in response to engagement activities.
SEE THE CASE STUDY - Over two years (FY18 and FY19), patient experience data showed a 6.2% increase in Children Mercy’s ability to ease child discomfort after engaging PFAs in identifying strategies to minimize pain associated with needle use.
SEE THE CASE STUDY - HRHCare reduced patient wait times by changing its protocol for scheduling provider appointments to review lab results.
SEE THE CASE STUDY
- Combine qualitative and quantitative analysis strategies to capture numerical data, as well as rigorously elicited narrative feedback on advisors’ experience with engagement infrastructure, function, and processes
- Allow your evaluation approaches to evolve over time. Focus evaluation approaches on process measures early in the implementation of the engagement strategy and incorporate more outcome-focused measures as the strategy matures.
- Measure for diversity, inclusion and equity, for example, by seeking to understand how well your advisors represent the full diversity of the populations your organization serves, whether your advisors feel included and how often the topics of engagement addressed issues of health equity. Ultimately, health care organizations should seek to understand whether they are making measurable reductions in health disparities.
- Reinforce the importance of the engagement strategy by incorporating reviews of the evaluation findings into senior leadership meetings.
It is essential to communicate results of consumer and family engagement efforts both internally and externally. Patient and family advisors need the information to understand the value of their work on behalf of the organization and to bolster their commitment to continuing their efforts. Communicating impacts of engagement activities to the staff of the entire organization helps to build the case for ongoing investment in engagement structures, and greater understanding of the ROI. To the extent possible, describe the connection between the feedback you received from advisors and the ultimate outcome. Finally, communicating impacts externally conveys to the communities you serve the value your organization places on quality improvement and consumer engagement.
Self-Assessment Questions
On a scale of 1 to 5 (with 1 indicating “strongly disagree” and 5 indicating “strongly agree”), rate your organization’s performance on the following questions:
- My organization has established goals and a clear framework for how we will define success in its consumer engagement activities.
- Our Engagement Team collaborates with consumer advisors to select specific measures of success for each engagement activity. We consider process measures, outcome measures and patient-reported outcome measures and work with advisors to monitor progress on those measures.
- We stratify quality data by race, ethnicity, and language in order to uncover and respond to health disparities.
- My organization regularly communicates the results of our consumer engagement activities to leadership.
- My organization regularly communicates the results of our consumer engagement activities to external audiences.
Resources
- Foundational Resources
- American Institutes for Research: Principles for Making Health Care Measurement Patient-Centered
- NAM Guiding Framework for Patient and Family Engaged Care
- AIR: A Roadmap for Patient and Family Engagement in Healthcare (Measurement and Research)
- NQF: Patient-Reported Outcomes Environmental Scan
- Aligning Forces for Quality: Using Data to Reduce Disparities and Improve Quality: A Guide for Health Care Organizations
- Schlesinger et al., A Rigorous Approach to Large-Scale Elicitation and Analysis of Patient Narratives (Medical Care Research and Review)
- Dukhanin, et al., Metrics and Evaluation Tools for Patient Engagement in Healthcare Organization- and System-Level Decision-Making: A Systematic Review
- Tools and Tips
