Chip Kahn (00:11):
Hello and welcome to another edition of Hospitals In Focus. I’m your host, Chip Kahn. Today, we’re unveiling a very important project we’ve been working on here at the Federation for a number of months, finding ways to improve the patient experience survey. For more than a decade, the Hospital Consumer Assessment of Healthcare Providers and Systems Survey, commonly known as HCAHPS, has been used by the government to measure the inpatient hospital experience.

You may remember in our last episode, our own Dr. Claudia Salzburg, outlined the results of a study released by the major hospital associations, that surveyed patient experience leaders about the effectiveness of the 32 question HCAHPS survey, and the need for updating that survey. Recommendations in that study included improving a few of the questions, and matching questions to the evolution of care delivery in hospitals, as well as taking the survey online. It currently is administered by phone and mail.

We wanted to put these ideas to the test, so we enlisted Bill McInturff to help. Bill is a cofounder and the managing partner of Public Opinion Strategies, one of the nation’s premier political and public affairs research firms. Bill and his team worked with us to undertake this effort, to see if the patient experience survey could be recast and improved and successfully administered online, as well as by the other modalities.

Bill unveiled his findings recently at a briefing on Capitol Hill, hosted by the Alliance for Healthcare Policy and the Federation. He joins us here today for a deeper dive into his project. Welcome, Bill.

Bill McInturff (02:01):
Hi, Chip. Glad to be here. Thank you.

Chip Kahn (02:03):
So let’s get started learning more about you, Bill. Now our listeners can’t see Bill, but I’m sure they will recognize your voice. You have appeared often over the years on Meet the Press, and other cable news. Give us a bit about your background, and the work that you and your colleagues do at Public Opinion Strategies.

Bill McInturff (02:22):
Well, thank you. Public Opinion Strategies is one of the 50 largest research firms in the country, but we specialize in public affairs and in politics. Half our work is our political work. We represent over 60 members of Congress, 12 senators, but the other half is the kind of work we’re doing here, specifically in healthcare. We’ve spent years doing healthcare research for health insurers, for doctors, for the pharmaceutical industry and of course for hospitals. We’ve worked with the American Hospital Association since the mid 1990s, and we work with other hospitals around the country.

And then the other part I do is my media polling. I do the work that does the monthly NBC Wall Street Journal Poll, and our firm also does CNBC. We do quarterly economic tracking about American’s perception of the economy.

Chip Kahn (03:10):
Great, Bill, so you are no stranger to healthcare. How does your work translate specifically to a project like ours? What kind of work do you do in the area of consumer experience?

Bill McInturff (03:21(:
Well, more of what we do is public policy, but in terms of consumer experience, we’ve asked a lot of questions about your thoughts about hospitals, your hospital experience, what things you like, what you don’t like. We’ve done a lot of work with women consumers about their experience seeing a physician, and we’ve done some very specialized disease work.

For example, for the CDC, we did work with women who have fibromyalgia, and other disease work to look what those people with those special needs have from the medical community. So across the years of research, we’ve done a lot to look at how consumers interact with today’s healthcare system.

Chip Kahn (04:03):
So it sounds like you’re really suited for our effort here.

Bill McInturff (04:07):
Thank you.

Chip Kahn (04:08):
Your research found that HCAHPS survey is important to patients to evaluate their experiences within the healthcare system. What is it about the current survey that you found works, and how well is it measuring the patient experience from hospitalizations?

Bill McInturff (04:25):
First, it’s really important to convey HCAHPS is a very well- designed, very successful survey. We have a lot of statistical measures that look at does this survey instrument predict people’s overall satisfaction, and does it predict their ability to recommend this hospital to their friend? And HCAHPS does that very well. It’s been very carefully designed. The things it does very well I think, is it looks at and helps measure the interactions between people and their doctors and nurses. It does and and it looks well at whether or not people believe that their physicians and nurses and medical team are effectively communicating and talking, and communicating clearly, and whether or not there’s a prompt response from nurses.

However, when we started this project, we were surprised that this survey was not being administered online. So one functional question was whether or not an HCAHPS like survey, or HCAHPS itself could be administered online. Short answer, yes, and it’s been years since that survey was first really in the design. And so guess what, we know this, look at every other consumer market, people change. And so there was an open question about in what ways could we look at, find and measure other things that may be important to people that would further strengthen and further enhance the HCAHPS survey?

Chip Kahn (05:48):
Specifically, Bill, I think you found a few measures that could be improved from the current survey. What were those?

Bill McInturff (05:55):
Well, number one, I think there was a more direct measures about their confidence in doctors and nurses, and also this concept of team care. When we did special work to spend a half hour, hour, with individual patients, we heard a frustration about whether or not the doctors, the nurses and other care staff were all coordinating. And they were very dissatisfied when that didn’t happen and very frustrated.

So one, confidence in nurses and doctors, two, team care, did I feel like there was coordinated care in terms of my treatment in the hospital? And three, they’re today’s consumers. And as today’s consumers, they want to be heard, they want somebody to pay attention, and they want someone to input in their own care. And then I think another really important area we found was the relationship between satisfaction, and how long it took to be admitted in the hospital, and the discharge process.

And I think that among our findings is the importance, it’s sort of the beginning, how do I get in the hospital, and the end, what was the process for me to be checked out of the hospital? And I think that our work has demonstrated that those areas could need additional exploration and additional measurement to understand the hospital experience.

Chip Kahn (07:13):
Bill, Dr. Claudia Salzberg, Federation’s VP of quality, who we heard from in our last podcast, worked very closely with you on creating questions to evaluate some of the topics we just
discussed. What were some of those questions that you created?

Bill McInturff (07:29):
Well, I want to first thank FAH. Thank you, thank Claudia, and the leadership you’ve demonstrated, because this project started with Claudia spending enormous amounts of time with 27 people, who provide and monitor care at individual hospitals. And so she approached us and said, “Look, with that as an information base, here’s what our experts in these hospitals are telling us they think are most important to patients.” We then met with and did 16 interviews, one-on- one’s, spending a half hour, to hour with patients, to bounce off those observations, and they worked. That’s sort of the list we just gave you.

So some of the questions that we developed from that iterative research process were, during this hospital stay how often do the doctors, nurses and staff work together to provide you timely efficient care? Again, this notion of was their communication across the entire care team inside the hospital. During this hospital stay, how often did you receive accurate updates about the timing of any delays in your care plan? Again, here’s what frustrates people. They know healthcare is not an exact science. They know there’s going to be ups and downs.

They know they might not … That things can happen to postpone their care, but they want to be told, they want to know, they want to understand. And that question went a long way to to help to identify that.

When I left the hospital, I clearly understood the purpose of taking each of my medications? During this hospital stay, how often did you understand your care plan thoroughly? And again, what people were telling us, and what we’re measuring here is whether or not people feel like they understood the care they were being provided.

And then another area that came out of our one-to-one exploration and these interviews were, how often was the medication you were prescribed before you were admitted to the hospital, administered on time? There was in these interviews and other work we did, a great deal of distress that they, again, you have to remember a lot of these patients are over 55. We know almost everybody over 65 tells us in surveys they’re on some medication, and so they want to know, and they want to make sure that there are folks in the hospital that understand this, and that if it’s not counter indicated, that they’re maintaining their existing prescriptions. This is an area that I think is, it may seem small, but it certainly seemed important to people we talk to and interviewed.

Chip Kahn (09:56):
Bill, I have a feeling that you took a very unique approach here in terms of developing this research. Could you talk a bit about the methodology?

Bill McInturff (10:06):
Sure, but let me also indicate again, the CMS survey, researcher’s department is very skilled. I’ve worked with them in the past. They are really good, and we all have to understand that no private entity has the resources that CMS has. This was a test project. This was exploration. This was simply looking at new ideas to put on the table, recognizing that more would need to be done to independently validate those results, and before they are extended into something like an HCAHPS survey.

But I believe the research is iterative, meaning you learn layer by layer by layer and you build on that. So importantly it started off with Dr. Salzberg’s work to tell us based on hospital expertise, what they think patients are most concerned about. We then administered and we had 16 interviews, 45 minutes to an hour a piece, with recently hospitalized adults, half men, half women. And then from that we did an exploratory survey. We did 500 interviews online, where we took what we thought would be provocative, new questions, that seemed to capture elements not in the current HCAHPS, to see if they worked, and if those questions were predictive of people’s overall satisfaction and predictive of their wanting to recommend this hospital.

And then we did sort of a really cool kind of final test, where online we administered two different surveys at the same time, one a 500 interviews amongst recently hospitalized adults using the exact HCAHPS questionnaire, slightly modified to be online, and then we changed the HCAHPS questionnaire. We cut seven or eight questions, and we inserted new questions that we felt had worked well in our test survey, and then we could compare the existing HCAHPS survey to a new revised survey that we thought was going to be enhanced and strengthen and show additional insight. And it did and that’s the positive result that
came out of this bundle of work.

Chip Kahn (12:11):
Bill, I want to find out about your conclusions from this work.
Initially, let me just ask, what did you sense from the work about moving this survey online, as well as continuing to provide it to patients both through the mail and over the phone?

Bill McInturff (12:30):
Well, I’m a very strong advocate for this survey being administered online as one of the modalities and one of the options. 87% of Americans tell us that they either have a laptop, they have a computer, or they have a smartphone. You have to remember by the way, 3% of Americans tell us they don’t have telephones. So you’re never going to get to a hundred, but in today’s world, 87% is a big number. And obviously for the generation Z and millennials, that’s how they spend their life. And so it’s possible that offering this option might even expand the range of collecting data from younger respondents.

So because the survey worked very well online, we would hope that the results of this survey would help lead CMS, and others to say there’s every indication that the careful testing that CMS would normally do should be done. And this survey clearly should be considered as an online option, as one of the ways to take the survey.

Chip Kahn (13:37):
That’s good to hear, Bill, even though this survey is done under the auspices of Medicare, I think it’s important for our listeners to understand that the survey is done of a sample of all patients who had hospitalizations, not simply seniors.

Let me move on a bit though. Beyond the survey administration itself, what were the other findings you think are really important for our audience to hear?

Bill McInturff (14:01):
Well, number one, and look, it’s a big deal. The survey that we re crafted was actually shorter than the existing HCAHPS survey by a statistical amount. We shaved about about a minute, point three, and why is that important? Because hospitals tend to add, want to add a few questions of their own about their individual hospital. And the shorter the overall survey, the easier it is to add a handful of questions, and still get very high response rates.

So number one, the survey was shorter. Number two, even though we reduced the number of items, we know using statistical models that a very strong HCAHPS survey, that the new survey worked even better. Even better to tell us that this group of questions was more accurate, and more precise in measuring satisfaction and the ability to recommend the hospital. Again, I mean that was really exciting for us as the people who worked to help design the work, and again it reinforced what we were hearing through those other rounds of research.

What did it reinforce? Number one, please, I want my care team to be carefully working together on my behalf. I want to be heard, I want to understand my care, and as I said the primacy of this concern about the efficiency of the admission and the discharge process. And among your listeners and if again, if I were taking this data and applying it to real world, it would make me as a hospital, a person at a hospital level, want to really think through how we are handling admission and discharge.

Chip Kahn (15:44):
Well thanks, Bill. I am really gratified about your findings of this and I’ll emphasize demonstration. I hope that it does contribute to the ongoing evolution of HCAHPS. And I’ll just ask in conclusion, do you have anything else that you would add that’s important for the CMS and ARC developers, and administrators of HCAHPS, in terms of where you think it should go next from the current survey?

Bill McInturff (16:14):
Well, again, I have enormous respect for the capability. And so number one, I would do the careful test to measure online respondents versus phone, versus the the people who take it through paper and pencil. I would start with that. I would not have changed the existing HCAHPS. I would just start with the same HCAHPS, and then compare it in terms of the responses and response rates. That’s number one.

Number two, I use the phrase exploratory surgery. We did a very small number of one-on-one interviews across a range of patients, but a a little more basic work to go back and say, “Look, it’s been 15 years since this was first designed or more. What are today’s patients like?” And more importantly, as you said, it’s not just seniors, it’s all patients. Millennials and generation Z aren’t baby boomers. They’ve grown up with different expectations, and spending some time to talk to people in different age brackets, I think is important.

And then three, I hope that we’ve pointed the way, like admissions and discharge, like care team. I hope we’ve pointed the way to look at a handful of areas that we found to be very important, that worked very well in the survey, that worked very well in statistical models, and help validate and see whether or not these questions are replicable across larger surveys, and whether or not they do in a larger survey, with more respondents, what they did here. And what they did here was work as well or better than the questions we draw from HCAHPS to predict satisfaction. And I think that we’ve hopefully in a small demonstration sort of way, opened the door for the discussion about further strengthening and improving the HCAHPS product.

Chip Kahn (18:08):
Bill, this has really been great, and I so appreciate all your work because from the standpoint of hospitals as caregivers, that patient experience, and making sure we understand it, and then know how to improve it is so critically important. And I hope that your work and the further work that I know the agencies at HHS will do, will help us not only better understand, but really make that experience in the hospital better for patients.

Bill McInturff (18:39):
My hope so too, and if it does, what an exciting thing for me and my firm to be a very small part of. That’s again, this is one of the reasons I love and enjoy my job, and thank you a lot to have the opportunity to see if this is something we could contribute to that good goal.

Chip Kahn (18:56):
Thanks so much, Bill.

Bill McInturff (18:57):
Thank you, sir.

Chip Kahn (18:58):
Thanks so much for listening to our podcast today. We hope you found this conversation interesting and will join us every other week as we talk with leaders on a broad spectrum of healthcare issues. Don’t forget to subscribe to Hospitals In Focus. We can be found on Apple Podcasts, Google Play, and now Spotify. Be sure to rate and review us and tell a friend. Thanks for listening.

Bill McInturff, Partner, Public Opinion Strategies

Bill McInturff is a partner of Public Opinion Strategies, one of the nation’s largest political and public affairs survey research firms. The firm has been described as a “GOP powerhouse” and currently represents seven governors, 11 U.S. Senators and 50 members of Congress. Bill, along with Hart Research Associates, conducts The NBC News/Wall Street Journal Poll.

One nationally recognized political reporter described Bill as “one of the smartest most clear-thinking people in politics.” Another said he “has more influence on Capitol Hill than any other Republican strategist.”

Over a more than 30-year career in survey research and over 5,000 projects, he has been devoted to what he describes as “combat message development,” not simply monitoring public opinion, but developing messages to defend and promote client interests on complex public policy issues. The focus of much of Bill’s work has been health care, with the firm having conducted over 1700 focus groups and more than 1300 surveys for health care stakeholders and about health care policy.

Bill has done the polling for 30 successful initiative campaigns over the past decade, including conducting the polling for Prop 22 in California, the successful effort to assure app-based drivers remain independent contractors.

Bill is a frequently quoted source on the topic of American politics. He has appeared frequently on Meet the Press and MTP Daily, and he is quoted in a variety of national news magazines and major newspapers.

Bill lives in Alexandria, Virginia, with his wife, where he watches a lot of movies and is looking forward to returning to being a youth baseball coach.