Learning from Experience the Quality Improvement tool

Welcome to Wachter's World Sign in | Join | Help
Search


Today’s Big ACGME and Joint Commission Announcements: The Courage To – and Not To – Change

One of the mantras of performance improvement is that caregivers and provider organizations should learn from their experiences. That’s all well and good, but how about policy-setting organizations?

A few moments ago in the on-line version of the New England Journal of Medicine, two of the Biggest Kahunas in the safety and quality worlds – the Joint Commission (TJC) and the Accreditation Council for Graduate Medical Education (ACGME) – announced bold new policies. To their credit, both organizations have learned from their experiences… and their mistakes.

Let's start with the Joint Commission. In a gutsy move, three TJC leaders (led by CEO Mark Chassin) call for raising the bar on measures whose purpose is public accountability (i.e., measures used for pay for performance or public reporting programs). Mark was nice enough to invite me to co-author the manuscript, and I was proud to do so. I think it’s an important statement.

We begin today’s article by tracing the surprisingly brief history of national quality measurement. “Over the past decade we have learned that standardized data can be collected by thousands of hospitals to identify and implement substantial improvements in care,” we write. “We believe that the ‘proof of concept’ phase of national quality measurement and public reporting has now been completed.”

But, while quality measurement has led to real progress, there have been several speed bumps along the way – particularly when flawed measures have become national standards. To move the field forward, we propose that any measure being used for accountability purposes must:


1.…be based on a strong research foundation (at least two studies using robust methods, often randomized trials);
2.…capture whether the evidence-based care has actually been delivered; (“Organizations that wish to improve their performance record [on things like smoking cessation or discharge counseling] may be tempted to create clever… forms with just the right check-boxes… to satisfy the chart reviewers rules, instead of doing the hard work of improving their clinical care,” we observe);
3.…address a process relatively proximate to the desired outcome; and
4.…have minimal or no adverse consequences.

We then point to six existing TJC/Medicare measures that do not stack up, including measures of smoking cessation counseling and discharge instructions (which fail to capture the care process of interest and are subject to gaming); the measure of LV systolic function in hospitalized patients with heart failure (not proximate enough to an outcome of interest); and the measure of door-to-antibiotics time for patients with pneumonia (unanticipated consequence of unnecessary antibiotics for patients who don’t actually have pneumonia).

On the glass-is-half-full side of the ledger, of the 28 core measures publicly reported in 2010 by TJC and Medicare, 22 of them do meet all four criteria. Reassuringly, since the advent of public reporting, we’ve seen striking improvement in performance on these measures: whereas only 20% of US hospitals nailed more than 90% of all 22 measures in 2002, nearly 71% did so in 2008.

While the NEJM article describes a new philosophy around quality measurement, this is not just an academic exercise. In today’s issue of the Joint Commission’s online newsletter, TJC announced that it would embrace this approach for all its present and future quality and safety measures, and it challenged other stakeholders to do the same. Bravo.

We concluded our article this way:


Eliminating measures that do not pass these accountability tests and replacing them with ones that do will reduce unproductive work on the part of hospitals, enhance the credibility of the program with physicians and other key stakeholders, and increase the positive effect that all these programs will have on health outcomes for patients.


Let’s turn now to the ACGME, which today released its long-awaited revised residency duty hour regulations (they are subject to public comment for about 6 weeks before they become the Law of the Land). You know the background: in 2003, the organization famously made the “80-hour work week” a medical term of art. (I recently met Bertrand Bell, whose “Bell Commission” originally crafted New York state's 80-hour limits following the death of Libby Zion. “How did you choose 80?” I asked the curmudgeonly Dr. Bell, now in his 80s. “Sixty seemed too little, and 100 seemed too much,” he told me. “So we split the difference.”)

In their article in today's NEJM, ACGME leaders, led by CEO Tom Nasca, chronicle the positive changes that the 2003 regs were supposed to usher in, and why so many of them failed to materialize. For example, concerns have been raised that the regulations created a shift-work mentality among residents, have overemphasized duty hours over the equally important issue of housestaff supervision, and failed to account for the maturation stages of residents as they move through their years of training. Most damning, while there is evidence that residents’ quality of life improved after the 2003 duty hours limits, there is no evidence that the regulations resulted in better quality or safety.

In light of continued national anxiety about patient safety, some fairly aggressive recommendations in a 2008 IOM report, the fact that most other industrialized countries have duty hour limits of 50-60 hours/week, and substantial push from sleep researchers and other experts, the betting was that the ACGME would slash the duty hours limits again. For the last several months, the buzz among residency directors and teaching hospital leaders has crescendoed – like a World Cup crowd armed with noisy Vuvuzelas. Might the number of duty hours indeed be cut, perhaps down to as few as 60?

Cue the drum roll…..

And the answer is: the duty hour limits will remain 80 per week.

Yet the new regulations do call for substantial changes. For example, they:


•Insist on direct, in-house attending-level supervision of interns (either at the bedside or “on site and available to provide direct supervision”)
•Allow housestaff workload and autonomy to escalate as residents become more senior.
•Promote the primacy of education over service in curricular decision-making.
•Forbid 30-hour shifts for interns; the maximum intern shift will now be 16 hours (more senior residents can still do overnight shifts of up to 28 hours, with “strategic napping” encouraged).

I’m pleased that the ACGME resisted the pressure to cut the weekly duty hours further. In addition to the massive costs of replacing resident labor (with hospitalists or allied health professionals), I believe that lower hours would be detrimental to training: residents would be forced to pack more work into less time, shorter hours would further promote a run-for-the-doors mentality, we’d be stuck with even more risky handoffs, and – I know I sound like an old fogey – I worry that even good residency programs are graduating residents who aren’t ready to be practicing doctors because they haven’t cared for enough patients, exercised enough autonomy, or developed their professional compass around when it is in their patient’s interest for their physician to work while tired.

By retaining the 80-hour work week, I'm guessing that the ACGME hopes to give programs and hospitals some breathing room to focus on some of the more challenging, but ultimately more important, issues, including how to promote a culture of safety among residents and their programs, how to enhance supervision of trainees early in their training cycle while allowing graduated autonomy as residents move up the food chain, and how to emphasize education over service in resident rotations.

At UCSF Medical Center, anticipating the need for around-the-clock supervision and a ban on 24-hour shifts (we predicted that they would be forbidden for all trainees, not just interns), we will launch overnight hospitalist attending coverage starting next week. Although it will take a few organizational back flips to comply with the new intern 16-hour limits, today’s ACGME announcement means that the amount of transformation and associated costs will be significantly less than we anticipated. I’m hoping that we – and teaching hospitals everywhere – take advantage of this break by focusing some of our energy and resources on relieving housestaff of many of their clerical tasks that soak up their limited hours, rebalancing rotations to emphasize education over service, and improving handoffs. I think these moves would honor the spirit of what the ACGME intended to do.

Kudos to Tom Nasca and the ACGME for not taking the easy path: cutting duty hours to give the appearance of acting decisively in the name of safety. While one might arguably improve safety in the short term by slashing resident hours and replacing trainees with senior faculty (assuming we could find, and afford, enough of the latter), we would ultimately pay the price in safety and quality as a generation of undertrained individuals grew up to become our future physician workforce. The ACGME’s choices reflect an appropriate balancing of safety today and safety tomorrow, which is as it should be.


Taken together, both of today’s announcements are the products of courageous executives helping their organizations to make hard choices – choices that will be controversial but strike me as well considered and generally wise. As important as the individual decisions, both Chassin and Nasca have demonstrated that – even as they require that their accredited hospitals and training programs engage in continuous improvement and learn from their mistakes – they are guiding their own organizations to do the same.


Published Wed, Jun 23 2010 7:18 PM by Bob Wachter
Filed under: Quality Improvement, Hospital Care, Pay-for-performance, Transparency and Reporting, Quality Measurement, Medical Education/Academia, Hospitalists/Hospital Medicine, Patient Safety/Medical Errors, Health Policy
Comment Notification
If you would like to receive an email when updates are made to this post, please register here

Subscribe to this post's comments using RSS
9 Comments

Josh Adler MD said:
This is a very encouraging direction for The Joint Commission and I second your characterization as courageous. On the ACGME work hours announcement, this is encouraging in some respects. Not lowering the 80 hours does provide breathing room and recognizes the potential training value inherent in the long hours. But, the ACGME is noticibly silent on the profound effects that modifcations to duty hours have on the ability of hospitals and clinics to provide care. This is not to say that reducing hours was wrong, in fact it was certainly the correct direction to move. But, there has been little attention paid to the unintended (or perhaps intended) consequences for care delivery, including the vast number of hand offs created, the marked increases in allied health professionals employed (and their associated costs), and the impact on supervising physicians.

Thu, Jun 24 2010 1:26 AM
Bob Wachter said:
I can't resist this – my sports-nut of a son just sent me this link in honor of the World Cup (plus I think he was proud of the fact that I worked "vuvuzela" into my blog). His subject line was "Your New Homepage." It's pretty funny.

http://www.vuvuzela-time.co.uk/www.wachtersworld.com

You'll need your sound on, but not too loud.

-- Bob

Thu, Jun 24 2010 2:48 AM
Brian Clay, MD said:
@ Dr. Adler --

On the contrary, the new proposed regulations also include a Common Program Requirement that rotations "should be structured to minimize the number of handoffs required." I'm still attempting to determine how one does that with a reduction in maximum shift length (one would almost certainly cede that there will be more handoffs now than previously).

There are also new program requirements about training our housestaff in the skills of communication around handoffs and transitions of care. As an academic hospitalist and an associate program director for an internal medicine residency, I welcome these changes, as it gives our own division of hospital medicine standing to advance this curriculum and skill set to the residency at large, rather than just as individual attending physicians on our ward teams.

There is an additional upside to the new rules: if programs take the new rules as effectively closing the door on overnight call (and go to night team structures as a result), it affords an opportunity for many programs to move toward a unit-based, geographically cohorted model of care, where phenomena like multidisciplinary rounding with nursing and case management can occur much more easily.

We in hospital medicine should continue to take as a task for ourselves the research and advancement of effective and efficient handoffs -- our residents are going to need those skills.

Thu, Jun 24 2010 2:11 PM
Sumant said:
Actually, it's not clear if the proposed regulations require in-house attending-level supervision for interns. The regulations do say that interns require in-house supervision at all time, but also state that "for many aspects of patient care, the supervising physician may be a more advanced resident".

Fri, Jun 25 2010 4:39 AM
Guido Candiotti, MD said:
Based on the 4 measures to evaluate a quality measure, EHRs and CPOEs would fail on several to be useful, yet, the JC is serving out CPOE as though it is safe. Readers take note that two Harvard studies already report no improvement in costs or outcomes at highly wired hospitals. I am surprised you joined Dr. Chassin in this report.

Fri, Jun 25 2010 5:12 PM
Bob Wachter said:
Re: Sumant's comment: "not clear" is correct. I read the NEJM article, as well as the ACGME website, several times and still couldn't be sure I understood what kind of in-house supervision is needed for interns. I spoke to a friend who consulted with ACGME on the new regulations and is a senior educational leader at a major academic program; she too was not certain. This will obviously need to be clarified...

-- Bob

Sat, Jun 26 2010 6:05 AM
jqyoung said:
Brian: can you say more about how these changes might move us "toward a unit-based, geographically cohorted model of care." And what do you mean by this? I found this comment intriguing. Thanks.

Sat, Jun 26 2010 6:09 AM
Brian Clay, MD said:
jqyoung --

When residents admit new patients under the construct of a 30-hour shift, more often than not those patients are distributed among several patient care units in the hospital. This creates inefficiency on rounds (multiple floors to round on each day); it also hampers the ability of the housestaff (and their attending physicians) to work in conjunction with all of the ancillary services caring for the patient (nursing, therapies, case management) that, for the most part, are centered on a given patient care unit.

It is often the case that, on any hospital floor, the nurses, therapists, social workers, and case managers all know each other and work closely together, but the patients on that floor might have eight or nine separate physician service teams with primary responsibility for that set of 25 or so patients.

When overnight call goes away, the natural next construct for covering the hospital is to have several day shift teams, and one (or more if needed) night shift teams, with handoff moments in the early morning and early evening. All patients admitted at night will be passed off to their primary team in the morning.

If you set up a system where multiple day teams can admit concurrently, it is possible to actually match a resident team to an actual geographic set of beds (say, a hospital floor). In this way, that team would care for the patients on that floor, and that floor only. When admissions come to that floor (whether during the day, or passed off from the night team), a single resident team takes care of them.

Now you have a resident team structured the same way as the nurses, therapists, etc. -- based on a hospital unit. This gives the greatest opportunity to have physicians, nurses, and other team members rounding together on the patients and putting everyone on the same page in terms of the plan for the patient.

Mon, Jun 28 2010 9:30 AM
wrs said:
There has never been a randomized controlled study to show that parachutes work. Would the proposed framework support a hypothetical quality improvement project to measure the use of parachutes?

With The Joint Commission supporting an evidence-based practice framework, does this mean that there will be an elimination of current standards for which good evidence is lacking?

Mon, Jun 28 2010 10:34 PM

Leave a Comment
Comments to Wachter's World are welcome provided that they are:
•On topic, i.e. you have read the post and are commenting on its topic.
•Adding something to the post, i.e. "you stink" and "you rock" are equally useless.
•Polite, or at least professional.
•Not overly promotional or outright spam.
I may remove any comment that does not meet these criteria.