Skip to Content


Secretarial Review and Publication of the Annual Report to Congress Submitted by the Contracted Consensus-Based Entity Regarding Performance Measurement

Document Details

Information about this document as published in the Federal Register.

Published Document

This document has been published in the Federal Register. Use the PDF linked in the document sidebar for the official electronic format.

AGENCY: Office of the Secretary of Health and Human Services, HHS.



SUMMARY: This notice acknowledges the Secretary of the Department of Health and Human Services' (HHS) receipt and review of the annual report submitted to the Secretary and Congress by the contracted consensus-based entity as mandated by section 1890(b)(5) of the Social Security Act, as added by section 183 of the Medicare Improvements for Patients and Providers Act of 2008 (MIPPA) and section 3014 of the Affordable Care Act of 2010. The statute requires the Secretary to publish the report in the Federal Register together with any comments of the Secretary on the report not later than six months after receiving the report. This notice fulfills those requirements.

FOR FURTHER INFORMATION CONTACT: Stephanie Mika (202) 260-6366 .

I. Background

Rising health care costs coupled with the growing concern over the level and variation in quality and efficiency in the provision of health care raise important challenges for the United States. Section 183 of MIPPA also required the Secretary of the Department of Health and Human Services (HHS) to contract with a consensus-based entity to perform various duties with respect to health care performance measurement. These activities support HHS's efforts to achieve value as a purchaser of high-quality, patient-centered, and financially sustainable health care. The statute mandates that the contract be competitively awarded for a period of four years and may be renewed under a subsequent competitive contracting process.

In January, 2009, a competitive contract was awarded by HHS to the National Quality Forum (NQF) for a four-year period. The contract specified that NQF should conduct its business in an open and transparent manner, provide the opportunity for public comment and ensure that membership fees do not pose a barrier to participation in the scope of HHS's contract activities, if applicable.

The HHS four-year contract with NQF includes the following major tasks:

Formulation of a National Strategy and Priorities for Health Care Performance— NQF shall synthesize evidence and convene key stakeholders on the formulation of an integrated national strategy and priorities for health care performance measurement in all applicable settings. NQF shall give priority to measures that: Address the health care provided to patients with prevalent, high-cost chronic diseases; provide the greatest potential for improving quality, efficiency and patient-centered health care and may be implemented rapidly due to existing evidence, standards of care or other reasons. NQF shall consider measures that assist consumers and patients in making informed health care decision; address health disparities across groups and areas; and address the continuum of care across multiple providers, practitioners and settings.

Implementation of a Consensus Process for Endorsement of Health Care Quality Measures— NQF shall implement a consensus process for endorsement of standardized health care performance measures which shall consider whether measures are evidence-based, reliable, valid, verifiable, relevant to enhanced health outcomes, actionable at the caregiver level, feasible to collect and report, and responsive to variations in patient characteristics such as health status, language capabilities, race or ethnicity, and income level and is consistent across types of providers including hospitals and physicians.

Maintenance of Consensus Endorsed Measures—NQF shall establish and implement a maintenance process to ensure that endorsed measures are updated (or retired if obsolete) as new evidence is developed.

Promotion of Electronic Health Records— NQF shall promote the development and use of electronic health records that contain the functionality for automated collection, aggregation, and transmission of performance measurement information.

Focused Measure Development, Harmonization and Endorsement Efforts to Fill Critical Gaps in Performance Measurement— NQF shall complete targeted tasks to support performance measurement development, harmonization, endorsement and/or gap analysis.

Development of a Public Web site for Project Documents— NQF shall develop a public Web site to provide access to project documents and processes. The HHS contract work is found at:

Annual Report to Congress and the Secretary— Under section 1890(b)(5)(A) of the Act, by not later than March 1 of each year (beginning with 2009, NQF shall submit to Congress and the Secretary of HHS an annual report. The report shall contain a description of the implementation of quality measurement initiatives under the Act and the coordination of such initiatives with quality initiatives implemented by other payers; a summary of activities and recommendations from the national strategy and priorities for health care performance measurement task; and a discussion of performance by NQF of the duties required under the HHS contract. Section 1890(b)(5)(B) of the Social Security Act requires the Secretarial review of the annual report to Congress upon receipt and the publication of the report in the Federal Register together with any Secretarial comments not later than 6 months after receiving the report.

The first annual report covered the performance period of January 14, 2009 to February 28, 2009 or the first six weeks post contract award. Given the short timeframe between award and the statutory requirement for the submission of the first annual report, this first report provided a brief summary of future plans. In March 2009, NQF submitted the first annual report to Congress and the Secretary of HHS. The Secretary published a notice in the Federal Register in compliance with the statutory mandate for review and publication of the annual report on September 10, 2009 (74 FR 46594).

In March 2010, NQF submitted to Congress and the Secretary the second annual report covering the period of performance of March 1, 2009 through February 28, 2010. The second annual report was published in the Federal Register on October 22, 2010 (75 FR 65340) to comply with the statutorily required Secretarial review and publication.

In March 2011, NQF submitted the third annual report to Congress and Secretary of HHS. This notice complies with the statutory requirement for Secretarial review and publication of the third annual report covering the period of performance of January 14, 2010 through January 13, 2011. The third annual report was published in the Federal Register on September 7, 2011 (76 FR 55474).

Affordable Care Act was signed into law on March 23, 2010. Section 3014 of this Act included a time-sensitive requirement for NQF to provide input into the national priorities for consideration under for the National Strategy for Quality for Improvement in Healthcare. The NQF convened the National Priorities Partnership and developed a consensus report on input to HHS on the development of the National Quality Strategy.

Section 3014 of the Affordable Care Act also required NQF to: convene multi-stakeholder groups to provide input on the selection of quality measures, such as for use in reporting performance information to the public; and transmit multi-stakeholder input to the Secretary. It also amended the requirements for the Annual Report to include identifying gaps in quality measures, including measures in the priority areas identified by the Secretary under the national strategy and areas in which evidence is insufficient to support evidence of quality measures in priority areas. Activities required by the Affordable Care Act will be carried out from 2010 throughout 2014.

In March 2012, NQF submitted its fourth annual report to the Congress and the Secretary. The report covers the period of performance of January 14, 2011 through January 13, 2012. This notice complies with the statutory requirement for Secretarial review and publication of the fourth NQF annual report.

II. March 2012—NQF Report to Congress and the HHS Secretary

Submitted in March 2012, the fourth annual report to Congress and the Secretary spans the period of January 14, 2011 through January 13, 2012.

A copy of NQF's submission of the March 2012 annual report to Congress and the Secretary of HHS can be found at:

The 2012 NQF annual report is reproduced in section III of this notice. This year's annual report has two sections. The first is entitled 2012 NQF Report to Congress Changing Healthcare by the Numbers. The second section is entitled NQF Report on Measure Gaps and Inadequacies. Both sections were reviewed by the Secretary.

III. NQF March 2012 Annual Report

2012 NQF Report to Congress Changing Healthcare by the Numbers

Report to the Congress and the Secretary of the U.S. Department of Health and Human Services, Covering the Period of January 14, 2011, to January 13, 2012 Pursuant to Public Law 110-275 and Contract #HHSM-500-2009-00010C


Letter From William Roper and Janet Corrigan

Executive Summary

Building Consensus About What and How To Improve

Endorsing Measures for Use in Accountability and Performance Improvement

Aligning Payment and Public Reporting Programs That Reward Value

National Quality Forum: Background

Bridging Consensus About Improvement Priorities and Approaches

National Priorities Partnership

NQF's Focus on Safety

Endorsing Measures and Developing Related Tools

NQF Endorsement in 2011

Culling the NQF Portfolio

Enhancing NQF Endorsement

The Information Technology Accelerant

Aligning Accountability Programs To Enhance Value

Growing Use of NQF-Endorsed Measures

Measure Application and Alignment

Achieving Results

Looking Forward


Appendix A: 2011 Accomplishments: January 14, 2011 to January 13, 2012

Appendix B: NQF Board and Leadership Staff

Appendix C: Overview of Consensus Development Process

Appendix D: Map Measure-Selection Criteria

Appendix E: NQF Membership

Appendix F: 2011 NQF Volunteer Leaders

Letter From William Roper and Janet Corrigan

Over the last decade, Members of Congress from both parties, as well as federal and private-sector leaders, have increasingly supported the use of standardized quality measures as part and parcel of a larger healthcare value agenda. Agreed-upon strategies for improving value—healthier individuals and communities, as well as better, lower-cost care—include public reporting of standardized performance measures and linking measures to payment.

Evidence of support for this agenda includes the fact that approximately 85 percent of measures currently used in public programs are endorsed by the National Quality Forum (NQF),1 as well as the significant use of NQF-endorsed measures by private health plans and employers. In addition, recent statutes—the 2008 Medicare Improvements for Patients and Providers Act (MIPPA) and the 2010 Affordable Care Act (ACA)—reinforce preferential use of NQF-endorsed measures on federal healthcare Compare Web sites, and linkage of endorsed measures to payment for clinicians, hospitals, nursing homes, health plans, and other entities.

In 2011, this commitment to a value agenda was significantly accelerated. Under the auspices of NQF, and in a historic first, private-sector organizations voluntarily worked in a more coordinated and collaborative fashion with each other and with the public sector to forge consensus about how to further this accountability environment. Specifically, innovations in convening and rulemaking facilitated the private sector bringing its real-world experience to inform guidance to the Department of Health and Human Services (HHS) on implementing the first-ever National Quality Strategy (NQS), and provided advice on selecting the best measures for use across an array of federal health programs. Forward-thinking leaders—including those on Capitol Hill and within HHS—understand that the public and private sectors working independently will not yield improvements quickly or comprehensively enough in our unorganized and complex healthcare system.

We are grateful to Congress, HHS, and private-sector leaders for their vision and tenacity in designing and advancing this ambitious value agenda, and for the progress we collectively are making against it each and every day. These advancements are made possible because of the ever-expanding number of organizations and individuals who are committing themselves to work in partnership, including our colleagues at HHS; the more than 450 institutional members of NQF; the hundreds of experts who volunteer to serve on NQF committees; the NQF staff; and the many, many organizations that constitute the quality movement. We are privileged to work at the intersection of so many committed and diverse organizations that are increasingly rowing in the same direction to improve both our nation's health and healthcare for the benefit of the American public.

We are changing healthcare by the numbers.

William L. Roper, MD, MPH

Chair, Board of Directors

National Quality Forum

Janet M. Corrigan, Ph.D., MBA

President and Chief Executive Officer

National Quality Forum

Executive Summary

The U.S. healthcare system is among the most innovative in the world and patients with very serious and/or unusual conditions are particularly appreciative of the range of therapies, interventions, and clinical talent it offers to treat them and restore them to health. That said, it is also one of the most fragmented, unorganized, and uncoordinated systems as compared to its counterparts in the industrialized world—which contributes to less-than- optimal quality outcomes, serious patient safety problems, and very high per-capita costs.2, 3, 4 Consequently, Members of Congress, business leaders from small and large companies, patients, physicians, nurses, and many others have come to the conclusion that Americans are not deriving enough value for the substantial dollars they spend.

Important strides have been made toward improving this value proposition over the last decade, starting with the sine qua non of using standardized performance measures to assess “how we are doing” on an array of healthcare quality and cost dimensions, making the measure results public, and then linking those results to provider payment. And while establishing this accountability environment is critical foundational work, it is not sufficient for achieving the kind of substantial improvements that the National Quality Strategy (NQS) envisions. Released by the Department of Health and Human Services (HHS) in March 2011 and supported by public- and private-sector healthcare leaders, the NQS is built around three compelling aims focused on healthy people and communities, better care, and more affordable care. To achieve these ambitious aims also will take fundamental reform of care delivery and payment, which, while underway, will still require time, effort, and perseverance to realize.

That said, the accountability environment's basic infrastructure is moving into place. A key lesson learned in constructing it is that neither the public nor private sectors, nor any single stakeholder, can meaningfully shape it on their own. Healthcare is too large and complex, with too many interrelated parts, for a go-it-alone strategy to be fully effective. Recent actions of healthcare leaders demonstrate that they understand that sustainable solutions to our nation's healthcare challenges are ones that all stakeholders embrace. Over the last year, significant progress has been made toward forging a shared sense of priorities for improvement; an agreed-upon way to set, continuously enhance, and implement strategies to achieve these priorities; and standardized methods for measuring progress along the way. Without such agreements, competing strategies and a plethora of near-identical measures run the risk of whipsawing providers and overburdening them with redundant and sometimes conflicting reporting requirements. In addition, such an environment can confuse consumers who increasingly seek to better inform themselves as they play a more active role in healthcare decision-making.

Congress, wisely understanding this need for a quality infrastructure and more public-private collaboration, passed two statutes that included this notion, and directed HHS to work with a consensus-based entity to act as a key convener and measurement standard setter. These statutes include the 2008 Medicare Improvements for Patients and Providers Act (MIPPA) (Pub. L. 110-275) and the 2010 Patient Protection and Affordable Care Act (ACA) (Pub. L. 111-148). HHS awarded contracts related to the consensus-based entity to the National Quality Forum (NQF).

NQF has prepared this third Annual Report to Congress which covers highlights of work related to these statutes conducted under federal contract between January 14, 2011 and January 13, 2012. See appendix A for a complete listing of deliverables worked on and completed during the contract year.

Building Consensus About What and How To Improve

In the fall of 2010, as HHS was developing the first-ever NQS, the National Priorities Partnership (NPP), convened by NQF, was asked to provide initial input on the overarching aims and priority areas and published a report. Subsequently, in response to a second request from HHS, NPP identified three goals for each of the NQS six priorities in a second report, along with appropriate performance measures, and “strategic opportunities” to accelerate progress. These opportunities require leveraging the reach of the many public and private stakeholder groups participating in NPP, which balances the interests of consumers, purchasers, health plans, clinicians, providers, federal agency leaders, community alliances, states, quality organizations, and suppliers. In 2011, NPP focused further on enhancing patient safety, one of the six NQS priorities and a very important focus for HHS. More specifically, NPP worked collaboratively with HHS on its Partnership for Patients initiative, through hosting quarterly meetings and an interactive webinar series, which brought tools and ideas for reducing patient harm to nearly 10,000 front-line clinicians, hospitals, and other stakeholders across the country. Moving forward in 2012, NPP will draw on the real-world experience of its partners to develop implementation strategies, likely targeting patient safety in maternity care and readmissions.

Endorsing Measures for Use in Accountability and Performance Improvement

NQF completed 11 endorsement projects during the course of the contract year—using both the NQS priorities that cross conditions and leading health conditions with respect to prevalence and cost as a way to prioritize its efforts. In total, NQF committees evaluated 353 submitted measures and endorsed 170 new measures—or 48 percent of those submitted. While the number of measures endorsed is considerably higher than in previous years, the endorsement rate is lower due to the enhanced rigor of the review criteria. At the same time, NQF placed emphasis on reducing providers' reporting burden by harmonizing specifications related to similar measures.

Currently, the portfolio of NQF-endorsed measures includes more than 700 measures, of which 30 percent assess patient outcomes and experience with care. Considerable progress also has been made in specifying measures for use with electronic health records. NQF worked with 18 measure developers to create eMeasure specifications for 113 existing endorsed measures, and released an initial and updated Measure Authoring Tool (MAT). The re-tooled measures and MAT are innovations that enable the field to get substantially closer to having electronic health records with the capacity to capture and report performance information during routine care.

Aligning Payment and Public Reporting Programs That Reward Value

A significant proportion—about 85 percent—of the measures used in federal programs are NQF-endorsed. Further, NQF-endorsed measures are used extensively by private health plans, state governments, and others. Such alignment can simultaneously reduce reporting burdens for providers and accelerate improvement because of the common signals that payers send. The NQF-convened Measure Applications Partnership (MAP), launched in the spring of 2011, fostered further alignment with its series of three performance measurement coordination strategy reports: Clinician Performance Measurement, Dual-Eligible Beneficiaries, and Healthcare-Acquired Conditions and Readmissions Across Public and Private Payers. As a part of these reports, MAP also developed a framework and criteria to guide the selection of the best measures for use in numerous payment and public reporting programs. Building on these reports, MAP then provided pre-rulemaking guidance to HHS, including input on measure sets pertaining to 17 HHS programs, as well as strategies for enhancing consistency and minimizing reporting burden across federal programs and between public- and private-sector efforts. Leaders from nine different HHS agencies are actively participating in MAP.

This advice from MAP—provided many months in advance of relevant rules—represents a true innovation in rulemaking, with the public and private sectors now having forums for substantive back-and-forth dialogue that cuts across program silos, and a unique opportunity to build a shared perspective and consensus about measure selection. Measures related to care coordination—essential to making care more patient centered—are an object lesson for what is possible with pre-rulemaking convening and endorsement. More specifically, MAP recommended that an existing care transitions measure focused on hospitals also be used in other settings, and suggested a broadening of a readmission measure to include all ages and applicability to additional kinds of providers. MAP also advised the Center for Medicare & Medicaid Services (CMS) to require reporting of medication reconciliation measures at the time of transition between settings. As it turns out, NQF has already endorsed measures for medication reconciliation, readmission, and care transitions that apply to additional settings and populations so these measures can move right into other federal programs.

Taken together, the reports are important stepping stones for MAP as the Partnership works on a comprehensive measurement strategy it will recommend to guide HHS measure selection for federal programs in the coming years. This strategy will be informed by the Partnership's in-depth understanding of current measures and their use in relevant programs, opportunities for potential coordination and integration, growing collaboration across the public and private sectors, and a vision for the future.

Numbers are an essential guidepost for gauging healthcare performance, and measures may be a powerful motivator of change when paired with public reporting and payment. But alone, they cannot drive achievement of the value agenda. Rather, implementation of innovative measures needs to go hand-in-glove with fundamental redesign of delivery and payment systems to achieve the NQS' three, interconnected aims. And while local communities are changing the way care is organized and paid for to break down existing silos, facilitate integration and coordination of care, and connect healthcare to other sectors (e.g., employment, education), such innovations have not yet swept the country. When they do, and are coupled with accountability strategies embraced by the public and private sectors, we will be able to achieve our goals of healthier people and communities, and better, less-costly patient care. We will have then changed healthcare by design and by the numbers.

1 National Quality Forum: Background

More than a decade after their publication, the Institute of Medicine's (IOM's) landmark Quality Chasm and To Err is Human reports still resonate: Our healthcare system continues to fall short on quality, safety, and affordability. That said, recent years have seen a re-energized commitment to improving care and constraining healthcare costs. HHS, NQF, and the increasing number of private-sector organizations that constitute the quality movement are at the center of that resurgence.

Established in 1999 as the standard-setting organization for healthcare performance measures, NQF today has a much-broadened mission to:

  • Build consensus on national priorities and goals for performance improvement, and work in partnership with the public and private sectors to achieve them.
  • Endorse and maintain best-in-class standards for measuring and publicly reporting on healthcare performance quality.
  • Promote the attainment of national goals and the use of standardized measures through education and outreach programs.

NQF is governed by a 27-member Board of Directors (see Appendix B) from a diverse array of public- and private-sector organizations. A majority of seats on the board is held by consumers, employers, and other organizations that purchase healthcare services on consumers' behalf. In 2011, NQF convened hundreds of experts across every stakeholder group on its priority-setting, measure-review, and measure-selection committees—individuals who volunteered their time, talents, experience, and insights (see Appendix F). NQF also directly reached some 10,000 frontline clinicians, hospitals, and others with educational programming via webinars. And its endorsed performance standards touched the care delivered to millions of patients every day.

In recent years, the number and variety of NQF-endorsed measures has greatly expanded. More than 700 NQF-endorsed measures now address most settings of care, conditions, and types of providers. The measures portfolio includes clinical process measures, patient experience of care, the actual outcomes of care, the costs and resources that go into providing care, as well as select structural measures. The portfolio is being enhanced with advanced measures, such as functional outcome and crosscutting care-coordination measures. At the same time, the NQF portfolio is being carefully culled to retire measures that no longer meet the more rigorous criteria. In the last year alone, 353 measures were submitted to NQF and 170, or nearly half, were endorsed. This endorsement rate—or ratio of submitted-to-endorsed measures—reflects NQF's efforts to systematically raise the bar on performance measurement, even as it seeks to reduce the burden on providers by eliminating duplicative measures.

To be NQF endorsed, a measure must be a process or outcome that is important to measure and report, be scientifically acceptable, be feasible to collect, and provide useful results. NQF conducts an eight-step, consensus-based process that has been continually improved over a decade (see Appendix C). Review committees are comprised of multiple stakeholders; consumer organizations are equal partners with clinicians and other stakeholders throughout the process. There is a strong commitment to transparency and NQF invites public participation at every step, ranging from nominations for committees, to decisions on specific measures. Endorsed measures are re-evaluated every three years to ensure their actual use and usefulness in the field and their continuing relevance with current science, and to determine whether they continue to represent the best in class.

Measures included in the NQF portfolio are developed and maintained by about 65 different organizations. The following gives a sense of the range of organizations NQF works with: CMS, the National Committee on Quality Assurance (NCQA), the American Medical Association-Physician Consortium for Performance Improvement (AMA PCPI), Ingenix, the Joint Commission, American College of Surgeons (ACS), Bridges to Excellence, Cleveland Clinic, Minnesota Community Measurement, and Pharmacy Quality Alliance.

In recognition of its skill in building consensus across multiple stakeholders in the measure-endorsement realm, NQF has been asked to convene diverse committees to advise the public and private sectors on priorities for improvement, related implementation strategies, and selection of measures to both drive these strategies and gauge results. The NQF-convened NPP and MAP and their published reports are tangible outcomes of this work. An equally important outcome of these partnerships is the ongoing alignment across stakeholder groups and across public- and private-sector leaders about what levers to use to both improve healthcare performance and move the delivery system to be more patient centered.

NQF has been fortunate to have received support from the federal government for over 10 years, with more substantial support starting in 2008 when federal leaders strongly committed themselves to designing and implementing a value agenda. More specifically:

  • MIPPA has provided NQF with $10 million annually over a four-year period starting in 2009. These funds—awarded to NQF through a competitive process—are supporting the organization's efforts to identify priority areas for improvement, endorse and update related performance measures, foster the transition to an electronic environment, and report annually to Congress on the status and progress to date of this effort.
  • ACA has provided NQF with support of about $10 million, starting in 2011. Under section 3014, Congress directed HHS to contract with “the consensus-based entity under contract” to provide multi-stakeholder input into the NQS, as well as advice to the Secretary of HHS on the selection of measures for use in various quality programs that utilize the federal rulemaking process for measure selection. With federal leadership and support, as well as the support of foundations and over 450 NQF member organizations, much has been collectively accomplished since NQF's founding in 1999. With more substantial and predictable support from the federal government over the last three years, and an enhanced commitment on the part of the public and private sectors to work together, the basic infrastructure for performance measurement is moving into place and our ability to shape and further an environment of accountability has grown. NQF's accomplishments during 2011 will be described against that backdrop.

Sidebar 1—Working With NQF Helped Spur Rapid Evolution of Ophthalmology Measures

There are many intangible benefits from the endorsement activities supported under the HHS contract. One of these is that it provides valuable input to measure developers which helps focus measure development resources on important gap areas. The efforts of the American Academy of Ophthalmology (AAO) are a case in point.

As early as the 1980s, and before many other specialty societies, AAO developed “preferred practice patterns” to provide practice guidance for ophthalmologists. These guidelines proved to be a solid foundation to draw from when, in 2006, AAO began developing related quality measures for quality improvement feedback and public reporting purposes. Over the last five years, AAO has developed ever more sophisticated performance measures—evolving from process, to outcome, to functional status—and credits involvement with the NQF review process as an important catalyst in this evolution.

More specifically:

  • AAO—in collaboration with the AMA-PCPI—first worked to develop process measures focused on eye-care issues such as diabetic retinopathy (damage to the eye's retina as a result of long-term diabetes), and performance of optic nerve exams in primary open-angle glaucoma (chronic, progressive optic-nerve damage) patients.
  • Recognizing that measures that evaluate actual results of care are more critical to improving quality, NQF encouraged AAO to shift its focus to developing clinical outcome measures. As a result, NQF later endorsed a measure focused on reducing glaucoma patients' eye pressure (which can lead to optic-nerve damage or blindness) by 15 percent.
  • More outcome measures were later developed and endorsed under the HHS-funded outcomes project, focusing on issues such as complications within 30 days following cataract surgery, as well as 20/40 or better visual acuity within 90 days of cataract surgery.
  • Recently, the NQF board has approved measures related to patient functional status, attempting to measure improvement in patients' visual functional status and their overall satisfaction within 90 days following cataract surgery. These measures are currently under NQF review, and have been included in the 2012 Physician Quality Reporting System (PQRS) measure set.

Dr. Flora Lum, executive director of AAO's H. Dunbar Hoskins Jr., MD Center for Quality Eye Care, noted that NQF's ability to bring patient and consumer perspectives to the Steering Committee responsible for evaluating measures has been invaluable over the years. AAO's efforts to advance healthcare quality continue, with the organization now striving to develop appropriateness-of-care measures.

The evolution of AAO's measures over a short time period is noteworthy and the information that results from the measures provides physicians with multi-faceted feedback about the care they deliver. Ideally, such information is available in rapid-response reports, with educational interventions to help facilitate improvements at the practice level, and over time, so that ophthalmologists and patients can gauge progress. As AAO has gone on this journey to develop ever-increasingly sophisticated and meaningful measures, NQF has been pleased to be a part of it. [End of Sidebar 1]

Sidebar 2—Resource-Use Measures: Critical to the Value Agenda

U.S. healthcare per-capita spending is greater than that in any other country, yet it has not resulted in better health for Americans. With costs increasing beyond annual inflation, spending is largely focused on treating acute and chronic illnesses rather than prevention and health promotion.

Deriving more value from health spending is predicated on having both quality and cost (or resource use) information. To date, limited information about resource use exists. CMS and many measure developers are working to change that, and in 2009, NQF was tasked with further defining resource-use measures and identifying important attributes to consider when evaluating them. NQF also endorsed its first-ever resource-use measures during the 2011 contract year.

As defined by NQF, resource-use measures are comparable measures of actual dollars or standardized units of resources applied to the care given to a specific population or event—such as a specific diagnosis, procedure, or type of medical encounter. The endorsed measures:

  • Relative Resource Use for People with Diabetes
  • Relative Resource Use for People with Cardiovascular Conditions
  • Total Resource Use Population-Based Per-Member Per-Month (PMPM) Index
  • Total Cost of Care Population-Based PMPM Index

“The endorsement of standardized measures of healthcare resource use and cost fills a huge void that has kept the nation from measuring the value of healthcare in a consistent way,” said Steering Committee member Dolores Yanagihara, director, pay for performance, at the Integrated Healthcare Association. “That said, it is a complex process, both technically and from an accountability standpoint. The measures recommended for endorsement give us a broader picture of healthcare—overall and related to specific conditions.” [End of Sidebar 2]

2 Bridging Consensus About Improvement Priorities and Approaches

Released by HHS in March 2011, the country's NQS focuses the public and private sectors on an inspiring set of three, interconnected aims—better care, more affordable care, and healthier people and communities—as well as six related priority areas (see Figure 1). While the field has long targeted improving clinical care, the NQS gives significant, equal heft to the notion of health/wellbeing and affordability.

The NQS provides a critical framework for the efforts of the multiple-stakeholder committees convened by NQF. These efforts range from discussions at the highest, most conceptual levels about a three-to-five-year measurement strategy to undergird the evolving value agenda; to committees working in a new measurement area and developing consensus about what and how to measure; to those simultaneously enhancing and culling a set of measures in an established area, while considering their larger context within the NQF-endorsed measurement portfolio.

National Priorities Partnership

Development of the landmark NQS was informed by the collective input of the NQF-convened National Priorities Partnership (NPP), a collaboration of 51 public- and private-sector organizations uniquely qualified to represent the array of stakeholders needed to improve the nation's healthcare system. As the NQS was being formulated, HHS sought multi-stakeholder input from NPP on its aims and priorities. After publication of the NQS in March 2011, HHS again reached out to NQF to convene NPP to provide input on further specifying goals, measures, and implementation pathways to move the national strategy and related priorities forward, drawing upon the real-world experience of its stakeholder participants.

The NPP recommendations are captured in a follow-up report to the HHS Secretary, Priorities for the National Quality Strategy, published in September 2011. This second report identifies goals and measure concepts that address the three NQS aims and six priorities simultaneously. For example, there are suggestions for goals and measurement areas related to care coordination that cut across clinical conditions. This would encourage better, more integrated care delivery, enhanced health outcomes, and fewer wasted resources. The NPP report also acknowledges that successful implementation of NQS-related goals and measures are predicated on strategic and technical measure alignment—or agreement—across various levels of accountability in our healthcare system. This starts at the most granular level—the patient and physician—and moves in a linked chain across a family of measures and levels of increasing aggregation. Without agreement about strategic direction and concordance on measure selection, a predictable cacophony results, frustrating clinicians and confusing consumers. The cholesterol-control example (Figure 2) provides an illustration of a family of measures with linkages across levels and illustrates this crucial strategy of alignment. Further, these NQF-endorsed measures are included in HHS's newly launched and broad-based Million Hearts Campaign—a public-private initiative that aims to prevent one million heart attacks and strokes in five years.

In addition to NPP's consultative role as it relates to the NQS, NPP has served as a catalyst in developing implementation strategies—working across diverse stakeholder groups to spur collective action—focused on improving patient safety and reducing patient harm. Such a focus also can reduce costs, with the IOM estimating that decreasing healthcare-associated infections (HAIs), complications, and unnecessary readmissions by 10 to 20 percent could result in $2.4 billion to $4.9 billion annual savings for the U.S. healthcare system.5

NQF's Focus on Safety

In 2011, NQF's work in the safety realm spanned updating of measures and serious reportable events (SREs), a recommended approach for further aligning public- and private-sector patient-safety measurement strategies, and development of implementation strategies in support of HHS's Partnership for Patients Initiative.

Partnership for Patients is engaging stakeholders from the private and public sectors to reduce all-cause harm (i.e., all forms of harm that can affect patients) and hospital readmissions. More specifically, NPP partnered with the Partnership for Patients to host 11 webinars that attracted about 10,000 frontline clinicians, hospitals, and others across the country and provided education, tools, resources, and insight on key safety issues. These webinars ranged from big-picture interventions (e.g., how to get your Board on board when it comes to improving patient safety), to those with a more laser focus on clinical teams (e.g., reducing surgical-site infections [SSIs]). Nearly 90 percent of webinar participants, who came from every region of the country, reported that they would be able to implement something new in their institutions as a result of this novel public-private programming. Moving forward in 2012, NPP is developing two action pathways, which its multiple partners can implement and spread. These pathways are focused on the health of mothers and babies by reducing elective deliveries before 39 weeks, and reducing avoidable admissions and re-admissions across all settings of care. These represent 2 of the 10 areas Partnership for Patients is pursuing to achieve its global safety and harm-reduction goals. Reaching these goals also will substantially reduce costs.

In addition, MAP released a report, Coordination Strategy for Healthcare-Acquired Conditions and Readmissions Across Public and Private Payers, in October 2011, detailing the ways in which public and private healthcare providers can align performance measurement to enhance patient safety. Specifically, the report makes three recommendations: (1) There needs to be a national set of core safety measures applicable to all patients; (2) Data need to be collected on all patients to inform these national core safety measures; and (3) Public and private entities need to coordinate their efforts to make care safer. MAP's recent pre-rulemaking report further emphasizes the importance of safety measures by supporting their inclusion in federal public reporting and performance-based payment programs, and MAP will focus on alignment of core safety measures across programs in 2012. With respect to measure review, NQF endorsed numerous patient-safety measures, including healthcare-associated infections (HAIs), which now address long-term, acute-care and rehabilitation hospitals, and radiation-safety measures, to name a few.

NQF also updated its list of SREs, a compilation of serious, harmful, and largely—if not entirely—preventable patient-safety events, designed to help the healthcare field assess, measure, and report performance in providing safe care. In the 2011 update, the events were broadened in focus to explicitly include hospitals, office-based practices, ambulatory surgery centers, and skilled nursing facilities to reflect the various settings in which patients receive care and could experience harm. Based on input from users, the implementation guidance for each event was expanded, and a glossary was added to facilitate uniformity in reporting of the events. The list includes wrong-site surgery; death or serious injury associated with medication errors or unsafe blood products; and failure to follow up on lab, pathology, or radiology test results. Public and private purchasers have drawn heavily from the SRE list in identifying healthcare-associated conditions for use in payment and reporting programs. (See Sidebar 3.)

Sidebar 3—NQF and Patient Safety

Patient-Safety Measures

NQF's inventory of endorsed measures includes more than 100 patient-safety measures, with several focused specifically on healthcare-associated infections or HAIs. Preventing HAIs has become a national priority for public health and patient safety. To date, 27 states are requiring public reporting of certain HAIs. Further, the NQS has identified safer care as one of its primary aims and, in 2013, hospitals' annual Medicare payment updates will be tied to submission of infection data, including central line-associated bloodstream infections and surgical-site infections (SSIs).

In this past year, NQF endorsed four additional patient-safety measures focused on HAIs, including a successfully harmonized measure from the American College of Surgeons and the Centers for Disease Control and Prevention focused on SSIs, and updates of existing HAIs addressing urinary tract infections and bloodstream infections. These efforts were completed under federal contract.

Serious Reportable Events

Preventing adverse events in healthcare is also central to NQF's patient-safety efforts. To ensure that all patients are protected from injury while receiving care, NQF has developed and endorsed a set of serious reportable events (SREs). This set is a compilation of serious, harmful, and largely—if not entirely preventable—patient safety events, designed to help the healthcare field assess, measure, and report performance in providing safe care. The SREs focus on the following areas:

  • Surgical or invasive-procedure events
  • Product or device events
  • Patient-protection events
  • Care-management events
  • Environmental events
  • Radiologic events
  • Potential criminal events

Originally envisioned as a set of events that would form the basis for a national state-based reporting system, the SREs continue to serve that purpose. To date, 26 states and the District of Columbia have enacted reporting systems to help stakeholders identify and learn from SREs. The majority of those states incorporate at least some portion of NQF's list to help align reporting efforts and encourage learning across healthcare systems. [End of Sidebar 3]

Finally, NQF launched a project in 2011 that will leverage health IT data to address patient safety and quality concerns associated with medical devices, such as pumps used to deliver intravenous medications at home. This project, which continues in 2012, will determine what data needs to be collected and shared to improve quality and safety related to devices. It also will focus on ways to identify and report adverse events associated with the use of such devices.

3 Endorsing Measures and Developing Related Tools

With its extensive evaluation (see Sidebar 4) and multi-stakeholder input, NQF is recognized as a voluntary consensus standards-setting organization under the National Technology Transfer and Advancement Act of 1995. In addition, NQF adheres to the Office of Management and Budget's formal definition of consensus.6 Consequently, NQF-endorsed measures have special legal standing allowing federal agencies to readily adopt them into their programs, which they have done at a striking rate. About 85 percent of measures in federal health programs are currently NQF-endorsed, including those that apply to hospitals, clinicians, nursing homes, patient-centered medical homes, and many other settings.

In 2011, NQF completed 11 endorsement projects—reviewing 353 submitted measures and endorsing 170, or 48 percent. Enhancements to the endorsement process over the last year included strengthening its rigor by requiring testing of measures prior to measure review, initiation of a project to reduce endorsement cycle time, integration of review of existing measures with new measures to ensure harmonization and best-in-class assessment, and creation of an expedited review process to respond to important regulatory or legislative requests. In addition, NQF worked with 18 measure developers to update 113 electronic measures, or eMeasures, so they could be more readily collected through EHRs, and introduced and updated tools to respectively facilitate development and collection of eMeasures.

Sidebar 4—What does it take for a measure to get endorsed?

With the enhanced rigor of NQF's endorsement criteria, only about 50 percent of submitted measures were endorsed this past year.

The leading reason that measures do not pass the grade is failure to meet the “must pass” importance-to-measure-and-report criterion. This includes being able to demonstrate that the proposed measure or related data is focused on a high-impact health goal or priority; there is less-than-optimal performance; and there is strong scientific evidence for the measure, with respect to quality, quantity, and consistency. NQF expert committees rate the evidence based on specific guidance.

The second “must pass” criterion is scientific acceptability of measure properties. In other words, do the data from testing the measure show that it is reliable and valid and precisely specified? Expert committees look for moderate-to-high ratings so they are confident the measure results are reliably consistent and can be compared across providers and analyzed longitudinally. Other important criteria include usability and feasibility—assessing whether intended audiences can understand the results and find them helpful for decision-making and quality improvement. The criteria also consider whether providers can collect data without undue burden. See Appendix C for more detail. [End of Sidebar 4]

NQF Endorsement in 2011

The overall framework used to guide the NQF measures portfolio is multi-dimensional. It includes the NQS crosscutting priorities, as well as leading health conditions with respect to prevalence and cost that affect an array of populations. Figure 3 provides a snapshot of how the current NQF-endorsed measures portfolio stacks up against the NQS, with the percentages reflecting the proportion of NQF-endorsed measures against the six priorities. Some measures are counted in multiple priority areas. The chart shows gaps in emerging measurement areas, including patient-family centered care, measures related to community health and wellbeing, and affordability. These gaps require significant foundational work to understand what to focus on for measurement and how to best overcome technical barriers. NQF has undertaken this foundational work over the last year, and has started to bring in measures in all of these areas for endorsement review.

The 170 measures newly endorsed by NQF in 2011 include many outcome measures; measures that focus on populations previously under-represented, including pregnant women and children; a number of patient-safety measures—given the importance of reducing patient harm; measures in new areas that fill important gaps, such as cost (resource use); as well as the updating of measures related to highly prevalent conditions, (e.g., cardiac and surgical care). More specifically:

Outcome Measures

NQF has made great strides over the past year to endorse measures that evaluate results of care, particularly in the patient-safety, nursing-home, and surgical-care areas. Outcome measures are considered most relevant to patients and providers looking for improved quality and patient experience, as opposed to measures that assess process or structure. Examples of outcome measures endorsed in 2011 include potentially avoidable complications for select conditions (i.e., stroke, pneumonia), remission of symptoms in patients with depression, and patient experience in nursing homes and dialysis facilities.

Patient-Safety Measures

Long a focus of NQF, these new patient-safety measures span settings and types of conditions. They include measures focused on HAIs (urinary tract, central-line-associated bloodstream, and SSIs), and measures focused on issues such as standardized data collection and reporting of radiation doses.

Maternal and Child-Health Measures

These populations have been underrepresented in performance measurement. NQF has worked to fill these gaps through two endorsement projects over the past year—child health, and perinatal and reproductive health. Child-health measures focus on important screenings and access to care, including immunizations, hearing assessments, and well-child visits. Other measures address population health outcomes, including the number of school days missed due to illness and birth outcomes. Proposed perinatal measures (this project is still underway) address procedures such as cesarean sections and elective delivery prior to 39 weeks.

New and Existing Measurement Areas

NQF reviewed measures related to resource use, both those related to conditions (e.g., diabetes and cardiovascular disease), and those related more to global resource use. Endorsement projects in 2011 also focused on reviewing existing measurement areas for high-prevalence conditions or areas (palliative care and end-of-life care, cardiovascular disease and kidney disease), adding new measures, and retiring others as the expert committees saw fit. More specifically, NQF endorsed or maintained measures focused on optimal vascular care, complications or death for specific surgical procedures, and assessment of post-dialysis weight by nephrologists for kidney disease patients. Although NQF has made considerable progress in endorsing outcome measures—which constitute about 30 percent of the portfolio—differences exist with respect to outcome and process measures across conditions, which is illustrated in Figure 4. For example, there are more outcome measures for surgery and perinatal care than for mental health and cancer care. Also, HAIs are reflected under surgery, not infectious disease.

When NQF begins to address a new measurement area, the relevant expert committee will often start by developing a framework report to guide its future measurement review. These reports may include a scan of existing measures, a discussion about where there are key opportunities for improvement, and consideration of potential technical barriers. For example, NQF is developing a population health-measurement framework aimed at aligning delivery system, public health, and community stakeholder efforts to improve health outcomes and the social determinants of health. Historically, there has been little coordination across these sectors. NQF is also developing a patient-centric measurement framework for assessing the efficiency of care provided to individuals with multiple chronic conditions. This report will inform NQF's future efforts to endorse measures that apply respectively to population health and care for people who have more than one chronic condition.

Culling the NQF Portfolio

A key part of NQF's review process is focusing on endorsing best-in-class measures and eliminating similar or even identical measures that create confusion and burden across clinical settings and providers. This alignment of very similar measures—or measure harmonization—can reduce reporting burden for providers and enhance comparability of results for patients and payers, thereby reducing confusion and enabling decision-making. The harmonization of the surgical site infection measures from the Centers for Disease Control and Prevention and the ACS is a case in point (see Sidebar 5). Further, NQF's maintenance process retires existing measures that no longer meet the higher endorsement bar, thereby further culling the portfolio.

Sidebar 5—Harmonizing Surgical-Site Infection Measures

As part of NQF's federally funded Patient-Safety Measures project, similar and competing surgical-site infection (SSI) measures from the Centers for Disease Control and Prevention (CDC) and the American College of Surgeons (ACS) were reviewed. The CDC SSI measure has been in use since 2005; the ACS measure since 2004.

As a result of NQF member and public comments, and requests by the Steering Committee, the developers worked with NQF support to harmonize these two competing approaches to measurement. The result is a newly harmonized SSI measure, which is currently focused on abdominal hysterectomies and colon surgeries. CDC and ACS will jointly maintain the measure. The two organizations have also committed to developing harmonized measures for other procedures and will incorporate them into the combined SSI measure.

Notably, CMS has selected this harmonized measure for inclusion in the 2012 final rule of the Inpatient Prospective Payment System (IPPS).

Dr. Clifford Ko, director of ACS's National Surgical Quality Improvement Program, was directly involved in this effort. Dr. Ko noted that the resulting measure—Harmonized Procedure-Specific Surgical-Site Infection Outcome Measure—will now be available to literally thousands of hospitals that want to measure and improve their surgical-site infection rates.

Dr. Daniel Pollock, surveillance branch chief in CDC's Division of Healthcare Quality Promotion, says CMS' decision to include this measure will significantly increase SSI reporting rates in hospitals throughout the country. With increased reporting, providers will have more opportunities to identify areas for improvement. In addition, patients and payers will have SSI rate information when they are choosing between hospitals in a community.

While both Drs. Ko and Clifford noted that some characteristics of the original measures may be diminished or lost, they agreed that harmonized measures help eliminate the confusion non-comparable measures create and that, ultimately, providers, payers, and the public benefit. [End of Sidebar 5]

The recent Cardiovascular Project illustrates how NQF expert committees now consider new measures against existing endorsed measures. Using the measure evaluation criteria and guidance on evaluating related and competing measures, the Cardiovascular Committee reviewed proposed new measures and those undergoing maintenance, focusing on measures that address the broadest patient population or settings, while avoiding duplication whenever possible. Based on this rigorous vetting, 39 out of 65 measures (7 new and 32 undergoing maintenance) were endorsed (see Figure 5). When all is said and done, between 2010 and 2011 this represents approximately 13 percent fewer NQF-endorsed cardiovascular measures in this project.

Enhancing NQF Endorsement

As NQF's measures portfolio evolves, so too does its endorsement process. In 2011, NQF enhanced the rigor of its process by requiring that measures be tested before they are reviewed. This requirement now ensures that expert committees have crucial information about measure reliability and validity as they consider endorsement. In addition, NQF also established an approach that added greater consistency to review of the underlying evidence for measures, and created an expedited endorsement pathway to be responsive to key regulatory or legislative requests. Finally, NQF embarked upon a number of efforts to enhance effectiveness of the review process, including a lean effort to further reduce endorsement cycle time. This effort, which got underway in late 2011, maps each of the steps of the endorsement process to drive out redundancy, waste, and ultimately costs for measure developers, NQF, and HHS.

The Information Technology Accelerant

A future healthcare system that fully embraces health information technology (HIT) will allow for performance data to be collected in real time across settings, integrated, and regularly fed back to providers to inform practice and decision-making. It also will allow performance information to be made accessible in aggregated, de-identified, and timely public reports for payers and patients. Recent federal efforts—to simultaneously wire ambulatory practices and hospitals and assess providers' “meaningful use” of electronic health records (EHRs)—have been important steps on the path to a future HIT-enabled system.

Such milestones have been augmented by a number of NQF efforts that are helping the field move to a common electronic data platform that allows for the collection of more clinically relevant and actionable performance-measurement data. These HIT-enabled environments hold out the promise of reducing reporting burden for clinicians and other providers, and enhancing the precision and comparability of results.

In the past year, NQF has worked with measure developers to re-specify paper-based measures for EHRs, and developed tools that allow measure developers to marshal the building blocks necessary for their successful implementation. In both cases, these efforts broke new ground. To the best of NQF's knowledge, they have never been attempted—or accomplished—before. More specifically:


In 2010, at the request of HHS, NQF worked with 18 measure developers to re-tool 113 existing, endorsed measures for the electronic environment—that is, to develop electronic specifications that allow an EHR to calculate the measure—so they could be included in the Meaningful Use program. These eMeasures were further updated and enhanced in 2011. The measure stewards and NQF found that re-tooling measures for a new (electronic) platform was not a simple, straightforward matter; rather it involved the stewards re-conceptualizing each of the measures, with the support of NQF.

Quality Data Model (QDM)

This information model provides measure developers with a first-ever “grammar,” which defines data elements. These data elements can then be efficiently assembled and re-assembled into performance measures to be read by EHRs. Work on the QDM began in 2007, with funding from the Agency for Healthcare Research and Quality (AHRQ). In 2011, the third version of the QDM was released, which includes data elements to enable development of measures in gap areas, including patient/consumer engagement and disparities, as well as new methods of data capture and use. In summary, this effort makes a substantial contribution toward being able to more readily leverage existing electronic health-record data to produce clinically relevant, advanced measures.

Measure Authoring Tool (MAT)

This non-proprietary, web-based tool makes it easier and more efficient for measure developers to specify, submit, and maintain electronic measures, or eMeasures. Introduced in 2011, there are now more than 35 organizations using this tool for eMeasure development.

Work that began in 2011 and carries over into 2012 includes a project focused on sharing data across settings, convening a forum for stakeholders to share best practices related to implementation of eMeasures, and a project that will leverage health IT data to address patient safety and quality concerns associated with medical devices, which was described previously. More specifically, with respect to the first two projects:

HIT Systems To Support Care Coordination Measurement: Data Sources and Readiness

This project is analyzing the current process for identifying and sharing data on significant patient factors, planned interventions, and expected outcomes (care goals) to support quality measurement related to transitions of care. It will recommend a critical path forward with specific action steps that the government can take to enable electronic measurement around care plans.

E-Measure Collaborative

The eMeasure Collaborative, a public forum convened by NQF, is bringing together stakeholders from across the quality enterprise. The eMeasure Collaborative's goal is to promote shared learning and advance knowledge and best practices related to the development and implementation of eMeasures.

4 Aligning Accountability Programs To Enhance Value

At the request of HHS, NQF commissioned RAND Health to conduct an initial evaluation to better understand who is using NQF-endorsed measures and for what purposes. The RAND studies—coupled with NQF's own internal tracking efforts to understand measure use—have helped to provide some important context for HHS, NQF, and the NQF-convened MAP discussions.

Growing Use of NQF-Endorsed Measures

RAND interviews of key stakeholders using NQF-endorsed measures and online research across approximately 75 varied organizations found that nearly all used NQF-endorsed measures, although the extent varied as did the particular measures selected for use. Further, the study showed that most organizations used endorsed measures in quality-improvement efforts, followed closely by public reporting, then payment programs. The 2011 study also found that there is a strong preference to use NQF-endorsed measures where they exist because they are vetted, evidence-based, and seen as more credible within the provider community

NQF's additional research outside of the HHS contract indicates that about 90 percent of the portfolio of NQF-endorsed measures is being used in varied programs across the public and private sectors. Figure 6 is an estimation of the use of NQF-endorsed measures by: federal programs; private payers such as health plans and employers; states; and an amalgamation of other key stakeholders such as national registries, accrediting and specialty board certifying organizations, and community alliances. The gold-colored, hatched, and dotted areas on the chart represent alignment in use of the same measures by key sectors—specifically the overlap between private payers (health plans and employers) and federal programs, and the overlap between state and federal efforts. Alignment holds out the promise of reducing data-collection burden for providers and associated costs, while simultaneously accelerating improvement by sending the same message about where providers should be focusing improvement resources.

Overall use of NQF-endorsed measures by the federal government is high—about 85 percent of measures used in federal programs are NQF-endorsed. Yet the proportion of NQF-endorsed measures in use by various federal programs does differ. Sometimes it is a matter of timing. For example, the federal government has recently moved some non-endorsed measures into the Physician Quality Reporting System (PQRS) to better address the range of physician specialties. NQF is poised to quickly review such measures.

States also are heavy users of NQF-endorsed measures, in part due to federal programs that encourage or require standardized reporting at the state level, such as AHRQ's Health Care Utilization Project (HCUP), CDC measures and surveys, CHIPRA, and Medicaid. For example, 81 percent of CHIPRA measures and 88 percent of core adult Medicaid measures are NQF-endorsed. In the safety realm, more than half of states and the District of Columbia have implemented reporting systems for SREs, as well as reporting of key patient-safety indicators such as bloodstream and SSI measures.

Sidebar 7—AF4Q: Alignment at the Community Level

At the community level it is more challenging to get a comprehensive picture of use of NQF-endorsed measures. That said, leading multi-stakeholder alliances in communities across the country use NQF-endorsed measures, including the Robert Wood Johnson Foundation's Aligning Forces for Quality (AF4Q) alliances. To support community interest in aligning the measures they are using, a recent analysis conducted by NQF outside of the HHS contract has shown that at least 170 NQF-endorsed measures are being used in one or more of the 16 AF4Q alliances. In addition, NQF endorsed measures are being used by many of the Chartered Value Exchange (CVE) collaboratives, the federally-funded Beacon communities, other communities and a number of states. Given that there is no national requirement to use standardized measures at this level, communities/states have shown leadership in adopting such measures into their local programs.

The Robert Wood Johnson Foundation's Aligning Forces for Quality initiative seeks to increase the quality of healthcare and reduce racial and ethnic disparities in 16 diverse communities—with the involvement and collaborative efforts of physicians, patients, consumer groups, hospitals, health plans, and others.

The U.S. Agency for Healthcare Research and Quality (AHRQ) supports 24 Learning Network Chartered Value Exchanges. The CVEs are experimenting with new ways to bring healthcare stakeholders together to collect data and improve the quality of care.

The federal Beacon Community Cooperative Agreement program provides 17 communities with funding to improve quality, cost-efficiency, and population health using electronic health records and other health information technology tools to collect and analyze clinical data. The program's goal is to demonstrate the ability of health IT to transform local healthcare systems.

i Geographic reach of these efforts varies, e.g., state-wide, county-specific [End of Sidebar 7]

Measure Application and Alignment

Convened by NQF in the spring of 2011, the Measure Applications Partnership (MAP) is a public-private partnership made up of 60 organizations representing major stakeholder groups, 9 federal agencies, and 40 subject-matter experts. It was established to provide HHS with thoughtful, pre-rulemaking input about which performance measures to use in public reporting and payment within and across 17 federal programs. Simultaneously, MAP is informing the thinking and decisions of private-sector leaders with respect to their measure-selection strategies.

Federal Agencies Participating in Map

  • Agency for Healthcare Research and Quality
  • Centers for Disease Control and Prevention
  • Centers for Medicare & Medicaid Services
  • Health and Human Services' Office on Disability
  • Health Resources and Services Administration
  • Office of the National Coordinator for Health Information Technology
  • Office of Personnel Management
  • Substance Abuse and Mental Health Services Administration
  • Veterans Health Administration

MAP represents an important innovation in the regulatory process made possible by ACA statute. In contrast to traditional federal rulemaking—where there are limited, unidirectional forums for input before draft rules are issued and no forums that cross programmatic areas—MAP enables public- and private-sector leaders to work together on creating a measurement strategy and implementation plan that is crosscutting and coordinated across settings of care; federal, state, and private programs; levels of measurement analysis; payer type; and points in time. This is not an overnight prospect, but important, unprecedented steps in the direction of strategic alignment were taken.

In 2011, MAP consisted of four programmatic-oriented workgroups—clinician, hospital, LTC/PAC, and dual-eligible beneficiaries—and an ad-hoc safety workgroup, each of which makes recommendations to the MAP Coordinating Committee. This independent committee then integrates and aligns these recommendations across the four programmatic areas—which represent 17 different federal programs—and advises HHS directly. (See Sidebar 8)

Sidebar 8—Measure Applications Partnership Workgroup Leadership

MAP Coordinating Committee Co-Chairs

George Isham, MD, MS, Chief Health Officer, Health Partners

Elizabeth McGlynn, Ph.D., MPP, Director Center of Effectiveness and Safety Research (CESR), Kaiser Permanente

MAP Advisory Workgroups

Ad-Hoc Safety Workgroup:

Frank G. Opelka, MD FACS, Chair, Vice Chancellor for Clinical Affairs and Professor of Surgery, Louisiana State University

Clinician Workgroup:

Mark McClellan, MD, Ph.D., Chair, Director, Engelberg Center for Health Care Reform, Senior Fellow, Economic Studies, Brookings Institution, Leonard D. Schaeffer Chair in Health Policy Studies

Dual-Eligible Beneficiaries Workgroup:

Alice R. Lind, MPH, BSN, Chair, Senior Clinical Officer, Center for Health Care Strategies

Hospital Workgroup:

Frank G. Opelka, MD FACS, Chair, Vice Chancellor for Clinical Affairs and Professor of Surgery, Louisiana State University

Post-Acute/Long-Term Care (PAC/LTC) Workgroup:

Carol Raphael, MPA, Chair, President and Chief Executive Officer, Visiting Nurse Service of New York [End of Sidebar 8]

In the fall of 2011, and in advance of future measure-selection recommendations, MAP issued reports offering advice to HHS about how the agency might better coordinate its measure strategies as it relates to efforts focused on improving safety and clinician performance. Its reports include MAP Coordination Strategy for Clinician Performance Measurement and MAP Coordination Strategy for Healthcare-Acquired Conditions and Readmissions Across Public and Private Payers. In 2011, MAP also released the first of two reports focusing on dual-eligible beneficiaries who are enrolled in both Medicare and Medicaid programs: MAP Strategic Approach to Performance Measurement for Dual-Eligible Beneficiaries. Despite many of these individuals being the sickest and poorest patients enrolled in any federal program, not to mention among the most expensive, there has been little effort to date to use measurement as a tool to improve their care. For more detail about NQF's efforts to address vulnerable populations, see sidebar 6.

Sidebar 6—NQF Focuses on Vulnerable Populations

Vulnerable populations—from the disabled, to veterans, to special needs kids, to low-income individuals and racial/ethnic minorities, among others—often require a different and frequently higher level of care. Over the past year, NQF has taken on two major projects with a prime focus on such vulnerable individuals—The Measure Applications Partnership (MAP) Strategic Report: Performance Measurement for Dual Eligible Beneficiaries Interim Report to HHS, and measurement work focused on disparities in healthcare.

The interim MAP report provides multi-stakeholder input on performance measures to assess and improve the quality of care delivered to individuals who are eligible for both Medicare and Medicaid (dual-eligible). An estimated 8.9 million individuals are classified as dual-eligible, a population that includes many of the poorest and sickest individuals in our communities. This particular population frequently experiences fragmented care and accounts for a disproportionate share of total healthcare costs.

In its initial phase of work, MAP has developed a strategic approach to performance measurement and identified opportunities to promote significant improvement in the quality of care provided to these vulnerable populations. The core of the strategic approach is composed of:

A vision for high-quality care. Centered on the needs and preferences of an individual and his or her loved ones, this relies on holistic supports to maximize function and quality of life.

Guiding principles. These include desired effects, measurement design, and data.

A discussion of high-need subgroups. MAP deliberations suggested that there is not yet an established taxonomy for classifying subgroups of the dual-eligible population. MAP members observed that combinations of particular risk factors lead to high levels of need in an additive or synergistic manner.

High-leverage opportunities for improvement through measurement. MAP reached consensus on five areas where measurement could drive significant positive change, including quality of life, care coordination, screening and assessment, mental health and substance use, and structural measures of coordination between Medicare and Medicaid benefits.

In addition to the four primary elements, MAP also considered issues related to data sources and program alignment as inputs to the strategic approach. MAP will next consider gaps in currently available measures and may propose new measure concepts for development. A final report with MAP's input on improving the quality of care delivered to dual-eligible beneficiaries, including recommendations related to measures, is due to HHS on June 1, 2012.

NQF's healthcare disparities measurement efforts are multi-faceted. For example, measure developers are required to submit measure results stratified by race and ethnicity at the time of measure evaluation. NQF has also worked to endorse measures that address vulnerable populations, including measures used for the Children's Health Insurance and Reauthorization Act (CHIPRA) and Medicaid, as well as measures that fulfill important needs for vulnerable populations, including frail elders, pregnant women, children, and those who suffer from mental illness. With respect to already endorsed measures, NQF is working to identify measures across all settings that should be routinely stratified by race and ethnicity in order to identify conditions and populations that require targeted improvement efforts to improve quality and eliminate disparities. [End of Sidebar 6]

MAP's initial pre-rulemaking report published on February 1, 2012, and based on the consensus of 60 organizations:

  • Recommends that 40 percent of the measures CMS was considering move into federal programs targeting clinicians, hospitals, dual-eligible beneficiaries, and PAC/LTC settings via rules issued in 2012, with another 15 percent targeted for future consideration after further development, testing, and feasibility issues are worked out. MAP did not support inclusion of about 45 percent of other measures proposed by CMS. CMS submitted a large number of measures and measure concepts to get early, detailed feedback about them from key stakeholders. Consequently, many of the measures submitted did not have enough information to guide MAP measure evaluation and selection. See Appendix D for the criteria MAP used to guide measure selection.
  • Expresses clear preference for use of NQF-endorsed measures and feedback loops Nearly 87 percent of measures MAP supported for inclusion are currently endorsed by NQF, and many more are likely eligible for expedited review. That said, assessing the qualitative and quantitative impact of NQF-endorsed measures in the field would provide new and important information for future MAP analyses and decision-making.
  • Considers how to further align measures across programs and with the private sector with the goal of more targeted, interrelated sets of measures that are reported by different kinds of providers, in different settings and sectors, and across time. A good example is care-coordination measures contained within existing programs—care transitions, readmissions, and medication reconciliation—which MAP recommends be applied to additional kinds of providers, types of settings, and, consequently, to span and be integrated across federal programs. See Figure 7 to get a more detailed sense for MAP's crosscutting recommendations for care coordination.
  • Lays out guiding principles for a future three-to-five-year measurement strategy that supports movement towards a healthcare system that enhances value for patients, communities, and those that pay the bills on their behalf. In this future 21st century system, priority is placed on measures that drive the system toward meeting the NQS; measurement is person- rather than clinician- or setting-focused; and measures span settings, time, and types of clinicians. Person-centered measurement provides information about what matters to patients (e.g., “Will I be able to run after I recover from knee surgery?”) and measures that are specific to patient populations or care over time, (e.g., “Did I get the care and support needed to manage my diabetes so that I did not lose my vision or my mobility?”). This kind of measurement is predicated on a redesigned delivery and payment system, and an HIT-enabled environment that facilitates both coordination and integration of care for a range of patients across the continuum.

Figure 7—Aligning Care Coordination Measures Across Programs

ClinicianHospitalPost-acute care/long-term care
Care TransitionsSupport CTM-3 (NQF #0228) if successfully developed, tested, and endorsed at the clinician levelSupport immediate inclusion of CTM-3 measure and urge for it to be included in the existing HCAHPS surveySupport CTM-3 if successfully developed, tested, and endorsed in PAC-LTC settings.
Support several discharge planning measures (i.e., NQF #0338, 0557, 0558)Identify specific measure for further exploration for its use in PAC-LTC settings (i.e., NQF #0326, 0647).
ReadmissionsReadmission measures are a priority measure gap and serve as a proxy for care coordinationSupport the inclusion of both a readmission measure that crosses conditions and readmission measures that are condition-specificIdentify avoidable admissions/readmissions (both hospital and ER) as priority measure gaps.
Medication ReconciliationSupport inclusion of measures that can be utilized in a health IT environment including medication reconciliation measure (NQF #0097)Recognize the importance of medication reconciliation upon both admission and discharge, particularly with the dual eligible beneficiaries and psychiatric populationsIdentify potential measures for further exploration for its use across all PAC-LTC settings (i.e., NQF #0097).

The MAP proposed guiding principles support the direction of many public- and private-sector leaders who are innovating to move the nation's care delivery system towards more organization and shared accountability for patient welfare, community health, and stewardship of scarce resources. Where appropriate, they are encouraging transitioning from solo-physician practices to actual and virtual patient-centered medical homes, from stand-alone hospitals to those working collaboratively with an array of providers in an integrated delivery system or Accountable Care Organization (ACOs), and from single-specialty to multi-specialty physician groups working more closely with public health oriented organizations. Figure 8 details some key principles to guide measure selection, measurement tactics, the providers the measures are focused on, and the related federal programs.

Implementation of more advanced measures will be possible once care is more organized and integrated, payment crosses settings and providers, and HIT infrastructure is widely in place. Advanced measures could include how well patient care is coordinated between primary and specialty care and across specialists; whether patients are free of pain and can return to work, school, and other daily obligations; the degree to which patient preferences are incorporated into care decisions; and whether recommended care was appropriate in the first place and delivered cost effectively. Progress is being made as it relates to the development and implementation of such advanced measures, but is predicated on more integrated payment and delivery systems, as well as robust, common electronic data platforms.

Achieving Results

Those working to improve performance of the healthcare system are impatient for results, which take time to demonstrate and are influenced by many factors beyond measurement. Nevertheless, there are promising examples, particularly for hospitals and health plans that have been collecting, reporting, and acting on performance measures for a number of years. The case studies included in this section of the report were selected to provide illustrative examples of different kinds of programs and providers using NQF-endorsed measures (although they are efforts conducted outside of the federal contracts.) Taken together, and reflecting upon NQF's accomplishments over the last year, the case studies provide a clear sense that there is forward momentum, as well as a growing commitment on the part of healthcare leaders to enhance healthcare value for patients, communities, and payers.

Eight Years of Hospital Reporting Show Results

In 2002, three hospital industry associations demonstrated leadership by joining with HHS, The Joint Commission, consumer organizations, and other stakeholders to create a more unified approach to reporting hospital performance information to the public. They launched the Hospital Quality Initiative—later re-named the Hospital Quality Alliance (HQA)—and defined its role as:

  • Identifying measures for reporting that are meaningful, relevant and understood by consumers;
  • Rallying hospitals to participate in the initiative and act on the performance results; and
  • Aligning stakeholders to reduce redundant and wasteful data collection and reporting.

From the beginning, HQA recommended NQF-endorsed measures because of the organization's transparent, rigorous multi-stakeholder consensus process and strong evidence-based approach to endorsement.

In 2003, performance results for over 400 hospitals were reported on the CMS Web site for the first time. A year later, CMS began penalizing hospitals financially if they did not report to CMS the same performance information they were required to send to The Joint Commission to maintain hospital accreditation. Between 2003 and 2004, the number of hospitals reporting their results to CMS tripled—from over 400 to more than 1,400 hospitals. In 2005, CMS launched Hospital Compare. Today, over 4,000 hospitals simultaneously report performance data to CMS and The Joint Commission, and the number of measures collected has steadily increased. In 2012, The Joint Commission will incorporate hospital performance into its accreditation determinations for the first time.

Performance results improved steadily over the last eight years. A recent analysis of hospitals shows marked improvement based on NQF-endorsed measures between 2002 and 2009.7 More specifically, in 2002, about 20 percent of hospitals exceeded 90 percent performance on 22 key measures; by 2009 that percentage had climbed significantly to 86 percent. Key NQF-endorsed measures include measures related to heart attack and heart failure care, surgical care, children's asthma care, and pneumonia care, among others.

This tight alignment between HQA, CMS and The Joint Commission regarding use and reporting of NQF-endorsed measures is a likely contributor to hospitals improving their performance over time. At the end of 2011, HQA decided to close its doors—noting that it had accomplished what it had set out to do: establishing a unified approach to collection and public reporting of hospital performance information. HQA also acknowledged that recommendations for measure selection going forward would be best left to the NQF-convened MAP, which is constituted to look across all federal programs to foster alignment and a clear strategic direction for measurement use.

Linking Quality Measurement to Payment Reform

Blue Cross Blue Shield Massachusetts' Alternative Quality Contract

In January 2009, Blue Cross Blue Shield of Massachusetts (BCBS) piloted the Alternative Quality Contract, a pay-for-performance model directly linking payment to meeting quality and cost benchmarks. The private-payer program provides financial bonuses to participating provider organizations such as multispecialty groups, independent practice associations, and physician-hospital organizations that stay within a specified annual budget and meet clinical quality targets. The budget takes into account the entire spectrum of care, ranging from inpatient and outpatient services to long-term care and prescription drug costs.

Performance was evaluated on the quality of care delivered in several clinical settings based on NQF-endorsed measures. More specifically:

Seven participating clinical groups were eligible for bonus payments as high as five percent based on 32 NQF-endorsed ambulatory and office-based quality measures. Measures included and focused on conditions and procedures such as diabetes testing and controlled LDL-C levels; breast, cervical, and colorectal cancer screenings; and patient experience with accessing and understanding care options.

Providers were eligible for another five percent bonus payment based on 32 NQF-endorsed hospital-based measures. These measures focused on surgical site and wound infections, in-hospital mortality rates, and patient satisfaction communicating with doctors and nurses.

Initial performance evaluations showed that across the board, provider groups delivered care within the scope of their budgets and performed well on clinical quality measures, allowing them to receive financial rewards of up to 10 percent of the total per-member per-month payments.8

The results illustrate that programs like the Alternative Quality Contract can offer providers strong incentives to control healthcare spending across the continuum while continuing to provide high-quality care. This idea is in line with recent policy proposals to design payment systems that reward high-quality, efficient, and integrated care.

National Priorities Focus North Carolina Hospitals

The North Carolina Center for Hospital Quality and Patient Safety (NCQC) was established by the North Carolina Hospital Association (NCHA) in 2004. The two organizations worked in partnership to conduct quality improvement collaborative projects across the state for about four years, but progress had grown stagnant. With North Carolina ranking as only the 35th healthiest state, NCQC's director embraced the NPP's 2008 National Priorities and Goals report recommendations as a way to focus, spur action, and benchmark North Carolina hospitals against national goals. Subsequent NPP reports have built on this first report.

The NCQC targeted much of its initial efforts on patient safety, made sure that frontline staff understood how their actions related to the hospital-wide improvement goals, and focused on both culture change and building up quality improvement skills. The Central Line-Associated Bloodstream Infection (CLABSI) Collaborative, which involved 40 ICUs, was particularly successful. Using a separate intervention program that sought to learn from mistakes and improve safety, the CLABSI Collaborative achieved a 46 percent reduction in central-line infections over the 18-month time period. These results translated into saving approximately 18 lives (using a 15 percent fatality rate) and saving $4.5 million (using $40,000 as the extra cost to a hospital for a CLABSI) across 40 hospitals.9

It is important to note that although many individual hospitals had success, not all hospitals in North Carolina participated, and the state rate of CLABSIs did not decrease as much as NCQC had hoped. To address this, NCQC launched a Phase 2 of the initiative to continue its focus on reducing central-line infections, using the NQF-endorsed CLABSIs measure as a way to guide progress and benchmark themselves nationally. The NCQC has stated that it is too early to tell if alignment with the NPP priorities will enable it to meet its own performance goals, but does acknowledge measureable and exciting progress against benchmarks it set.

Performance of Thoracic Surgeons Published in Consumer Reports

More than two decades ago, The Society of Thoracic Surgeons (STS) launched the Adult Cardiac Surgery Database to track and improve surgical quality. It is the largest cardiothoracic surgery outcomes and quality improvement program in the world, containing more than 4.5 million surgical records and representing approximately 94 percent of all adult cardiac surgery centers throughout the U.S.

Twenty plus years after the launch of its database, STS made the bold decision to offer participating surgical groups the option of voluntarily reporting their performance data in Consumer Reports. More specifically, Consumer Reports began publicly reporting heart surgery ratings at the surgical group level starting in 2010—including survival rates, complication rates, and other key NQF-endorsed measures. These ratings are now available on a bi-yearly basis.

A variety of factors influenced STS's decision to begin publicly reporting surgical performance, including the organization's vast experience with collecting and analyzing performance measures; a desire to leverage public reporting to further accelerate improvements in thoracic surgeon performance; and wanting to exhibit leadership in an environment of enhanced accountability.

Doris Peter, manager, Consumer Reports' Health Ratings Center, notes that reaction to the reports has been very positive from cardiac surgery groups and consumers alike. Peter noted that the first time STS's data was published in Consumer Reports, there were 20 million web impressions on the ratings. Consumer Reports' readership is 8 million. Due to this success, the subsequent September 2011 release made the cover of Consumer Reports print edition. To date, 36 percent of STS surgery groups are participating in the Consumer Reports ratings, a 65 percent increase from the first release.

Looking Forward

A dozen years in existence, NQF has been able to make particularly strong strides in the last three years with the support of federal funding stemming from MIPPA and ACA, building very much upon the strong collaborative relationship that has been established between NQF, its hundreds of private sector partners, and HHS. At a high level, results over these three years include:

  • The ability of NQF to now set and implement a multi-year plan for measure endorsement that is cognizant of addressing gaps and focused on implementing a vision for where advanced measurement is heading in a 21st century healthcare system. Over the three years, NQF endorsed 184 measures under the federal contracts, and completed maintenance of 136 previously endorsed measures. Currently, there are 233 measures under maintenance review, another 157 measures undergoing updates to specifications, and 43 measures having testing results reviewed. These efforts involved approximately 65 measure developers and hundreds of experts who volunteered their time on review committees. In addition, NQF has developed tools that allow measure developers to more readily create and implement eMeasures so that providers can collect more meaningful and actionable clinical data that is both comparable for public reporting and valid for payment purposes.
  • Broad recognition that NQF is an effective and trusted convener of public- and private-sector leaders—reflected in the organization's multi-stakeholder membership, established processes for achieving consensus, and its commitment to scientific evidence and transparency. This recognition has translated into requests that NQF-convened committees advise HHS on the first-ever NQS and related measurement strategy, as well as detailed measure-selection recommendations. NQF deliverables to HHS have been in the form of reports. Less perceptible perhaps is the growing consensus between scores of public- and private-sector leaders about how to collaborate to improve performance, which is translating into alignment around quality-improvement priorities and measure use.

Looking ahead, NQF and the broader quality movement are at an exciting juncture. A robust measurement infrastructure is moving into place, and increasingly there is a shared commitment about what to improve and what measures to use in the process of doing so. Over the next couple of years, NQF will be:

  • Putting the patient first by facilitating efforts that move the field toward a focus on patient-oriented as opposed to clinician-oriented measurement. Implementation of patient reported measures—including those that address experience of care, functional status, patient reported outcomes and care coordination—can help put the patient at the center of care.
  • Helping drive waste out of the system by focusing on bringing more cost/resource use measures through NQF endorsement and understanding in more detail how existing NQF endorsed quality/safety measures—including readmission, medication reconciliation and care coordination measures—can contribute to a more cost-efficient system.
  • Facilitating a future measurement vision by supporting efforts of the NPP and MAP Partnerships to develop a 3-5 year comprehensive measurement strategy—with broad and strong backing from multiple stakeholders—to recommend to HHS. The intent is that this strategy will cross settings and levels of care, as well as types of clinicians, and will in essence drive a strategic plan for payers that moves the needle with respect to the NQS's six priorities.
  • Bringing the public and private sectors closer together by further strengthening collaboration and deepening their commitment to the value agenda, further aligning their respective measurement strategies to reduce redundant data collection, and dramatically accelerate improvements in performance of the U.S. healthcare system.

In the coming years, the country should be in the position of realizing many benefits from these efforts to change healthcare by the numbers.


1 Federal use of NQF-endorsed measures is based on an initial analysis by NQF during the Fall of 2011.

2 The Commonwealth Fund, Why Not the Best: Results from the National Scorecard on U.S. Health System Performance, 2008, New York, NY:Commonwealth Fund, 2008. Available at—Results-from-the-National-Scorecard-on-U-S—Health-System-Performance—2008.aspx. Last accessed February 2012.

3 Bodenheimer T, High and rising health care costs. Part 1: seeking an explanation, Ann Intern Med, 2005;142(10):847-854.

4 Bodenheimer T, Fernandez A, High and rising health care costs. Part 4: can costs be controlled while preserving quality? Ann Intern Med, 2005;143(1): 26-31.

5 Institute of Medicine (IOM), Roundtable on Value & Science-Driven Health Care—The Healthcare Imperative: Lowering Costs and Improving Outcomes: Workshop Series Summary, Washington, DC: National Academies Press; 2010. Available at Last accessed January 2012.

6 The White House, U.S. Office of Management and Budget (OMB). Circular No. A-119, February 10, 1998, Washington, DC:OMB; 1998. Available at Last accessed January 2012.

7 Chassin MR, Loeb JM, Schmaltz SP et al., Accountability measures—using measurement to promote quality improvement, New Engl J Med, 2010;363(7):683-688. Available at Last accessed February 2012.

8 Song Z. Safran DG, Landon BE et al., Health care spending and quality in year 1 of the Alternative Quality Contract, New Engl J Med, 2011;365(10):909-918. Available at Last accessed February 2012.

9 National Quality Forum (NQF), Evaluation of the National Priorities Partnership, Washington, DC:NQF, 2011. Available at Last accessed February 2012.

Appendix A: 2011 Accomplishments: January 14, 2011 to January 13, 2012

DescriptionOutputStatus (as of 1/13/12)Notes/scheduled or actual completion date
I. Priorities, Principles, and Coordination Strategies
Provision of input on priorities for the NQSInput to the Secretary of Health and Human Services on Priorities for the National Quality Strategy; final written report of Partnership and Subcommittee meeting deliberations and recommendationsCompletedSeptember 1, 2011.
MAP report recommending measures for use in the improvement of physician performanceMeasure Applications Partnership Coordination Strategy for Clinician Performance Measurement; final report including MAP Coordinating Committee recommendationsCompletedOctober 1, 2011.
MAP report recommending measures that address the quality issues identified for dual-eligible beneficiariesMeasure Applications Partnership Strategic Approach to Performance Measurement for Dual-Eligible Beneficiaries; interim report including MAP Coordinating Committee recommendationsCompletedOctober 1, 2011.
MAP report recommending measures to be used by private and public payers to reduce readmissions and healthcare-acquired conditions (HACs)Measure Applications Partnership Coordination Strategy for Healthcare-Acquired Conditions and Readmissions Across Public and Private Payers; final report including recommendations regarding the optimal approach for coordinating readmission and HAC measuresCompletedOctober 1, 2011.
Measures for use in quality reporting programs under MedicareMeasure Applications Partnership Pre-Rulemaking Report: Input on Measures Under Consideration by HHS for 2012 RulemakingIn progressCompleted February 2012 after close of reporting year.
MAP report recommending measures that address the quality issues identified for dual-eligible beneficiariesFinal report including potential new performance measures to fill gaps in measurement for dual-eligible beneficiariesIn progressJune 1, 2012.
II. Measure Endorsement
Cardiovascular measures and maintenance reviewTwo-phase project to endorse new cardiovascular measures and conduct maintenance on existing NQF-endorsed measuresCompleted39 measures endorsed in January 2012.
Emergency regionalization medical care measurement frameworkEnvironmental scan and white paper comparing how regions coordinate and perform on delivering emergency servicesCompletedFramework endorsed in January 2012.
Patient safety: SREsReviewed existing list of NQF SREs for hospitals to identify ones appropriate for other settings; considered potential new SREs for all settingsCompletedUpdated list of 29 SREs endorsed in May 2011.
Patient outcomes measuresThree-phase project endorsing measures specific to outcomes on Medicare high-impact conditions, child health, and mental healthCompleted38 measures endorsed: —30 measures endorsed in January and March 2011. —8 measures endorsed during previous contract year (September 2010).
Patient-safety measuresTwo-phase project endorsed new measures of patient safety (e.g., healthcare-associated infections, medication safety) and maintaining currently endorsed measuresCompletedPhase 1: 4 measures endorsed in January 2012. Phase 2: 2 measures endorsed in August and September 2011.
Nursing-home measuresEndorsed measures of nursing-home care qualityCompleted5 measures endorsed in February 2011.
Child-health measuresEndorsed measures specific to the care of childrenCompleted44 measures endorsed in September 2011.
Surgery measures and maintenance reviewTwo-phase project to endorse new surgery measures and conduct maintenance on existing NQF-endorsed measuresPhase 1 complete; Phase 2 in progressPhase 1: 18 measures endorsed in December 2011. NQF Board endorsed Phase 2 measures after the close of the contract year.
Phase 2 addendum report issued for public comment just after contract year closed.
Efficiency and resource-use measuresEndorsed measures of imaging efficiency; white paper drafted; endorsed measures of healthcare efficiencyCompleted In progress; completed just after contract yearImaging Efficiency (Complete) —6 imaging efficiency measures endorsed in February 2011. —1 imaging efficiency measure was recommended to be combined with an existing NQF measure and was endorsed in April 2011.
Efficiency—Resource Use (In Progress).
Cycle 1: 4 measures ratified by Board January 2012.
Cycle 2: 4 measures posted for public comment in December 2011; voting closed in February 2012.
Cancer measures and maintenance reviewProject to endorse new cancer measures and conduct maintenance on existing NQF-endorsed measuresIn progressCall for nominations completed in November 2011; call-for-measures deadline was January 2012.
Perinatal measures and maintenance reviewProject to endorse new perinatal measures and conduct maintenance on existing NQF-endorsed measuresIn progressSteering Committee reviewed 23 measures in December 2011.
Renal measures and maintenance reviewProject to endorse new renal measures and conduct maintenance on existing NQF-endorsed measuresIn progressSteering Committee reviewed 33 measures by December 2011; member and public commenting to conclude after close of reporting year.
Pulmonary/critical-care measures and maintenance reviewProject to endorse new pulmonary/critical-care measures, and conduct maintenance on existing NQF-endorsed measuresIn progressCall for nominations closed in December 2011. Call-for-measures deadline was January 2012.
Palliative and end-of-life careProject to endorse new palliative and end-of-life care measures and conduct maintenance on existing NQF-endorsed measuresIn progressNQF Board endorsed measures after close of reporting year.
Care-coordination measures and maintenance reviewSet of endorsed care-coordination measuresIn progressCall for measures closed January 9, 2012.
Population Health Phase 1: Prevention measures and maintenance measures reviewSet of endorsed measures for preventative servicesIn progressMember and public commenting period concluded February 2012.
Population health Phase 2: Population health measuresCommissioned paper addressing population health measurement issues and set of endorsed population health measuresIn progressDraft paper completed January 2012 after close of reporting year.
Behavioral health measures and maintenance reviewSet of endorsed measures for behavioral healthIn progressCall for nominations closed December 13, 2011. Call for measures closed February 14, 2012.
All-cause readmissions (expedited Consensus Development Process [CDP] review)Set of endorsed all-cause readmission measuresIn progressMember and public commenting concluded January 2012.
Multiple Chronic Conditions Measurement Framework report analyzing measures being used to gauge quality of care for people with multiple chronic conditionsWork plan completed; interim report available for public commentIn progressMay 30, 2012.
Patient-reported outcomes (PROs) workshops addressing prerequisites for endorsed PRO measuresTwo workshops discussing commissioned papers addressing methodological prerequisites for NQF consideration of PRO measures for endorsement (The Veterans Administration may fund the papers; proposal is pending their approval)In progressJune 30, 2012.
Oral healthReport that catalogs oral health measures, measure concepts, priorities and gaps in measurementIn progressJuly 6, 2012.
Rapid-cycle CDP improvement (measure-endorsement process)Summary of process improvement approach, events, and metrics used to enhance the quality and efficiency of CDP processIn progressFour rapid-cycle improvement events completed in November and December 2012; additional events planned during first quarter of 2012.
III. Health Information Technology
Retooled eMeasures, eMeasures Format Review Panel, and eMeasure UpdatesPublished 113 measures for an electronic environment eMeasure Format Review Panel reviewed retooled measures to ensure the electronic specifications or requirements of these measures are consistent with the original focus and intent of the measure Held 10 webinars/conference calls to solicit comments and proposed resolutions.CompletedAll updates and related activities completed by December 22, 2011. Completed first cycle of review in Fall 2010, following public comment period.
MATNon-proprietary, web-based tool that allows performance-measure developers to specify, submit, and maintain electronic measures in a more streamlined, efficient, and highly structured wayCompleted Contractor training; release of the MAT Basic Version on 9/2911; enhanced version on target for releaseTotal number of unique organizations using MAT: 32.
QDM maintenanceUpdated the QDM (Version 3, released in April 2011) to reflect additional types of data needed to support emerging measures (e.g., measures that include social determinants of health, patient/consumer engagement)Review and updates to QDM are ongoing based on annual cycleEach new version of the QDM will be published annually; NQF will post a draft of modifications for the next version; annual QDM updates and versions will be integrated into MAT and, moreover, enable incorporation of required data elements in electronic measures as new types and sources of data are recognized over time.
eMeasures process and technical assistanceProvided education, training, and ad-hoc support to HHS, HHS contractors, MAT users, QDM users, eMeasure developers, EHR vendors, providers implementing measures, and other relevant quality and health IT stakeholdersOngoingDeveloped and posted MAT User Guide to provide manual for MAT and eMeasure development. Completed 5 technical-assistance trainings to CMS' eMeasure contractors, focusing on topics such as QDM and in-depth MAT training.
Completed 7 public webinars (with as many as 740 attendees per webinar), focusing on topics such as eMeasures training for measure developers and IT vendors.
Patient-safety-complications measures and maintenance review (Phase 1)Set of endorsed measures on complications-related areasIn progressSteering Committee reviewed 27 measures in December 2011.
Commissioned paper on data sources and readiness of HIT systems to support care coordinationFinal report and commissioned paperIn progressDraft paper available for public comment in February 2012.
Critical pathExamine new measurement areas (e.g. care plans) to understand the feasibility of measuring such areas in an electronic environmentOngoingEnd of September 2012.
eMeasure Learning CollaborativeExamining issues related to implementation of eMeasures with a multi-stakeholder group in order to define best practices and recommendations to the Office of the National Coordinator's Federal Advisory CommitteesOngoingEnd of September 2012.
IV. Measure Use and Application
Patient safety: state-based reporting agencies initiativeConvened 27 state-based patient-safety reporting agencies to discuss safety reporting efforts and share “best practices”CompletedMajority of work completed during previous contract year; final HHS-funded call completed January 24, 2011.
RAND report analyzing uses of NQF-endorsed measuresAn Evaluation of the Use of Performance Measures in Health Care; work plan and list of research questions completed; report by independent researcher completedCompleted
Recommendations for measures to be implemented through the federal rulemaking process for public reporting and paymentMeasure Applications Partnership Pre-Rulemaking Report: Input on Measures Under Consideration by HHS for 2012 RulemakingIn progressCompleted in February 2012 after close of reporting year.
MAP report recommending measures for use in quality reporting for Prospective Payment System-exempt cancer hospitalsFinal report including MAP Coordinating Committee recommendationsIn progressJune 1, 2012.
MAP report recommending measures for use in quality reporting for hospice careFinal report including MAP Coordinating Committee recommendationsIn progressJune 1, 2012.
NPP support for Partnership for Patients' HHS initiative focused on patient safetyFirst round of work included 2 quarterly convenings and 8 webinars Content of meetings and webinars were captured in individual summaries Next round of work includes creating affinity groups to implement specific patient-safety strategies and webinarsIn progress.

Appendix B: NQF Board and Leadership Staff

Board of Directors

William L. Roper, MD, MPH (Chair), Dean, School of Medicine, Vice Chancellor for Medical Affairs and Chief Executive Officer, UNC Health Care System, University of North Carolina at Chapel Hill

Andrew Webber (Vice Chair), President and CEO, National Business Coalition on Health

Gerald M. Shea (Treasurer), Assistant to the President for External Affairs, AFL-CIO

Lawrence M. Becker, Director, HR Strategic Partnerships, Xerox Corporation

Judy Ann Bigby, MD, Secretary, Executive Office of Health & Human Services, Commonwealth of Massachusetts

Janet M. Corrigan, Ph.D., MBA, President and CEO, National Quality Forum

Maureen Corry, Executive Director, Childbirth Connection

Leonardo Cuello, Staff Attorney, National Health Law Program

Helen Darling, MA, President, National Business Group on Health

Robert Galvin, MD, MBA, Chief Executive Officer, Equity Healthcare, The Blackstone Group

Ardis Dee Hoven, MD, Chair, American Medical Association Board of Trustees, Medical Director, Bluegrass Care Clinic, Affiliated with the University of Kentucky School of Medicine

Karen Ignagni, MBA, President and CEO, America's Health Insurance Plans

Chris Jennings, President, Jennings Policy Strategies, Inc.

Charles N. Kahn III, MPH, President, Federation of American Hospitals

Donald Kemper, Chairman and CEO, Healthwise, Inc.

Mark B. McClellan, MD, Ph.D., Senior Fellow and Director, Engelberg Center for Health Care Reform and Leonard D. Schaeffer Chair in Health Policy Studies, The Brookings Institution

Sheri S. McCoy, Worldwide Chairman of the Pharmaceuticals Group, Johnson & Johnson

Harold D. Miller, President and CEO, Network for Regional Healthcare Improvement

Dolores L. Mitchell, Executive Director, Commonwealth of Massachusetts Group Insurance Commission

Mary Naylor, Ph.D., RN, FAAN, Director, New Courtland Center for Transitions & Health and Marian S. Ware Professor in Gerontology, University of Pennsylvania School of Nursing

Debra L. Ness, President, National Partnership for Women & Families

Samuel R. Nussbaum, MD, Executive Vice President and Chief Medical Officer, WellPoint, Inc.

J. Marc Overhage, MD, Ph.D., Chief Medical Informatics Officer, Siemens Medical Solutions, Inc.

Bernard M. Rosof, MD, Chair, Board of Directors, Huntington Hospital, Chair, Physician Consortium for Performance Improvement

John C. Rother, JD, President and CEO, National Coalition on Health Care

Joseph R. Swedish, FACHE, President and CEO, Trinity Health

John Tooker, MD, MBA, MACP, Associate Executive Vice President, American College of Physicians

Richard J. Umbdenstock, President and CEO, American Hospital Association


Don Berwick, MD, Administrator (until 12/2/11)

Marilyn Tavenner, BSN, MPA, Acting Administrator and Chief Operating Officer (12/5/11-present), Centers for Medicare & Medicaid Services

Designee: Patrick Conway, MD, Chief Medical Officer


Carolyn M. Clancy, MD, Director, Agency for Healthcare Research and Quality

Designee: Nancy Wilson, MD, MPH, Senior Advisor to the Director


Mary Wakefield, Ph.D., RN, Administrator, Health Resources and Services Administration

Designee: Terry Adirim, MD, Director, Office of Special Health Affairs


Thomas R. Frieden, MD, MPH, Director, Centers for Disease Control and Prevention

Designee: Peter A. Briss, MD, MPH, Captain, U.S. Public Health Service Medical Director

Ex Officio (Non-Voting):

Timothy Ferris, MD, (Chair, Consensus Standards Approval Committee), Associate Professor of Medicine, Massachusetts General Hospital

Paul C. Tang, MD, MS, (Chair, Health Information Technology Advisory Committee), Vice President and Chief Medical Information Officer, Palo Alto Medical Foundation

NQF Leadership Staff

Janet M. Corrigan, President and Chief Executive Officer

Karen Adams, Vice President, National Priorities

Heidi Bossley, Vice President, Performance Measures

Helen Burstin, Senior Vice President, Performance Measures

Floyd Eisenberg, Senior Vice President, Health Information Technology

Larry Gorban, Vice President, Operations

Ann Greiner, Vice President, External Affairs

Ann Hammersmith, General Counsel

Lisa Hines, Vice President, Member Relations

Connie Hwang, Vice President, Measure Applications Partnership

Rosemary Kennedy, Vice President, Health Information Technology

Laura Miller, Senior Vice President and Chief Operating Officer

Nicole Silverman, Vice President, Federal Program Management

Lindsey Spindle, Senior Vice President, Communications and External Affairs

Diane Stollenwerk, Vice President, Community Alliances

Jeffrey Tomitz, Chief Financial Officer, Accounting & Finance

Thomas Valuck, Senior Vice President, Strategic Partnerships

Kyle Vickers, Chief Information Officer

Appendix C: Overview of Consensus Development Process

For each Consensus Development Project (CDP), NQF follows a careful eight-step process that ensures transparency, public input, and discussion among representatives across the healthcare enterprise.

1. Call for Nominations allows anyone to suggest a candidate for the committee that will oversee the project. Committees are diverse, often encompassing experts in a particular field, providers, scientists, and consumers. After selection, NQF posts committee rosters on its Web site to solicit public comments on the composition of the panel and makes adjustments as needed to ensure balanced representation.

2. Call for Measures starts a 30-day period for developers to submit a measure or practice through NQF's online submission forms.

3. Steering Committee Review puts submitted measures to a four-part test to ensure they reflect sound science, will be useful to providers and patients, and will make a difference in improving quality. The expert steering committee conducts this detailed review in open sessions, each of which starts a limited period for public comment.

4. Public Comment solicits input from anyone who wishes to respond to a draft report that outlines the steering committee's assessment of measures for possible endorsement. The steering committee may request a revision to the proposed measures.

5. Member Vote asks NQF members to review the draft report and cast their votes on the endorsement of measures.

6. CSAC Review marks the point at which the NQF Consensus Standards Approval Committee (CSAC) deliberates on the merits of the measure and the issues raised during the review process, and makes a recommendation on endorsement to the Board of Directors. The CSAC includes consumers, purchasers, healthcare professionals, and others. It provides the big picture to ensure that standards are being consistently assessed from project to project.

7. Board Ratification asks for review and ratification by the NQF Board of Directors of measures recommended for endorsement.

8. Appeal opens a period when anyone can appeal the Board's decision.

Appendix D: MAP Measure-Selection Criteria

The Measure Applications Partnership (MAP) has developed measure-selection criteria to guide its evaluations of program measure sets. The term “measure set” can refer to a collection of measures—for a program, condition, procedure, topic, or population. For the purposes of MAP's pre-rulemaking analysis, we qualify the term measure set as a “program measure set” to indicate the collection of measures used in a given federal public reporting or performance-based payment program.

The measure-selection criteria are intended to facilitate structured discussion and decision- making processes. The iterative approach employed in developing the criteria allowed MAP in its entirety, as well as the public, to provide input on the criteria. Each MAP workgroup deliberated on draft criteria and advised the Coordinating Committee. Comments were received on the draft criteria through the public comment period for the Coordination Strategy for Clinician Performance Measurement report . A Measure-Selection Criteria Interpretive Guide also was developed to provide additional descriptions and direction on the meaning and use of the measure-selection criteria.

1. MAP measure-selection criteria and the interpretive guide were finalized at the November 1, 2011, Coordinating Committee in-person meeting The following criteria were then used as a tool during the pre-rulemaking task:

2. Measures within the program measure set are NQF-endorsed or meet the requirements for expedited review.

3. The program measure set adequately addresses each of the NQS priorities.

4. The program measure set adequately addresses high-impact conditions relevant to the program's intended populations (e.g., children, adult non-Medicare, older adults, or dual-eligible beneficiaries).

5. The program measure set promotes alignment with specific program attributes, as well as alignment across programs.

6. The program measure set includes an appropriate mix of measure types (e.g., process, outcome, structure, patient experience, and cost).

7. The program measure set enables measurement across the person-centered episode of care.

8. The program measure set includes considerations for healthcare disparities.

9. The program measure set promotes parsimony.

Public commenters supported the MAP measure-selection criteria and noted that the tool served MAP well in its pre-rulemaking activities.

Appendix E: NQF Membership

NQF members represent more than 450 organizations from across the country committed to advancing healthcare quality. Members of NQF participate in one of eight Member Councils organized by stakeholder group—consumers; health plans; health professionals; provider organizations; public-community health agencies; purchasers; quality measurement, research, and improvement; and supplier-industry—and are afforded a strong voice in crafting national solutions to quality concerns. Member organizations are from every region of the country as the map below indicates.

NQF Member Organizations

3M Health Care


Abbott Laboratories

ABIM Foundation

Academy of Managed Care Pharmacy

Academy of Medical-Surgical Nurses

Accreditation Association for Ambulatory Health Care Institute for Quality Improvement


Ada County Paramedics

Adventist Health System

Advocate Physician Partners


Affinity Health System


Agency for Healthcare Research and Quality

Albuquerque Coalition for Healthcare Quality

Aligning Forces for Quality-South Central Pennsylvania

Alliance for Health

Alliance of Community Health Plans

Ambulatory Surgery Foundation


American Academy of Allergy, Asthma and Immunology

American Academy of Dermatology

American Academy of Family Physicians

American Academy of Hospice and Palliative Medicine

American Academy of Neurology

American Academy of Nurse Practitioners

American Academy of Nursing

American Academy of Ophthalmology

American Academy of Orthopaedic Surgeons

American Academy of Otolaryngology-Head and Neck Surgery

American Academy of Pediatrics

American Academy of Physical Medicine and Rehabilitation

American Association of Birth Centers

American Association of Cardiovascular and Pulmonary Rehabilitation

American Association of Clinical Endocrinologists

American Association of Colleges of Nursing

American Association of Diabetes Educators

American Association of Neurological Surgeons

American Association of Nurse Anesthetists

American Association of Nurse Assessment Coordination

American Board of Medical Specialties

American Board of Optometry

American Case Management Association

American Chiropractic Association

American College of Cardiology

American College of Cardiology/American Heart Association Task Force on Performance Measures

American College of Emergency Physicians

American College of Gastroenterology

American College of Medical Quality

American College of Nurse-Midwives

American College of Obstetricians and Gynecologists

American College of Physician Executives

American College of Physicians

American College of Radiology

American College of Rheumatology

American College of Surgeons

American Data Network

American Dietetic Association

American Federation of Teachers Healthcare

American Gastroenterological Association Institute

American Geriatrics Society

American Health Care Association

American Health Information Management Association

American Health Quality Association

American Heart Association

American Hospice Foundation

American Hospital Association

American Medical Association

American Medical Association-Physician Consortium for Performance Improvement

American Medical Directors Association

American Medical Informatics Association

American Nurses Association

American Occupational Therapy Association

American Optometric Association

American Organization of Nurse Executives

American Osteopathic Association

American Pharmacists Association Foundation

American Physical Therapy Association

American Psychiatric Association for Research and Education

American Psychiatric Nurses Association

American Sleep Apnea Association

American Society for Gastrointestinal Endoscopy

American Society for Radiation Oncology

American Society of Anesthesiologists

American Society of Breast Surgeons

American Society of Clinical Oncology

American Society of Colon and Rectal Surgeons

American Society of Health-System Pharmacists

American Society of Hematology

American Society of Nuclear Cardiology

American Society of Pediatric Nephrology

American Society of Plastic Surgeons

American Urological Association

America's Health Insurance Plans

AmeriHealth Mercy Family of Companies


AmSurg Corp.

Anesthesia Quality Institute

Arkansas Medicaid

Ascension Health

Association for Professionals in Infection Control and Epidemiology

Association for the Advancement of Wound Care

Association of American Medical Colleges

Association of periOperative Registered Nurses

Association of Rehabilitation Nurses

Association of Women's Health, Obstetric and Neonatal Nurses


Atlantic Health

Aultman Health Foundation

Aurora Health Care

Avalere Health LLC

Baptist Health South Florida

Baptist Memorial Health Care Corporation

Baxter Healthcare

BayCare Health System

Baylor Health Care System

Betsy Lehman Center for Patient Safety and Medical Error Reduction

Better Health Greater Cleveland

BJC HealthCare

BlueCross BlueShield Association

Boehringer Ingelheim

Bon Secours St. Francis Health System

Booz Allen Hamilton

Bristol-Myers Squibb Company

Bronson Healthcare Group, Inc.

Buyers Health Care Action Group

California HealthCare Foundation

California Hospital Association

California Hospital Patient Safety Organization

California Maternal Quality Care Collaborative

California Office of Statewide Health Planning and Development

CareFirst BlueCross BlueShield


CaroMont Health

Case Management Society of America

Caterpillar Inc.

Catholic Health Association of the United States

Catholic Health Initiatives

Catholic Healthcare Partners

Cedars-Sinai Medical Center

Center for Health Care Quality, Department of Health Policy, George Washington University

Center to Advance Palliative Care

Centers for Disease Control and Prevention

Centers for Medicare & Medicaid Services

Childbirth Connection

Children's Hospital Boston

Children's Hospitals and Clinics of Minnesota


CIGNA HealthCare

Citizens for Patient Safety

City of Hope

Cleveland Clinic

Colorado Business Group on Health

Commission for Case Manager Certification

Community Health Accreditation Program

Community Health Alliance- Humboldt County Del-Norte

Community Health Foundation of Western and Central New York

Connecticut Center for Patient Safety

Connecticut Hospital Association

Consumer Coalition for Quality Health Care

Consumers Advancing Patient Safety

Consumers' Checkbook

Consumers Union

Coral Initiative, LLC

Core Consulting, Inc.

Council of Medical Specialty Societies

Crozer-Keystone Health System

Dallas-Fort Worth Hospital Council Education and Research Foundation

Dana-Farber Cancer Institute

Deloitte Consulting LLP, Health Sciences and Government

Dental Quality Alliance

Detroit Medical Center

Dialog Medical

Edwards Lifesciences

eHealth Initiative

Eisai, Inc.

Eli Lilly and Company

Elsevier Clinical Decision Support

Emergency Nurses Association

Employers' Coalition on Health

Englewood Hospital and Medical Center

Epstein Becker & Green, P.C.

Exeter Health Resources

Federation of American Hospitals

FirstWatch Solutions, Inc.

Florida Health Care Coalition

Florida Hospital

Florida State University, Center for Medicine and Public Health

Forest Laboratories, Inc.

Foundation for Informed Medical Decision Making

Fox Chase Cancer Center

Franciscan Alliance

GE Healthcare


Genesis HealthCare System

Gentiva Health Services


Good Samaritan Hospital

Greater Detroit Area Health Council

Greenway Medical Technologies

Group Health Cooperative

H. Lee Moffitt Cancer Center and Research Institute Hospital, Inc.

Hackensack University Medical Center

Harborview Medical Center

Health Action Council Ohio

Health Level Seven, Inc.

Health Management Associates, Inc.

Health Resources and Services Administration

Health Services Advisory Group

Health Services Coalition

Health Watch USA

HealthCare 21 Business Coalition

Healthcare Information and Management Systems Society

Healthcare Leadership Council



HealthSouth Corporation

Healthy Memphis Common Table

Heart Rhythm Society

Henry Ford Health System

Highmark, Inc.

Hoag Hospital

Horizon Blue Cross Blue Shield of New Jersey

Hospice and Palliative Nurses Association


Hospital Corporation of America

Hospital for Special Surgery

Hudson Health Plan

Humana Inc.

Huntington Memorial Hospital

Illinois Hospital Association

Infectious Diseases Society of America

Infusion Nurses Society

Inland Northwest Health Services

Institute for Clinical Systems Improvement

Institute for Safe Medication Practices

Integrated Healthcare Association

Intelligent Healthcare

Interim HealthCare, Inc.

Intermountain Healthcare

Iowa Healthcare Collaborative


Jefferson School of Population Health

Johns Hopkins Health System

Kaiser Permanente

Kansas City Quality Improvement Consortium

Kidney Care Partners

Lamaze International

Lehigh Valley Business Coalition on Health Care

LHC Group, Inc.

Long-Term Quality Alliance

Louisiana Health Care Quality Forum

Maine Health Management Coalition

Maine Quality Counts

Maine Quality Forum

Maryland Health Care Commission

Maryland Patient Safety Center

Massachusetts Health Quality Partners

Mayo Clinic

McKesson Corporation


MedeAnalytics, Inc.

Medisolv, Inc.

MedStar Health

Memorial Hermann Healthcare System

Memorial Sloan-Kettering Cancer Center

Merck & Co., Inc.

Mercy Medical Center

Meridian Health System

MHA Keystone Center for Patient Safety & Quality

Middlesex Hospital

Midwest Care Alliance

Milliman Care Guidelines

Minnesota Community Measurement

Mothers Against Medical Error

Mount Auburn Hospital

National Academy for State Health Policy

National Academy of Clinical Biochemistry

National Alliance of Wound Care

National Association for Behavioral Health

National Association for Healthcare Quality

National Association of Certified Professional Midwives

National Association of Children's Hospitals and Related Institutions

National Association of Dental Plans

National Association of EMS Physicians

National Association of Health Data Organizations

National Association of Pediatric Nurse Practitioners

National Association of Psychiatric Health Systems

National Association of Public Hospitals and Health Systems

National Association of State Medicaid Directors

National Breast Cancer Coalition

National Business Coalition on Health

National Business Group on Health

National Center for Healthcare Leadership

National Coalition for Cancer Survivorship

National Committee for Quality Assurance

National Consensus Project for Quality Palliative Care

National Consortium of Breast Centers

National Consumers League

National Council of State Boards of Nursing

National Council on Aging

National Forum for Heart Disease and Stroke Prevention

National Health Law Program

National Hospice and Palliative Care Organization

National Institute for Quality Improvement and Education

National Nursing Staff Development Organization

National Partnership for Women & Families

National Patient Safety Foundation

National Pressure Ulcer Advisory Panel

National Rural Health Association

National Sleep Foundation

NCH Healthcare System

Nemours Foundation

Neocure Group

New Jersey Health Care Quality Institute

New Jersey Hospital Association

New York Presbyterian Healthcare System

New York University College of Nursing

Next Wave

Niagara Health Quality Coalition

North Carolina Center for Hospital Quality and Patient Safety

North Mississippi Medical Center

North Shore-Long Island Jewish Health System

North Texas Specialty Physicians

Northeast Health Care Quality Foundation

Northwestern Memorial HealthCare

Norton Healthcare, Inc.


Nursing Alliance for Quality Care

Oakstone Medical Publishing

Oncology Nursing Society

Oregon Health Care Quality Corporation

Ortho-McNeill-Janssen Pharmaceutical, Inc.

OSUCCC-James Cancer Hospital

P2 Collaborative of Western New York

Pacific Business Group on Health

Park Nicollet Health Services

Partners HealthCare System, Inc.

Partnership for Prevention

Patient Centered Primary Care Collaborative

Pennsylvania Health Care Association


Pharmacy Quality Alliance


Phytel, Inc.


Premier, Inc.

Press Ganey Associates

Professional Research Consultants, Inc.

Providence Health & Services

Puget Sound Health Alliance

PULSE of New York

Quality Outcomes, LLC

Quantros, Inc.

Renal Physicians Association

Resolution Health, Inc.

Rhode Island Department of Health

Robert Wood Johnson University Hospital-Hamilton

Rockford Health System

Roswell Park Cancer Institute

Saint Barnabas Health Care System

Saint Francis Hospital and Medical Center

Sanofi Pasteur


Scott & White Healthcare

Seattle Cancer Care Alliance

Sharp HealthCare

Siemens Healthcare, USA

Sisters of Charity of Leavenworth Health System

SNP Alliance

Society for Academic Emergency Medicine

Society for Cardiovascular Angiography and Interventions

Society for Healthcare Epidemiology of America

Society for Maternal-Fetal Medicine

Society for the Advancement of Blood Management

Society for Vascular Surgery

Society of Behavioral Medicine

Society of Critical Care Medicine

Society of Gynecologic Oncology

Society of Hospital Medicine

Society of Thoracic Surgeons

Southeast Texas Medical Associates, LLP

St. Joseph Health System

St. Louis Area Business Health Coalition

Stamford Health System

State Associations of Addiction Services

Substance Abuse and Mental Health Services Administration

Summa Health System

Surgical Care Affiliates

Sylvester Comprehensive Cancer Center, University of Miami Hospitals and Clinics

Taconic IPA, Inc.

Takeda Pharmaceuticals North America, Inc.

Tampa General Hospital


Tenet Healthcare Corporation

Texas Health Resources

Texas Medical Institute of Technology

The Advanced Medical Technology Association

The Alliance

The Alliance for Home Health Quality and Innovation

The Commonwealth Fund

The Coordinating Center

The Empowered Patient Coalition

The Federation of State Medical Boards of the U.S., Inc.

The Health Alliance of Mid-America, LLC

The Health Collaborative

The Joint Commission

The Leapfrog Group

The National Consumer Voice for Quality Long-Term Care

The National Forum of ESRD Networks

The Partnership for Healthcare Excellence

Thomas Jefferson University Hospital

Thomson Reuters

Trauma Support Network

Trinity Health

Trust for America's Health

UCB, Inc.

UMass Memorial Medical Group, Inc.

United Surgical Partners International

UnitedHealth Group

Universal American Corp.

University HealthSystem Consortium

University of California-Davis Medical Group

University of Kansas School of Nursing

University of Michigan Hospitals & Health Centers

University of North Carolina-Program on Health Outcomes

University of Pennsylvania Health System

University of Texas Southwestern Medical Center

University of Texas-MD Anderson Cancer Center

University of Virginia Health System


Urgent Care Association of America

US Department of Defense-Health Affairs

UW Health

Vanderbilt University Medical Center

Vanguard Health Management

Verilogue, Inc

Veterans Health Administration

VHA, Inc.

Virginia Business Coalition on Health

Virginia Cardiac Surgery Quality Initiative

Virginia Mason Medical Center

Virtua Health


WellSpan Health

WellStar Health System

West Virginia Medical Institute

Wisconsin Collaborative for Healthcare Quality

Wisconsin Medical Society

Wound, Ostomy and Continence Nurses Society

Yale New Haven Health System

Zynx Health

Appendix F: 2011 NQF Volunteer Leaders

Stancel M. Riley, Chair, Ambulatory and Office-Based Surgery Technical Advisory Panel Serious Reportable Events in Healthcare Project

Chair, Patient Safety Serious Reportable Events Technical Advisory Panel, Massachusetts Board of Registration in Medicine

Mary George, Co-chair, Cardiovascular Endorsement Maintenance Steering Committee, Centers for Disease Control and Prevention

Raymond Gibbons, Co-chair, Cardiovascular Endorsement Maintenance Steering Committee, Mayo Clinic

Donald Casey, Co-chair, Care Coordination Endorsement Maintenance Steering Committee, Atlantic Health

Gerri Lamb, Co-chair, Care Coordination Endorsement Maintenance Steering Committee, Arizona State University

Thomas McInerny, Co-chair, Child Health Quality Measures Steering Committee, University of Rochester

Marina L. Weiss, Co-chair, Child Health Quality Measures Steering Committee

Co-chair, National Voluntary Standards for Patient Outcomes Child Health Steering Committee, March of Dimes

David Classen, Co-chair, Common Formats Expert Panel, University of Utah

Henry Johnson, Co-chair, Common Formats Expert Panel, ACS-MIDAS+

Timothy Ferris, Chair, Consensus Standards Approval Committee, Massachusetts General Hospital/Institute for Health Policy

Ann Monroe, Vice-chair, Consensus Standards Approval Committee, Community Health Foundation of Western and Central New York

Doris Lotz, Co-chair, Efficiency Resource Use Steering Committee, New Hampshire Department of Health and Human Services

Sally Tyler, Co-chair, Patient Safety SRE Steering Committee, AFSCME

Gregg S. Meyer, Co-chair, Patient Safety SRE Steering Committee, Massachusetts General Hospital

Paul C. Tang, Chair, Health Information Technology Advisory Committee, Palo Alto Medical Foundation and Stanford University

Dennis Andrulis, Co-chair, Healthcare Disparities and Cultural Competency Consensus Standards Committee, Texas Health Institute

Denice Cora-Bramble, Co-chair, Healthcare Disparities and Cultural Competency Consensus Standards Committee, Children's National Medical Center

Michael Doering, Co-chair, Improving Patient Safety through State-Based Reporting in Healthcare Workgroup, Pennsylvania Patient Safety Authority

Diane Rydrych, Co-chair, Improving Patient Safety through State-Based Reporting in Healthcare Workgroup, Minnesota Department of Health

Iona Thraen, Co-chair, Improving Patient Safety through State-Based Reporting in Healthcare Workgroup, Utah Department of Health

William Corley, Chair, Leadership Network, Community Health Network

George J. Isham, Co-chair, Measure Applications Partnership Coordinating Committee, HealthPartners, Inc.

Elizabeth A. McGlynn, Co-chair, Measure Applications Partnership Coordinating Committee, Kaiser Permanente Center for Effectiveness and Safety Research

Frank G. Opelka, Chair, Measure Applications Partnership Ad Hoc Safety Workgroup

Chair, Measure Application Partnership Hospital Workgroup, Louisiana State University Health Sciences Center

Mark McClellan, Chair, Measure Applications Partnership Clinician Workgroup, The Brookings Institution, Engelberg Center for Health Care Reform

Alice Lind, Chair, Measure Applications Partnership Dual Eligible Beneficiaries Workgroup, Center for Health Care Strategies

Carol Raphael, Chair, Measure Applications Partnership Post-Acute Care/Long-Term Care Workgroup, Visiting Nurse Service of New York

Michael Lieberman, Chair, Measure Authoring Tool Oversight and Testing Workgroup, Oregon Health and Science University

Caroline S. Blaum, Co-chair, Multiple Chronic Conditions Measurement Framework Steering Committee, University of Michigan Health System—Institute of Gerontology

Barbara McCann, Co-chair, Multiple Chronic Conditions Measurement Framework Steering Committee, Interim HealthCare

Helen Darling, Co-chair, National Priorities Partnership, National Business Group on Health

Margaret O'Kane, Co-chair, National Priorities Partnership, National Committee for Quality Assurance

Bernard Rosof, Co-chair, National Priorities Partnership, Physician Consortium for Performance Improvement convened by the American Medical Association

Peter Crooks, Co-chair, National Voluntary Consensus Standards for End Stage Renal Disease

Co-chair, Renal Endorsement Maintenance Steering Committee, Southern California Permanente Medical Group

Kristine Schonder, Co-chair, National Voluntary Consensus Standards for End Stage Renal Disease

Co-chair, Renal Endorsement Maintenance Steering Committee, University of Pittsburgh School of Pharmacy

Tom Rosenthal, Co-chair, National Voluntary Consensus Standards for Endorsing Performance Measures for Resource Use: Phase II, UCLA School of Medicine

Bruce Steinwald, Co-chair, National Voluntary Consensus Standards for Endorsing Performance Measures for Resource Use: Phase II

Co-chair, Efficiency Resource Use Steering Committee, Independent Consultant

G. Scott Gazelle, Co-chair, National Voluntary Consensus Standards for Imaging Efficiency, Massachusetts General Hosital

Eric D. Peterson, Co-chair, National Voluntary Consensus Standards for Imaging Efficiency, Duke University Medical Center

David A. Johnson, Chair, National Voluntary Consensus Standards for Patient Outcomes Biliary and Gastrointestinal Technical Advisory Panel, American College of Gastroenterology

Dianne Jewell, Chair, National Voluntary Consensus Standards for Patient Outcomes Bone/Joint Technical Advisory Panel, Virginia Commonwealth University

Lee Newcomer, Chair, National Voluntary Consensus Standards for Patient Outcomes Cancer Technical Advisory Committee, United HealthCare

Edward Gibbons, Chair, National Voluntary Consensus Standards for Patient Outcomes Cardiovascular Technical Advisory Panel, University of Washington School of Medicine

David Herman, Chair, National Voluntary Consensus Standards for Patient Outcomes Eye Care Technical Advisory Panel, Mayo Clinic

E. Patchen Dellinger, Chair, National Voluntary Consensus Standards for Patient Outcomes Infectious Disease Technical Advisory Panel, University of Washington School of Medicine

Sheldon Greenfield, Chair, National Voluntary Consensus Standards for Patient Outcomes Metabolic Technical Advisory Panel, University of California, Irvine

Barbara Yawn, Chair, National Voluntary Consensus Standards for Patient Outcomes Pulmonary Technical Advisory Panel, Olmstead Medical Center

Tricia Leddy, Co-chair, National Voluntary Consensus Standards for Patient Outcomes Mental Health Steering Committee, Rhode Island Department of Health

Jeffrey Sussman, Co-chair, National Voluntary Consensus Standards for Patient Outcomes Mental Health Steering Committee, University of Cincinnati

Charles Homer, Co-chair, National Voluntary Standards for Patient Outcomes Child Health Steering Committee, NICHQ

David Gifford, Co-chair, National Voluntary Standards for Nursing Homes, American Health Care Association and National Center for Assisted Living

Christine Mueller, Co-chair, National Voluntary Standards for Nursing Homes, University of Minnesota School of Nursing

June Lunney, Co-chair, Palliative Care and End-of-Life Care Endorsement Maintenance Steering Committee, Hospice and Palliative Nurses Association

Sean Morrison, Co-chair, Palliative Care and End-of-Life Care Endorsement Maintenance Steering Committee, Mount Sinai School of Medicine

Sherrie Kaplan, Co-chair, Patient Outcomes: All-Cause Readmissions Expedited Review Steering Committee, UC Irvine School of Medicine

Eliot Lazar, Co-chair, Patient Outcomes: All-Cause Readmissions Expedited Review Steering Committee, New York Presbyterian Healthcare System

Lisa J. Thiemann, Co-chair, Patient Safety Measures Steering Committee, Surgical Care Affiliates

William A. Conway, Co-chair, Patient Safety Measures Steering Committee

Co-chair, Patient Safety Measures: Complications Endorsement Maintenance Steering Committee, Henry Ford Health System

Darrell A. Campbell, Jr., Chair, Patient Safety Measures HAI Technical Advisory Panel, University of Michigan Hospitals & Health Centers

David Nau, Chair, Patient Safety Measures Medical Management Technical Advisory Panel, Pharmacy Quality Alliance

Steven Clark, Chair, Patient Safety Measures Perinatal Technical Advisory Panel, Hospital Corporation of America

Pamela Cipriano, Co-chair, Patient Safety Measures: Complications Endorsement Maintenance Steering Committee, University of Virginia Health System

Tejal Gandhi, Chair, Patient Safety Serious Reportable Events Technical Advisory Panel

Chair, Physician Office Technical Advisory Panel Serious Reportable Events in Heatlhcare, Partners Healthcare

Eric Tangalos, Chair, Patient Safety Serious Reportable Events Technical Advisory Panel

Chair, Skilled Nursing Facility Technical Advisory Panel Serious Reportable Events In Healthcare Project, Mayo Clinic

Laura Riley, Co-chair, Perinatal and Reproductive Health Endorsement Maintenance Steering Committee, Massachusetts General Hospital

Carol Sakala, Co-chair, Perinatal and Reproductive Health Endorsement Maintenance Steering Committee, Childbirth Connection

Paul Jarris, Co-chair, Population Health: Prevention Endorsement Maintenance Steering Committee, Association of State and Territorial Health Officers

Kurt Stange, Co-chair, Population Health: Prevention Endorsement Maintenance Steering Committee, Case Western Reserve University

David Bates, Co-chair, Quality Data Model Sub-committee, Partners Healthcare

Caterina Lasome, Co-chair, Quality Data Model Sub-committee, Ion Informatics

Arthur Kellermann, Co-chair, Regionalized Emergency Medical Care Services Steering Committee, The RAND Corporation

Andrew Roszak, Co-chair, Regionalized Emergency Medical Care Services Steering Committee, Department of Health and Human Services

James Weinstein, Chair, Resource Use Project: Phase II Bone/Joint Technical Advisory Panel, The Dartmouth Institute for Health Policy; Dartmouth-Hitchcock Clinic

David Penson, Chair, Resource Use Project: Phase II Cancer Technical Advisory Panel, Vanderbilt University Medical Center

Jeptha Curtis, Co-chair, Resource Use Project: Phase II Cardiovascular/Diabetes Technical Advisory Panel, Yale University School of Medicine

James Rosenzweig, Co-chair, Resource Use Project: Phase II Cardiovascular/Diabetes Technical Advisory Panel, Boston Medical Center and Boston University School of Medicine

Kurtis Elward, Co-chair, Resource Use Project: Phase II Pulmonary Technical Advisory Panel, Family Medicine of Albermarle

Janet Maurer, Co-chair, Resource Use Project: Phase II Pulmonary Technical Advisory Panel, American College of Chest Physicians

Arden Morris, Co-chair, Surgery Endorsement Maintenance Steering Committee, Ann Arbor Veterans Affairs Medical Center

David Torchiana, Co-chair, Surgery Endorsement Maintenance Steering Committee, Massachusetts General Physicians Organization


1030 15th Street NW., Suite 800

Washington, DC 20005

NQF Report on Measure Gaps and Inadequacies


The Affordable Care Act (ACA) (Pub. L. 111-148, sec. 3011), requires the Secretary of Health and Human Services to establish a National Strategy for Quality Improvement in Health Care, which serves as a strategic plan for improving the delivery of health care services, achieving better patient outcomes, and improving the health of the U.S. population. The strategy will be continually updated as the Affordable Care Act is implemented.

Section 3014 of ACA requires a report from the National Quality Forum (NQF) regarding the identification of gaps in endorsed quality measures—to include measures within the National Quality Strategy priority areas—to be provided to the Secretary by February 1, 2012 and annually thereafter. The report was also intended to identify areas where evidence was insufficient to support endorsement of quality measures in priority areas.


In order to prepare this report on measure gaps, NQF staff consulted numerous data sources to identify endorsed measure and evidence gaps. Staff reviewed approximately 750 endorsed measures within the NQF portfolio and identified the measures that address one or more of the National Quality Strategy (NQS) priority areas and areas where gaps remain. Staff also reviewed NQF-related efforts that address many of the priority areas, including NQF project consensus development project reports. NQF endorsement committees routinely identify gaps as part of the work of the consensus development process. The NQF report “Prioritization of High-Impact Medicare Conditions and Measure Gaps” developed by the Measure Prioritization Advisory Committee and published in May, 2010 was also used as a data source for gaps.

NQF has captured this information in a high-level matrix organized by priority area and the high impact clinical conditions which highlights where endorsed measures exist and gaps remain. Given the volume of clinical conditions and cross-cutting areas addressed within the NQF portfolio, a targeted list of clinical conditions is included.

It is anticipated that this analysis will continue to evolve over the coming years through the NQF National Priorities Partnership, the Measures Applications Partnership, endorsement maintenance projects, and other activities.

National Quality Strategy Overview

The NQF-convened National Priorities Partnership (NPP) proposed goals and measure concepts in its September 1, 2011 report “Input to the Secretary of Health and Human Services on Priorities for the National Quality Strategy” regarding the six national priorities:

1. Making Care Safer

2. Ensuring Person- and Family-Centered Care

3. Promoting Effective Communication and Coordination of Care

4. Promoting the Most Effective Prevention and Treatment of the Leading Causes of Mortality, Starting with Cardiovascular Disease

5. Working with Communities to Promote Wide Use of Best Practices to Enable Healthy Living

6. Making Quality Care More Affordable

The proposed goals and measure concepts are intended to “provide a set of clear aims with which the NQS can guide the nation to achieve safe, timely, effective, efficient, and equitable care,” and are discussed in more detail below. Some of the measure concepts identify important measurement gaps, while measure development may be limited by evidence gaps.

The Secretary's National Quality Strategy requires a wide array of quality and efficiency measures for implementation. While some of the strategy's priority areas may be well-supported by NQF-endorsed measures, others may have fewer, or in some cases, no endorsed measures aligned with them.

For the purposes of this report, we have expanded the applicability of the fourth priority area, related to prevention and treatment, beyond cardiovascular disease to the other conditions listed below. While there are numerous condition-specific clinical process measures, there are major gaps for some conditions (e.g., Alzheimer's). There are also important gaps in condition-specific measures that address critical national priorities (e.g., cost measures for high-cost conditions).

  • Alzheimer's Disease
  • Cancer
  • Cardiovascular
  • Cataract
  • Child Health
  • Depression
  • Diabetes
  • Glaucoma
  • Hip/Pelvic Fracture
  • Maternal Health
  • Osteoporosis
  • Pulmonary
  • Renal Disease
  • Rheumatoid Arthritis/Osteoarthritis
  • Serious Mental Illness
  • Stroke

Since there is a strong desire to move toward patient-focused outcomes of care, the report also identifies potential outcome gaps for clinical and cross-cutting areas. For example, while there are numerous cancer-related process measures, there are no endorsed cancer outcome measures. Recent work by NQF's Evidence Task Force identified a hierarchical preference for outcomes linked to evidence-based processes and structures (Figure 1). While there is still a need for process and structural measures, especially for quality improvement, they should be closely linked to outcomes. In the tables that follow, gaps for outcome measures in some high impact clinical areas are identified.

The NQF Evidence Task Force also emphasized the importance of assessing the quality, quantity and consistency of evidence underlying the measure focus. While endorsement of some clinical measures has been limited by empirical evidence, NQF provides an exception in cases for which expert opinion can be systematically assessed with agreement that the benefits to patients greatly outweigh potential harms. In some cross-cutting priority areas, such as pain management and patient engagement, Committee expert opinion has been used to satisfy the evidence requirement.

There has also been a strong interest from numerous stakeholders, including consumers and purchasers, in moving to composite measures. Composite measures are defined as one or more measures that are combined into a single score. Because composite measures provide a more comprehensive view of care and may be more understandable to end users, there has been a shift toward composite measures in many clinical areas. For example, an endorsed cardiovascular care composite encompasses the key secondary prevention elements critical for prevention of cardiac events (e.g., use of aspirin, non-smoking status, lipid control, and blood pressure control). Given the interest in these measures, gaps for composite measures are also noted in the tables that follow.

Gaps Across Cross-Cutting Areas

While many measures within the NQF portfolio relate to specific conditions or clinical areas, others address or are applicable to cross-cutting areas such as safety and care coordination. Currently NQF-endorsed measures are categorized by these cross-cutting areas when applicable, overlapping with many of the cross-cutting national priorities outlined within the NQS.

Figure 2 provides a graphic representation of the more than 750 measures across these areas. This figure provides information on NQF-endorsed measures by cross-cutting area, as well as the type of measure (structure, process, outcome, and composite).

As demonstrated in the figure below, population health/prevention and safety represent the cross-cutting areas with the largest number of measures, while there are clear measure gaps in cross-cutting areas such as care coordination and patient experience and engagement. In addition, for areas with a range of measures, many focus on processes of care. However, there has been an increased focus on outcome measures with outcome measures now representing approximately 30 percent of the NQF portfolio. Measure development is also evolving to new areas such as resource use/cost (an area for which NQF is now endorsing measures) and patient-reported outcomes. Planned NQF endorsement projects in the coming year in these high priority areas, such as patient engagement and population health, should help to fill some of these important gaps.

The following sections address measures and gaps related to each of the cross-cutting areas.

Making Care Safer

NQF has endorsed a robust set of patient safety measures. However, gaps remain. For example, there is a need for measures that assess broader, more cross-cutting issues of medication safety, rather than measures that apply to separate medications. There is also interest in “templates” for medication management and safety that could be applied to different medications or conditions. In addition, more research on standard medication monitoring and its effect on outcomes or complications are needed. There is also a recognized need to expand available patient safety measures beyond the hospital setting and harmonize safety measures across sites and settings of care. There have also been recognized patient safety gaps in potentially high leverage areas, such as healthcare associated infections (e.g., MRSA) and measures that assess the culture of safety.

The NPP provided guidance on proposed goals and measure concepts related to the National Quality Strategy. The following table provides the NPP-recommended goals and measure concepts on Priority Area #1, Making Care Safer. Under the identified measure concepts, there are gaps related to inappropriate medication use and polypharmacy. There are also continued efforts to expand all-cause safety measures.

Ensuring Person- and Family-Centered Care

There have been a growing number of standardized measures that assess patient experience in multiple care settings. However, as noted in the NPP measure concepts related to this priority area, there is a significant gap in measures that assess patient and family involvement in decisions about healthcare. There is a growing evidence base on decision quality and there is an expectation that these measures will be submitted to NQF in the coming year. The measurement of care planning and joint development of treatment goals has not been limited by available evidence. It has been difficult to construct meaningful measures that move beyond “checkbox” measures that assess whether a plan exists.

Promoting Effective Communication and Coordination of Care

In the area of care coordination, measures that focus on communication and transitions across setting (e.g., medication reconciliation and transitions from inpatient facilities to other settings) and healthcare home have been endorsed, leaving many areas outlined in the NQF care coordination framework (i.e., proactive plan of care and follow-up, information systems) without current endorsed measures. NQF is aware of some work to begin to leverage information systems to facilitate care coordination, but in a recent call for measures related to Care Coordination, NQF did not receive any new measures to address this area. Some limited development is underway, but much work remains.

The table below from the National Priorities Partnership's September report shows the NPP-recommended goals and measure concepts for Promoting Effective Communication and Coordination of Care, the third priority area in HHS' National Quality Strategy. Several of the measure concepts have associated endorsed measures, such as transition records and advanced care planning. These endorsed measures tend to be limited to certain populations and settings and there is a need for a measure development and testing that would move these measures to broader populations.

The NPP goals also specifically note the need for measures that assess symptom management and functional status. While there have been measures that assess patient function and well-being in certain settings, such as home health and nursing homes, measures that assess a change (or “delta”) in function have been limited. In addition, while there are many patient-level instruments/measures of health status and function, there are few performance measures that utilize these tools to assess the care provided by healthcare entities. In 2012, NQF will work with experts to address some of methodological challenges that have limited use of patient-reported outcomes across data platforms as performance measures.

Promoting the Most Effective Prevention and Treatment of the Leading Causes of Mortality, Starting With Cardiovascular Disease

The following table provides the NPP-recommended goals and measure concepts on Priority Area #4, Promoting the Most Effective Prevention and Treatment of the Leading Causes of Mortality, Starting with Cardiovascular Disease. While most of the identified cardiovascular prevention concepts relate to currently endorsed measures, there are some measurement gaps related to access to healthy foods and nutrition. Evidence will likely be strong for these cardiovascular prevention measures. The current NQF Population Health project may bring some of these measures forward for evaluation for endorsement.

Condition-specific measures and the gaps related to effective prevention and treatment of high impact conditions, including cardiovascular care, are discussed in the condition-specific section of this report.

Working With Communities To Promote Wide Use of Best Practices To Enable Healthy Living

Measures that can assess the health of populations are a growing area of interest in the measurement enterprise. Population health focuses not only on disease across multiple sectors, but also on prevention and health promotion. Identifying valid and reliable measures of performance across these multiple sectors can be challenging. The NPP-recommended goals and measure concepts for this priority area are noted below. The NPP recommended a three-tiered approach to population health to address the national priority of working with communities to promote the wide use of best practices to enable healthy living and well-being. While there have been endorsed measures that relate to the receipt of clinical preventive services and immunization measures across the lifespan, most, but not all, of these measures focused on clinical rather than community settings. There are measurement gaps in many of the population-level concepts below, including social support, unhealthy drinking, obesity, and dental health. In the current Population Health Project, NQF will evaluate submitted population-level measures that include a focus on healthy lifestyle behaviors and community interventions that improve health and well-being. A new oral health project will also help to prioritize dental concepts and identify gaps in both dental measures and evidence.

Making Quality Care More Affordable

A new area for NQF endorsement is related to cost and resource use. Currently, a small number of measures are under NQF review, examining some specific clinical conditions as well as the total cost of care for patients who interact with the healthcare system in a given year. While private payers have captured and reported the associated costs and resources used for patients within their systems, these measures had not yet been publicly vetted; the current NQF work can pave the way for increased transparency as well as the possibility of tracking costs in a consistent manner by multiple payers and other interested parties. Many challenges remain within this area, specifically enabling measurement and reporting of costs/resources at the individual provider level, and in the future, pairing these measures with those of quality to begin to capture efficiency.

The NPP's guidance on proposed goals and measure concepts related to this priority area appears in the table below. There are important measure gaps related to access, per capita expenditures and affordability. In addition, development of measures around potential overuse of specific procedures may be limited by the available evidence in clinical guidelines. However, the overuse measures that have failed endorsement to date primarily relate to the lack of availability of the detailed clinical information in claims data. Similarly, the ability to construct a measure of preventable emergency department use has been limited by the availability of data to assess the concept of preventability.

Identification of Gap Areas Based on Federal Programs' Measure Usage

The Measure Applications Partnership (MAP) is a public-private partnership convened by the National Quality Forum (NQF) for the primary purpose of providing input to the Department of Health and Human Services (HHS) on selecting performance measures for public reporting, performance-based payment programs, and other purposes. In its first year, the MAP focused on the availability of measures for federal programs and provided input on important measurement gaps. The MAP Pre-Rulemaking Report provides input on over 350 measures under consideration by HHS for nearly twenty clinician, hospital, and post-acute care/long-term care performance measurement programs, using the six NQS priorities to guide its recommendations. The findings of the MAP related to gaps in the federal programs reinforce the gap analysis presented in this report. For example, MAP found that most federal reporting programs lacked measures in the areas of person and family-centered care, and cost and appropriateness. Looking specifically at clinical areas, MAP also noted a lack of measures in the area of mental health. All these findings echo the lack of NQF-endorsed measures in these areas as described.

In part due to MAP's required focus on the federal programs, which to date have often been defined by setting of care, the MAP work identified gaps by setting or provider type for the clinician, hospital and Post-Acute Care/Long Term Care (PAC/LTC) federal reporting programs. The high-level measure development and implementation gaps in federal programs are included in the table below:

Clinician Programs
• Patient-reported outcomes, health-related quality of life.
• Shared decision-making, patient activation, care planning.
• Care coordination.
• Multiple chronic conditions.
• Palliative and end-of-life care.
• Cost including total cost, cost transparency, efficiency, and resource use.
• Appropriateness.
Hospital Programs
• Cost—total cost of care, episode, transparency, efficiency.
• Appropriateness—admissions, treatment.
• Care coordination—transitions of care, readmissions, hand-off communication, follow-up.
• Patient-reported outcomes—patient and family experience of care and engagement, patient and family preferences, shared decision-making.
• Disparities in care.
• Special populations—behavioral health, child health, maternal health.
• Quality of life/well-being.
• Pain.
• Malnutrition.
• Palliative Care—comfort, integration of patient values in care planning.
PAC/LTC Programs
• Functional status is a high-priority gap across all programs because assessing function and change in function over time is a baseline for tailoring care for individuals and population subsets.
• A second prominent gap is measures that incorporate the patient, family, and caregiver experience and their involvement in shared decision-making.
• Measures that assess if care goals are established using a shared decision making process and if those goals are attained.
• Measures understanding how providers use assessment information to tailor goals.
• Establishing and attaining care goals.
• Care coordination, including transitions.
• Cost.
• Mental health.
• Nutritional status.

Gaps Across National Priority Areas by Condition-Specific Areas

To better highlight gaps areas, NQF further grouped its endorsed measures by the following high impact conditions, and reported gaps by each condition, mapped to the NQS priority areas. The condition-specific areas map to the Prioritization of High-Impact Medicare Conditions and Measure Gaps report prepared for HHS in 2011, with additional high impact areas added to address younger populations (e.g., child health, maternal health, and serious mental illness). For example, NQF broadened the high-impact condition COPD to include other pulmonary conditions (such as asthma.) Finally, related conditions, such as acute myocardial infarction and congestive heart failure, have been grouped together under the broader term of cardiovascular.

  • Alzheimer's Disease
  • Cancer
  • Cardiovascular
  • Cataract
  • Child Health
  • Depression
  • Diabetes
  • Glaucoma
  • Hip/Pelvic Fracture
  • Maternal Health
  • Osteoporosis
  • Pulmonary
  • Renal Disease
  • Rheumatoid Arthritis/Osteoarthritis
  • Serious Mental Illness
  • Stroke

In addition to categorizing the measures by NQS priority area, the measure type (i.e., structure, process, outcome, and composite) have been included in these tables. Figure 3 offers a high level analysis of measures by clinical system. As evident in the table, there are many clinical areas that need further outcome measure development.

As a result, high-level information is presented below regarding gaps in endorsed quality measures within the priority areas identified in the NQS. While there are many reasons for the persistent gaps in performance measurement described below, many developers who submit measures to NQF report that the lack of adequate financial support for measure development is a major driver. In addition, measure gaps persist due to insufficient evidence (e.g., management and treatment of Alzheimer's disease) and methodological challenges related to emerging measurement areas (e.g., aggregation of patient-reported outcomes into measures appropriate for accountability and quality improvement).

Gaps Across National Priority Areas by Condition-Specific Areas

For each condition, the shaded spaces in the tables below represent areas where there are NQF-endorsed measures addressing NQS priority areas, by measure type. The blank spaces represent areas where there are gaps in NQF-endorsed measures.

Alzheimer's Disease

While Alzheimer's is recognized as a critical area for measurement, there is a gap in endorsed measures for this condition. There has been limited measure development in this area, which was evidenced through a request for measures by NQF that resulted in no submissions in 2010. Through recent discussions with several developers, NQF has learned that some development has begun. Future NQF measure endorsement projects will include an opportunity for submission of newly developed measures related to Alzheimer's disease.


The set of endorsed cancer measures is primarily oriented to cancer screening and effectiveness of treatment for specific cancers. For the priority area of prevention, there are process measures addressing breast, cervical, and colorectal cancer screening. For this topic, there are gaps across all measure types in the healthy living priority area. In the person and family centered care priority area, there are several process measures and there are measures that specifically address the quality of care received at the end of life through caregiver surveys. For safer care, there are several process measures and a small number of outcome measures. There is a gap in outcomes related to cancer survival. There are a small number of overuse measures related to affordable care. Gaps related to the quality of life and other critical outcomes of care related to patients diagnosed with cancer remain. No measures were brought forward to address these gap areas in the recent call for measures for the current NQF Cancer Endorsement Project.

Cardiovascular Care

NQF has a very large set of endorsed cardiovascular measures addressing conditions such as acute myocardial infarction, coronary artery disease, and congestive heart failure. There are also endorsed process, outcome, and composite measures related to healthy living and prevention, including measures that align with the CDC goals in its national initiative “Million Hearts” to prevent one million heart attacks and strokes. While each of the clinical conditions within the larger topic area of cardiovascular care has a robust set of measures of process and outcome measures, gaps remain in the area of person- and family-centered care. As a result of the NQF Patient Outcomes project completed in 2011, several composite measures that examine care transitions for cardiovascular care are now included in the NQF portfolio. In addition, measures that assess coordination of care, such as the recently endorsed measure that assesses referral to cardiac rehabilitation after a heart attack, are in development. Measures that begin to address affordable care are slowly increasing in numbers. For example, NQF recently endorsed measures of appropriate use of cardiac stress testing as well as measures that capture resources or costs associated with specific cardiovascular conditions, but many gap areas remain.


While only a handful of measures have been endorsed in the area of cataracts, these measures address the outcomes of cataract surgery. Complications following surgery and improvement in patients' visual function have been targeted. Currently, the measures focus on those patients who have had surgery. Future measures should address the appropriate selection of treatment of patients with cataracts, ensuring that only those patients whose visual function and quality of life is compromised receive surgery. There is also a need for measures that address cataract outcomes for patients with multiple co-morbid comorbidities, including diabetes. These may be examples where the evidence base may limit applicability of these measures to more complex patients.

Child Health

The number of endorsed measures focused on child health has grown in the last year—in part due to a targeted NQF Child Health project that was completed in 2011. The portfolio has also expanded to accommodate core measures for the CHIPRA program. Similar to Maternal Health discussed below, Child Health has many measures focused on screening, immunizations, well-child visits, and treatment for specific clinical conditions. While there are endorsed outcome measures for children, such as those that examine infection, mortality, and readmission in the intensive care units, they are primarily hospital focused rather than ambulatory. In terms of affordable care, there is a measure focused on length of stay in pediatric intensive care units and a measure of emergency department visits for children with asthma, both of which address use of resources.

An opportunity exists to increase the number of measures that apply to children by adapting adult-focused measures to apply to younger ages. This gap is very dependent on measure developers' willingness to apply measures to younger populations, but age-based population limits and this limitation should only occur when the evidence does not support the expansion to those under 18 years of age. In January 2011, NQF released a report from the Measure Prioritization Advisory Committee focused on measure development and endorsement agenda that identified child health gaps in the areas of care coordination (transitions, referrals, medical homes); acute and chronic management (health promotion, community resources, timely and appropriate follow-up of screening tests); and population health outcomes.

Depression and Serious Mental Illness

There is a growing set of endorsed outcome and process measures that address depression. There are some endorsed measures that address Healthy Living and Prevention (e.g., maternal depression screening, suicide risk assessment). In NQF's Patient Outcomes project, measures looking at whether remission of symptoms was achieved at 6 and 12 months were recently endorsed—a step toward assessing patient outcomes related to depression. Many gaps remain specific to person- and family-centered care. There are also a small number of endorsed process measures related to safer care in the areas of medication management and evaluation and assessment for major depressive disorder. There are a limited number of measures that assess coordination of care, such as persistent use of needed antidepressants, as well as follow-up care after hospitalization.

There are many measurement gaps for patients with serious mental illness. Currently, only measures specific to schizophrenia and bipolar disease are endorsed, leaving many other mental health conditions unaddressed. There are endorsed process measures that address prevention and safer care (e.g., screening for potential comorbidities for patients with bipolar disorder, use of multiple antipsychotic medications). However, gaps remain specific to other priorities. There is an endorsed patient experience of care measure for inpatient psychiatric care and a set of measures that assess transition from inpatient to outpatient care. Measure gaps relate to affordability, such as potential measures that assess overuse of multiple antipsychotic medications. There are also important population health gaps for serious mental illness, including measures that would address issue of social support and homelessness. NQF anticipates that additional measures related to serious mental illness will be submitted in the upcoming Behavioral Health project.


While NQF has endorsed multiple diabetes measures, they are primarily oriented to prevention and healthy living, including two composite measures that address both processes and intermediate outcomes for patients with diabetes. In healthy living, there are also population-level measures that assess potentially preventable admissions for diabetic complications. While there are measures that address the treatment of patients with the disease, measures have not yet been developed or endorsed that adequately address the pediatric population or primary screening and prevention of diabetes for high-risk individuals. Many of these gaps are due to the lack of consistent, strong evidence on appropriate screening and treatment. In the current NQF Resource Use project, a recently endorsed measure captures the relative resource use for patients with diabetes. This measure should allow implementers including payers to identify the costs and resources associated with this chronic illness.


Two measures have been endorsed in the area of glaucoma that address appropriate evaluations and the reduction of intraocular pressures. Many gaps remain, including addressing patients' quality of life, experience with care, care coordination, and education related to treatments.

Hip/Pelvic Fracture

There is a limited set of endorsed measures that address hip and pelvic fracture. Two outcome measures were recently endorsed that target the rate of complications and readmissions after hip surgery. There is also an endorsed measure that examines the mortality rate related to these fractures. Beyond these three outcomes measures, the NQF portfolio includes measures that address osteoporosis screening and treatment with several specifically targeting those patients who have had a hip or pelvic fracture. Those measures are captured within the discussion and analysis of osteoporosis and are not reflected in the table below. Many gaps remain related to the coordination of care and person/family centered care. For affordable care, resource use measures related to hip fracture are under consideration in the current NQF Resource Use Project.

Maternal Health

NQF has a growing set of endorsed measures that relate to maternal health. There are several important process measures, such as ensuring adequate screening, prenatal and postpartum visits, and appropriate treatment during delivery. Several measures related to appropriate processes or intermediate outcomes during labor and delivery (e.g., use of prophylactic antibiotics and health-care acquired infections in the newborn) are linked to the priority area of Safer Care. There are measures that relate to affordable care, such as the rate of Cesarean sections for first-time mothers and elective deliveries prior to 39 weeks. One significant area for which measures may be in development but have not yet been submitted to NQF is related to reproductive health.


Few measures have been endorsed in the area of osteoporosis. To date, those measures have focused on appropriate screening and treatment, such as endorsed measures that target appropriate screening or treatment following a fracture, or general screening of women at risk. Significant gaps remain in areas that assess patients' quality of life and functional status and care coordination, in addition to the dearth of outcomes measures and the lack of applicability of the current measures to men.


For the purpose of this report, pulmonary conditions include asthma, chronic obstructive pulmonary disease (COPD), and pneumonia. There are many process measures that examine care for adults and children with asthma, measures of appropriate use of medications to prevent and treat exacerbations of COPD, and outcome measures related to mortality and readmission for pneumonia. Several outcome measures for pulmonary conditions were recently endorsed through the NQF Patient Outcomes project, including care transitions for patients with pneumonia and quality of life for patients with COPD in pulmonary rehabilitation programs. While some measures looking at safer care and person/family centered care have now been endorsed, measures related to other pulmonary conditions or applicable to broader settings are needed.

Renal Disease

There is a broad set of measures related to End Stage Renal Disease (ESRD) and a small but emerging set of measures related to chronic renal disease. NQF has endorsed several process and outcome measures on this topic, in the priority area of Healthy Living and Prevention. As part of a recent End Stage Renal Disease (ESRD) endorsement project, a CAHPS measure was endorsed that assesses patient experience with in-center hemodialysis. There are also multiple outcome measures related to adequacy of dialysis and infection rates. Evidence continues to evolve regarding the appropriate target hemoglobin for patients with ESRD. Due to the black box warning issued by the FDA and continued changes to what hemoglobin levels are considered safe targets, NQF and its committees have been reluctant to endorse measures for which the evidence is not yet consistent to support a performance measure. Additional gaps remain related to care coordination and affordable care.

Rheumatoid Arthritis/Osteoarthritis

Few measures have been endorsed in the areas of rheumatoid arthritis and osteoarthritis. To date, those measures have focused on appropriate screening and treatment. For example, NQF has endorsed measures related to medication safety for patients with rheumatoid arthritis as well as measures that focus on ensuring appropriate follow-up and testing to prevent toxicity. Significant gaps remain in areas that assess patients' quality of life and functional status and care coordination. There is also an absence of outcomes measures such as functional status.


Within stroke, there are endorsed process and outcome measures related to prevention, safer care and care coordination. Within safer care, there are outcome measures related to potentially avoidable complications and mortality after stroke. NQF has also endorsed primary prevention related measures, such as anticoagulation for patients with atrial fibrillation and secondary prevention related measures, such as use of statins. There are multiple measures that assess the appropriate care and screening for patients after stroke, including issues related to anticoagulation and ongoing need for speech therapy. There is a single endorsed measure related to stroke education, but no endorsed measures that assess person and family centered care. There are also gaps in measures in the healthy living and affordable care priority areas. While NQF has not previously endorsed measures related to affordable care, there are stroke-related resource use measures currently in the NQF endorsement process.


While the NQF portfolio of endorsed measures can address many important priority area and high priority clinical conditions, there are many gaps that remain. While many measure gaps could be filled with measure development, there would be a small sub-set where development would be limited by available evidence. Another important impediment to measure development in many high priority areas relates to the lack high quality data for measurement. The move toward an electronic data platform should help increase capacity to measure some of these important concepts. Collectively, the NPP, MAP and endorsement-related work provide a roadmap to where measures are needed to fill many important gaps. This report can be used to target measure development resources to areas where there are critical development gaps.

Appendix of Measures Included Within the Condition-Specific Areas

Alzheimer's Disease

* There are no measures in the portfolio for this condition.

IV. Secretarial Comments on the Annual Report to Congress

The Secretary is pleased with the scope and vision of NQF's March 2012 annual report to Congress (the “annual report”). An internal multidisciplinary cross-component HHS team is working collaboratively with NQF to provide for a clear multi-year vision to ensure the most efficient and effective utilization of the HHS contract. The contract with NQF provides an important opportunity to further enhance HHS' efforts to foster a collaborative, multi-stakeholder approach to increase the availability of national voluntary consensus standards for quality and efficiency measures.

Over the past year NQF continued work on tasks outlined in the Statement of Work, including: Providing additional input on the development of a national strategy for performance measurement and prioritization of measures for development and endorsement; conducting measure endorsement projects focused on measure gap areas such as outcomes measures and patient safety measures; maintaining current NQF-endorsed measures; promoting Electronic Health Records through activities that include developing a measure authoring software tool; and retooling of a subset of existing NQF-endorsed measures into electronic measure format. NQF provided input on the implementation of the national priorities of the National Strategy for Quality Improvement in Healthcare (NQS). The NQF convened the National Priorities Partnership (NPP) and delivered a report that focused further on enhancing patient safety, one of the six NQS priorities. The NPP worked with HHS on the Partnership for Patients initiative. The NQF continued its endorsement of quality measures for use in accountability and performance improvement with a focus on crosscutting measures and measures addressing costly and prevalent health conditions. NQF convened the Measure Applications Partnership (MAP) to foster alignment of measures in order to reduce reporting burden and accelerate improvement in reporting. The MAP provided pre-rulemaking guidance to HHS, including input on the selection of quality and efficiency measures.

The Secretary has reviewed the annual report and has the following comments. First, the Secretary notes an inadvertent statement in the annual report. The statement appears in the third sentence of the first paragraph on page 16 of the Report to Congress under the section entitled “3. Endorsing Measures and Developing Related Tools”. It refers to NQF-endorsed measures and states they have “special legal standing”. The suggestion that NQF-endorsed measures enjoy “special legal standing” is ambiguous and could be misinterpreted. Numerous statutory provisions in the Social Security Act (the “Act”) require the Secretary to specify measures for quality programs that have been endorsed by the consensus-based entity with a contract under section 1890(a) of the Act. NQF currently holds this contract and the Secretary often selects NQF-endorsed measures for quality programs. Nonetheless, the suggestion that these measures “have special legal standing” does not describe the significance of NQF endorsement for measures the Secretary selects. In addition, this statement oversimplifies the complex intellectual property concerns that frequently attend federal agency use, adoption, and dissemination of NQF-endorsed measures.

Second, the Secretary wishes to clarify a statement that has the potential to be misleading. This statement appears in the final sentence of the first full paragraph on page 7 of the Report to Congress and states: “As it turns out, NQF has already endorsed measures for medication reconciliation, readmission, and care transitions that apply to additional settings and populations so these measures can move right into other federal programs.” This sentence is vague and the reference to measures moving `right into other federal programs' does not accurately describe the process by which measures are selected for use in quality programs.

Third, the Secretary also wishes to clarify a statement in the sentence in the middle of the second column in “Sidebar 5: Harmonizing Surgical-Site Infection Measures” on page 20 of the Report to Congress. The sentence states: “Notably, CMS has selected this harmonized measure for inclusion in the 2012 final rule of the Inpatient Prospective Payment System (IPPS).” This sentence suggests that the referenced measure—Surgical Site Infection—was included in Fiscal Year 2012 Inpatient Prospective Payment System (IPPS)/Long term Care Hospital Prospective Payment System final rule as part of the payment for the IPPS program, when in fact this measure was finalized in that rule for use in the Hospital Inpatient Quality Reporting (“Hospital IQR”) program.

Fourth, the section entitled “Eight Years of Hospital Reporting Show Results” on page 31 of the Report to Congress discusses simultaneous reporting on measures by hospitals to the Centers for Medicare & Medicaid Services (“CMS”), presumably for the Hospital IQR program, and to the Joint Commission for hospital accreditation. Although there may be some overlap in the measures on which hospitals report to CMS and the Joint Commission, this section suggests that CMS and the Joint Commission run the Hospital IQR program together, which is not the case.

Fifth, the Secretary notes some ambiguity with respect to the description of funding that NQF receives from the MIPPA and the Affordable Care Act. Specifically the language in the Report to Congress implies that the two laws directly appropriated funds to the NQF, which is not accurate. The NQF receives MIPPA and Affordable Care Act funding through a contract from HHS. In addition, regarding the first bullet point before the text box entitled `Working with NQF Helped Spur Rapid Evolution of Ophthalmology Measures,' the Secretary clarifies that section 3014 of the Affordable Care Act amended section 1890(b) of the Social Security Act by adding paragraphs (7) and (8), which require NQF to convene multi-stakeholder groups to provide input on the selection of quality and efficiency measures and national priorities for improvement in population health and the delivery of healthcare services for consideration under the national strategy, and to transmit the multi-stakeholder group input to the Secretary.

Sixth, the Secretary also wishes to note that section 3014 of the Affordable Care Act added additional items that must be included in the report that the consensus-based entity submits to Congress and the Secretary that are not included in the last bullet in the narrative prior to the next section, `2 Bridging Consensus About Improvement Priorities and Approaches,' of the Report to Congress. Section 3014 of the Affordable Care Act amended section 1890(b)(5)(A) of the Social Security Act to require that the report submitted to Congress and the Secretary identify gaps in endorsed quality and efficiency measures, including gaps in priority areas identified in the national strategy, instances where quality and efficiency measures are unavailable or inadequate to address such gaps, areas in which evidence is insufficient to support endorsement of quality and efficiency measures, including priority areas, as well as the input provided by multi-stakeholder groups on the selection of quality and efficiency measures and the national priorities.

Finally, the Secretary wishes to clarify the first sentence in the second paragraph on page 1 of the Overview section of the NQF Report on Measure Gaps and Inadequacies. Section 3014 of the Affordable Care Act amended section 1890(b)(5)(A) of the Act to add additional topics to the items that must be described in the Report to Congress, but these amendments did not change the date by which the entity with a contract is required to submit the Report to Congress and the Secretary. That date is March 1 of each year (beginning in 2009), not February 1, 2012 and annually thereafter, as the addendum states.

The Secretary is pleased with the progress and timeliness of the work outlined in the Annual Report.

V. Future Steps

HHS provided a four-year contract to NQF. During this performance year of the contract, NQF completed deliverables for each task required by section 183 in MIPPA and by section 3014 in Affordable Care Act. In the final year of the contract, HHS will continue to task NQF with projects than can be completed wholly or partially by the expiration of the current contract. In addition, HHS will develop a contract mechanism to support the Affordable Care Act-required work needed through FY2014.

Maintenance of Consensus-Based Endorsed Measures

During January 14, 2012 to January 13, 2013, NQF will maintain endorsed measures relevant to HHS-wide programs and will continue to maintain consensus-based endorsed measures as developed under the priority process. Maintenance of NQF-endorsed measures encompasses five areas: (1) Review of time-limited measure results, (2) annual updates, (3) endorsement maintenance projects, (4) ad hoc reviews, and (5) education to measure developers on endorsement maintenance activities. In 2012, 42 time-limited endorsed measures are expected to undergo NQF review while 276 measures will require annual updates. Measures in these topical areas are undergoing endorsement maintenance: Cardiovascular, surgery, palliative/end-of-life-care, renal, perinatal, cancer, and pulmonary/critical care measures. In addition, NQF will begin endorsement maintenance projects for the following four topics: Gastrointestinal/genitourinary; infectious diseases; neurology; head, ears, eyes, nose and throat (HEENT). Finally, NQF is prepared to undertake ad hoc endorsement reviews as needed and will be hosting web-based educational events on its endorsement maintenance activities.

Promotion of Electronic Health Records

In 2012, NQF will continue to support the promotion of electronic health records as part of HHS-wide efforts. NQF's contributions will include enhancements of the Quality Data Model, which specify the necessary data for electronic and personal health records. NQF will continue hosting and enhancing the Measure Authoring Tool, and will provide technical assistance and support to tool users. NQF will also maintain an online Knowledge Base of information gleaned during the eMeasure retooling process of 2011, the subsequent comment and updating process, and the ongoing consulting activities that began in 2011. The Knowledge Base will be available on the NQF Web site for public use and updated at a minimum on a monthly basis to highlight new critical issues that are identified. The content of the Knowledge Base will support educational requirements for measure developers, measure implementers, EHR vendors, clinician, health care organizations, health information exchanges, and others as new stakeholders are identified. In addition, NQF will help HHS transition the Measure Authoring Tool to HHS for continued hosting and enhancements.

Focused Measure Development, Harmonization, and Endorsement Efforts To Fill Critical Gaps in Performance Measurement

In 2012, NQF will finish endorsement efforts focused on efficiency/resource use measures and regionalized emergency care services. In addition, NQF will perform an assessment of need among key stakeholders for a measure registry, a system capturing the lifecycle of a measure with capability to track versions of measures as they proceed through their lifecycle. Such a registry could assist measure developers and users to better identify measures in development, especially those identified as filling critical gaps, and how measures are similar and different version to version. General issues/concerns regarding establishing, using, and maintaining a registry (e.g., intellectual property, data quality, incentives for use) will be explored specific to health care performance and cost measures.

Convening Multi-Stakeholder Groups

NQF will continue work to provide further input into the National Quality Strategy and annual selection of quality measures for use in public and private reporting programs and value-based purchasing programs.

V. Collection of Information Requirements

This document does not impose information collection and recordkeeping requirements. Consequently, it need not be reviewed by the Office of Management and Budget under the authority of the paperwork Reduction Act of 1995 (44 U.S.C. 35)

Dated: August 27, 2012.

Kathleen Sebelius,

Secretary, Department of Health and Human Services.



[FR Doc. 2012-22379 Filed 9-13-12; 8:45 am]