Skip to Content

We invite you to try out our new beta eCFR site at We’ve made big changes to make the eCFR easier to use. Be sure to leave feedback using the 'Help' button on the bottom right of each page!


Agency Information Collection Activities: Proposed Collection; Comment Request

Document Details

Information about this document as published in the Federal Register.

Document Statistics
Document page views are updated periodically throughout the day and are cumulative counts for this document including its time on Public Inspection. Counts are subject to sampling, reprocessing and revision (up or down) throughout the day.
Published Document

This document has been published in the Federal Register. Use the PDF linked in the document sidebar for the official electronic format.

Start Preamble


Agency for Healthcare Research and Quality, HHS.




This notice announces the intention of the Agency for Healthcare Research and Quality (AHRQ) to request that the Office of Management and Budget (OMB) approve the proposed information collection project: “Patient-Centered Outcomes Research Clinical Decision Support: Current State and Future Directions.” This proposed information collection was previously published in the Federal Register on March 25, 2020. AHRQ received no comments from members of the public. The purpose of this notice is to allow an additional 30 days for public comment.


Comments on this notice must be received by 30 days after date of publication of this notice.


Written comments and recommendations for the proposed information collection should be sent within 30 days of publication of this notice to​public/​do/​PRAMain. Find this particular information collection by selecting “Currently under 30-day Review—Open for Public Comments” or by using the search function.

Start Further Info


Doris Lefkowitz, AHRQ Reports Clearance Officer, (301) 427-1477, or by email at

End Further Info End Preamble Start Supplemental Information


Proposed Project

“Patient-Centered Outcomes Research Clinical Decision Support: Current State and Future Directions”

Research has shown that health care quality in the U.S. varies significantly and only half of adults receive evidence-based, recommended care. Individuals with multiple chronic conditions (42% of adults) and older adults are at particular risk for negative health outcomes. Current evidence shows that clinical decision support (CDS) systems improve adherence to evidence-based practices by analyzing patient data and making appropriate information available to the physician at the time they need it. CDS systems are usually electronic health record (EHR)-based, encompassing tools like alerts, clinical guidelines, patient reports and dashboards, diagnostic support, and workflow tools. These tools help reduce clinical errors and allow for customization to patient needs, improving quality of care and patient outcomes.

The AHRQ Patient-Centered (PC) CDS Learning Network (PC CDS LN) defines PC CDS as: “CDS that supports individual patients and their approved care givers and/or care teams in health-related decisions and actions by leveraging information from PCOR findings and/or patient-specific information (e.g., patient-generated health data).” Through PC CDS, AHRQ seeks to accelerate the movement of patient-centered outcomes research (PCOR) evidence into practice and to make CDS more shareable, standards-based, and publicly available.

Traditionally, CDS initiatives have focused on provider-directed guidelines and increasing the shareability of CDS artifacts; however, PC CDS targets both patients (and/or caregivers) and providers.

AHRQ's effort to support PC CDS has included efforts such as the PC CDS LN, CDS Connect, and other related grants and contracts. In this project, AHRQ seeks to conduct a comprehensive evaluation to assess the impact of AHRQ's PCOR CDS Initiative (the Initiative) on understanding of the current state of PC CDS and to identify gaps to guide AHRQ's future research.

This research has the following goal:

To assess the accomplishments and opportunities for the Initiative as a whole, and each of its four individual components: The PC CDS Learning Network, CDS Connect, Quantifying Efficiencies, and the U18 CDS Resource Grants.

This study is being conducted by AHRQ through its contractor, NORC at the University of Chicago, pursuant to AHRQ's statutory authority to disseminate government-funded research relevant to comparative clinical effectiveness research. 42 U.S.C. 299b-37(a)-(c).

Method of Collection

To achieve these goals, the evaluation team will use key informant interviews and a web-based survey to gather information about the programs from stakeholders, contributors, and users of the CDS Initiative programs.

Key Informant Interviews: The evaluation team will conduct semi-structured interviews with people involved in the Initiative's components, including representatives from academia, industry, health systems, and government. Key informants will include the following groups:

Leaders: Includes AHRQ project officers, contractor's senior staff, and senior consultants to Initiative components. Leaders are expected to have set the direction of the components or activities and to be familiar with the activities, the processes of implementation, and their outputs in their entirety.

Contributors: Includes lead authors or content developers for a product or output of a component, and may overlap with leaders. Examples of contributors from the PC CDS LN include lead authors of the Trust Framework, Opioid Action Plan, or Patient Blogs; examples from the CDS Connect include individuals who contributed CDS artifacts to the repository.

Participants: Includes individuals who participated in workgroups of either the PC CDS LN or CDS Connect, or participated in the development of one of the products.

Consumers: Includes individuals who have used a product developed by the Initiative, including artifacts found on Start Printed Page 37948the CDS Connect repository and the CDS Connect Authoring Tool in particular. Individuals will be identified from interviews with leaders, contributors, and participants, and through literature review for authors making references to Initiative products (i.e., reports or artifacts).

AHRQ and the evaluation contractor will create a list of eligible key informants that reflect the appropriate mix of roles and depth of experience to ensure comprehensive evaluation. Key informants will receive invitational emails that explain the scope and allow candidates to ask questions before declining or accepting the invitation. We will include clinical staff in our sample of participants in the Quantifying Efficiencies grant program, the U18 grants and the two opioid-related CDS projects. Involving staff at clinical sites will also be critical to understanding the value of PC CDS in the context of provider workflows and burdens.

Web Survey: The purpose of the web survey is to understand more about who the users of CDS Connect resources are, their reasons for using the resources, how they use these resources, and their perceptions about their value. The CDS Connect resources of interest include the CDS Authoring Tool, artifacts in the CDS Connect Repository and open-source CDS Connect resources available on Github, a platform for developing and sharing software. Respondents will be identified through a chain-referral methodology. The first set of survey invitations will be sent to a list of email addresses of known contributors or users of CDS Connect as well as a group of potential users of CDS Connect. At the end of the survey, each respondent will be asked to provide names and email addresses for up to four other users of CDS Connect resources. After the list of names from all referrals is deduplicated, a survey invitation will be sent to these referrals.

The survey instrument includes multiple choice questions that capture important data points about use of CDS Connect resources, specifically the CDS Authoring tool, GitHub resources, and artifacts from the CDS Repository. Respondents will only be presented with more detailed questions about CDS Connect resource usage based on their responses to initial screening questions. The survey will take ten minutes on average to complete based on in-house testing.

This mixed methods evaluation seeks to answer the following research questions about the Initiative as a whole:

1. To what extent has the Initiative promoted the dissemination and implementation of PCOR findings through sharable, standards-based, and publicly available CDS and how?

2. What activities carried out through each component (e.g., webinars, workgroups, in-person meetings, repositories, CDS artifacts and development tools, final reports or plans) were found to be most successful in furthering the various goals of the Initiative?

3. What do stakeholders perceive to be the impacts of the Initiative to date, including reflection on their own involvement in it, and current or potential achievements, such as the development of a common definition of PC CDS and growth of interest in and capacity for developing these types of CDS among stakeholders?

4. How does the Initiative address federal policies for the dissemination and implementation of evidence-based research funded by the PCOR Trust Fund, and how do they interact with other federal policy initiatives designed to promote widespread use, interoperability and patient access to information from EHRs with advanced CDS.

5. What can AHRQ learn from the Initiative that is relevant to other initiatives aimed at disseminating and implementing clinical evidence and evidence-based practices? How can the lessons learned here inform future research, implementation, and dissemination initiatives?

Information collected by the study will inform strategies to promote the adoption of PCOR evidence into practice through CDS developed by AHRQ and other Department of Health and Human Services agencies, including the Centers for Medicare & Medicaid Services (CMS) and the Office of the National Coordinator for Health IT, as well as state and local governments and private health care organizations. Findings from the evaluation can help identify and shape strategies to promote more effective implementation of PCOR CDS in order to accelerate the movement of evidence into clinical practice and support patient-centered decision making by clinicians with their patients.

Estimated Annual Respondent Burden

Key Informant Interviews. Key informant interviews will be conducted with up to 147 key informants across a variety of organizations involved in each component of the Initiative. NORC will use one of 14 interview protocols based on the component the key informant is involved in and their role in that component. As shown in Exhibit 1, the interview form names include the type of role of the key informant in the project. All interviews are expected to last one hour. Some key informants may serve multiple roles or work on multiple projects. In these cases, the relevant protocols will be combined and streamlined so that the informant only completes one interview. Some of the key informant interviews for the sites or Opioid-related grants may be conducted during the course of site visits at the implementation sites, either with individuals or small groups of respondents.

Web Survey. For the web survey, it is estimated that 453 CDS Connect users will respond to the 10-minute survey. The total annual burden hours for the key informant interviews and surveys is estimated to be 224 hours as shown in Exhibit 1.

Exhibit 1—Estimated Annualized Burden Hours

Form nameNumber of respondentsHours per responseTotal burden hours
PC CDS Learning Network—Leader717
PC CDS Learning Network—Governance/Non-Executive Steering Committee313
PC CDS Learning Network—Contributor818
CDS Connect—Leader515
CDS Connect—Contributor20120
CDS Connect—Consumer/Patient25125
CDS Connect—Participant10110
Quantifying Efficiencies—Leader515
Quantifying Efficiencies—Informaticist414
Quantifying Efficiencies—Clinician818
Start Printed Page 37949
PC CDS Projects—Site Leader18118
PC CDS Projects—Informaticist10110
PC CDS Projects—Clinician20120
PC CDS Projects—Patient414
Web Survey of CDS Connect Users453.1777

Exhibit 2 shows the estimated annual cost burden associated with the respondents' time to participate in this information collection, which comes to $14,371.85.

Exhibit 2—Estimated Annualized Cost Burden

Form nameNumber of interviews*Total burden hoursAverage hourly wage rate**Total cost burden
PC CDS Learning Network—Leader771 $59.54$416.78
PC CDS Learning Network—Governance/Non-Executive Steering Committee331 59.54178.62
PC CDS Learning Network—Contributor881 59.54476.33
CDS Connect—Leader551 59.54297.71
CDS Connect—Contributor20201 59.541,190.82
CDS Connect—Consumer25251 59.541,488.53
CDS Connect—Participant10101 59.54595.41
Quantifying Efficiencies—Leader551 59.54297.71
Quantifying Efficiencies—Informaticist441 59.54238.16
Quantifying Efficiencies—Clinician882 101.43811.46
PC CDS Projects—Site Leader18181 59.541,071.74
PC CDS Projects—Informaticist10101 59.54595.40
PC CDS Projects—Clinician20202 101.432,028.60
PC CDS Projects—Patient443 24.9899.93
Web Survey of CDS Connect Users453771 59.544,584.66
** Wage rates were calculated using the mean hourly wage from the U.S. Department of Labor, Bureau of Labor Statistics, May 2018 National Occupational Employment and Wage Estimates for the United States,​oes/​current/​oes_​nat.htm.
1 Average rate for Computer Information and Research Scientists
2 Average rate for Physicians and Surgeons
3 Average rate for All Occupations

Request for Comments

In accordance with the Paperwork Reduction Act, comments on AHRQ's information collection are requested with regard to any of the following: (a) Whether the proposed collection of information is necessary for the proper performance of AHRQ's health care research and health care information dissemination functions, including whether the information will have practical utility; (b) the accuracy of AHRQ's estimate of burden (including hours and costs) of the proposed collection(s) of information; (c) ways to enhance the quality, utility and clarity of the information to be collected; and (d) ways to minimize the burden of the collection of information upon the respondents, including the use of automated collection techniques or other forms of information technology.

Comments submitted in response to this notice will be summarized and included in the Agency's subsequent request for OMB approval of the proposed information collection. All comments will become a matter of public record.

Start Signature

Dated: June 18, 2020.

Virginia L. Mackay-Smith,

Associate Director.

End Signature End Supplemental Information

[FR Doc. 2020-13574 Filed 6-23-20; 8:45 am]