Agency for Healthcare Research and Quality, HHS.
This notice announces the intention of the Agency for Healthcare Research and Quality (AHRQ) to request that the Office of Management and Budget (OMB) approve the proposed information collection project: “Eisenberg Center Voluntary Customer Survey Generic Clearance.” In accordance with the Paperwork Reduction Act, 44 U.S.C. 3501-3521, AHRQ invites the public to comment on this proposed information collection.
This proposed information collection was previously published in the Federal Register on May 27, 2016 and allowed 60 days for public comment. AHRQ received no substantive comments. The purpose of this notice is to allow an additional 30 days for public comment.
Comments on this notice must be received by September 22, 2016.
Written comments should be submitted to: AHRQ's OMB Desk Officer by fax at (202) 395-6974 (attention: AHRQ's desk officer) or by email at OIRA_submission@omb.eop.gov (attention: AHRQ's desk officer).
Start Further Info
FOR FURTHER INFORMATION CONTACT:
Doris Lefkowitz, AHRQ Reports Clearance Officer, (301) 427-1477, or by email at doris.lefkowitz@AHRQ.hhs.gov.
End Further Info
Start Supplemental Information
Eisenberg Center Voluntary Customer Survey Generic Clearance
The Agency for Healthcare Research and Quality (AHRQ) requests that the Office of Management and Budget (OMB) renew under the Paperwork Reduction Act of 1995 AHRQ's Generic Clearance to collect information from users of work products and services initiated by the John M. Eisenberg Center for Clinical Decisions and Communications Science (Eisenberg Center). Since September 2008, the Eisenberg Center has been operated by Baylor College of Medicine (BCM), located in Houston, Texas.
AHRQ is the lead agency charged with supporting research designed to improve the quality of health care, reduce its cost, improve patient safety, decrease medical errors, and broaden access to essential services (see 42 U.S.C. 299). The Eisenberg Center, funded by AHRQ, is an innovative effort aimed at improving communication of findings to a variety of audiences (“customers”), including consumers, clinicians, and health care policymakers. The Eisenberg Center compiles research results into a variety of useful formats for customer stakeholders.
This research has the following goals:
(1) Conduct research into effective communication of research findings in order to improve the usability and rapid incorporation of findings into medical practice and decision making.
(2) Conduct research into effective strategies for disseminating evidence-based products, tools, and resources to consumers, clinicians, and other health care professionals, and policymakers.
(3) Evaluate outcomes reported by clinicians and other health care professionals resulting from participation in continuing medical education (CME) initiatives and activities.
(4) Conduct research into factors associated with successful collaboration between AHRQ and partnering institutions and organizations in synthesizing, translating, and disseminating evidence-based research.
Clearance is being requested to cover a three-year period in which differing numbers of information collections concerning products and research activities may be conducted within each contract year. The collections proposed include activities to assist in the development of materials to be disseminated through the Eisenberg Center and to provide feedback to AHRQ on the extent to which these products meet customer needs. These materials include documents that summarize and translate the findings of research reports for various decision-making audiences, such as consumers, clinicians, or policymakers. The summaries are designed to help these decision makers use research evidence to maximize the benefits of health care, minimize harm, and optimize the use of health care resources. In addition, each year of the contract a unique research project will be undertaken to study successful approaches to disseminating AHRQ products in various health care settings and clinical environments. Also each year the Eisenberg Center will develop one interactive decision aid for clinical problems identified from selected research reports. The intent is for the decision aid to increase the customer's knowledge of the health condition, options, and risk/benefits; lead to greater assurance in making a decision; increase the congruence between values and choices; and enhance involvement in the decision making process. Information collections conducted under this generic clearance are not required by regulation and will not be used to regulate or sanction customers. Data collections will be entirely voluntary, and information provided by respondents will be combined and summarized so that no individually identifiable information will be released.
This study is being conducted by AHRQ through its contractor, Baylor College of Medicine, pursuant to AHRQ's statutory authority to conduct and support research on health care and on systems for the delivery of such care, including activities with respect to the quality, effectiveness, efficiency, appropriateness and value of health care services and with respect to quality measurement and improvement. 42 U.S.C. 299a(a)(1) and (2).
Method of Collection
The data collections listed below will be implemented to achieve project goals. Note: Assessments such as interviews and surveys are here denoted formative if conducted prior to product development or determination of dissemination channels; usability testing or pretesting if conducted while reviewing a draft product, proposed dissemination approach, or other proposed content/strategy; and evaluation if conducted for summative evaluation or to assess satisfaction after the product has been in use or the dissemination campaign, learning activity, or other initiative undertaken.Start Printed Page 57584
Data collections will include the following:
(1) Interviews for Product and Decision Aid Development, Testing, and Use. Individual interviews will be conducted with clinical professionals, patients, or other health care consumers, or health policymakers. In some cases focus groups may be substituted for patient interviews. These formative and pretesting/cognitive interviews will allow for (1) collecting input from target audiences regarding the development of summary products and decision aids; (2) determining if intended information and messages are being delivered effectively through products that are developed and disseminated through the Eisenberg Center; (3) assessing whether changes in topical knowledge levels can be identified following exposure to Eisenberg Center informational or instructional products or aids; (4) identifying product strengths and weaknesses to facilitate improvements that are practical and feasible; and (5) assessing decision support from the perspective of each audience. In addition, the Eisenberg Center will conduct a new research project annually to inform the enhancement of existing health information products, beyond what is currently being provided. The accompanying assessments will likely consist of interviews conducted with target audience members and may be integrated into the existing product interviews discussed above. If new assessments are required, the interview scripts and data collection particulars will be submitted as addenda.
(2) Interviews for Dissemination Activities. Interviews will be conducted with leadership and staff of health systems, hospitals, and/or clinics in which dissemination activities are conducted to explore, prior to initiating the project, those pathways holding the greatest potential for successful uptake of the AHRQ materials. Interviews will be conducted again after project conclusion with administrators and product users (e.g., consumers, clinicians) to assess success of dissemination efforts, perceptions around product access, challenges that arose, and strategies to facilitate future successful dissemination initiatives. Interview scripts will be developed and submitted as addenda.
(3) Survey for Decision Aids. Following delivery of the decision aid, a user survey will be completed to explore subjects' impressions of the tool, including ease of use, clarity of presentation, length, balance of information, rating of interactive features, and overall satisfaction. Both clinicians and patients/consumers will be surveyed. For patients, the customer satisfaction survey may include decisional outcome measures (e.g., decisional conflict, desire for involvement in decision-making), measures of attitudes and self-efficacy, and indicators of choice intention or actual choice made. If the aid is evaluated within a clinical context, measures of physician-patient interaction will also be considered. Additionally, clinicians may be interviewed about the impact of the aid on decision making, clinical flow, and patient outcomes. A user survey will be developed and submitted as an addendum.
(4) Survey for Summary Products (initial, follow up). Very brief surveys will be offered to health care professionals, consumers, and policymakers that use the online summaries. Immediately upon accessing the summaries, visitors will be asked to complete a brief survey assessing for whom they were seeking information, how the product might be used, and an email address for a follow-up survey. Respondents will subsequently be sent an email asking them to complete a follow-up online survey assessing how the information has been used, whether it influenced health care practices, and any barriers to use or suggestions for improvement.
(5) Survey of Patient and Consumer Advocacy Organizations. Each project year, representatives from consumer and patient advocacy organizations will be invited to attend a meeting and participate in ongoing activities to facilitate engagement in AHRQ systematic review, translation, and dissemination activities. Surveys by phone or online questionnaire will be used to assess the quality of the in-person meeting and ongoing activities, the impact and value of engaging with AHRQ, the value of research and translation products for the target audiences, how partners and their constituents are using the products, and ways to make the products and partnerships with AHRQ more useful for partners and have a broader reach. The survey and any additional assessment mechanisms that may be useful in evaluating these relationships with advocacy organizations will be developed and submitted as an addendum.
(6) Survey of AHRQ Partners. AHRQ, through the Evidence-based Practice Center (EPC) Program and Eisenberg Center, partners with organizations when developing, translating, and/or disseminating research reports and related products. AHRQ partners include developers of clinical practice guidelines, payers, other Government agencies, private companies, consumer and patient advocacy groups, and health care systems. Surveys by phone or online questionnaire, followed by targeted interviews, will be used to assess the impact and value of AHRQ research products for the target audiences, determine how partners are using the products, and identify ways to make the products and partnerships more useful for partners and have a broader reach. Survey and interview script will be developed and submitted as addenda.
(7) CME Outcomes Survey. AHRQ through the Eisenberg Center will offer AMA PRA Category 1 continuing medical education (CME) credit for certain products that it develops. Clinicians wishing to claim credit must complete an outcome assessment survey delivered online two months after completing the activity.
(8) Interviews and Surveys for Dissemination Research Project. Each project year the Eisenberg Center will propose and conduct a unique research project aimed at disseminating products. As part of that project, formative interviews and potentially cognitive testing will be conducted with consumers, clinicians, and administrators from participating health systems, hospitals, and/or clinics for purposes of assessing current dissemination initiatives, similar products available to their consumers, ways to optimize dissemination, and other indicators as determined by the project aims. These three audiences may also be asked to complete follow-up surveys and/or participate in interviews to document project outcomes and lessons learned from the study. Survey and interview scripts will be developed and submitted as addenda.
The information will be used to develop, improve and/or maintain high quality health care informational products and services to lay public and health care professionals. Each product previously developed by the Eisenberg Center was proposed, drafted, tested, and revised with heavy reliance on data collected in a manner similar to those approaches described in this clearance. This includes data collected at the formative stage when ideas for the product and its information parameters are being developed, through draft testing and revisions, and finally product implementation and evaluation of its usefulness in practice. Work on implementing and evaluating dissemination strategies and approaches will complement the development Start Printed Page 57585activities in optimizing delivery to the targeted audiences.
Estimated Annual Respondent Burden
Exhibit 1 shows the estimated total burden for the respondents' time to participate in this research. These estimates assume a maximum of 141 Summary products over 3 years with separate products developed for clinicians, policymakers, and consumers.
Formative interviews, and in some cases focus groups, will be used to conduct needs assessment and will be held with clinicians and consumers for development of the products and decision aids, and additionally with policymakers for those products in which policy recommendations are applicable. Interviews will be conducted with no more than 2,115 persons for product development, 180 persons for decision aid development, and 180 persons for development of dissemination initiatives over 3 years, and each will last about 60 minutes.
Once the products are developed they will be subjected to in-person or telephone interviews for purposes of usability and product testing with clinicians, policymakers and consumers. In-person/telephone interviews will be conducted with about 2,115 persons for products and 180 persons for decision aids over 3 years and will take about 60 minutes on average. A second round of interviews will be conducted only occasionally with one or more of the targeted populations if necessary due to substantial product revisions. These interviews may also be used to inform product enhancements in relation to the annual enhancement study. Because these specifications cannot be determined in advance, clearance is being requested for two testing rounds with every product and every audience.
Evaluation surveys will be conducted with approximately 6,000 representatives across the targeted audiences (i.e., consumer, clinician, policymaker) for the health information products and 2,400 persons who have used the decision aids over the 3-year period. The product surveys will take about 5 minutes to complete, and the decision aid surveys about 10 minutes. A follow-up survey will be completed for the product evaluations, which will also last about 5 minutes, while a subset of 180 of those having used the decision aids will be asked to participate in a follow-up evaluation interview lasting an hour.
Those involved in or targeted by the dissemination initiatives will be asked to participate in evaluation interviews, which will include up to 480 persons completing interviews across the 3 project years. Note: Because the timing of interviews with persons at the 6 total partner organizations has not yet been finalized, AHRQ is requesting that all dissemination-related interviews be approved for the first project year. For simplicity, the interviews are presented as annualized in Exhibits 1 and 2.
The unique dissemination research project to be proposed and completed annually will include 135 formative interviews with consumers, clinicians, and administrators, with each lasting 1 hour. Follow-up evaluation surveys and interviews will be conducted with 360 and 180 persons, respectively.
AHRQ partners will be asked to complete surveys and interviews in relation to their prior or ongoing collaborative work with AHRQ. These will include 150 persons completing surveys and 60 follow-up interviews. Similar types of surveys designed with the goal of improving products and expanding their research will be completed by 90 representatives of advocacy organizations across the 3 years, with each survey lasting about 10 minutes.
Clinicians that have completed CME accrediting requirements and are requesting CME credit will be asked to complete a follow-up outcomes survey two months following completion of the online activity. These will be completed by no more than 27,000 clinicians over 3 years and will require 5 minutes to complete.
The total burden hours are estimated to be 13,875 annually or 41,625 over 3 years. The total annual cost burden is $237,604.
Exhibit 1—Estimated Annualized Burden Hours
|Form name||Number of respondents||Number of responses per
respondent||Hours per response||Total burden hours|
|Product Formative Interviews||705||1||1||705|
|Product Pretesting Interviews||705||2||1||1,410|
|Product Evaluation Surveys||2,000||2||5/60||333|
|Dissemination Formative Interviews||40||1||1||40|
|Dissemination Evaluation Interviews||120||1||1||120|
|Decision Aid Formative Interviews||60||1||1||60|
|Decision Aid Pretesting Interviews||60||1||1||60|
|Decision Aid Evaluation Interviews||60||1||1||60|
|Decision Aid Evaluation Surveys||800||1||10/60||133|
|Research Project Formative Interviews||45||1||1||45|
|Research Project Evaluation Surveys||120||1||10/60||20|
|Research Project Evaluation Interviews||60||1||1||60|
|Partnership Evaluation Surveys||50||1||10/60||8|
|Partnership Evaluation Interviews||20||1||1||20|
|Advocacy Meeting Evaluation Surveys||30||1||10/60||5|
|CME Outcomes Surveys||9,000||1||5/60||750|
|Start Printed Page 57586|
|* For the 3-year contract period, product formative interviews and product testing interviews will each comprise 300 consumers, 300 clinicians, and 105 policymakers; product evaluation surveys will include 800 consumers, 800 clinicians, and 400 policymakers; dissemination-related formative interviews will include 40 health system/hospital/clinic administrators; dissemination-related evaluation interviews will include 40 consumers, 40 clinicians, and 40 administrators; formative interviews, pretesting interviews, and evaluation interviews for the decision aids will each include 30 consumers and 30 clinicians; evaluation surveys for the decision aids will include 400 consumers and 400 clinicians; formative interviews for the annual dissemination research project will include 15 consumers, 15 clinicians, and 15 administrators; evaluation surveys for the research project will include 50 consumers, 50 clinicians, and 20 administrators; evaluation interviews for the research project will include 20 consumers, 20 clinicians, and 20 administrators; the AHRQ partner surveys will include 50 partners; the AHRQ partner evaluation interviews will include 20 partners; the health advocates surveys will include 30 participants; and CME outcomes surveys will include 500 clinicians for each of 18 CME activities.|
Exhibit 2—Estimated Annualized Cost Burden
|Form name||Number of respondents||Total burden hours||Average hourly wage
rate *||Total cost burden|
|Product Formative Interviews||705||705||a $54.81||38,641|
|Product Pretesting Interviews||705||1,410||a 54.81||77,282|
|Product Evaluation Surveys||2,000||333||a 54.00||17,982|
|Dissemination Formative Interviews||40||40||a 49.84||1,994|
|Dissemination Evaluation Interviews||120||120||a 54.74||6,568|
|Decision Aid Formative Interviews||60||60||a 57.19||3,431|
|Decision Aid Pretesting Interviews||60||60||a 57.19||3,431|
|Decision Aid Evaluation Interviews||60||60||a 57.19||3,431|
|Decision Aid Evaluation Surveys||800||133||a 57.19||7,606|
|Research Project Formative Interviews||45||45||b 54.74||2,463|
|Research Project Evaluation Surveys||120||20||b 55.96||1,119|
|Research Project Evaluation Interviews||60||60||b 54.74||3,284|
|AHRQ Partner Evaluation Surveys||50||8||c 54.50||436|
|AHRQ Partner Evaluation Interviews||20||20||c 54.50||1,090|
|Advocacy Meeting Evaluation Surveys||30||5||d 21.21||106|
|CME Outcomes Surveys||9,000||750||e 91.66||68,745|
|* National Compensation Survey: Occupational wages in the United States May 2014, “U.S. Department of Labor, Bureau of Labor Statistics.”|
|a Based on the mean and/or weighted mean wages for various combinations of consumers (00-0000 all occupations), clinicians (29-1060 physicians and surgeons, 29-1062 family and general practitioners), and health policymakers (11-0000 management occupations, 11-3111 compensation & benefits managers, 13-1141 compensation, benefits & job analysis specialists, 11-9111 medical and health service managers, 13-2053 insurance underwriters and 15-2011 actuaries).|
|b Based on the mean and/or weighted mean wages for various combinations of consumers (00-0000 all occupations), clinicians (29-1060 physicians and surgeons, 29-1062 family and general practitioners), and health system/hospital/clinic administrators (11-9111 medical and health services managers).|
|c Based on the mean wages for AHRQ partners (25-1071 health specialties teachers, postsecondary, 11-1021 general and operations managers, 21-0091 health educators, 21-1093 social and human service assistants, 11-9111 medical and health services managers).|
|d Based on the mean wages for health advocacy organizations (21-1093 social and human service assistants [social advocacy organizations], 21-0091 health educators).|
|e Based on the mean wages for clinicians (29-1060 physicians and surgeons, 29-1062 family and general practitioners).|
Exhibit 2 depicts the estimated total cost burden associated with the respondent's time to participate in this research. The cost burden is estimated to be $237,604 annually.
Request for Comments
In accordance with the Paperwork Reduction Act, comments on AHRQ's information collection are requested with regard to any of the following: (a) Whether the proposed collection of information is necessary for the proper performance of AHRQ health care research and health care information dissemination functions, including whether the information will have practical utility; (b) the accuracy of AHRQ's estimate of burden (including hours and costs) of the proposed collection(s) of information; (c) ways to enhance the quality, utility, and clarity of the information to be collected; and (d) ways to minimize the burden of the collection of information upon the respondents, including the use of automated collection techniques or other forms of information technology.
Comments submitted in response to this notice will be summarized and included in the Agency's subsequent request for OMB approval of the proposed information collection. All comments will become a matter of public record.
End Supplemental Information
Sharon B. Arnold,
[FR Doc. 2016-20035 Filed 8-22-16; 8:45 am]
BILLING CODE 4160-90-P