Agency for Healthcare Research and Quality, HHS.
This notice announces the intention of the Agency for Healthcare Research and Quality (AHRQ) to request that the Office of Management and Budget (OMB) approve the proposed information collection project: “Evaluation of the Educating the Educator (EtE) Workshop.” In accordance with the Paperwork Reduction Act of 1995, Public Law 104-13 (44 U.S.C. 3506(c)(2)(A)), AHRQ invites the public to comment on this proposed information collection.
Comments on this notice must be received by May 27, 2014.
Written comments should be submitted to: Doris Lefkowitz, Reports Clearance Officer, AHRQ, by email at email@example.com.
Copies of the proposed collection plans, data collection instruments, and specific details on the estimated burden can be obtained from the AHRQ Reports Clearance Officer.
Start Further Info
FOR FURTHER INFORMATION CONTACT:
Doris Lefkowitz, AHRQ Reports Clearance Officer, (301) 427-1477, or by email at firstname.lastname@example.org.
End Further Info
Start Supplemental Information
Evaluation of the Educating the Educator (EtE) Workshop
AHRQ's Educating the Educator (EtE) workshop training project is an Agency knowledge translation and dissemination project that aims to increase knowledge about and use of AHRQ's EHC Program products among health care professionals. For the EtE project, AHRQ is sponsoring the development of an accredited, in-person, train-the-trainer workshop program for health care professionals to educate them on how to use AHRQ's EHC Program materials and resources in shared decision making (SDM) with patients/caregivers. As a train-the-trainer program, the workshop also provides education on effectively training other health care professionals to facilitate the dissemination of the key competencies taught by the program. Additionally, as part of the EtE project, a collection of new stand-alone tools are being developed to facilitate the implementation and use of AHRQ EHC Program materials. The new tools will be integrated into the EtE workshop training program and made available to workshop participants. These new tools also will be publicly-accessible through the AHRQ Web site for easy referral, access, and use by both workshop participants and other health care professionals.
AHRQ recognizes the importance of ensuring that its dissemination activities are useful, well implemented, and effective in achieving their intended goals. Therefore, an evaluation is associated with the EtE project. The EtE evaluation is comprised of two key components. One component has been designed to support both a process-oriented formative evaluation and a summative (impact) evaluation of the EtE train-the-trainer workshop program. Start Printed Page 17543The other component is designed to assess the impact of new tools developed through this project in supporting the implementation of AHRQ EHC Program materials.
The specific goals of the EtE train-the-trainer workshop evaluation (component 1) are to examine the following:
- Who is participating in both the primary train-the-trainer sessions, and in subsequent, secondary trainings offered by primary trainees?
- The uptake of and confidence among primary trainees in training others on the key competencies of the curriculum
- How the workshop implementation or course content should be modified to improve the quality of the training (e.g., instructor, materials, modules, etc.)?
- The extent to which workshop participants have been able to conduct additional trainings, start new PCOR education programs based on the workshop curriculum, or integrate the workshop curriculum into existing training programs in their local settings
- What the results of subsequent trainings by workshop participants were among secondary participants (i.e., individuals who received training from a workshop participant) in terms of their use of PCOR information and the practice of SDM with patients?
- Whether workshop participants have participated in other project activities, such as ongoing webinars or the learning network that are planned as part of the EtE project
- How workshop participants are using what they have learned from the training program in their own practice?
The specific goals of the EtE new tools evaluation (component 2) are to examine the following:
- If and how workshop trainees and other health care professionals are using the new tools developed during this project to support their implementation of AHRQ EHC Program resources?
- How useful clinicians find AHRQ EHC Program resources to be in their practice?
- How frequently new tools are being accessed and used by workshop trainees and other health care professionals?
- Suggestions for improving tools to meet health care professionals' needs when serving their patients?
This study is being conducted by AHRQ through its contractor, AFYA, Inc., and The Lewin Group, pursuant to AHRQ's statutory authority to support the agency's dissemination of comparative clinical effectiveness research findings. 42 U.S.C. 299b-37(a)-(c).
Method of Collection
To achieve the goals of this project the following data collections will be implemented:
(1) Pre-Training Survey of Primary Participants. This pen and paper survey will be administered to train-the-trainer workshop participants (also referred to as Primary Workshop Participants) immediately prior to the start of the in-person train-the-trainer workshop sessions. Information collected includes (1) non-identifying demographic information about respondents (e.g., type of clinician; practice setting); (2) participants knowledge of core concepts and objectives of the workshop; and (3) their confidence in training others. This instrument will also collect information about participants' use of and exposure to AHRQ EHC Program products for comparison at later time points.
(2) Post-Training Survey of Primary Participants. This pen and paper survey will be administered to train-the-trainer workshop participants immediately following the conclusion of the in-person train-the-trainer workshop sessions. Information collected includes (1) post-training knowledge of core concepts presented in workshop; and (2) post-training confidence in training others. The post-training instrument will also collect information about participants' reaction to the training (e.g., instructor, the content, the presentation style, the schedule, etc.), a requirement for accreditation purposes.
(3) Six-Month Post-Training Survey of Primary Participants. This survey will be administered to primary workshop participants six months following their participation in the train-the-trainer workshop. The survey will be Web-enabled, and a link to the survey will be emailed to participants. Information to be collected includes (1) behaviors and experiences of primary workshop participants in training others (i.e., secondary participants); (2) the numbers of individuals they have trained; and (3) barriers they have encountered in training others. This instrument will also collect (4) data on primary participants' early experiences in applying what they learned in the workshop training in their own clinical practice with patients; and (5) their use of AHRQ EHC resources and tools which will be compared to baseline measures.
(4) One-Year Post-Training Survey of Primary Participants. This survey will be administered to primary workshop participants one year following their participation in the train-the-trainer workshop. The survey will be Web-enabled, and a link to the survey will be emailed to participants. This survey will collect the same information as collected in the 6-month survey. This instrument will also collect new information from participants about their use/participation in continued training that will be offered (e.g., participating in training and technical assistance webinars and the learning network that will be created).
(5) One-Year Post Survey of Secondary Workshop Participants. This survey will be administered to secondary workshop participants one year following their receipt of continuing education (CE) credits for participating in locally-delivered workshops by primary workshop participants. The questions of interest include (1) non-identifying demographic information about respondents (e.g., type of clinician; practice setting); (2) their use of AHRQ EHC program products; (3) how useful they thought the training they received was in developing their patient engagement and SDM skills; (4) barriers they have encountered when implementing what they learned in practice; (5) the types of changes they or their organization have made related to involving patients in health care decision making and their use of decision support tools, since participating in the workshop; and (6) any changes that they have observed in their patients since they participated in the training.
(6) New Tool Users. This survey will be deployed on the AHRQ Web site on a quarterly basis. More specifically, it will be made available on the new tools Web landing page on the AHRQ Web site so that it targets users of the new tools from this project. Information to be collected includes (1) non-identifying demographic information about respondents (e.g., type of clinician; practice setting); (2) whether or not they have participated in the workshop training associated with this project; (3) how often respondents use tools on the AHRQ tools landing page; and (4) how useful respondents find the tools to be and new tools that they would like to see added.
AHRQ and the EHC Program staff will use the information collected through this Information Collection Request to assess the short- and long-term progress in achieving the dissemination and implementation aims of the EtE project. The information collected will facilitate real-time adjustments in the strategies and tactics that are used to promote and deliver the new tools and workshop training. The summative evaluation will assess the impact of this EtE workshop training program and new tools on Start Printed Page 17544increased awareness, understanding, and use of AHRQ's EHC Program products in clinical practice with patients.
The specific purpose and use of each of the data collection instruments is described below.
(1) Pre- and Post-Training Surveys of Primary Workshop Participants—These data collections will be used to assess the effectiveness of the training in transferring course concepts to train-the-trainer participants. They will be used to measure what participants learned during the training relative to their knowledge of core concepts and objectives of the workshop, and their confidence in training others as assessed prior to the training (pre-training survey). The pre-training survey also will establish a baseline level regarding workshop participants' use and exposure to AHRQ EHC Program products for comparison at later time points. The post-training assessment also will be used to assess workshop participants' reaction to the training. This is important for the process evaluation component of this project as it will provide information on participants' reactions to specific components of the program (e.g., instructor, the content, the presentation style, the schedule, etc.), a requirement for accreditation purposes, and help to identify where minor tweaking of the program may be needed to better meet participants' needs.
(2) Six-Month Post-Training Survey of Primary Participants—This data collection will be used to assess the behaviors and experiences of workshop participants in training others (i.e., secondary participants), and whether the training has promoted changes to participants' use of PCOR resources in SDM with their patients. This survey will also be used to assess whether the use of AHRQ EHC Program products has increased since participating in the survey.
(3) One-Year Post-Training Survey of Primary Participants—This data collection will be used to assess the long-term impact of the train-the-trainer workshop on participants' use of PCOR resources in SDM with patients in clinical practice. The assessment will determine if the training results in or contributes to changes in participants' continued use of key training concepts relative to baseline and 6-month assessments. This assessment also will provide information on the numbers of other individuals (i.e., secondary participants) who have received training at subsequent time points by the train-the-trainer workshop participants and the impact of training those secondary participants on their organizational practices regarding using AHRQ EHC Program products in SDM with their patients.
(4) One-Year Post Training Survey of Secondary Workshop Participants—This data collection will be used to assess the effectiveness of the train-the-trainer format on disseminating knowledge among the health care community. The questions of interest include the following:
○ Are participants from the train-the-trainer workshop able to effectively transfer or share key competencies from their training to other locally-based health care professionals (i.e., secondary participants)?
○ Do secondary participants taught by AHRQ-sponsored train-the-train workshop participants increase their use of AHRQ EHC Program products in SDM with their patients?
(5) New Tool Survey—This data collection will be used to gather information on AHRQ Web site users experiences with the available new tools including who uses these tools and if they are useful.
Estimated Annual Respondent Burden
Exhibit 1 shows the estimated annualized burden hours for the respondents' time to participate in this evaluation. For the longitudinal evaluation, four questionnaires will be completed by approximately 1,500 primary trainees who participate in the AHRQ-sponsored EtE train-the-trainer workshop, at the specified intervals, and each will require 10 or 15 minutes to complete. The annual survey of secondary participants will be completed by 3,000 secondary trainees (individuals who receive training from primary trainees) over the 3 years. Based on previous experience with convenience-based Web-based surveys, we estimate that the quarterly Web-based survey of new tool users will be completed by approximately 1,200 respondents over the 3-year period.
Exhibit 2 shows the estimated annualized cost burden based on the respondents' time to participate in this project. The total cost burden is estimated to be $91,668.
Exhibit 1—Estimated Annualized Burden Hours
|Form name||Number of respondents||Number of responses per
respondent||Hours per response||Total burden
|Pre-training survey (primary trainees) (time point #1)||* 1500||1||15/60||375|
|Post-training survey (time point #2)||* 1500||1||15/60||375|
|6-month post training survey (time point #3)||* 1500||1||10/60||250|
|12-month post training survey (time point #4)||* 1500||1||10/60||250|
|Annual survey (one-time survey of secondary trainees)||3000||1||10/60||500|
|Quarterly survey of new tool users||1200||1||5/60||100|
|* These individuals are the same 1500 individuals (primary trainees) and will be assessed at four different time points.|
|** Estimated total number of unique respondents.|
Exhibit 2—Estimated Annualized Cost Burden
|Form name||Number of respondents||Total burden
wage rate *||Total cost
|Pre-training survey (primary trainees) (time point #1)||1500||375||* $49.55||$18,581|
|Post-training survey (time point #2)||1500||375||* 49.55||18,581|
|6-month post training survey (time point #3)||1500||250||* 49.55||12,388|
|12-month post training survey (time point #4)||1500||250||* 49.55||12,388|
|Annual survey (one-time survey of secondary trainees)||3000||500||* 49.55||24,775|
|Start Printed Page 17545|
|Quarterly survey of new tool users||1200||100||* 49.55||4,955|
|* Average hourly wage based on the weighted average of wages for 1 Family and General Practitioner (29-1062, $81.78), 1 Internist (29-1063, $86.20), 1 Physician Assistant (29-1071, $44.96), 1 Psychiatrist (29-1066, $95.33), 1 Nurse Practitioner (29-1171, $44.48), 3 Registered Nurses (29-1141, $34.23), 1 Pharmacist (29-1051, $59.87), 1 Licensed Practical or Licensed Vocational Nurse (29-2061, $21.17), 1 Health Educator (21-1091, $20.52), and 1 Administrative Services Manager (11-3011, $37.61). Data Source: National Occupational Employment and Wage Estimates in the United States, May 2012, “U.S. Department of Labor, Bureau of Labor Statistics” (available at http://www.bls.gov/oes/current/naics4_621400.htm).|
|** Estimated total number of unique respondents.|
Request for Comments
In accordance with the above-cited Paperwork Reduction Act legislation, comments on AHRQ's information collection are requested with regard to any of the following: (a) Whether the proposed collection of information is necessary for the proper performance of AHRQ health care research and information dissemination functions, including whether the information will have practical utility; (b) the accuracy of AHRQ's estimate of burden (including hours and costs) of the proposed collection(s) of information; (c) ways to enhance the quality, utility, and clarity of the information to be collected; and (d) ways to minimize the burden of the collection of information upon the respondents, including the use of automated collection techniques or other forms of information technology.
Comments submitted in response to this notice will be summarized and included in the Agency's subsequent request for OMB approval of the proposed information collection. All comments will become a matter of public record.
End Supplemental Information
Dated: March 19, 2014.
[FR Doc. 2014-06880 Filed 3-27-14; 8:45 am]
BILLING CODE 4160-90-P