Skip to Content

We invite you to try out our new beta eCFR site at We’ve made big changes to make the eCFR easier to use. Be sure to leave feedback using the 'Feedback' button on the bottom right of each page!


Request for Information; Comment Request; Department of Labor Research and Evaluation Plan for 2016

Document Details

Information about this document as published in the Federal Register.

Document Statistics
Document page views are updated periodically throughout the day and are cumulative counts for this document. Counts are subject to sampling, reprocessing and revision (up or down) throughout the day.
Published Document

This document has been published in the Federal Register. Use the PDF linked in the document sidebar for the official electronic format.

Start Preamble


Office of the Assistant Secretary for Policy, Chief Evaluation Office.


Request for information.


The Department of Labor (DOL), as part of its continuing effort to improve the quality and use of research and evaluation, is requesting comments from the public on its 2016 Research and Evaluation Plan.


Written comments must be received by the office listed in the addressee section below on or before February 8, 2016.


A copy of this research and evaluation plan may be obtained free of charge by contacting Jonathan Simonetta, Chief Evaluation Office, U.S. Department of Labor, Room S-2312, 200 Constitution Avenue NW., Washington, DC 20210, Email:

You may submit comments by one of the following methods: Email:; Mail or Courier: Jonathan Simonetta, Chief Evaluation Office, U.S. Department of Labor, Room S-2312, 200 Constitution Avenue NW., Washington, DC 20210. Instructions: Please submit one copy of your comments, preferably by email. We continue to experience delays in receiving mail in the Washington, DC area, therefore commenters are strongly encouraged to transmit their comments electronically via email or to submit them early by mail.

Start Further Info


Jonathan Simonetta, Chief Evaluation Office, U.S. Department of Labor, Room S-2312, 200 Constitution Avenue NW., Washington, DC, 20210, by telephone at 202-693-5959 (this is not a toll-free number), or by email at

End Further Info End Preamble Start Supplemental Information


U.S. Department of Labor, Chief Evaluation Office, Fiscal Year 2016 Evaluation Plan Priorities and Themes.

The U.S. Department of Labor's Chief Evaluation Office (CEO) directly funds and sponsors evaluations and also collaborates with other DOL agencies and programs to design and conduct evaluations that those agencies sponsor. The Department's annual evaluation plan is based mainly on agencies' priorities, the Department's Strategic Plan priorities, statutory requirements for evaluations, and continuing

Start Printed Page 968

discussions with agency leadership and program staff.

In addition to funds appropriated for Departmental Program Evaluations (DPE), Division G, Title I, Section 107 of Public Law 113-235 of the Consolidated and Further Continuing Appropriations Act, 2015 (the Act) authorizes the Secretary of Labor to reserve not more than 0.5 percent from specific budget accounts for transfer to and use by the Office of the Chief Evaluation Officer for departmental program evaluation. The accounts referred to in subsection (a) of the Act are: Training and Employment Services, Job Corps, Community Service Employment for Older Americans, State Unemployment Insurance and Employment Service Operations, Employee Benefits Security Administration, Office of Workers' Compensation Programs, Wage and Hour Division, Office of Federal Contract Compliance Programs, Office of Labor-Management Standards, Occupational Safety and Health Administration, Mine Safety and Health Administration, funding made available to the Bureau of International Affairs and Women's Bureau within the Departmental Management, Salaries and Expenses account, and Veterans Employment and Training. Set-aside funds are transferred to CEO and are available for evaluations of programs administered by the agencies responsible for those budget accounts.

Evaluation funding (core and set-aside) must be obligated within two years. The following sections present principles followed in developing the evaluation plan and a summary of the priorities and themes for potential evaluation projects expected to be initiated in FY 2016. Of particular note is that the Workforce Innovation and Opportunity Act (WIOA) of 2014 requires several specific evaluations, which will be carried out collaboratively by CEO and the Employment and Training Administration (ETA); some WIOA-specific studies are included in this plan and others will be included in subsequent years' plans.

Guiding Principles

Three principles guide the Department's overall evaluation plan and all studies initiated by the CEO:

1. Prioritize studies that focus on measuring the effectiveness of key program outputs and outcomes consistent with Departmental priorities, the Departmental Strategic Plan, Agency Learning Agendas, and Agency Operating Plans.

2. Encourage the most rigorous evaluation designs possible to address the evaluation question of interest, particularly experimental designs, but also non-experimental designs, in a manner that is realistic given the programmatic missions/goals, programmatic maturity, data availability, and analytic capability.

3. Expand the capacity, knowledge, and utilization of high quality evaluation designs and methods department-wide; and improve the quality of data that can be used for evaluations.

Agency Learning Agendas identify priorities for evaluations that can help agencies measure their effectiveness, their progress towards goals and outcomes, continuous improvement, and, in some cases, meet Congressional requirements for reports and evaluations. Evaluations focus on program performance and outcomes, measuring the impacts of core programs and services, evaluating new programs and initiatives, and testing the relative effectiveness of alternative program practices, using the most rigorous methodologies possible.

Evaluation Priorities and Themes for FY 2016

These themes reflect a diverse mix of potential activities designed to build evidence about what works and the factors that influence or are related to Departmental programs. Broadly, four types of projects are considered in pursuing the priorities:

  • Statistical Analyses of Trends in Programs, Labor Supply and Demand, Economic Conditions, and the Labor Market as they Relate to DOL Programs;
  • Exploratory, Formative and Implementation Evaluations, and Designs and Evaluability Assessments for Pilots and Demonstrations;
  • Formal Evaluations of Programs and Demonstrations; and
  • Research and Evaluation Capacity Building Activities.

A. Statistical Analysis of Trends and Surveys

In FY 2016, CEO is exploring the following themes, possibly through statistical analyses using agency administrative data, surveys or statistical databases:

  • Labor Market and Occupational Trends in Selected Industry Sectors;
  • Gender Patterns and Pay in Occupations and Industries;
  • Caregiving and Women's Retirement Security;
  • Unemployment and Unemployment Insurance;
  • Labor Enforcement Program Data (OSHA, WHD, OFCCP);
  • Immigration, Immigrants, and Work Visas;
  • Employment of Veterans; and
  • Analysis of Adult Skills and Competencies.

B. Exploratory, Formative and Implementation Evaluations, Evaluability Assessments, Feasibility Studies, and Structured Evidence/Literature Reviews

In FY 2016, CEO is exploring the following priorities and types of studies, possibly through exploratory evaluations using formative and implementation analysis methods, and evidence reviews.

  • Evidence and Literature Reviews

○ CLEAR Reviews. Structured literature and evidence reviews will be conducted using the review standards and guidelines established for the Clearinghouse for Labor Evaluation and Research (CLEAR) ;

○ Active Labor Market Policies and Livelihood Services in Developing Countries;

○ Policies and Strategies to Address Child Labor and Forced Labor; and

○ Education, Training and Certification Pathways.

  • Job Driven Skills and Training

○ Models for Improving Basic Skills and Career Preparation (e.g., GED and new testing mode, career education);

○ Occupational Credentialing and Training Program Practices;

○ Characteristics of, Services to, and Employment Outcomes for Unemployed and Dislocated Workers;

○ Employment Effects of Soft Skills Training and Job Search Strategies for Adults and Youth; and

○ Models of Engagement with, and Effect of Programs on, Businesses and Employers (e.g., Job Corps and other youth programs, adult workforce programs, job training programs, veterans programs).

  • WIOA Implementation

○ Implementation of WIOA; and

○ Strategies and Services Delivery in One Stop Centers/American Job Centers.

  • Veterans

○ Strategies and Models of Employment Services for Serving Veterans and Alternative Models; and

○ Models for Improving the Transition of Individuals from Active Military Duty to Civilian Employment.

  • Other Special Populations

○ DOL Programs and Services in Native American, Tribal, and Pacific Islander Urban and Rural Communities;

○ Employer Practices Regarding Accommodation and Talent-development of Employees with Disabilities; and

○ Role of Intermediaries, including Non-Farm Labor Contractors, in the Hiring of Farmworkers.Start Printed Page 969

  • Labor Standards, Worker Safety and Health, Compliance, and Compliance Assistance

○ Worker Rights in Developing Countries;

○ Child Labor Information and Technical Assistance Efforts in Developing Countries; and

○ Labor Standards in Supply Chains in Selected Industries.

C. Formal Evaluations of Programs and Demonstrations

In FY 2016, CEO is exploring the following themes, possibly through formal evaluations to test promising strategies, replicate proven models, and estimate the effectiveness of program components and service delivery approaches:

  • Youth

○ National Guard Youth ChalleNGe Job ChalleNGe Demonstration;

○ Performance Partnership Pilots (P3) for Disconnected Youth;

○ Job Corps Innovations Pilots; and

○ Youth Build.

  • Job-Driven Skills and Training

○ American Apprenticeship Initiative Grants;

○ Employment and Training Services for Adults, Dislocated Workers, Out of School Youth, and Foster Youth;

○ Innovative Career Pathways Models; and

○ Subsidized Employment and Tax Credit Strategies to Increase Employment.

  • Employment and Reemployment

○ Innovative Strategies for Improving Employment Outcomes for Incarcerated and Formerly Incarcerated Individuals and

○ Effective Reemployment Strategies for Unemployed Workers and Recipients of Unemployment Insurance.

  • Behavioral Economics and Insights

○ Evaluations Using Behavioral Insights to Improve Program Outcomes in DOL Employment and Training and Worker Protection Programs.

  • Labor Standards, Worker Health and Safety, Compliance and Compliance Assistance

○ Deterrence Strategies for Improving Compliance with Labor Standards Laws and Regulations;

○ Evaluation of Voluntary Compliance with Labor Standards Laws and Regulations;

○ Effectiveness of Various Methods and Strategies for Inspection, Compliance, and Enforcement; and

○ Improving Injury and Illness Reporting.

  • Worker Security, Benefits, and Tax Strategies

○ Effectiveness of Financial Literacy Strategies;

○ Effect of Worker Benefits on Family, Worker, and Child Well-being; and

○ Effectiveness of Tax Credits and Wage Subsidy Strategies on Employment Outcomes.

  • Outreach, Information, Training, and Technical Assistance

○ Effective Translation and Adoption of Federal Policies by States and Localities;

○ Effectiveness of Inspector Training Programs; and

○ Effectiveness of Technical Assistance and Outreach.

D. Research and Evaluation Capacity Building Activities

It is important to complement evaluation studies with other activities designed to continuously reinforce the role of evaluation at DOL: The importance of evaluation for achieving performance goals and objectives; the integration of evaluation into ongoing management; and the expectation of high quality products and reports. Dissemination of evaluation reports and access to accumulating evidence is also essential, as is the commitment to developing a pipeline of labor-focused young evaluators/scholars. This category includes various activities to continue to build DOL's evaluation capacity, such as:

  • DOL Scholars Research Program (with priority given to young scholars);
  • Collaborative Cross-Agency Statistical Analysis; and
  • Wage Record Data Exchanges for Evaluations.


BLS, Bureau of Labor Statistics

CEO, Chief Evaluation Office

EBSA, Employee Benefits Security Administration

ETA, Employment and Training Administration

ILAB, Bureau of International Labor Affairs

MSHA, Mine Safety and Health Administration

OASP, Office of the Assistant Secretary for Policy

ODEP, Office of Disability Employment Policy

OFCCP, Office of Federal Contract Compliance Programs

OLMS, Office of Labor-Management Standards

OSEC, Office of the Secretary

OSHA, Occupational Safety and Health Administration

OWCP, Office of Workers' Compensation Programs

SOL,  Office of the Solicitor

UI, Unemployment Insurance

VETS, Veterans' Employment & Training Service

WHD, Wage and Hour Division

WB, Women's Bureau

Instructions, Who Should Respond: We invite practitioners, policy makers, program directors, business and industry associations, nonprofit organizations, and researchers to provide feedback on the priorities, the themes and the types of studies and approaches included in the 2016 plan, and suggestions for related evaluation areas topics, for example:

  • Are there other themes or topics that should be considered for inclusion in the evaluation plan?
  • What types of evaluations or topics would be of most relevance to program practitioners?
  • Are there any particular data or resource constraints that should be considered?

Guidance for Submitting Documents

On page one of your submission, please indicate your name, the name of your organization (if applicable), and your contact information (including phone number, postal address, and email address). While not required, it would assist us in reviewing your information if you also included the type of organization you represent (public, private, not-for-profit, or philanthropic), the field(s) in which you work and the level at which you operate (national, state, regional, local or tribal).

Rights to Materials: By submitting material in response to this RFI, the respondent is agreeing to grant the Department a worldwide, royalty-free, perpetual, irrevocable, non-exclusive license to use the material and to make it publicly available. Further, the respondent agrees that it owns, has a valid license, or is otherwise authorized to provide the material to the Department. The Department will not provide any compensation for material submitted in response to this RFI.

Electronic Access to This Document: The official version of this document is the document published in the Federal Register. Free Internet access to the official edition of the Federal Register and the Code of Federal Regulations is available via the Federal Digital System at:​fdsys. At this site you can view this document, as well as all other documents of the Department published in the Federal Register, in text or Adobe Portable Document Format (PDF). To use PDF you must have Adobe Acrobat Reader, which is available free at the site.

You may also access documents of the Department published in the Federal Register by using the article search feature at: Specifically, through the advanced search feature at this site, you can limit Start Printed Page 970your search to documents published by the Department.

Start Signature

Signed: at Washington, DC this 17th day of December, 2015.

Sharon I. Block,

Principal Deputy Assistant Secretary for Policy, U.S. Department of Labor.

End Signature End Supplemental Information

[FR Doc. 2016-00170 Filed 1-7-16; 8:45 am]