This evaluation policy statement presents key principles that govern the Department of Labor's (DOL) planning, conduct, and use of program evaluations. The policy represents a commitment to conducting rigorous, relevant evaluations and to using evidence from evaluations to inform policy and practice. DOL seeks to promote rigor, relevance, transparency, independence, and ethics in the conduct of evaluations. This policy addresses each of these principles.

The mission of the Department of Labor is to "...foster, promote, and develop the welfare of the wage earners, job seekers, and retirees of the United States; improve working conditions; advance opportunities for profitable employment; and assure work-related benefits and rights." The importance of these goals demands that we continually innovate and improve, and that we evaluate the performance and effectiveness of our programs and activities. Through evaluation, DOL can learn systematically so that we can make our services as effective as possible.

Evaluation produces one type of evidence. A learning organization with a culture of continual improvement requires many types of evidence, including not only evaluation but also descriptive research studies, performance measurement, financial and cost data, survey statistics, and program administrative data. Although this policy focuses on evaluation, the principles apply to the development and use of other types of evidence as well. The emphasis on evidence is meant to support, not inhibit, innovation, improvement, and learning. The intent is to integrate the use of evidence and opportunities for further learning into all activities. Where an evidence base is lacking, evidence will be developed through strong evaluations and analysis. Where evidence exists, it will be used to encourage replication studies.

While much of DOL's evaluation activity is overseen by the Chief Evaluation Office (CEO) in the Office of the Assistant Secretary Policy (OASP), agencies and program offices within DOL also sponsor evaluations through dedicated contracts or as part of their grant-making, typically in coordination with the CEO. Discretionary grant funding opportunity announcements from DOL will include evidence provisions that specify how activities will be evaluated. Successful applicants will be required to cooperate with and participate in evaluations. As legally allowed, programs with waiver authorities will require evaluations as a condition of waivers.

Rigor

DOL is committed to using the most rigorous methods that are appropriate and feasible within statutory, budget, and other constraints. Rigor is required for all types of evaluations, including impact and outcome evaluations, implementation and process evaluations, descriptive studies, and formative evaluations. Rigor requires ensuring that inferences about cause and effect are well founded (internal validity); requires clarity about the populations, settings, or circumstances to which results can be generalized (external validity); and requires the use of measures that accurately capture the intended information (measurement reliability and validity).

DOL maintains an evaluation workforce with training and experience appropriate for planning and overseeing a rigorous evaluation portfolio, recruiting staff with advanced academic degrees and experience in disciplines such as public policy, economics, and sociology. DOL also provides professional development opportunities so staff can keep their evaluation and methodological skills current.

Relevance

Evaluation priorities should take into account legislative requirements and the interests and needs of leadership, specific agencies, and programs; program office staff and leadership; and DOL partners such as states, territories, tribes, and grantees; the populations served; researchers; and other stakeholders. Evaluations should be designed to address DOL's diverse programs, customers, and stakeholders; and DOL should encourage diversity among those carrying out the evaluations.

Transparency

DOL will make information about evaluations and findings from evaluations broadly available and accessible, typically on the Internet. This includes identifying the evaluator, releasing study plans, and describing the evaluation methods. DOL will release results of all evaluations that are not specifically focused on internal management, legal, or enforcement procedures or that are not otherwise prohibited from disclosure. Evaluation reports will present all results, including favorable, unfavorable, and null findings. DOL will release evaluation results timely — usually within two months of a report's completion — and will archive evaluation data for secondary use by interested researchers (e.g., public use files with appropriate data security protections).

Independence

Independence and objectivity are core principles of evaluation. Agency and program leadership, program staff, stakeholders, and others should participate in setting evaluation priorities, identifying evaluation questions, and assessing the implications of findings. However, it is important to insulate evaluation functions from undue influence and from both the appearance and the reality of bias. To promote objectivity, DOL protects independence in the design, conduct, and analysis of evaluations. After technical peer review, the Chief Evaluation Officer has authority to approve, release, and disseminate evaluation reports.

Ethics

DOL-sponsored evaluations will be conducted in an ethical manner and safeguard the dignity, rights, safety, and privacy of participants. Evaluations will comply with both the spirit and the letter of relevant requirements such as regulations governing research involving human subjects.

Effective November 2013

Footnotes

i American Evaluation Association, An Evaluation Roadmap for a More Effective Government, September 2010, https://www.eval.org/EPTF/aea10.roadmap.101910.pdf, accessed 18 June 2012, and Government Accountability Office, Employment and Training Administration: Increased Authority and Accountability Could Improve Research Program, GAO-10-243, January 2010, https://www.gao.gov/products/GAO-10-243, accessed 18 June 2012.

 

The Department of Labor's (DOL) Chief Evaluation Office (CEO) sponsors independent evaluations and research, primarily conducted by external, third-party contractors in accordance with the Department of Labor Evaluation Policy. CEO’s research development process includes extensive technical review at the design, data collection and analysis stage, including: external contractor review and OMB review and approval of data collection methods and instruments per the Paperwork Reduction Act (PRA), review by academic peers (e.g., Technical Working Groups), and inputs from relevant DOL agency and program officials and CEO technical staff. Final reports undergo an additional independent expert technical review and a review for Section 508 compliance prior to publication. The resulting reports represent findings from this independent research and do not represent DOL positions or policies.