Glossary of Evidence Terms

The department's Annual Evaluation Plan describes key information collections or acquisitions and key questions for each significant evaluation study that the agency plans to begin in the next fiscal year, led by the Chief Evaluation Officer, per Evidence Act requirements. Learn more about the department's Evaluation Plans.

The department's Capacity Assessment documents the coverage, quality, methods, effectiveness, and independence of the department's statistics, evaluation, research, and analysis efforts, led by the Chief Evaluation Officer, per Evidence Act requirements.
Learn more about the Capacity Assessment.

A role required by the Evidence Act, the Chief Data Officer is responsible for data governance and lifecycle data management, among other things.
Learn more about the Department of Labor's Chief Data Officer.

This interagency council includes all Chief Data Officers, the Administrator of the Office of Electronic Government, the Administrator of the Office of Information and Regulatory Affairs, and an Ex Officio Member who represents all Chief Information Officers and Evaluation Officers. The Council meets regularly to establish government-wide best practices for the generation, use, protection, and dissemination of data. The Council also promotes data sharing agreements between agencies and identifies ways agencies can better produce evidence for use in policymaking.

A role required by the Evidence Act, the Chief Evaluation Officer is responsible for coordinating and managing the department's evaluation efforts, and has a leading role in implementing a number of Evidence Act requirements.

Learn more about the Department of Labor's Chief Evaluation Officer, and about the Chief Evaluation Office.

The CFO Act provides guidance on financial management in the federal government. It created chief financial officer positions in 23 federal agencies, including the Department of Labor. Most of the guidance provided in the Evidence Act is required for CFO Act agencies and suggested for agencies not included in the CFO Act.

CIPSEA provides data protection and use guidance for statistical agencies, including the Bureau of Labor Statistics.

A departmental body required by the Evidence Act, the Department of Labor's Data Governance Board sets and coordinates implementation priorities for managing data that supports the department's overall mission.

A policy required by the Evidence Act, the Department of Labor's long-standing Evaluation Policy guides evaluations conducted at the Department of Labor with the principles of rigor, transparency, independence, ethics, and relevance.

Descriptive Studies can be quantitative or qualitative in nature, and seek to describe a program, policy, organization, or population without inferring causality or measuring effectiveness. While not all descriptive studies are evaluations, some may be used for various evaluation purposes, such as to understand relationships between program activities and participant outcomes, measure relationships between policies and particular outcomes, describe program participants or components, and identify trends or patterns in data.

The Evidence Act defines an evaluation as an assessment using systematic data collection and analysis of one or more programs, policies, and organizations intended to assess their effectiveness and efficiency. Evaluations can look beyond the program, policy, or organizational level to include assessment of particular projects or interventions within a Department of Labor program. There are different types of evaluation – formative evaluation, impact evaluation, and process/implementation evaluation – that address different questions. Learn more about the department's evaluation work.

An evaluation design is a plan for conducting research. Designs can be more or less rigorous and determine what research questions can be answered, for example experimental designs can answer questions that non-experimental study designs cannot. Evaluation designs are separate from data collection methods.

An interagency council of Evaluation Officers from various federal agencies. Members exchange information, collaborate with the Office of Management and Budget on issues that impact evaluations, and serve as a leader and resource for the Federal evaluation community.

OMB defines 'evaluators' as department staff and associated partners who are trained-through advanced education and evaluation experience to properly plan, implement, manage, and/or oversee evaluation activities and evaluations. Evaluators can also be others engaged in conducting evaluations, including grantees, academics, and non-governmental entities.

OMB defines 'evidence' as the available body of facts or information indicating whether a belief is true or valid. Evidence can be quantitative or qualitative and may come from a variety of sources, including foundational fact finding, performance measurement, policy analysis, and program evaluation.

The Evidence Act requires agencies to designate three Evidence Officials: the Chief Evaluation Officer, Statistical Official, and Chief Data Officer. Within the Department of Labor, the following individuals oversee the implementation of the Evidence Act and coordinate evidence-building activities: Chief Evaluation Officer, Chief Data Officer, Commissioner of Labor Statistics, and Director of the Performance Management Center.

Findings are the outcomes of an evaluation and include the data collected and the analysis and interpretation of that data.

Visit the Chief Evaluation Office's Completed Reports for Department of Labor research findings.

A formative evaluation is typically conducted to assess whether a department program, policy, or organizational approach is feasible, appropriate, and acceptable before it is fully implemented. It may include process and/or outcome measures but usually focuses solely on learning and improvement and does not answer questions of overall effectiveness.

One of the four types of evidence defined in OMB's Phase 1 Guidance, foundational fact finding includes foundational research and analysis, such as aggregate indicators, exploratory studies, descriptive statistics, and basic research.

Enacted in 2019, the Evidence Act emphasizes collaboration and coordination to advance data and
evidence-building functions in the Federal Government. It mandates Federal evidence-building activities, open government data, and confidential information protection and statistical efficiency. The Office of Management and Budget defines four phases of Evidence Act implementation (find all guidance here):
Phase 1: Learning Agendas, Personnel, & Planning
Phase 2: Open Data Access & Management
Phase 3: Data Access for Statistical Purposes
Phase 4: Program Evaluation Ongoing Implementation & Reporting

An impact evaluation tries to assess the causal impact of a program, policy, or organization, or aspect of them on outcomes, relative to a counterfactual. Impact evaluations can include both experimental (i.e., randomized controlled trials) and quasi-experimental designs. Learn more about how the Department of Labor assesses causal studies on the Clearinghouse for Labor Evaluation and Research.

This interagency council advises and assists the Chief Statistician of the United States to coordinate the Federal Statistical System and to implement statistical policies, principles, standards, and guidelines. The council also assists the Chief Statistician in the evaluation of statistical program performance.

An intervention is a combination of program elements or strategies related to the design and
implementation of programs and policies that aim to produce specific results.

A learning agenda is a plan that describes the department's agency-level evidence-building priorities, which roll up into the development of departmental planning activities, including annual evaluation and evidence-building plans. The Evidence Act refers to the Evidence-Building Plan as a learning agenda, but DOL's agency-specific learning agendas are internal-use documents.

The Evidence Act requires that the department develop an Open Data Plan annually to describe progress in opening government data to the public.

An outcome evaluation measures the extent to which a program, policy, or organization has achieved its intended outcome(s) and focuses on outputs and outcomes to assess effectiveness. Outcome evaluations cannot attribute cause.

Performance management is the use of goals, measurement, evaluation, analysis, and data-driven reviews to improve results of programs and the effectiveness and efficiency of agency operations. The primary purpose of performance management is to improve performance and then to find lower cost ways to deliver effective programs.

One of the four types of evidence defined in the Evidence Act, performance measurement includes ongoing, systematic tracking of information relevant to policies, strategies, programs, goals, and/or other activities.

One of the four types of evidence defined in the Evidence Act, policy analysis includes analysis of data, such as general-purpose survey or program specific data, to generate and inform policy.

A process or implementation evaluation assesses how the program or service is delivered relative to its intended theory of change, and often includes information on content, quantity, quality, and structure of services provided.

A program refers to a set of projects or activities that support a higher level objective or departmental goal. Generally, a program is an organized set of activities directed toward a common purpose or goal that an agency undertakes or proposes to carry out its responsibilities. Within this broad definition, agencies and their stakeholders currently use the term "program" in different ways. Agencies have widely varying missions and achieve these missions through different programmatic approaches.

One of the four types of evidence defined in the Evidence Act, a program evaluation includes systematic analysis of a program, policy, organization to assess effectiveness and efficiency.

A research method is a strategy or method used to implement the evaluation design. Methods can be quantitative, qualitative, or mixed.

A role required by the Evidence Act, the Statistical Official is responsible for advising the Department on statistical policy, techniques, and procedures. The department's Statistical Official is the Commissioner of Labor Statistics at the Bureau of Labor Statistics.

Learn more about the Department of Labor's Statistical Official.

The DATA Act guides transparency on government spending. Chief Data Officers, required by the Evidence Act, are responsible for key components of the DATA Act.