Development of PY 2020-2021 WIOA Models

Overview

Overview of the Development of the Statistical Adjustment Models

This document provides an overview of the statistical analysis that was conducted in developing and evaluating the statistical adjustment models that will be used for the 2020 and 2021 program years (PY). Beginning with PY 2020, the statistical models will be used as a factor in performance negotiations and to assess state performance after each program year in accordance with WIOA performance accountability provisions for the following performance indicators and WIOA title I and title III programs: Employment Rate 2nd Quarter after Exit (Q2ER), Median Earnings 2nd Quarter after Exit (ME), and Measurable Skill Gains Rate (MSG) for the Adult, Dislocated Worker, Youth, and Wagner-Peyser programs (note: the MSG indicator does not apply to the Wagner-Peyser program).

Previously, the statistical adjustment models underwent several rounds of development. The initial methodology of the statistical adjustment models was developed by the U.S. Department of Labor’s Chief Evaluation Office and is explained in the WIOA Statistical Adjustment Model Methodology Report. The models recommended at that time were implemented as a preliminary test beginning with PY 2017 using WIA data and proxy variables where necessary. Subsequently, the model specifications (i.e., which variables were included in the models) changed slightly beginning in PY 2018 as a result of stakeholder feedback and ongoing statistical analysis by ETA. However, through PY 2019 the models had a limited role being used as a proxy method of assessing performance because of the structural differences in data collection between WIA and WIOA.

Beginning with PY 2020, there is sufficient WIOA data to generate usable model estimates for the Q2ER, ME, and MSG indicators. As a result, the Q2ER and ME models have been refined and the MSG models have been developed. The changes to the model were largely limited to the selection of explanatory variables, the logic regarding how some variables were calculated, and determinations on the data used. The MSG indicator is a new performance indicator under WIOA (collection and reporting of this indicator began in PY 2016), so the models for this indicator are newly developed yet they follow the same basic framework as the previous models.

The following analysis is centered on three main sections that are the focus for this model development period. The sections include:

  • WIOA vs. WIA model performance - how model performance is affected by the data
  • Refinement of current model specifications - assessing the Q2ER and ME models
  • MSG model development

This report also lists the final model specifications, resulting model estimates, and model predictions that will be used as a factor in the negotiation of PY 2020-2021 performance levels.

WIOA vs. WIA Proxy Models

Comparison of WIOA Models and WIA Proxy Models

The first PY where sufficient WIOA data (i.e., at least 2 years of data) are available to generate statistical adjustment model estimates for negotiations is PY 2020. Up until this point, model estimates were generated using WIA data as a proxy. This section analyzes and compares the performance of the WIOA models (i.e., models using the WIA Proxy model specifications but fit with only WIOA data) and the WIA Proxy models (i.e., models using the WIA Proxy model specifications and fit with both WIA and WIOA data).

Comparing the performance of this specified model fit with WIA data to the performance of the model fit with WIOA data provides several insights. First, it shows how well the WIOA model is performing. While some aspects of the data collected under WIA differ compared to how it is collected under WIOA, the underlining relationships regarding how the various participant characteristic variable and economic condition variables interact with the performance outcome variables should be similar. Second, it shows the degree in which the limited amount of WIOA data is the primary factor affecting model performance. To date, there are only 2 full program years of data available for 2nd quarter after exit indicators compared to 10 years of data which were used to fit the WIA model.

All the models used in comparing WIOA and WIA Proxy models have the same specifications (i.e., the same variables) as the models originally developed for PY 2017 and PY 2018-2019 using WIA data.

Distribution of Model Residuals

Residuals of the fitted models are expected to be centered near zero with a normal distribution. As the plots below for select program indicators show, both the WIOA and WIA Proxy models fit this criteria. The biggest difference between those model types is that the WIA Proxy models tend to have wider tails with more outlier observations with the exception of the Wagner-Peyser models where both model types have noticeable outliers. Overall, the WIOA models have residuals that are reasonable. The figures below show the distribution of residuals for each of the program’s Q2ER and ME models.

Adult-Q2ER

Distribution of residuals for the Adult, Employment Rate 2nd Quarter after Exit models.

Adult-ME

Distribution of residuals for the Adult, Median Earnings 2nd Quarter after Exit models.

DW-Q2ER

Distribution of residuals for the Dislocated Worker, Employment Rate 2nd Quarter after Exit models.

DW-ME

Distribution of residuals for the Adult, Median Earnings 2nd Quarter after Exit models.

Youth-Q2ER

Distribution of residuals for the Youth, Employment Rate 2nd Quarter after Exit models.

WP-Q2ER

Distribution of residuals for the Wagner-Peyser, Employment Rate 2nd Quarter after Exit models.

WP-ME

Distribution of residuals for the Wagner-Peyser, Median Earnings 2nd Quarter after Exit models.

Predictability of the Models

To test how predictive the models are, all WIOA and WIA Proxy models were used to predict PY 2018 outcomes. The plots of the predictions vs. the actual outcomes for select program outcomes are shown below.

These plots clarify a few things. The predictions for both model types are generally comparable and perform similarly in their ability to predict the PY 2018 outcomes. In fact, for some of the program indicators, the WIOA model appears to predict slightly better than the WIA Proxy models. However, the WIOA model also did much better at predicting the extreme outliers (e.g., the Hawaii outcome for the Adult-Q2ER and DW-Q2ER models). While this may seem like a positive result, it may also indicate that, due to the brevity of WIOA data to date, the WIOA models are over-fitting the data. As a result, there is reason to believe that the WIOA model may not perform as well as the WIA proxy model in the more distant future (i.e., PY 2020 and PY 2021) when the resulting data may be more different than the data used to fit the models. The next section (i.e., Refine Current Models) discusses modifications to the model specifications (primarily the participant characteristic variables) that will help to reduce the degree to which the WIOA model is over-fitting the data. However, much of this effect is driven by the economic condition variables and can only be addressed with more data. Future models that are fit with more WIOA data will be less influenced by outlier data.

Adult-Q2ER

Plot of the PY 2018 prediction results for the Adult, Employment Rate 2nd Quarter after Exit models.

Adult-ME

Plot of the PY 2018 prediction results for the Adult, Median Earnings 2nd Quarter after Exit models.

DW-Q2ER

Plot of the PY 2018 prediction results for the Dislocated Worker, Employment Rate 2nd Quarter after Exit models.

DW-ME

Plot of the PY 2018 prediction results for the Dislocated Worker, Median Earnings 2nd Quarter after Exit models.

Youth-Q2ER

Plot of the PY 2018 prediction results for the Youth, Employment Rate 2nd Quarter after Exit models.

WP-Q2ER

Plot of the PY 2018 prediction results for the Wagner-Peyser, Employment Rate 2nd Quarter after Exit models.

WP-ME

Plot of the PY 2018 prediction results for the Wagner-Peyser, Median Earnings 2nd Quarter after Exit models.

Model Coefficients

Since this comparison is between models with the same variables fitted on similar data, the coefficients (or weight of a particular variable) should be comparable between the models. Looking at the plots of the scaled model coefficients (shown below), it is clear that the coefficients are fairly consistent between the models. The most noticeable difference between the two model types for all program outcome models is that the WIOA model has larger confidence intervals for the coefficient estimates. This is an expected result since the WIOA model were fit with much less data than the WIA Proxy models. However, for most of the variables the point estimate for the WIOA model is close to the WIA Proxy model.

The coefficients that differ the most are those for economic conditions variables and, in particular, the industry share variables. This is not surprising given that the economic conditions (i.e., unemployment rate and industry share) data for the states is noticeably different when using only the 2016 to present data instead of the 2005 to present data. The large confidence intervals for the industry share variables is also influenced by the fact that those variables are interdependent (e.g., a change in the share of manufacturing is going to be reflected in a change to one of the other industry types). The relatively larger confidence intervals are also present in the education level variables (i.e., hsdropout, hsgrad, collegedropout, etc.) for a similar reason that that they are inter-related. In addition, the coefficients for those variables have deviated some in the WIOA model as compared to the WIA Proxy model likely as a reflection of the changes in recent years with respect to the characteristics of the participants being served and the labor market conditions. These variables still follow the expected relationship of higher levels of educational attainment being associated with better outcomes on average.

The Wagner-Peyser coefficients for both model types are less stable and have larger confidence intervals. This is primarily due to the limitations in specifying the Wagner-Peyser models with less available data on participants in the WP program prior to WIOA.

Adult-Q2ER

Scaled model coefficients for the Adult, Employment Rate 2nd Quarter after Exit models.

Adult-ME

Scaled model coefficients for the Adult, Median Earnings 2nd Quarter after Exit models.

DW-Q2ER

Scaled model coefficients for the Dislocated Worker, Employment Rate 2nd Quarter after Exit models.

DW-ME

Scaled model coefficients for the Dislocated Worker, Median Earnings 2nd Quarter after Exit models.

Youth-Q2ER

Scaled model coefficients for the Youth, Employment Rate 2nd Quarter after Exit models.

WP-Q2ER

Scaled model coefficients for the Wagner-Peyser, Employment Rate 2nd Quarter after Exit models.

WP-ME

Scaled model coefficients for the Wagner-Peyser, Median Earnings 2nd Quarter after Exit models.

Summary of WIOA vs. WIA Proxy Models

The comparison of the WIOA and WIA Proxy models show that in many aspects the different model types perform similarly, but the WIA Proxy models give slightly better estimates. This is primarily due to the limited data available under WIOA for the 2nd quarter after exit measures (i.e., only two program years of data) and is resulting in the model over-fitting the data and being overly influenced by the outliers. It also results in the coefficient estimates being less precise.

However, despite these limitations with the WIOA models, it is best to move forward with the models that are fit with only WIOA data. Taking this approach will allow for improvements to be made to the the model specifications (i.e., modify which variables are included in the models) and may improve the performance of the models, but is especially needed for the Youth, Median Earnings 2nd Quarter after Exit model (which deficiencies in available WIA data make that model impossible to fit as currently specified) and the Wagner-Peyser models (which are currently poorly specified). In addition, whenever possible the differences between the models should be limited to reduce the burden and improve the understanding of the models by stakeholders. As a result, in balance the benefits of using only WIOA data outweigh the benefits of using the WIA data to increase the number of available observations.

Refine Current Models

Refine Model Specifications for 2nd Quarter after Exit Outcomes

The upcoming 2020 and 2021 program years will be the first years where the WIOA models will be fully implemented for the 2nd Quarter after Exit outcomes (i.e., Employment Rate and Median Earnings). The Measurable Skill Gains model will also be implemented and is discussed further in the next section. Last section assessed how the model estimates changed when the models were fit with only WIOA data as opposed to including WIA data (which they were for the proxy models being used through PY 2019). This section will assess possible changes to the specifications for the 2nd Quarter after Exit models.

If the determination of which variables to include began from scratch it may be best to take a systematic approach (e.g., step-wise model building) to evaluate which variables should be included in each model. However, for this model development period, the scope is relatively small by only considering modifications to the WIA proxy model that are being used for PY 2018 and PY 2019. The potential changes to the specifications will be limited to responding to changes that have occurred to the definitions of data elements, the data collected for the variables, and new variables added since WIOA implementation.

The section includes subsections which consider:

  • Removing original model variables
  • Adding new variables or modifying existing variables
  • Restricting the data used

Removing Original Model Variables

The goal of this step was to refine the original models defined under WIA as much as possible.

Overall Model Performance

The overall performance of different specified models were assessed for both 2nd Quarter after Exit outcomes (i.e., Employment Rate and Median Earnings) for each program. The same model specifications were used for the Adult, Dislocated Worker, and Wagner-Peyser programs (except for the wp variable which is not in the Wagner-Peyser models) in an attempt to simplify the models as much as possible. The model specifications of the Youth program were slightly different than the other programs due to the difference in regards to which variables apply to that program.

The three metrics used to assess the different models include:

  • Adjusted R-squared - a measure of the degree to which the outcome is explained by the model. Note: the within r-squared (i.e., the amount within a state that the variables explain the outcome) is used here because the overall r-squared is approximately 1 for all models with state fixed effects included.
  • Akaike information criterion (AIC) - a measure to evaluate the trade-offs between over-fitting and under-fitting the models
  • Root-mean-square error (RMSE) - a measure of how predictive the model is while penalizing large errors (outliers)

Three different model specifications were assessed for each program-outcome model. The first model includes the same specifications as what was developed using the WIA data with only small modifications as needed (i.e., the EFL variables removed because that data element changed under WIOA). The second model (i.e., without_sd) removes the variables that are associated with service delivery (i.e., rectraining, recindvidualcs, and recprevoc). The rational for removing those variables, even if they are statistically significant, is because they are not aligned with the purpose of the models to adjust based on things out of the control of the states (i.e., the characteristics of the participants served and the economic conditions within a state) and not account for those elements that the state can control. The edstatexit variable in the Youth models were also removed because it isn’t a participant characteristic at program entry. The third model removes additional variables that are partly related to service delivery or are sparsely reported in addition to those removed in the second models.

Adult
Overall performance of assessed models for the Adult program.
Adult, Employment Rate 2nd Quarter after Exit
Model Adjusted R-squared AIC RMSE
wia_adultdw 0.2751032 -1592.286 0.0254146
wioa_without_sd 0.2447853 -1577.928 0.0260713
wioa_without_allsd 0.2385916 -1576.699 0.0263086
Adult, Median Earnings 2nd Quarter after Exit
Model Adjusted R-squared AIC RMSE
wia_adultdw_me 0.5632051 6090.033 357.4050
wioa_me_without_sd 0.5572866 6093.467 361.6129
wioa_me_without_allsd 0.5510872 6097.042 365.9342
Dislocated Worker
Overall performance of assessed models for the Dislocated Worker program.
Dislocated Worker, Employment Rate 2nd Quarter after Exit
Model Adjusted R-squared AIC RMSE
wia_adultdw 0.1049317 -1387.942 0.0310242
wioa_without_sd 0.0718016 -1375.791 0.0317584
wioa_without_allsd 0.0645250 -1374.754 0.0320477
Dislocated Worker, Median Earnings 2nd Quarter after Exit
Model Adjusted R-squared AIC RMSE
wia_adultdw_me 0.3104516 6571.809 589.8376
wioa_me_without_sd 0.2963792 6578.045 598.7491
wioa_me_without_allsd 0.2992222 6574.348 600.4413
Youth
Overall performance of assessed models for the Youth program.
Youth, Employment Rate 2nd Quarter after Exit
Model Adjusted R-squared AIC RMSE
wia_youth 0.1979713 -1234.144 0.0383671
wioa_y_without_sd 0.2004396 -1237.974 0.0385722
wioa_y_without_allsd 0.2037610 -1240.942 0.0386232
Youth, Median Earnings 2nd Quarter after Exit
Model Adjusted R-squared AIC RMSE
wia_youth 0.2884837 5990.121 322.2224
wioa_y_without_sd 0.2778053 5993.434 326.7884
wioa_y_without_allsd 0.2800030 5990.845 327.3624
Wagner-Peyser
Overall performance of assessed models for the Wagner-Peyser program.
Wagner-Peyser, Employment Rate 2nd Quarter after Exit
Model Adjusted R-squared AIC RMSE
wia_adultdw_wp 0.5094250 -1960.612 0.0163874
wioa_wp_without_sd 0.4906749 -1947.547 0.0167806
wioa_wp_without_allsd 0.4806622 -1941.773 0.0170282
Wagner-Peyser, Median Earnings 2nd Quarter after Exit
Model Adjusted R-squared AIC RMSE
wia_adultdw_wp_me 0.7131032 5613.670 191.9427
wioa_wp_me_without_sd 0.7045601 5623.544 195.7414
wioa_wp_me_without_allsd 0.6946800 5634.817 199.9605
All Programs (Full Table)
Overall performance of assessed models for all program-outcome models.
Program Outcome Model Adjusted R-squared AIC RMSE
Adult Employment Rate 2nd Quarter after Exit wia_adultdw 0.2751032 -1592.286 0.0254146
wioa_without_sd 0.2447853 -1577.928 0.0260713
wioa_without_allsd 0.2385916 -1576.699 0.0263086
Median Earnings 2nd Quarter after Exit wia_adultdw_me 0.5632051 6090.033 357.4050478
wioa_me_without_sd 0.5572866 6093.467 361.6129347
wioa_me_without_allsd 0.5510872 6097.042 365.9342243
Dislocated Worker Employment Rate 2nd Quarter after Exit wia_adultdw 0.1049317 -1387.942 0.0310242
wioa_without_sd 0.0718016 -1375.791 0.0317584
wioa_without_allsd 0.0645250 -1374.754 0.0320477
Median Earnings 2nd Quarter after Exit wia_adultdw_me 0.3104516 6571.809 589.8376337
wioa_me_without_sd 0.2963792 6578.045 598.7491077
wioa_me_without_allsd 0.2992222 6574.348 600.4412713
Youth Employment Rate 2nd Quarter after Exit wia_youth 0.1979713 -1234.144 0.0383671
wioa_y_without_sd 0.2004396 -1237.974 0.0385722
wioa_y_without_allsd 0.2037610 -1240.942 0.0386232
Median Earnings 2nd Quarter after Exit wia_youth 0.2884837 5990.121 322.2224120
wioa_y_without_sd 0.2778053 5993.434 326.7884012
wioa_y_without_allsd 0.2800030 5990.845 327.3623679
Wagner-Peyser Employment Rate 2nd Quarter after Exit wia_adultdw_wp 0.5094250 -1960.612 0.0163874
wioa_wp_without_sd 0.4906749 -1947.547 0.0167806
wioa_wp_without_allsd 0.4806622 -1941.773 0.0170282
Median Earnings 2nd Quarter after Exit wia_adultdw_wp_me 0.7131032 5613.670 191.9427226
wioa_wp_me_without_sd 0.7045601 5623.544 195.7414064
wioa_wp_me_without_allsd 0.6946800 5634.817 199.9605428

The removal of the service delivery variables minimally impacts the performance of the models. It is not consistent in regards to which metrics improve or decrease with the different specified models. In fact, most of the metrics diverge with only small differences. As a result, it is safe to remove the variables directly related to service delivery based on theoretical reasoning discussed above, but keep the additional variables (at this stage).

Multicolleanarity Issues

In addition to re-assessing the theoretical justification of the current variables, the previous tests have revealed other potentially problematic variables. One obvious issue for the models is the high standard errors for some coefficient estimates. This could just be an unavoidable issue resulting from the limited amount of data available to fit the models or it could be a specification issue due to multicollinearity.

To test for collinearity issues in the models, we evaluated the variables based on how correlated they are with other variables and the variance inflation factor (VIF) score. The figures below show those results with the correlation plots filtered to those variables with correlations greater than 0.6 and the VIF tables filtered to VIF scores above 5.

Adult-Q2ER

Variable correlation and VIF measures for the Adult, Employment Rate 2nd Quarter after Exit model.

Adult-ME

Variable correlation and VIF measures for the Adult, Median Earnings 2nd Quarter after Exit model.

DW-Q2ER

Variable correlation and VIF measures for the Dislocated Worker, Employment Rate 2nd Quarter after Exit model.

DW-ME

Variable correlation and VIF measures for the Dislocated Worker, Median Earnings 2nd Quarter after Exit model.

Youth-Q2ER

Variable correlation and VIF measures for the Youth, Employment Rate 2nd Quarter after Exit model.

Youth-ME

Variable correlation and VIF measures for the Youth, Median Earnings 2nd Quarter after Exit model.

WP-Q2ER

Variable correlation and VIF measures for the Wagner-Peyser, Employment Rate 2nd Quarter after Exit models.

WP-ME

Variable correlation and VIF measures for the Wagner-Peyser, Median Earnings 2nd Quarter after Exit model.

This analysis identified several variables that should be removed due to multicollinearity. The estabita variable (Established Individual Training Account) was removed from all models except the Youth models. Those variables had multicollinearity issues together and with other variables already in the model including edstatentry and resuppserv and additionally because estabita was one of the service delivery variables under consideration for removal above. Also, the osy (Out-of-school Youth) and uiexhaustee (UI Exhaustee) variables were removed from the Youth models. While these are both important variables for the Youth models, the effects of these variables are already captured by the edstatentry (Education Status at Entry) and recneeds (Received Needs-related Payments) variable respectively.

Other variables with collinearity problems include the various categorical variables included in the models (i.e., age, race, and education level). Little can be done with the age variables since they are already reasonably grouped and exclude an appropriate group. Likewise, no changes can be made to the race variables. The issue is that the smaller race categories (i.e., asian, hpi, multi, etc.) are highly correlated. It is unsurprising that more diverse states have higher levels of these subgroups, however, these subgroups cannot be eliminated or combined because we want to be able to account for the differences in different states. The situation is similar for the education attainment variables, however, one small modification was made to these variables in the models. A large amount of the variance is a result of the hsdropout variable being a relatively large percentage of the participants. The estimates were improved by dropping the hsdropout variable and instead adding a gradschool variable for the Adult, DW, and WP models and the associateorba variable for the Youth models (these were the categories previously dropped). It also makes the education level variables slightly easier to interpret as the effect of each higher level of attainment with the base being less than a high school graduate.

The industry share category variables also have high VIF scores and large standard errors in the coefficient estimates. However, this is not due to multicollinearity and instead is just a result of the nature of those variables given the limited available data (i.e., limited time frame).

Modifying Existing Variables and Adding New Variables

This section identifies additional variables considered for the specified models. First, some modifications to variables that were previously included in the models were assessed. Then, some new variables were considered for inclusion in the models.

Modified Variable: Days in Program

One the variables that could use some modification is the dayinprog variable. In its basic form, this variable is calculated as the median number of days a participant is in the program (i.e., entry date to exit date or quarter end date). Using the median instead of the average number of days is a change from past models and this change was made to better account for outliers. However, outliers are still a bit of a problem even with the median being used. There are still some observations with very high values for daysinprog and a large percentage of the observations had a zero value. The presence of the zero is complicating factor because the variable includes service delivery characteristics (i.e., how a state records participants).

To account for these issues, the calculation for daysinprog has changed to now be the median of participant with greater than 0 days and less than 1000. The resulting distribution of the variable are shown in the figures below (this variable calculated in the new way is labeled as daysinprog2). The variable now better represents its intended purpose and meaning.

Adult

Dislocated Worker

Youth

Wagner-Peyser

Modified Variable: Educational Functioning Level

The educational functional level (EFL) variables were also assessed. These variables changed from WIA to WIOA and were variables originally included in the Youth models. However, the definition of the EFL variables now include ABE and ESL elements that are not comparable to be included in the same variable.

Separate variables for both the ABE and ESL elements were created, however, as the figure below shows these elements are not being fully reported at a sufficient enough level to use in the models. The red signifies missing data for that element in the Youth data set. The figure on the right also shows that only a low numbers of participants have values for either the ABE or ESL (i.e., the EFL is the combination of those elements). It would not be appropriate to use EFL scores or levels when less than 20% of all participants have data on these elements.

New Variables

Other PIRL elements not included in the WIA models were also considered. In some cases these were new data elements under WIOA and in other cases it may have just been an element that was not consistently reported under WIA and therefor not previously considered. The figures below show the data on the variables tested for each program.

Adult

Characteristics of the new variables considered in the Adult program.

Dislocated Worker

Characteristics of the new variables considered in the Dislocated Worker program.

Youth

Characteristics of the new variables considered in the Youth program.

Wagner-Peyser

Characteristics of the new variables considered in the Wagner-Peyser program.

The graphs above show that some of these variables are not frequently reported. As a result of the limited data, the msfw (Migrant and Seasonal Farm Worker) and culturalbarriers (Cultural Barriers at Program Entry) variables were not included in the updated models. The recsnap (Supplemental Nutrition Assistance Program) variable was also not included because the inclusion of the variable degraded the models’ performance due to multicollinearity issues. The longtermunemp (Long-Term Unemployed at Program Entry), recssi (Received SSI or SSDI), and dishomemaker (Displaced Homemaker) were added to all models (except the dishomemaker is not included in the Youth models) for further testing.

Assessing Model Performance

The different model specifications were tested to determine which models performed the best. At this stage, three models were evaluated for each program indicator model. The first models (with the suffix refined) are the result of the refined basic WIA models after removing the service delivery variables and the variables with collinearity concerns. The second models (with the suffix refined2) are the refined models with the new calculations for the daysinprog variable. Finally, the third models (with the suffix plus) are the refined2 models with the addition of the new variables. The tables with model performance by different metrics for each of the specifications are below.

Adult
Overall performance of assessed models for the Adult program.
Adult, Employment Rate 2nd Quarter after Exit
Model Adjusted R-squared AIC RMSE
wioa_refined 0.2443291 -1579.036 0.0261660
wioa_refined2 0.2440585 -1578.893 0.0261707
wioa_plus 0.2386225 -1574.018 0.0261338
Adult, Median Earnings 2nd Quarter after Exit
Model Adjusted R-squared AIC RMSE
wioa_me_refined 0.5510145 6097.788 365.3654
wioa_me_refined2 0.5583284 6091.169 362.3773
wioa_me_plus 0.5880836 6065.078 348.2325
Dislocated Worker
Overall performance of assessed models for the Dislocated Worker program.
Dislocated Worker, Employment Rate 2nd Quarter after Exit
Model Adjusted R-squared AIC RMSE
wioa_refined 0.0711278 -1376.834 0.0318797
wioa_refined2 0.0710441 -1376.799 0.0318811
wioa_plus 0.0894479 -1382.583 0.0314006
Dislocated Worker, Median Earnings 2nd Quarter after Exit
Model Adjusted R-squared AIC RMSE
wioa_me_refined 0.2931761 6578.539 602.0557
wioa_me_refined2 0.2961037 6576.846 600.8076
wioa_me_plus 0.2963543 6578.733 597.7869
Youth
Overall performance of assessed models for the Youth program.
Youth, Employment Rate 2nd Quarter after Exit
Model Adjusted R-squared AIC RMSE
wioa_y_refined 0.2083116 -1243.183 0.0385126
wioa_y_refined2 0.2058370 -1241.963 0.0385728
wioa_y_plus 0.2087813 -1242.075 0.0383705
Youth, Median Earnings 2nd Quarter after Exit
Model Adjusted R-squared AIC RMSE
wioa_y_refined 0.2883151 5986.177 325.4672
wioa_y_refined2 0.2843003 5988.438 326.3840
wioa_y_plus 0.2969928 5982.609 322.4181
Wagner-Peyser
Overall performance of assessed models for the Wagner-Peyser program.
Wagner-Peyser, Employment Rate 2nd Quarter after Exit
Model Adjusted R-squared AIC RMSE
wioa_wp_refined 0.4720240 -1934.453 0.0171412
wioa_wp_refined2 0.4606693 -1925.900 0.0173246
wioa_wp_plus 0.4812551 -1939.505 0.0169072
Wagner-Peyser, Median Earnings 2nd Quarter after Exit
Model Adjusted R-squared AIC RMSE
wioa_wp_me_refined 0.6942076 5636.131 199.7911
wioa_wp_me_refined2 0.6933761 5637.231 200.0626
wioa_wp_me_plus 0.7026605 5626.814 196.0486
All Programs (Full Table)
Overall performance of assessed models for all program-outcome models.
Program Outcome Model Adjusted R-squared AIC RMSE
Adult Employment Rate 2nd Quarter after Exit wioa_refined 0.2443291 -1579.036 0.0261660
wioa_refined2 0.2440585 -1578.893 0.0261707
wioa_plus 0.2386225 -1574.018 0.0261338
Median Earnings 2nd Quarter after Exit wioa_me_refined 0.5510145 6097.788 365.3653784
wioa_me_refined2 0.5583284 6091.169 362.3772562
wioa_me_plus 0.5880836 6065.078 348.2325177
Dislocated Worker Employment Rate 2nd Quarter after Exit wioa_refined 0.0711278 -1376.834 0.0318797
wioa_refined2 0.0710441 -1376.799 0.0318811
wioa_plus 0.0894479 -1382.583 0.0314006
Median Earnings 2nd Quarter after Exit wioa_me_refined 0.2931761 6578.539 602.0556713
wioa_me_refined2 0.2961037 6576.846 600.8075503
wioa_me_plus 0.2963543 6578.733 597.7868740
Youth Employment Rate 2nd Quarter after Exit wioa_y_refined 0.2083116 -1243.183 0.0385126
wioa_y_refined2 0.2058370 -1241.963 0.0385728
wioa_y_plus 0.2087813 -1242.075 0.0383705
Median Earnings 2nd Quarter after Exit wioa_y_refined 0.2883151 5986.177 325.4672432
wioa_y_refined2 0.2843003 5988.438 326.3839737
wioa_y_plus 0.2969928 5982.609 322.4180717
Wagner-Peyser Employment Rate 2nd Quarter after Exit wioa_wp_refined 0.4720240 -1934.453 0.0171412
wioa_wp_refined2 0.4606693 -1925.900 0.0173246
wioa_wp_plus 0.4812551 -1939.505 0.0169072
Median Earnings 2nd Quarter after Exit wioa_wp_me_refined 0.6942076 5636.131 199.7911177
wioa_wp_me_refined2 0.6933761 5637.231 200.0625629
wioa_wp_me_plus 0.7026605 5626.814 196.0485700

Overall, the plus models performed the best under this criteria. Making the modifications to the daysinprog variable results in a small improvement in most of the models without negatively impacting any model. The addition of the new variables results in a noticeable improvement in a few of the models (particularly the median earnings models) without affecting the other models significantly.

Restricting the Data Used

Outlier data can have a significant impact on the model estimates. Each model was assessed to determine the degree to which there are extreme outliers in the data and/or the degree to which an individual observation is overly influential. The plots below show this analysis for each model and highlight the observations which may be problematic.

Adult-Q2ER

Model plots for the fitted Adult, Employment Rate 2nd Quarter after Exit model.

Adult-ME

Model plots for the fitted Adult, Median Earnings 2nd Quarter after Exit model.

DW-Q2ER

Model plots for the fitted Dislocated Worker, Employment Rate 2nd Quarter after Exit model.

DW-ME

Model plots for the fitted Dislocated Worker, Median Earnings 2nd Quarter after Exit model.

Youth-Q2ER

Model plots for the fitted Youth, Employment Rate 2nd Quarter after Exit model.

Youth-ME

Model plots for the fitted Youth, Median Earnings 2nd Quarter after Exit model.

WP-Q2ER

Model plots for the fitted Wagner-Peyser, Employment Rate 2nd Quarter after Exit model.

WP-ME

Model plots for the fitted Wagner-Peyser, Median Earnings 2nd Quarter after Exit model.

This analysis revealed that there were some observations that were causing issues. A few observations in some of the models were removed after further examination revealed inconsistent data for one or more of the model variables for an observation.

There were also some observations with outlier reported outcome data. Placing thresholds on the minimal and maximum level of outcome data (e.g., drop all observations with a Q2ER level under 0.50) were considered and tested. Placing these type of restrictions resulted in a few outlier observations being dropped in each model and improved overall performance of the models. One downside is that the resulting fit models end up giving predictions for a few states that were higher than some past reported outcome levels (e.g., a state that had reported Q2ER outcomes in the 40%-60% range would result in a prediction close to 70%). However, the predicted levels of performance, derived from theses statistical adjustment models, are only one factor in negotiations. Therefore, the negotiated levels of performance, agreed to by ETA and the state, may be different from the model predictions. Using this approach will result in more reliable post-PY adjustments.

The thresholds for the outcome levels of the observations are as follows:

  • Q2ER minimum
    • Adult - 0.6
    • Dislocated Worker - 0.6
    • Wagner-Peyser - 0.45
  • ME minimum
    • Adult - 4000
    • Youth - 1700
  • ME maximum
  • Adult - 9500
  • Youth - 7000
  • Wagner-Peyser - 9000

Comparison of Model Predictions

With the final model specifications and final data used to fit the model determined, the models were tested by observing how well they predict performance. The plots below show how well the new model predicts PY 2018 performance compared to the old model which was originally specified using WIA data. Overall, the predictions of the models for PY 2018 are similar with the most significant difference being that some of the lower-bound outliers had higher predictions in the new model.

Adult-Q2ER

Plot of the PY 2018 prediction results for the Adult, Employment Rate 2nd Quarter after Exit models.

Adult-ME

Plot of the PY 2018 prediction results for the Adult, Median Earnings 2nd Quarter after Exit models.

DW-Q2ER

Plot of the PY 2018 prediction results for the Dislocated Worker, Employment Rate 2nd Quarter after Exit models.

DW-ME

Plot of the PY 2018 prediction results for the Dislocated Worker, Median Earnings 2nd Quarter after Exit models.

Youth-Q2ER

Plot of the PY 2018 prediction results for the Youth, Employment Rate 2nd Quarter after Exit models.

Youth-ME

Plot of the PY 2018 prediction results for the Youth, Median Earnings 2nd Quarter after Exit models.

WP-Q2ER

Plot of the PY 2018 prediction results for the Wagner-Peyser, Employment Rate 2nd Quarter after Exit models.

WP-ME

Plot of the PY 2018 prediction results for the Wagner-Peyser, Median Earnings 2nd Quarter after Exit models.

Model Coefficients

Finally, the coefficient estimates were evaluated to see how much they changed due to the changes to the models. The figures below compare the coefficient estimates of the original models to those of the newly specified models. In all models, the coefficients stabilized for the participant characteristic variables with the new model. There was little change in the economic condition variables. Unfortunately, this is just the result of the limited amount of WIOA data available and there is little that can be done as they are required variables. However, the economic condition variable estimates should improve in future years once additional years of data are available.

Adult-Q2ER

Scaled model coefficients for the Adult, Employment Rate 2nd Quarter after Exit models.

Adult-ME

Scaled model coefficients for the Adult, Median Earnings 2nd Quarter after Exit models.

DW-Q2ER

Scaled model coefficients for the Dislocated Worker, Employment Rate 2nd Quarter after Exit models.

DW-ME

Scaled model coefficients for the Dislocated Worker, Median Earnings 2nd Quarter after Exit models.

Youth-Q2ER

Scaled model coefficients for the Youth, Employment Rate 2nd Quarter after Exit models.

Youth-ME

Scaled model coefficients for the Youth, Median Earnings 2nd Quarter after Exit models.

WP-Q2ER

Scaled model coefficients for the Wagner-Peyser, Employment Rate 2nd Quarter after Exit models.

WP-ME

Scaled model coefficients for the Wagner-Peyser, Median Earnings 2nd Quarter after Exit models.

MSG Models

Measurable Skill Gains (MSG) Model Development

The MSG performance indicator was first implemented under WIOA and collection for this indicator began in PY 2016.

This section includes subsections which discuss:

  • Modeling challenges specific to MSG
  • Data challenges and how the reported MSG data affects modeling
  • The determination of which model specifications to use
  • Assessment of the overall performance of the MSG models
  • Resulting model predictability and estimates

How is the MSG outcome different?

MSG is different than the other performance indicators in that it is not exit based. This is an important distinction for the modeling process for two reasons:

First, individual participants aren’t in a discreet quarter and PY. Instead, an individual participant can be included in multiple quarters and PY’s. As long as the participant is still in a training or education program they remain in the denominator. As a result, a defined state observation (e.g., a state-quarter) only partially consists of different participants than the nearest neighbor observations (i.e., the state observation for the quarter before and after).

Second, the annual MSG rates are calculated with each participant who meets the definition only being in the denominator and numerator once. As a result, treating each quarter as independent will result in MSG rates that are not comparable to the annual rates since the same participant will be in the denominator for multiple quarters, but may not have a MSG gain (i.e., a numerator count) during all those quarters.

The 2nd Quarter after Exit models were fit with state-quarter observations since it was necessary to use state-quarters in order to increase the number of observations and get better model estimates. This was possible for the exit outcomes since the quarterly rates are comparable to the annual rates. If it is not possible to get MSG rates that are comparable to the annual rates and, as a result, annual-state observations have to be used it would severely reduce the number of observations available to fit the MSG models.

Difference between quarterly MSG rates and annual MSG rates

Due to the reasons discussed above, treating an MSG quarter as a discreet observation results in MSG rates that are much lower than the annual rates. In an attempt to eliminate (or at least reduce) this difference, the quarterly MSG rates used in this analysis were calculated differently. In the new calculation, the MSG rate is calculated by including all participants in training or education in a quarter in the denominator and all participants in the denominator who had an MSG gain in that quarter or the previous 3 quarters in the numerator. This fixed-denominator and rolling-numerator is more comparable to the annual rates, however, it is still considerably lower than the annual rates on average.

Another alternative is to calculate the quarterly observations as rolling-4 quarters. Using this approach, the denominator and numerator are both calculated by including in the observation each participant who meets the respective criteria in a given quarter and the previous 3 quarters. This approach contrasts with the approach above in that it also expands the scope of who is included in the denominator. While using rolling-4 quarters results in MSG rates that are more comparable to the annual rates, it has the downside of creating observations that are less independent since approximately 75% of the data that make up an observation will be same to the nearest neighbor observations (i.e., the previous or next state quarter).

The plots below show the difference in the MSG rates depending on the time period selected and how it is calculated. In the first plot, it is clear that the rolling-4 quarter observations are distributed more closely to the annual observations. The second plot shows how the average of the observations for each period type for each state compare. Clearly the rolling-4 quarters are still lower than the annual rates on average, but they are closer than the quarterly observations calculated with a fixed denominator.

Distribution of MSG Rates for Different Time Period Types

Relationship of MSGS Rates by Period Type

Given how the average quarterly MSG rate (even when using 4 quarters for the numerator) is lower than the average annual MSG rate, the quarterly MSG data is not ideal as a perfect replacement for the annual rates. Using the quarterly data to fit the models will result in predictions that are at least slightly lower than the average annual rates.

All three methods of calculating MSG rates will be used in the rest of this section. To distinguish between the different quarter calculations, the rolling-numerator and fixed denominator method will just be referred to as a fixed-quarter and the rolling-4 method will be referred to as a rolling-quarter.

MSG Data Challenges

The MSG outcome data also has a linear trend of significant increases in the reported values over time. This is not surprising given that this is a new performance indicator for states, so it took states time to improve their ability to collect and report data on this indicator.

MSG Rates over Time

The plots below show the trend over time and how the trend is less obvious if you remove the PY 2016 data.

PY 2016-2018 MSG Rates

PY 2017-2018 MSG Rates

Given the significant MSG data issues in PY 2016, it was determined that PY 2016 data should be removed from all the analysis that follows. While the available data is already limited, including PY 2016 only added problems.

Additional MSG Data Challenges

The trend of MSG rate over time is not only a problem in regards to establishing predicted performance. It is also a problem in regards to getting good estimates on the effects of variables that will be included in the model. Without reliable outcome data, it is difficult to determine if effects detected are just the result of spurious correlations due to data reporting improvements.

The plots below show a simple regression of the unemployment rate on the MSG outcome. The unemployment rate was the variable chosen because it has an obvious inverse relationship with how the MSG rates have been reported over time. While the MSG rates have been steadily improving (primarily due to data collection and reporting improvements), the unemployment rates for the states have been continuing the decade long decline as the economy has continued to improve following the Great Recession. As a result, this simple regression shows an effect that does not match theoretical expectations. The second plot shows how much this is driven by the data trends with a large decrease in the effect when PY 2016 data is excluded.

UR Effect with PY 2016 Data (Adult)

UR Effect without PY 2016 Data (Adult)

Unfortunately, there is little that can be done to counteract this concern–it is just a result of the quantity, quality, and short-time frame of available data at this point. It does suggest that the data used to fit the model needs to be closely examined so these type of effects can be reduced as much as possible.

Determination of MSG Model Specifications

A step-wise selection using AIC selection criteria was used to determine which subset of variables resulted in the best performing model. Taking this approach provided a starting point to determine the chosen model specifications. The step-wise selection was conducted for each program model using the fixed-quarter data. The results from this process were used as the baseline model for each program MSG model.

The best performing models resulting from the step-wise procedures couldn’t be used as the final models without modification because, at a minimum, only some of variables of the categorical groups were selected in each program model. As a result, the models selected were refined by filling them out with all the variables for each category group. This specification of the MSG model was the second type evaluated for each program. Finally, a third specification of the model was also tested which sought to make the different models more similar by making the variables consistent across the programs where appropriate.

Overall Model Performance

The overall performance of the different specified models for each program based on the step-wise results were assessed. The tables below show the performance metrics of the different specified models. Each program tab has two tables: One for the specified models fit with the fixed-quarter data and one with the models fit with the rolling-quarter data.

Adult
Adult, Fixed-Quarters
Model Adjusted R-squared AIC RMSE
msg_adult_step 0.3978707 -870.1829 0.0784769
msg_adult_step_refined 0.4000363 -864.2039 0.0772926
msg_full 0.4175262 -875.6161 0.0758466
Adult, Rolling-Quarters
Model Adjusted R-squared AIC RMSE
msg_adult_step 0.6496709 -1018.099 0.0675887
msg_adult_step_refined 0.6552582 -1017.910 0.0661642
msg_full 0.6565872 -1017.471 0.0657697
Dislocated Worker
Dislocated Worker, Fixed-Quarters
Model Adjusted R-squared AIC RMSE
msg_dw_step 0.2641378 -781.6881 0.0854047
msg_dw_step_refined 0.2865711 -789.6398 0.0831861
msg_full 0.2963273 -791.4465 0.0819335
Dislocated Worker, Rolling-Quarters
Model Adjusted R-squared AIC RMSE
msg_dw_step 0.6164557 -971.4091 0.0714088
msg_dw_step_refined 0.6226749 -972.8382 0.0700858
msg_full 0.6371084 -986.5098 0.0681816
Youth
Youth, Fixed-Quarters
Model Adjusted R-squared AIC RMSE
msg_y_step 0.3035358 -1015.804 0.0795030
msg_y_step_refined 0.3005403 -1008.659 0.0791396
msg_y_full 0.3336144 -1032.731 0.0770713
Youth, Rolling-Quarters
Model Adjusted R-squared AIC RMSE
msg_y_step 0.4305718 -1155.388 0.0719624
msg_y_step_refined 0.4450060 -1164.451 0.0705826
msg_y_full 0.4461222 -1163.933 0.0703572

The results show that there is some variance in the metrics assessed. Overall, adding more variables to the step-wise selected models resulted in a higher AIC (which is expected since the step-wise criteria was based on finding the specified model with the lowest AIC) which is a worse performing model, but higher Adjusted R-squared and lower RMSE scores which is a better performing model. It is safe to assume that the models are not significantly impacted by adding the additional variables to make the category groups whole and make the program models more consistent.

Restricting the MSG Data Used to Fit the Models

With the model specifications chosen, the next step was to further refine the data used to fit the models. As previously discussed, the PY 2016 data was removed due to data quality concerns. However, even with the removal of the PY 2016 data, there were still a significant number of observations with data issues that were impacting model performance and/or overly influential in determining model estimates. The model plots below show the observation data for each program model fit with both fixed-quarter and rolling-quarter data.

Adult (Fixed)

Model plots for the fitted Adult MSG model fit with Fixed-quarter data.

Adult (Rolling)

Model plots for the fitted Adult MSG model fit with Rolling-quarter data.

DW (Fixed)

Model plots for the fitted Dislocated Worker MSG model fit with Fixed-quarter data.

DW (Rolling)

Model plots for the fitted Dislocated Worker MSG model fit with Rolling-quarter data.

Youth (Fixed)

Model plots for the fitted Youth MSG model fit with Fixed-quarter data.

Youth (Rolling)

Model plots for the fitted Youth MSG model fit with Rolling-quarter data.

After a detailed examination of the data used for the models, it was clear that there are a number of issues with the data that needed to be corrected. First, there were still some observations with very low MSG rates (some at zero) in PY 2017 and PY 2018. Second, a number of states had some observations with very low counts in the MSG denominator. In addition to this affecting the MSG rates, it is also problematic because many of the participant characteristic variables are the proportion of participants (i.e., the denominator) that meet a particular criteria. The low counts result in a lot of variance for those variables with small changes causing big swings.

To reduce the impact of these data issues, the data was restricted to include only observations with an MSG higher than 10% and an MSG denominator count above 100 (denominator count about 50 in the DW program). The largest impact of making this fix is that it was not possible to generate estimates for one state in some of the models, however, a solution to this issue is discussed below.

There were also issues with some variables in some of the models for some states. As a result, the recneeds (Received Need-related Payments) variable was removed from all models, the hpi (Race: Hawaiian or Pacific Islander) was removed from all models, and the daysinprog (Days in Program) variable was removed from the Youth model. The hpi variable was causing issues because it is reported at very low levels for all states except Hawaii. The daysinprog variable was highly correlated with the daysenrolled (Days Enrolled in Training or Education) in the youth program but not the other programs.

MSG Model Predictions by Period Type

The final models were tested against the actual reported data for PY 2018 to see how well the models predict those results. The plots of the predicted MSG rates compared to the actual MSG rates in PY 2018 are shown below for each program. There is a plot for each model fit with fixed-quarter data and one for each fit with rolling-quarter data. The solid black line is where the predictions would fall if they were exact and perfectly predicted the actual values. The blue dashed line is the linear fit of the model predictions from the model(s).

As expected, rolling-quarter data needs to be used because the fixed-quarter data just results in low predicted values. The models fit with the rolling-quarter data clearly predict the actual outcomes better as the blue line shifts closer to the black line. It is also generally good that the blue line is above the black line for the lower values as a better model should predict higher values than the unusually low values actually reported by some states in PY 2018.

Adult

Plot of the PY 2018 prediction results for the Adult MSG models.

Dislocated Worker

Plot of the PY 2018 prediction results for the Dislocated Worker MSG models.

Youth

Plot of the PY 2018 prediction results for the Youth MSG models.

The variance in the residuals is higher than what would be ideal. However, this is primarily the result of a large variance in the data and the low number of observations. In future years this will improve with more quality data. Attempts to reduce the variance by placing additional restrictions on the data were not worth the trade-offs as those efforts resulted in more extreme predictions for some states. Plus, the minimum restrictions on the data used to fit the models that were employed (i.e., MSG rate about 10% and MSG denominator counts above 100) already resulted in the models being unable to generate estimates for one state in the Adult and Dislocated Worker programs. Additional restrictions would have increased that number. In fact, the restriction on the minimum denominator count was reduced to 50 for the Dislocated Worker program because the higher threshold made it impossible to generate estimates for 5 states in that model. The trade-off of being unable to generate estimates for 5 states was greater than the improvement in overall model performance.

In the case where fixed effects estimates for a state could not be generated in a particular model, the model will still be used to provide predicted levels of performance and to assess performance. These values will be determined by using the model estimates, the particular state’s actual participant and economic data, and the state fixed effect that is closest to zero for the model.

MSG Model Coefficients

Plots of the coefficients for each program model, fit with both the fixed-quarter and rolling-quarter data, are shown below. Overall, the estimates seem reasonable and similar for both fixed-quarter and rolling-quarter fit models. The biggest concern is the high variance of the economic factor coefficients, however, this was an expected result given the brevity of data and is similar to the situation in the 2nd Quarter after Exit models.

Adult

Scaled MSG model coefficients for the Adult program.

Dislocated Worker

Scaled MSG model coefficients for the Dislocated Worker program.

Youth

Scaled MSG model coefficients for the Youth program.

Final Model Estimates

The tables below are the final model estimates for all models. The model estimates are combined by program and each program tab includes a statistical model table and a condensed full table with all the estimates.

Final Model Estimates

Adult

Final model coefficient estimates for all performance indicator models in the Adult program.

Statistical Table of Model Estimates
  Adult - Q2ER Adult - ME Adult - MSG
Coefficient Estimates Estimates Estimates
female 0.130185 ** -2841.066829 *** -0.361395
age2544 -0.030832 -862.093012 0.521779 *
age4554 -0.154534 -3144.208889 ** -0.070214
age5559 -0.077898 -5290.321561 ** -0.997461
age60 -0.599337 *** -6059.006213 ** 2.921663 *
hispanic 0.081516 232.425400 -1.445551 **
raceasian -0.233320 * -4413.857765 *** 2.131024 ***
raceblack 0.086113 -2324.859301 *** -0.567085 *
racehpi -0.132045 -6352.731998 ***
raceai 0.050076 -2692.432592 0.851962
racemulti -0.118292 6983.794471 *** 1.975918
hsgrad -0.125661 * 362.021729 -0.121827
collegedropout -0.122101 * 826.890232 -0.184205
certotherps 0.097792 -1324.304957 0.073085
associate -0.077305 5643.185267 *** -0.854405
ba 0.172173 4052.079709 *** -0.200492
gradschool -0.143026 8539.936486 *** 1.838685
empentry 0.196416 *** 965.080078 * 0.364707 *
edstatentry 0.126544 3623.201217 *** -0.304498 *
disabled -0.181335 * -989.223683 -0.261099
veteran 0.290099 * -1349.308924 0.250823
englearner -0.030616 -4419.892216 *** 0.881011
singleparent -0.094244 145.762987 0.213482
lowinc 0.008103 -332.406681
homeless -0.053384 -446.426195
offender 0.154999 * 2013.303091 * 0.780876 *
dishomemaker -0.184172 -1947.918453
recwages2qprior 0.188944 *** 807.624555 -0.001255
longtermunemp 0.015713 2011.822653 *** 0.065221
uiclaimant -0.014782 685.889125
uiexhaustee 0.139437 -2567.350384 * 0.010359
recsuppserv 0.062045 912.913762 * -0.129651
recneeds 0.488581 15112.528861 *
recotherasst -0.049419 * 107.529877
recssi -0.020526 -5911.850988 ** 0.492933
rectanf 0.043761 840.864110 -0.284812
wp 0.022043 -205.492791 0.073185
daysinprog -0.000183 *** 3.248927 *** 0.000355
natresources 2.126552 24063.844354 10.015508 *
construction 0.861537 32326.493768 * 8.928656 *
manufacturing 0.189725 39237.262531 *** 12.123992 ***
information -5.331239 -48189.256506 -43.831254 *
financial -4.866359 4074.290117 31.723399 **
business 3.757520 * 96754.448401 *** 7.575823
edhealthcare 0.823523 56163.154748 *** 9.928560 **
leisure -0.892258 57668.001059 *** 2.581261
otheremp 4.527386 10767.793542 32.868475 *
publicadmin 2.214851 39658.638787 * -0.143088
ur 0.682199 -6106.382702 -13.553496 ***
factor(state_id)AK -0.086946 -32367.989182 *** -5.233818 *
factor(state_id)AL -0.013178 -35318.625497 *** -6.243113 *
factor(state_id)AR 0.003793 -36413.251660 *** -6.152248 *
factor(state_id)AZ 0.030568 -35794.335723 *** -6.338014 *
factor(state_id)CA -0.054998 -35005.499238 *** -5.302395 *
factor(state_id)CO -0.006261 -35664.993742 *** -5.984425 *
factor(state_id)CT 0.152489 -34836.156582 *** -7.182500 *
factor(state_id)DC -1.024650 -41019.123387 *** -5.728475
factor(state_id)DE 0.284043 -35261.347846 *** -8.404191 **
factor(state_id)FL 0.019789 -35518.564479 *** -6.137751 *
factor(state_id)GA 0.050946 -33977.061260 *** -5.331660 *
factor(state_id)HI 0.116767 -35784.457779 *** -7.589716 **
factor(state_id)IA 0.231409 -30554.478015 *** -6.967691 *
factor(state_id)ID 0.033907 -34894.290617 *** -6.753516 **
factor(state_id)IL 0.062813 -34259.817329 *** -6.707190 *
factor(state_id)IN 0.082654 -34399.512037 *** -6.894756 **
factor(state_id)KS 0.025044 -35066.863296 *** -6.736759 *
factor(state_id)KY -0.002993 -35027.134595 *** -6.567522 *
factor(state_id)LA -0.053157 -33698.053814 *** -5.796540 *
factor(state_id)MA 0.072857 -36609.279273 *** -6.648469 *
factor(state_id)MD -0.246789 -37688.282109 *** -6.277424 *
factor(state_id)ME 0.094147 -35071.625154 *** -7.004326 **
factor(state_id)MI 0.029523 -36047.982225 *** -7.028562 **
factor(state_id)MN 0.091833 -34187.982219 *** -7.159454 **
factor(state_id)MO 0.072022 -35727.201890 *** -6.552094 *
factor(state_id)MS 0.126536 -33531.244588 *** -5.698464 *
factor(state_id)MT -0.012095 -34202.671561 *** -6.250535 *
factor(state_id)NC -0.042217 -35931.686645 *** -6.081801 *
factor(state_id)ND 0.049953 -29321.145830 *** -6.759703 **
factor(state_id)NE 0.119913 -33901.301466 *** -7.007566 **
factor(state_id)NH 0.163777 -33153.628387 *** -6.300129 *
factor(state_id)NJ -0.150775 -35129.817597 *** -6.188207 *
factor(state_id)NM -0.255770 -37537.754235 *** -5.186512 *
factor(state_id)NV 0.118321 -38025.993895 *** -5.204861 *
factor(state_id)NY 0.051754 -35142.017728 *** -6.738470 *
factor(state_id)OH 0.136464 -35324.431302 *** -6.583216 *
factor(state_id)OK -0.070585 -35569.705696 *** -6.189815 *
factor(state_id)OR -0.048197 -35003.850755 *** -6.888634 *
factor(state_id)PA 0.011066 -34687.748020 *** -7.036487 **
factor(state_id)RI 0.053566 -37119.140621 *** -7.470101 **
factor(state_id)SC -0.030764 -35586.111072 *** -6.192289 *
factor(state_id)SD 0.293089 -31581.497204 *** -7.472990 **
factor(state_id)TN 0.043064 -34811.163540 *** -6.158717 *
factor(state_id)TX -0.003894 -34471.929344 *** -6.245256 *
factor(state_id)UT 0.047697 -35030.767126 *** -6.160372 *
factor(state_id)VA -0.284129 -38750.791817 *** -6.395937 *
factor(state_id)VT 0.093728 -35886.042243 *** -6.810016 **
factor(state_id)WA 0.096416 -31323.317851 ***
factor(state_id)WI 0.119407 -34142.727667 *** -7.188241 **
factor(state_id)WV -0.115471 -34762.834421 *** -6.221128 *
factor(state_id)WY 0.016736 -32458.715924 *** -5.474147 *
wages2qprior 0.365281 ***
daysenrolled -0.000244 **
daysenrolled_under30 -0.008693
Observations 399 403 403
R2 / R2 adjusted 0.999 / 0.998 0.997 / 0.996 0.977 / 0.970
  • p<0.05   ** p<0.01   *** p<0.001
Condensed Table of Model Estimates
Employment Rate 2nd Quarter after Exit
Median Earnings 2nd Quarter after Exit
Measurable Skill Gains
Model Coefficients
State Fixed Effects
Model Coefficients
State Fixed Effects
Model Coefficients
State Fixed Effects
Variable Estimate State Fixed.Effect Variable Estimate State Fixed.Effect Variable Estimate State Fixed.Effect
female 0.1301854612 AK -0.086946 female -2841.0668290 AK -32367.99 female -0.3613951778 AK -5.233818
age2544 -0.0308320972 AL -0.013178 age2544 -862.0930118 AL -35318.63 age2544 0.5217790469 AL -6.243113
age4554 -0.1545339948 AR 0.003793 age4554 -3144.2088889 AR -36413.25 age4554 -0.0702137617 AR -6.152248
age5559 -0.0778982189 AZ 0.030568 age5559 -5290.3215612 AZ -35794.34 age5559 -0.9974608798 AZ -6.338014
age60 -0.5993371457 CA -0.054998 age60 -6059.0062129 CA -35005.50 age60 2.9216631919 CA -5.302395
hispanic 0.0815159795 CO -0.006261 hispanic 232.4254005 CO -35664.99 hispanic -1.4455511356 CO -5.984425
raceasian -0.2333197600 CT 0.152489 raceasian -4413.8577655 CT -34836.16 raceasian 2.1310241545 CT -7.182500
raceblack 0.0861127041 DC -1.024650 raceblack -2324.8593012 DC -41019.12 raceblack -0.5670845229 DC -5.728475
racehpi -0.1320450806 DE 0.284043 racehpi -6352.7319982 DE -35261.35 raceai 0.8519623159 DE -8.404191
raceai 0.0500758118 FL 0.019789 raceai -2692.4325920 FL -35518.56 racemulti 1.9759178873 FL -6.137751
racemulti -0.1182919095 GA 0.050946 racemulti 6983.7944710 GA -33977.06 hsgrad -0.1218266290 GA -5.331660
hsgrad -0.1256612564 HI 0.116767 hsgrad 362.0217289 HI -35784.46 collegedropout -0.1842046933 HI -7.589716
collegedropout -0.1221006420 IA 0.231409 collegedropout 826.8902324 IA -30554.48 certotherps 0.0730846244 IA -6.967691
certotherps 0.0977921347 ID 0.033907 certotherps -1324.3049571 ID -34894.29 associate -0.8544051098 ID -6.753516
associate -0.0773050296 IL 0.062813 associate 5643.1852667 IL -34259.82 ba -0.2004916133 IL -6.707190
ba 0.1721732169 IN 0.082654 ba 4052.0797092 IN -34399.51 gradschool 1.8386849236 IN -6.894756
gradschool -0.1430257271 KS 0.025044 gradschool 8539.9364862 KS -35066.86 empentry 0.3647069923 KS -6.736759
empentry 0.1964164066 KY -0.002993 empentry 965.0800780 KY -35027.13 edstatentry -0.3044984079 KY -6.567522
edstatentry 0.1265440825 LA -0.053157 edstatentry 3623.2012166 LA -33698.05 disabled -0.2610991873 LA -5.796540
disabled -0.1813346748 MA 0.072857 disabled -989.2236831 MA -36609.28 veteran 0.2508229156 MA -6.648469
veteran 0.2900985046 MD -0.246789 veteran -1349.3089242 MD -37688.28 englearner 0.8810110159 MD -6.277424
englearner -0.0306155425 ME 0.094147 englearner -4419.8922163 ME -35071.63 singleparent 0.2134819507 ME -7.004326
singleparent -0.0942438927 MI 0.029523 singleparent 145.7629872 MI -36047.98 offender 0.7808762839 MI -7.028562
lowinc 0.0081029046 MN 0.091833 lowinc -332.4066813 MN -34187.98 recwages2qprior -0.0012554715 MN -7.159454
homeless -0.0533836309 MO 0.072022 homeless -446.4261948 MO -35727.20 longtermunemp 0.0652205888 MO -6.552094
offender 0.1549994736 MS 0.126536 offender 2013.3030910 MS -33531.24 uiexhaustee 0.0103585869 MS -5.698464
dishomemaker -0.1841722221 MT -0.012095 dishomemaker -1947.9184526 MT -34202.67 recsuppserv -0.1296512548 MT -6.250535
recwages2qprior 0.1889437789 NC -0.042217 recwages2qprior 807.6245553 NC -35931.69 recssi 0.4929333729 NC -6.081801
longtermunemp 0.0157127638 ND 0.049953 wages2qprior 0.3652811 ND -29321.15 rectanf -0.2848115800 ND -6.759703
uiclaimant -0.0147822729 NE 0.119913 longtermunemp 2011.8226527 NE -33901.30 wp 0.0731854412 NE -7.007566
uiexhaustee 0.1394365795 NH 0.163777 uiclaimant 685.8891250 NH -33153.63 daysenrolled -0.0002438943 NH -6.300129
recsuppserv 0.0620452606 NJ -0.150775 uiexhaustee -2567.3503837 NJ -35129.82 daysenrolled_under30 -0.0086930493 NJ -6.188207
recneeds 0.4885811671 NM -0.255770 recsuppserv 912.9137620 NM -37537.75 daysinprog 0.0003546398 NM -5.186512
recotherasst -0.0494189936 NV 0.118321 recneeds 15112.5288608 NV -38025.99 natresources 10.0155075639 NV -5.204861
recssi -0.0205263477 NY 0.051754 recotherasst 107.5298771 NY -35142.02 construction 8.9286564094 NY -6.738470
rectanf 0.0437611488 OH 0.136464 recssi -5911.8509885 OH -35324.43 manufacturing 12.1239920647 OH -6.583216
wp 0.0220432421 OK -0.070585 rectanf 840.8641098 OK -35569.71 information -43.8312538939 OK -6.189815
daysinprog -0.0001830449 OR -0.048197 wp -205.4927906 OR -35003.85 financial 31.7233990782 OR -6.888634
natresources 2.1265518891 PA 0.011066 daysinprog 3.2489272 PA -34687.75 business 7.5758232969 PA -7.036487
construction 0.8615369971 RI 0.053566 natresources 24063.8443545 RI -37119.14 edhealthcare 9.9285604785 RI -7.470101
manufacturing 0.1897251830 SC -0.030764 construction 32326.4937683 SC -35586.11 leisure 2.5812606605 SC -6.192289
information -5.3312391420 SD 0.293089 manufacturing 39237.2625312 SD -31581.50 otheremp 32.8684751740 SD -7.472990
financial -4.8663587200 TN 0.043064 information -48189.2565064 TN -34811.16 publicadmin -0.1430876383 TN -6.158717
business 3.7575197029 TX -0.003894 financial 4074.2901169 TX -34471.93 ur -13.5534964996 TX -6.245256
edhealthcare 0.8235226171 UT 0.047697 business 96754.4484010 UT -35030.77 UT -6.160372
leisure -0.8922582707 VA -0.284129 edhealthcare 56163.1547482 VA -38750.79 VA -6.395937
otheremp 4.5273863467 VT 0.093728 leisure 57668.0010588 VT -35886.04 VT -6.810016
publicadmin 2.2148506684 WA 0.096416 otheremp 10767.7935418 WA -31323.32 WI -7.188241
ur 0.6821987086 WI 0.119407 publicadmin 39658.6387868 WI -34142.73 WV -6.221128
WV -0.115471 ur -6106.3827023 WV -34762.83 WY -5.474147
WY 0.016736 WY -32458.72 WA* -5.186512
* In the MSG model, the state fixed effect applied to WA is the state fixed effect closest to zero. An estimate could not be calculated for this state in the MSG model because there were not sufficient usable observations.

Dislocated Worker

Final model coefficient estimates for all performance indicator models in the Dislocated Worker program.

Statistical Table of Model Estimates
  DW - Q2ER DW - ME DW - MSG
Coefficient Estimates Estimates Estimates
female 0.059589 -1901.341529 ** -0.150912
age2544 0.018934 1115.515382 -0.065496
age4554 -0.016926 -125.987305 0.241436
age5559 0.105989 -2126.378454 0.457897
age60 -0.190542 -2492.931194 0.913940
hispanic 0.118518 -857.755025 -0.664582
raceasian -0.290982 * -4684.971253 * -0.533993
raceblack -0.035840 -1536.602735 -0.329320
racehpi 0.879207 ** -3269.175345
raceai -0.098328 -3522.213767 2.646459 ***
racemulti -0.194744 -3712.059377 0.050258
hsgrad -0.025949 -1400.096972 -0.192154
collegedropout -0.194218 ** -1902.904768 -0.238369
certotherps -0.179935 83.415133 -0.260748
associate -0.090731 1526.240169 0.262508
ba -0.144749 * 1169.417875 -0.112254
gradschool -0.120999 2155.049710 -0.427505
empentry 0.106102 1700.779426 -0.021509
edstatentry -0.025436 3787.510280 * -0.264344
disabled -0.052683 279.593052 -1.448056 *
veteran 0.005588 1445.834444 -0.923471 *
englearner -0.252089 * -2976.132771 0.451362
singleparent 0.044648 -784.434832 0.369141
lowinc -0.051809 -538.709660
homeless 0.030619 7893.825015 *
offender 0.377490 ** 1805.978310 0.469289
dishomemaker -0.227365 192.756366
recwages2qprior 0.113108 * 21.081713 0.028672
longtermunemp 0.057435 1348.368234 0.388125 *
uiclaimant 0.020818 68.696206
uiexhaustee 0.073688 -2493.013164 * 0.356100
recsuppserv 0.049649 176.162786 -0.061515
recneeds -0.493771 6660.190623
recotherasst -0.125927 470.045073
recssi 0.813373 * -2105.801403 -0.435632
rectanf -0.530095 -4222.301090 -3.871620 **
wp -0.051242 * -403.342473 -0.096296
daysinprog 0.000008 2.222186 ** -0.000321 *
natresources -2.022414 -27241.594110 -8.080607
construction -0.466957 36651.674165 4.449278
manufacturing -1.706402 47186.585809 ** 9.855141 **
information -9.899828 -260263.704113 * -58.705607 **
financial -6.274409 85893.195715 12.197709
business -3.602686 95022.132013 ** 16.055431 **
edhealthcare -1.994586 * 51172.308282 ** 7.163421 *
leisure -2.851943 ** 43978.650647 * 1.512235
otheremp 3.042764 -4546.688846 65.315399 ***
publicadmin 1.229505 22271.277967 -2.847844
ur 0.411812 -5795.681566 -10.823728 ***
factor(state_id)AK 2.255656 ** -17842.662709 -3.133246
factor(state_id)AL 2.526852 *** -29700.571783 * -5.150761
factor(state_id)AR 2.550669 *** -29847.758315 * -5.027428
factor(state_id)AZ 2.705233 *** -29301.463752 * -5.047204
factor(state_id)CA 2.707397 *** -23265.964736 -4.177181
factor(state_id)CO 2.779950 *** -25132.566492 -4.611606
factor(state_id)CT 2.813566 *** -28634.356139 -6.167643 *
factor(state_id)DC 2.204652 * -27058.779363 -7.754375 *
factor(state_id)DE 2.907736 *** -34357.367024 * -6.268757 *
factor(state_id)FL 2.746479 *** -29067.128955 * -5.213093
factor(state_id)GA 2.762194 *** -25251.462662 -4.574801
factor(state_id)HI 2.520500 ** -23950.914535 -5.473445
factor(state_id)IA 2.682558 *** -26531.750621 -5.298796
factor(state_id)ID 2.526904 *** -26836.631069 -4.871504
factor(state_id)IL 2.686033 *** -28278.047749 * -6.115683 *
factor(state_id)IN 2.506091 *** -29367.230194 * -5.864722 *
factor(state_id)KS 2.625629 *** -27632.213253 -5.170015
factor(state_id)KY 2.430896 ** -29740.840610 * -5.081511
factor(state_id)LA 2.404281 ** -26884.997792 -4.388795
factor(state_id)MA 2.911248 *** -27252.918782 -5.861500 *
factor(state_id)MD 2.616000 ** -30283.372385 * -5.721748 *
factor(state_id)ME 2.494570 ** -29474.372478 * -5.600062 *
factor(state_id)MI 2.730832 *** -30107.722800 * -6.504235 *
factor(state_id)MN 2.716874 *** -25694.728811 -5.829576 *
factor(state_id)MO 2.692714 *** -29525.593851 * -5.301886
factor(state_id)MS 2.449448 ** -28227.877437 * -4.342688
factor(state_id)MT 2.339445 ** -25333.254751 -4.845451
factor(state_id)NC 2.650518 *** -29547.862854 * -5.082503
factor(state_id)ND 2.468655 *** -19932.057974 -4.056875
factor(state_id)NE 2.711318 *** -27666.672993 -5.019336
factor(state_id)NH 2.630676 *** -26200.902004 -5.382512 *
factor(state_id)NJ 2.529944 ** -28927.127440 * -5.902824 *
factor(state_id)NM 2.494748 ** -27658.088064 -4.036266
factor(state_id)NV 2.681365 *** -28910.669910 * -4.364276
factor(state_id)NY 2.720284 ** -27682.094898 -5.433295
factor(state_id)OH 2.682824 *** -30035.658236 * -5.750927 *
factor(state_id)OK 2.495556 ** -25738.991734 -4.440662
factor(state_id)OR 2.462849 ** -26861.196616 -5.747610 *
factor(state_id)PA 2.652898 *** -28927.578436 * -6.137101 *
factor(state_id)RI 2.792597 *** -31760.137463 * -6.641260 *
factor(state_id)SC 2.604135 *** -29656.083185 * -5.270634
factor(state_id)SD 2.472435 ** -27166.779422 -4.790769
factor(state_id)TN 2.686909 *** -28458.263874 * -5.290114
factor(state_id)TX 2.650290 *** -26460.180737 -4.810925
factor(state_id)UT 2.730831 *** -26934.348889 -4.649212
factor(state_id)VA 2.736584 *** -30690.586191 * -6.248055 *
factor(state_id)VT 2.532891 *** -26226.276274 -5.281759
factor(state_id)WA 2.779306 *** -17680.226401
factor(state_id)WI 2.589262 *** -27774.944455 * -5.542222 *
factor(state_id)WV 2.367434 ** -24115.052192 -4.621858
factor(state_id)WY 2.348828 ** -18812.619466 -2.794552
wages2qprior 0.091661 *
daysenrolled -0.000274 **
daysenrolled_under30 0.161981
Observations 388 408 398
R2 / R2 adjusted 0.998 / 0.998 0.995 / 0.993 0.975 / 0.968
  • p<0.05   ** p<0.01   *** p<0.001
Condensed Table of Model Estimates
Employment Rate 2nd Quarter after Exit
Median Earnings 2nd Quarter after Exit
Measurable Skill Gains
Model Coefficients
State Fixed Effects
Model Coefficients
State Fixed Effects
Model Coefficients
State Fixed Effects
Variable Estimate State Fixed.Effect Variable Estimate State Fixed.Effect Variable Estimate State Fixed.Effect
female 0.059588641619 AK 2.255656 female -1901.34152928 AK -17842.66 female -0.1509121899 AK -3.133246
age2544 0.018934089688 AL 2.526852 age2544 1115.51538163 AL -29700.57 age2544 -0.0654958006 AL -5.150761
age4554 -0.016926055107 AR 2.550669 age4554 -125.98730475 AR -29847.76 age4554 0.2414357025 AR -5.027428
age5559 0.105989312358 AZ 2.705233 age5559 -2126.37845407 AZ -29301.46 age5559 0.4578970572 AZ -5.047204
age60 -0.190542106198 CA 2.707397 age60 -2492.93119410 CA -23265.96 age60 0.9139404298 CA -4.177181
hispanic 0.118518012212 CO 2.779950 hispanic -857.75502470 CO -25132.57 hispanic -0.6645815390 CO -4.611606
raceasian -0.290982207833 CT 2.813566 raceasian -4684.97125327 CT -28634.36 raceasian -0.5339926914 CT -6.167643
raceblack -0.035839687022 DC 2.204652 raceblack -1536.60273499 DC -27058.78 raceblack -0.3293196913 DC -7.754375
racehpi 0.879207184863 DE 2.907736 racehpi -3269.17534480 DE -34357.37 raceai 2.6464587912 DE -6.268757
raceai -0.098328416676 FL 2.746479 raceai -3522.21376701 FL -29067.13 racemulti 0.0502575952 FL -5.213093
racemulti -0.194744448737 GA 2.762194 racemulti -3712.05937683 GA -25251.46 hsgrad -0.1921538099 GA -4.574801
hsgrad -0.025948791754 HI 2.520500 hsgrad -1400.09697196 HI -23950.91 collegedropout -0.2383691632 HI -5.473445
collegedropout -0.194217896498 IA 2.682558 collegedropout -1902.90476823 IA -26531.75 certotherps -0.2607481850 IA -5.298796
certotherps -0.179934797129 ID 2.526904 certotherps 83.41513308 ID -26836.63 associate 0.2625079774 ID -4.871504
associate -0.090730941207 IL 2.686033 associate 1526.24016950 IL -28278.05 ba -0.1122537341 IL -6.115683
ba -0.144748840181 IN 2.506091 ba 1169.41787505 IN -29367.23 gradschool -0.4275047921 IN -5.864722
gradschool -0.120999266648 KS 2.625629 gradschool 2155.04970999 KS -27632.21 empentry -0.0215091763 KS -5.170015
empentry 0.106102070723 KY 2.430896 empentry 1700.77942576 KY -29740.84 edstatentry -0.2643440753 KY -5.081511
edstatentry -0.025435701687 LA 2.404281 edstatentry 3787.51027980 LA -26885.00 disabled -1.4480556180 LA -4.388795
disabled -0.052683338588 MA 2.911248 disabled 279.59305156 MA -27252.92 veteran -0.9234707728 MA -5.861500
veteran 0.005587873244 MD 2.616000 veteran 1445.83444443 MD -30283.37 englearner 0.4513616982 MD -5.721748
englearner -0.252088847252 ME 2.494570 englearner -2976.13277112 ME -29474.37 singleparent 0.3691411509 ME -5.600062
singleparent 0.044648366364 MI 2.730832 singleparent -784.43483166 MI -30107.72 offender 0.4692889981 MI -6.504235
lowinc -0.051809084873 MN 2.716874 lowinc -538.70966045 MN -25694.73 recwages2qprior 0.0286724335 MN -5.829576
homeless 0.030618561111 MO 2.692714 homeless 7893.82501466 MO -29525.59 longtermunemp 0.3881248214 MO -5.301886
offender 0.377490440594 MS 2.449448 offender 1805.97830954 MS -28227.88 uiexhaustee 0.3560996463 MS -4.342688
dishomemaker -0.227365225997 MT 2.339445 dishomemaker 192.75636558 MT -25333.25 recsuppserv -0.0615149823 MT -4.845451
recwages2qprior 0.113108383341 NC 2.650518 recwages2qprior 21.08171273 NC -29547.86 recssi -0.4356315764 NC -5.082503
longtermunemp 0.057434962249 ND 2.468655 wages2qprior 0.09166108 ND -19932.06 rectanf -3.8716204394 ND -4.056875
uiclaimant 0.020818050298 NE 2.711318 longtermunemp 1348.36823384 NE -27666.67 wp -0.0962955150 NE -5.019336
uiexhaustee 0.073688134757 NH 2.630676 uiclaimant 68.69620574 NH -26200.90 daysenrolled -0.0002735278 NH -5.382512
recsuppserv 0.049648685679 NJ 2.529944 uiexhaustee -2493.01316378 NJ -28927.13 daysenrolled_under30 0.1619814185 NJ -5.902824
recneeds -0.493770910754 NM 2.494748 recsuppserv 176.16278640 NM -27658.09 daysinprog -0.0003213372 NM -4.036266
recotherasst -0.125926659001 NV 2.681365 recneeds 6660.19062273 NV -28910.67 natresources -8.0806067874 NV -4.364276
recssi 0.813373207857 NY 2.720284 recotherasst 470.04507279 NY -27682.09 construction 4.4492778351 NY -5.433295
rectanf -0.530094942190 OH 2.682824 recssi -2105.80140279 OH -30035.66 manufacturing 9.8551414932 OH -5.750927
wp -0.051242081120 OK 2.495556 rectanf -4222.30108971 OK -25738.99 information -58.7056068532 OK -4.440662
daysinprog 0.000008016196 OR 2.462849 wp -403.34247271 OR -26861.20 financial 12.1977093582 OR -5.747610
natresources -2.022413998658 PA 2.652898 daysinprog 2.22218592 PA -28927.58 business 16.0554307218 PA -6.137101
construction -0.466957288822 RI 2.792597 natresources -27241.59411020 RI -31760.14 edhealthcare 7.1634213255 RI -6.641260
manufacturing -1.706402420226 SC 2.604135 construction 36651.67416461 SC -29656.08 leisure 1.5122345603 SC -5.270634
information -9.899827801574 SD 2.472435 manufacturing 47186.58580939 SD -27166.78 otheremp 65.3153989283 SD -4.790769
financial -6.274409283037 TN 2.686909 information -260263.70411317 TN -28458.26 publicadmin -2.8478442549 TN -5.290114
business -3.602686058283 TX 2.650290 financial 85893.19571481 TX -26460.18 ur -10.8237278353 TX -4.810925
edhealthcare -1.994586164291 UT 2.730831 business 95022.13201293 UT -26934.35 UT -4.649212
leisure -2.851943150828 VA 2.736584 edhealthcare 51172.30828224 VA -30690.59 VA -6.248055
otheremp 3.042763681784 VT 2.532891 leisure 43978.65064705 VT -26226.28 VT -5.281759
publicadmin 1.229505049349 WA 2.779306 otheremp -4546.68884590 WA -17680.23 WI -5.542222
ur 0.411811663529 WI 2.589262 publicadmin 22271.27796717 WI -27774.94 WV -4.621858
WV 2.367434 ur -5795.68156618 WV -24115.05 WY -2.794552
WY 2.348828 WY -18812.62 WA* -2.794551
* In the MSG model, the state fixed effect applied to WA is the state fixed effect closest to zero. An estimate could not be calculated for WA in the MSG model because there were not sufficient usable observations.

Youth

Final model coefficient estimates for all performance indicator models in the Youth program.

Statistical Table of Model Estimates
  Youth - Q2ER Youth - ME Youth - MSG
Coefficient Estimates Estimates Estimates
female 0.059470 -1877.494316 *** -0.275838
age1415 0.122600 -92.301309 -0.810620
age1617 -0.143635 -1309.058743 * -0.902512 ***
age1819 -0.205360 ** -1066.476193 -0.699767 **
age2021 0.010543 649.493056 -1.640423 ***
hispanic -0.062767 1913.458483 *** -0.016985
raceasian 0.198885 649.712221 -0.016157
raceblack -0.041385 -886.370347 * 0.004210
racehpi -0.534210 -3388.423241 ***
raceai -0.334087 ** -184.471998 -0.157793
racemulti 0.150781 933.113352 1.972659 **
hsgrad 0.069148 1383.940816 *** -0.269244 *
collegedropout -0.312708 * -828.191307 1.051262 **
certotherps 1.146930 ** 173.095537 -0.644904
associateorba 0.493456 6672.333019 ** 1.900030
empentry 0.274766 *** 613.785692
edstatentry 0.035622 546.199408 0.022015
disabled -0.046860 -495.181142
englearner -0.139164 2456.302336 **
lowinc 0.037494 -305.798486
homeless -0.200764 * 983.904380
offender 0.063461 -1284.659618 *
yparent -0.071614 854.512806
basiclitdeficient 0.034935 -283.477457 0.197613 **
yfoster -0.010019 1009.829347
longtermunemp -0.086714 -630.266364
uiclaimant -0.043278 -462.583776 0.019842
recsuppserv 0.044177 161.175000 -0.071243
recotherasst -0.150971 *** -184.078603 0.274159 ***
rectanf -0.034137 -539.650932
recneeds 0.765976 2823.223979
recpell 0.036806 104.184311 -0.886366 **
recssi 0.074260 -1658.754548 * 0.553619
ynaa 0.000494 -4.334071
wp 0.014843 -27.873083 -0.050273
daysinprog -0.000040 0.594200
natresources -6.787195 *** -3172.195765 7.628195 *
construction -1.880014 10994.477166 9.573994 **
manufacturing -1.360190 21559.959277 * 5.831283
information -7.297438 -55465.649263 -42.813580 ***
financial -2.136676 44805.105549 -14.243336
business -2.556361 14219.016076 14.476898 **
edhealthcare 0.024683 20372.044401 * 7.063440 *
leisure -0.394391 7088.247680 6.299267 *
otheremp -10.794028 57026.850481 50.839071 ***
publicadmin 3.299306 43573.513926 ** -7.340849
ur -1.487183 * -10090.219185 -1.166276
factor(state_id)AK 1.778303 -10894.280354 -2.755678
factor(state_id)AL 1.672114 -11643.816417 -3.506829
factor(state_id)AR 1.812431 -10250.624612 -3.609990
factor(state_id)AZ 1.931691 -11264.280022 -3.153583
factor(state_id)CA 2.092489 -10619.162727 -3.341792
factor(state_id)CO 1.968332 -10706.010226 -3.305029
factor(state_id)CT 1.995423 -12652.406922 -3.711476
factor(state_id)DC 1.781112 -19273.055146 -5.082130
factor(state_id)DE 1.924112 -12061.962673 -3.253938
factor(state_id)FL 1.916048 -10696.498143 -3.625961
factor(state_id)GA 1.847547 -9638.602678 -2.993399
factor(state_id)HI 2.163985 * -7779.592323 -5.346696 *
factor(state_id)IA 1.755305 -11755.010247 -2.936913
factor(state_id)ID 2.046427 * -10224.733199 -4.041549
factor(state_id)IL 2.058533 -11148.403425 -4.148114
factor(state_id)IN 1.821538 -11445.098143 -4.034736
factor(state_id)KS 1.815744 -11595.098263 -3.481154
factor(state_id)KY 1.651570 -11143.778775 -3.407749
factor(state_id)LA 1.793602 -9829.931840 -3.734316
factor(state_id)MA 1.971289 -11178.638932 -3.851665
factor(state_id)MD 1.746257 -12269.707079 -3.997743
factor(state_id)ME 1.724300 -10598.462754 -3.764947
factor(state_id)MI 2.052941 -10771.344953 -4.721371
factor(state_id)MN 1.938173 -11191.103581 -3.546914
factor(state_id)MO 1.830118 -11168.150320 -3.518775
factor(state_id)MS 1.701017 -10538.771649 -3.097804
factor(state_id)MT 1.802918 -11780.998179 -3.723848
factor(state_id)NC 1.764420 -11340.657458 -3.483601
factor(state_id)ND 1.974100 * -8756.751046 -3.196737
factor(state_id)NE 1.813342 -11739.421022 -3.001942
factor(state_id)NH 1.838808 -10759.530310 -3.209695
factor(state_id)NJ 1.806310 -11806.910117 -3.796792
factor(state_id)NM 1.865882 -11810.227323 -3.751942
factor(state_id)NV 1.739620 -9163.206101 -4.157771
factor(state_id)NY 1.906071 -12141.968292 -2.975206
factor(state_id)OH 1.846957 -11184.057868 -3.900284
factor(state_id)OK 1.850224 -10555.034386 -3.327440
factor(state_id)OR 2.096850 * -10608.385138 -4.547164
factor(state_id)PA 1.866849 -11932.317723 -4.081972
factor(state_id)RI 1.874416 -12936.219071 -4.423481
factor(state_id)SC 1.761247 -10743.952742 -3.613004
factor(state_id)SD 1.755884 -11722.994442 -2.833668
factor(state_id)TN 1.806815 -10723.887065 -3.359392
factor(state_id)TX 2.009688 -11149.198656 -3.718638
factor(state_id)UT 1.877098 -10355.189020 -2.992210
factor(state_id)VA 1.891817 -11882.813100 -4.395266
factor(state_id)VT 1.715269 -10324.717654 -3.973769
factor(state_id)WA 2.158747 * -8860.221978 -2.877611
factor(state_id)WI 1.894133 -11313.129960 -3.683783
factor(state_id)WV 1.690998 -11618.953418 -3.713823
factor(state_id)WY 1.928539 -10304.393001 -3.027734
daysenrolled -0.000326 ***
daysenrolled_under30 -0.344096 ***
Observations 391 402 472
R2 / R2 adjusted 0.997 / 0.996 0.991 / 0.988 0.972 / 0.966
  • p<0.05   ** p<0.01   *** p<0.001
Condensed Table of Model Estimates
Employment Rate 2nd Quarter after Exit
Median Earnings 2nd Quarter after Exit
Measurable Skill Gains
Model Coefficients
State Fixed Effects
Model Coefficients
State Fixed Effects
Model Coefficients
State Fixed Effects
Variable Estimate State Fixed.Effect Variable Estimate State Fixed.Effect Variable Estimate State Fixed.Effect
female 0.05947039425 AK 1.778303 female -1877.4943160 AK -10894.28 female -0.2758376671 AK -2.755678
age1415 0.12260008755 AL 1.672114 age1415 -92.3013088 AL -11643.82 age1415 -0.8106200342 AL -3.506829
age1617 -0.14363454073 AR 1.812431 age1617 -1309.0587427 AR -10250.62 age1617 -0.9025120849 AR -3.609990
age1819 -0.20535954096 AZ 1.931691 age1819 -1066.4761934 AZ -11264.28 age1819 -0.6997672056 AZ -3.153583
age2021 0.01054343918 CA 2.092489 age2021 649.4930559 CA -10619.16 age2021 -1.6404229232 CA -3.341792
hispanic -0.06276666323 CO 1.968332 hispanic 1913.4584829 CO -10706.01 hispanic -0.0169846889 CO -3.305029
raceasian 0.19888509384 CT 1.995423 raceasian 649.7122212 CT -12652.41 raceasian -0.0161566715 CT -3.711476
raceblack -0.04138479463 DC 1.781112 raceblack -886.3703469 DC -19273.06 raceblack 0.0042098289 DC -5.082130
racehpi -0.53420962626 DE 1.924112 racehpi -3388.4232412 DE -12061.96 raceai -0.1577929793 DE -3.253938
raceai -0.33408650594 FL 1.916048 raceai -184.4719976 FL -10696.50 racemulti 1.9726593742 FL -3.625961
racemulti 0.15078149394 GA 1.847547 racemulti 933.1133520 GA -9638.60 hsgrad -0.2692438376 GA -2.993399
hsgrad 0.06914787476 HI 2.163985 hsgrad 1383.9408156 HI -7779.59 collegedropout 1.0512621285 HI -5.346696
collegedropout -0.31270753030 IA 1.755305 collegedropout -828.1913074 IA -11755.01 certotherps -0.6449042461 IA -2.936913
certotherps 1.14692954778 ID 2.046427 certotherps 173.0955374 ID -10224.73 associateorba 1.9000298176 ID -4.041549
associateorba 0.49345550942 IL 2.058533 associateorba 6672.3330187 IL -11148.40 edstatentry 0.0220145534 IL -4.148114
empentry 0.27476631252 IN 1.821538 empentry 613.7856919 IN -11445.10 basiclitdeficient 0.1976130714 IN -4.034736
edstatentry 0.03562153926 KS 1.815744 edstatentry 546.1994079 KS -11595.10 uiclaimant 0.0198422243 KS -3.481154
disabled -0.04686038667 KY 1.651570 disabled -495.1811418 KY -11143.78 recsuppserv -0.0712433485 KY -3.407749
englearner -0.13916373069 LA 1.793602 englearner 2456.3023358 LA -9829.93 recotherasst 0.2741585147 LA -3.734316
lowinc 0.03749439722 MA 1.971289 lowinc -305.7984864 MA -11178.64 recpell -0.8863661686 MA -3.851665
homeless -0.20076439017 MD 1.746257 homeless 983.9043796 MD -12269.71 recssi 0.5536193850 MD -3.997743
offender 0.06346133826 ME 1.724300 offender -1284.6596184 ME -10598.46 wp -0.0502733193 ME -3.764947
yparent -0.07161419545 MI 2.052941 yparent 854.5128061 MI -10771.34 daysenrolled -0.0003263213 MI -4.721371
basiclitdeficient 0.03493477008 MN 1.938173 basiclitdeficient -283.4774565 MN -11191.10 daysenrolled_under30 -0.3440958977 MN -3.546914
yfoster -0.01001924299 MO 1.830118 yfoster 1009.8293471 MO -11168.15 natresources 7.6281947392 MO -3.518775
longtermunemp -0.08671394055 MS 1.701017 longtermunemp -630.2663644 MS -10538.77 construction 9.5739942414 MS -3.097804
uiclaimant -0.04327776360 MT 1.802918 uiclaimant -462.5837759 MT -11781.00 manufacturing 5.8312825525 MT -3.723848
recsuppserv 0.04417748993 NC 1.764420 recsuppserv 161.1750004 NC -11340.66 information -42.8135803504 NC -3.483601
recotherasst -0.15097058116 ND 1.974100 recotherasst -184.0786032 ND -8756.75 financial -14.2433364630 ND -3.196737
rectanf -0.03413650525 NE 1.813342 rectanf -539.6509316 NE -11739.42 business 14.4768978378 NE -3.001942
recneeds 0.76597588083 NH 1.838808 recneeds 2823.2239794 NH -10759.53 edhealthcare 7.0634398864 NH -3.209695
recpell 0.03680563174 NJ 1.806310 recpell 104.1843112 NJ -11806.91 leisure 6.2992665029 NJ -3.796792
recssi 0.07425987099 NM 1.865882 recssi -1658.7545482 NM -11810.23 otheremp 50.8390705608 NM -3.751942
ynaa 0.00049382198 NV 1.739620 ynaa -4.3340708 NV -9163.21 publicadmin -7.3408494944 NV -4.157771
wp 0.01484303822 NY 1.906071 wp -27.8730831 NY -12141.97 ur -1.1662757628 NY -2.975206
daysinprog -0.00004013644 OH 1.846957 daysinprog 0.5941999 OH -11184.06 OH -3.900284
natresources -6.78719531927 OK 1.850224 natresources -3172.1957655 OK -10555.03 OK -3.327440
construction -1.88001445079 OR 2.096850 construction 10994.4771662 OR -10608.39 OR -4.547164
manufacturing -1.36019014757 PA 1.866849 manufacturing 21559.9592771 PA -11932.32 PA -4.081972
information -7.29743845842 RI 1.874416 information -55465.6492630 RI -12936.22 RI -4.423481
financial -2.13667591106 SC 1.761247 financial 44805.1055492 SC -10743.95 SC -3.613004
business -2.55636064242 SD 1.755884 business 14219.0160763 SD -11722.99 SD -2.833668
edhealthcare 0.02468281436 TN 1.806815 edhealthcare 20372.0444007 TN -10723.89 TN -3.359392
leisure -0.39439053978 TX 2.009688 leisure 7088.2476795 TX -11149.20 TX -3.718638
otheremp -10.79402818106 UT 1.877098 otheremp 57026.8504807 UT -10355.19 UT -2.992210
publicadmin 3.29930646598 VA 1.891817 publicadmin 43573.5139263 VA -11882.81 VA -4.395266
ur -1.48718344784 VT 1.715269 ur -10090.2191848 VT -10324.72 VT -3.973769
WA 2.158747 WA -8860.22 WA -2.877611
WI 1.894133 WI -11313.13 WI -3.683783
WV 1.690998 WV -11618.95 WV -3.713823
WY 1.928539 WY -10304.39 WY -3.027734

Wagner-Peyser

Final model coefficient estimates for all performance indicator models in the Wagner-Peyser program.

Statistical Table of Model Estimates
  WP - Q2ER WP - ME
Coefficient Estimates Estimates
female 0.080068 -2774.846031 ***
age2544 0.108555 1526.770908
age4554 -0.086029 223.621230
age5559 -0.006958 1725.953416
age60 -0.062930 3989.386301 *
hispanic 0.232562 ** 1506.273645
raceasian -0.235368 -1285.758948
raceblack -0.160946 * -2926.831878 ***
racehpi 0.970283 -2473.035153
raceai -0.306159 ** -5567.849724 ***
racemulti 0.247057 10678.096770 **
hsgrad -0.017186 -1763.879669 ***
collegedropout 0.038606 -2177.452606 *
certotherps -0.042201 -2180.318953
associate 0.349586 2095.547070
ba -0.676754 *** 72.812827
gradschool -0.524585 -5012.837572
empentry 0.086474 456.190569
edstatentry -0.090273 -1155.207158
disabled -0.355660 ** -5107.069158 **
veteran 0.216994 * -913.896731
englearner -0.018463 1563.751187
singleparent 0.202731 *** 660.101794
lowinc 0.092574 *** 634.463334 **
homeless -0.066680 -3513.094811
offender 0.185040 * 1920.767940
dishomemaker -0.230393 -10834.380354 *
recwages2qprior 0.317363 *** -219.479177
longtermunemp -0.154069 *** 771.284878
uiclaimant -0.038477 454.796655
uiexhaustee -0.089666 * 247.988913
recsuppserv -0.102577 -636.385548
recneeds -9.894965 -21804.706709
recotherasst -0.116322 -1174.579280
recssi 1.087342 * 10874.758703
rectanf -0.567987 *** 1657.839301
daysinprog -0.000276 ** 0.871737
natresources 1.685595 37057.607854 ***
construction 1.541115 * 42760.771028 ***
manufacturing 1.012654 47700.870793 ***
information -0.459548 11314.808576
financial 2.964851 62614.679742 **
business 0.943068 67885.340234 ***
edhealthcare 1.283098 * 51491.776355 ***
leisure 0.591872 43018.430501 ***
otheremp 3.902084 41629.744276
publicadmin 1.700646 * 51356.260184 ***
ur 0.343995 -5089.390952
factor(state_id)AK -0.708056 -32949.142856 ***
factor(state_id)AL -0.565902 -34497.621524 ***
factor(state_id)AR -0.562478 -34164.338226 ***
factor(state_id)AZ -0.769326 -36793.194211 ***
factor(state_id)CA -0.690562 -35815.286288 ***
factor(state_id)CO -0.668760 -36433.328364 ***
factor(state_id)CT -0.647342 -36108.616356 ***
factor(state_id)DC -0.865351 -41421.393991 ***
factor(state_id)DE -0.714074 -36243.987976 ***
factor(state_id)FL -0.580527 -34581.329579 ***
factor(state_id)GA -0.447274 -33318.906028 ***
factor(state_id)HI -1.001776 -35415.003571 ***
factor(state_id)IA -0.634677 -34067.010730 ***
factor(state_id)ID -0.603241 -35202.150413 ***
factor(state_id)IL -0.601913 -35097.686701 ***
factor(state_id)IN -0.500366 -34143.908935 ***
factor(state_id)KS -0.649165 -35650.227571 ***
factor(state_id)KY -0.597450 -34197.788936 ***
factor(state_id)LA -0.605414 -33284.142173 ***
factor(state_id)MA -0.533469 -36768.472525 ***
factor(state_id)MD -0.583142 -35807.023357 ***
factor(state_id)ME -0.630777 -35052.854680 ***
factor(state_id)MI -0.535439 -35044.267607 ***
factor(state_id)MN -0.489997 -34700.474567 ***
factor(state_id)MO -0.532255 -34773.373972 ***
factor(state_id)MS -0.502331 -32796.104860 ***
factor(state_id)MT -0.639835 -33561.890382 ***
factor(state_id)NC -0.510584 -34669.184256 ***
factor(state_id)ND -0.596882 -31349.402684 ***
factor(state_id)NE -0.576455 -34225.079857 ***
factor(state_id)NH -0.500244 -33729.700008 ***
factor(state_id)NJ -0.625696 -34972.438119 ***
factor(state_id)NM -0.730247 -36190.112324 ***
factor(state_id)NV -0.496929 -34452.056361 ***
factor(state_id)NY -0.657590 -35840.068238 ***
factor(state_id)OH -0.461737 -34137.183737 ***
factor(state_id)OK -0.652116 -34608.641068 ***
factor(state_id)OR -0.709444 -34863.843873 ***
factor(state_id)PA -0.646653 -35967.890228 ***
factor(state_id)RI -0.640388 -36740.415563 ***
factor(state_id)SC -0.504929 -34096.424319 ***
factor(state_id)SD -0.584772 -33752.183802 ***
factor(state_id)TN -0.483926 -33351.010346 ***
factor(state_id)TX -0.683578 -35060.191279 ***
factor(state_id)UT -0.587069 -34795.814661 ***
factor(state_id)VA -0.535480 -36329.453285 ***
factor(state_id)VT -0.638672 -35826.742810 ***
factor(state_id)WA -0.528872 -33331.307684 ***
factor(state_id)WI -0.591372 -35088.396684 ***
factor(state_id)WV -0.692899 -34505.369383 ***
factor(state_id)WY -0.686791 -33343.579149 ***
wages2qprior 0.261150 ***
Observations 402 405
R2 / R2 adjusted 0.999 / 0.999 0.999 / 0.998
  • p<0.05   ** p<0.01   *** p<0.001
Condensed Table of Model Estimates
Employment Rate 2nd Quarter after Exit
Median Earnings 2nd Quarter after Exit
Model Coefficients
State Fixed Effects
Model Coefficients
State Fixed Effects
Variable Estimate State Fixed.Effect Variable Estimate State Fixed.Effect
female 0.0800683012 AK -0.708056 female -2774.8460306 AK -32949.14
age2544 0.1085553541 AL -0.565902 age2544 1526.7709080 AL -34497.62
age4554 -0.0860285708 AR -0.562478 age4554 223.6212301 AR -34164.34
age5559 -0.0069581357 AZ -0.769326 age5559 1725.9534161 AZ -36793.19
age60 -0.0629302685 CA -0.690562 age60 3989.3863007 CA -35815.29
hispanic 0.2325624981 CO -0.668760 hispanic 1506.2736447 CO -36433.33
raceasian -0.2353682178 CT -0.647342 raceasian -1285.7589482 CT -36108.62
raceblack -0.1609463153 DC -0.865351 raceblack -2926.8318783 DC -41421.39
racehpi 0.9702829924 DE -0.714074 racehpi -2473.0351534 DE -36243.99
raceai -0.3061591649 FL -0.580527 raceai -5567.8497244 FL -34581.33
racemulti 0.2470568722 GA -0.447274 racemulti 10678.0967701 GA -33318.91
hsgrad -0.0171855941 HI -1.001776 hsgrad -1763.8796690 HI -35415.00
collegedropout 0.0386057375 IA -0.634677 collegedropout -2177.4526060 IA -34067.01
certotherps -0.0422008235 ID -0.603241 certotherps -2180.3189533 ID -35202.15
associate 0.3495856565 IL -0.601913 associate 2095.5470705 IL -35097.69
ba -0.6767542823 IN -0.500366 ba 72.8128269 IN -34143.91
gradschool -0.5245846844 KS -0.649165 gradschool -5012.8375722 KS -35650.23
empentry 0.0864738192 KY -0.597450 empentry 456.1905685 KY -34197.79
edstatentry -0.0902727807 LA -0.605414 edstatentry -1155.2071578 LA -33284.14
disabled -0.3556599380 MA -0.533469 disabled -5107.0691576 MA -36768.47
veteran 0.2169939131 MD -0.583142 veteran -913.8967313 MD -35807.02
englearner -0.0184630669 ME -0.630777 englearner 1563.7511871 ME -35052.85
singleparent 0.2027309106 MI -0.535439 singleparent 660.1017942 MI -35044.27
lowinc 0.0925740938 MN -0.489997 lowinc 634.4633341 MN -34700.47
homeless -0.0666795065 MO -0.532255 homeless -3513.0948113 MO -34773.37
offender 0.1850396808 MS -0.502331 offender 1920.7679399 MS -32796.10
dishomemaker -0.2303925451 MT -0.639835 dishomemaker -10834.3803542 MT -33561.89
recwages2qprior 0.3173634626 NC -0.510584 recwages2qprior -219.4791767 NC -34669.18
longtermunemp -0.1540688760 ND -0.596882 wages2qprior 0.2611502 ND -31349.40
uiclaimant -0.0384766581 NE -0.576455 longtermunemp 771.2848776 NE -34225.08
uiexhaustee -0.0896659724 NH -0.500244 uiclaimant 454.7966545 NH -33729.70
recsuppserv -0.1025769829 NJ -0.625696 uiexhaustee 247.9889134 NJ -34972.44
recneeds -9.8949647026 NM -0.730247 recsuppserv -636.3855475 NM -36190.11
recotherasst -0.1163215596 NV -0.496929 recneeds -21804.7067095 NV -34452.06
recssi 1.0873416857 NY -0.657590 recotherasst -1174.5792805 NY -35840.07
rectanf -0.5679873752 OH -0.461737 recssi 10874.7587034 OH -34137.18
daysinprog -0.0002756013 OK -0.652116 rectanf 1657.8393012 OK -34608.64
natresources 1.6855951071 OR -0.709444 daysinprog 0.8717370 OR -34863.84
construction 1.5411152033 PA -0.646653 natresources 37057.6078537 PA -35967.89
manufacturing 1.0126536458 RI -0.640388 construction 42760.7710277 RI -36740.42
information -0.4595484910 SC -0.504929 manufacturing 47700.8707928 SC -34096.42
financial 2.9648506066 SD -0.584772 information 11314.8085760 SD -33752.18
business 0.9430684446 TN -0.483926 financial 62614.6797424 TN -33351.01
edhealthcare 1.2830977047 TX -0.683578 business 67885.3402345 TX -35060.19
leisure 0.5918722288 UT -0.587069 edhealthcare 51491.7763554 UT -34795.81
otheremp 3.9020836429 VA -0.535480 leisure 43018.4305008 VA -36329.45
publicadmin 1.7006456021 VT -0.638672 otheremp 41629.7442762 VT -35826.74
ur 0.3439948537 WA -0.528872 publicadmin 51356.2601838 WA -33331.31
WI -0.591372 ur -5089.3909515 WI -35088.40
WV -0.692899 WV -34505.37
WY -0.686791 WY -33343.58

Model Predictions

The tables below show the predicted outcomes (Estimate0) in PY 2020 for each program indicator. The predictions are calculated by applying the model estimates to the most recent reported PY data on each state’s participant characteristics for each program indicator (i.e., PY 2018) and the most recent economic conditions data for states (i.e., time period 7/1/2018 - 6/30/2019).

Adult

Employment Rate 2nd Quarter after Exit
Median Earnings 2nd Quarter after Exit
Measurable Skill Gains
State PY.2020.Prediction State PY.2020.Prediction State PY.2020.Prediction
AK 0.8164787 AK 7083.283 AK 0.7517289
AL 0.8047171 AL 5657.800 AL 0.6143881
AR 0.8404348 AR 6193.263 AR 0.6284781
AZ 0.7128072 AZ 6538.696 AZ 0.5500868
CA 0.6883760 CA 6473.101 CA 0.4411489
CO 0.8072060 CO 7144.959 CO 0.5207939
CT 0.7607674 CT 5765.008 CT 0.5484195
DC 0.7004922 DC 7322.537 DC 0.3895057
DE 0.8445629 DE 6603.287 DE 0.2754979
FL 0.8618310 FL 8230.993 FL 0.4685323
GA 0.8302541 GA 7292.802 GA 0.3804062
HI 0.6394134 HI 4216.438 HI 0.2661061
IA 0.7067325 IA 5605.405 IA 0.6791101
ID 0.8347291 ID 7191.612 ID 0.4594629
IL 0.7823407 IL 6642.588 IL 0.3813954
IN 0.8267457 IN 6927.270 IN 0.5625524
KS 0.7619632 KS 5726.641 KS 0.5409826
KY 0.7235237 KY 5357.272 KY 0.2163479
LA 0.6793483 LA 6000.126 LA 0.4834976
MA 0.7819356 MA 6811.542 MA 0.4600679
MD 0.7964423 MD 7099.311 MD 0.6174405
ME 0.7299724 ME 4933.813 ME 0.4142601
MI 0.8771348 MI 6598.461 MI 0.3332028
MN 0.7985756 MN 7942.697 MN 0.5722650
MO 0.7430956 MO 5359.842 MO 0.5097699
MS 0.8512732 MS 6165.545 MS 0.5299495
MT 0.7339384 MT 6158.892 MT 0.5246451
NC 0.7924633 NC 5880.283 NC 0.4289536
ND 0.7496168 ND 6615.913 ND 0.5743638
NE 0.8002379 NE 6239.630 NE 0.4097797
NH 0.8574332 NH 6674.831 NH 0.8501801
NJ 0.6429710 NJ 5486.516 NJ 0.3596216
NM 0.7589391 NM 7359.106 NM 0.6364653
NV 0.7765551 NV 5706.459 NV 0.6396032
NY 0.7123658 NY 5801.571 NY 0.3013736
OH 0.8500969 OH 6203.800 OH 0.6518568
OK 0.6738391 OK 5958.156 OK 0.7946679
OR 0.6987326 OR 6618.953 OR 0.2762616
PA 0.7752511 PA 6105.649 PA 0.4249332
RI 0.8467991 RI 7718.441 RI 0.2714941
SC 0.8003207 SC 5847.952 SC 0.5513215
SD 0.7770874 SD 5730.613 SD 0.2664900
TN 0.8617498 TN 7208.479 TN 0.5820277
TX 0.7749454 TX 5589.668 TX 0.5361561
UT 0.7920153 UT 6389.916 UT 0.3163659
VA 0.8106564 VA 5284.609 VA 0.6360440
VT 0.7394967 VT 5605.719 VT 0.5199079
WA 0.6899364 WA 8595.834 WI 0.4309455
WI 0.7906891 WI 6410.067 WV 0.3281916
WV 0.6726378 WV 6524.125 WY 0.7931004
WY 0.8506173 WY 7224.125 WA* 0.3923855
PR** 0.7994361 PR** 3510.049 PR** 0.2397416
* The prediction for WA in the MSG model was estimated by applying the model estimates and the state fixed effect closest to zero to WA’s reported PY 2018 data. WA has not reported sufficient MSG data in the Adult program to generate a state fixed effect in this model.
** The predictions for PR in all Adult models were estimated using PY 2018 Q2 rolling-4 quarter participant data. PR has not reported annual PY 2018 data. Also, PR did not have sufficient data to generate a state fixed effect in all Adult models. As a result, the following fixed effects were applied to PR for the following models to get a reasonable prediction for PY 2020: Q2ER - average fixed effect, ME - average fixed effect, MSG - fixed effect closest to zero plus a 0.7 constant value.

Dislocated Worker

Employment Rate 2nd Quarter after Exit
Median Earnings 2nd Quarter after Exit
Measurable Skill Gains
State PY.2020.Prediction State PY.2020.Prediction State PY.2020.Prediction
AK 0.8185589 AK 11546.993 AK 0.6728140
AL 0.7876597 AL 8227.126 AL 0.5460117
AR 0.8804750 AR 6898.437 AR 0.7657738
AZ 0.7623136 AZ 8801.941 AZ 0.6713217
CA 0.7186528 CA 8276.650 CA 0.5006176
CO 0.7794149 CO 9742.673 CO 0.5213016
CT 0.8320354 CT 9191.612 CT 0.5534386
DC 0.7641469 DC 8481.942 DC 0.5122724
DE 0.8133746 DE 8754.485 DE 0.4368916
FL 0.8720673 FL 8009.776 FL 0.5669791
GA 0.8356788 GA 9587.069 GA 0.2895076
HI 0.9090301 HI 7758.669 HI 0.2616420
IA 0.8842619 IA 10538.423 IA 0.2805707
ID 0.7977404 ID 8267.188 ID 0.6060717
IL 0.8359204 IL 9934.597 IL 0.4297292
IN 0.8050636 IN 8098.021 IN 0.5059666
KS 0.8227726 KS 9855.048 KS 0.7423998
KY 0.7479011 KY 7745.148 LA 0.6469805
LA 0.6462780 LA 7812.581 MA 0.3550353
MA 0.7780038 MA 10276.797 MD 0.6329335
MD 0.8401327 MD 9336.069 ME 0.5045354
ME 0.7484926 ME 6825.255 MI 0.3901416
MI 0.9236149 MI 8571.317 MN 0.5353802
MN 0.8690173 MN 12782.951 MO 0.5876926
MO 0.7818109 MO 7899.572 MS 0.4916061
MS 0.7472844 MS 6045.974 MT 0.6430875
MT 0.7344932 MT 8473.681 NC 0.5096223
NC 0.7499365 NC 7482.871 ND 0.2907938
ND 0.7878607 ND 10103.240 NE 0.4496678
NE 0.8521738 NE 8356.893 NH 0.8575353
NH 0.8493176 NH 9476.719 NJ 0.3714163
NJ 0.6693220 NJ 7871.918 NM 0.6197596
NM 0.7227988 NM 7812.559 NV 0.6415843
NV 0.8332900 NV 7731.757 NY 0.3553391
NY 0.6671397 NY 7139.220 OH 0.6696439
OH 0.8792684 OH 8647.654 OK 0.7245306
OK 0.7645725 OK 8715.907 OR 0.3296442
OR 0.7314598 OR 7041.619 PA 0.3298018
PA 0.8384760 PA 8561.337 RI 0.5249953
RI 0.8728774 RI 8475.079 SC 0.5707150
SC 0.8050421 SC 7899.846 SD 0.5046922
SD 0.7653045 SD 7907.538 TN 0.6066316
TN 0.8547761 TN 8504.468 TX 0.5622338
TX 0.7446708 TX 8593.863 UT 0.4111860
UT 0.8713778 UT 8212.156 VA 0.5854623
VA 0.8592566 VA 8337.695 VT 0.6167383
VT 0.8304739 VT 10119.029 WI 0.5045018
WA 0.7519854 WA 9578.959 WV 0.2748939
WI 0.8484184 WI 8532.724 WY 0.7440796
WV 0.7881374 WV 9937.554 KY** 0.3089124
WY 0.7619601 WY 8467.598 WA* 0.7284369
PR*** 0.7152446 PR*** 7011.821 PR*** 0.1645206
* The prediction for WA in the MSG model was estimated by applying the model estimates and the state fixed effect closest to zero to WA’s reported PY 2018 data. WA has not reported sufficient MSG data in the Adult program to generate a state fixed effect in this model.
** The prediction for KY in the MSG model was estimated by applying the model estimates to KY’s reported PY 2017 data for the participant characteristics variables and the PY 2018 data for the economic conditions variables. KY did not report MSG data for the Dislocated Worker program in PY 2018.
*** The predictions for PR in all Dislocated Worker models were estimated using PY 2018 Q2 rolling-4 quarter participant data. PR has not reported annual PY 2018 data. Also, PR did not have sufficient data to generate a state fixed effect in all Dislocated Worker models. As a result, the following fixed effects were applied to PR for the following models to get a reasonable prediction for PY 2020: Q2ER - average fixed effect, ME - maxium fixed effect minus a constant of 2000, MSG - maximum fixed effect (fixed effect closest to zero).

Youth

Employment Rate 2nd Quarter after Exit
Median Earnings 2nd Quarter after Exit
Measurable Skill Gains
State PY.2020.Prediction State PY.2020.Prediction State PY.2020.Prediction
AK 0.5648550 AK 3030.285 AK 0.5733615
AL 0.6728323 AL 2742.253 AL 0.5140564
AR 0.8010851 AR 3288.171 AR 0.6061060
AZ 0.7545993 AZ 4648.187 AZ 0.5520768
CA 0.6797595 CA 3490.826 CA 0.5644029
CO 0.6768336 CO 3576.988 CO 0.4232997
CT 0.7467132 CT 3108.161 CT 0.6013266
DC 0.5715724 DC 3113.788 DC 0.5275873
DE 0.6777796 DE 2652.963 DE 0.5994263
FL 0.8178977 FL 3474.087 FL 0.4454891
GA 0.7300643 GA 2988.292 GA 0.4257251
HI 0.7497791 HI 4030.451 HI 0.2480702
IA 0.7234625 IA 3724.285 IA 0.4418699
ID 0.8005803 ID 3985.925 ID 0.3965426
IL 0.7750926 IL 3625.272 IL 0.2967923
IN 0.7842821 IN 3340.667 IN 0.4407808
KS 0.7564540 KS 3080.189 KS 0.5460994
KY 0.6146822 KY 2677.962 KY 0.5702171
LA 0.7281663 LA 3140.789 LA 0.4109971
MA 0.7166891 MA 3583.800 MA 0.4705547
MD 0.7747258 MD 3548.445 MD 0.5022664
ME 0.6691110 ME 3906.654 ME 0.3701062
MI 0.8128240 MI 3674.253 MI 0.2403944
MN 0.7788851 MN 3930.467 MN 0.5339856
MO 0.7172689 MO 3126.775 MO 0.3213523
MS 0.7863453 MS 2814.833 MS 0.5834045
MT 0.6304383 MT 2842.210 MT 0.2958767
NC 0.7111621 NC 2968.101 NC 0.4138277
ND 0.7747160 ND 4630.507 ND 0.4563457
NE 0.7872294 NE 3544.885 NE 0.4281692
NH 0.8158748 NH 4321.629 NH 0.7771365
NJ 0.6173459 NJ 2375.119 NJ 0.4494824
NM 0.6364223 NM 3197.618 NM 0.4810706
NV 0.6503394 NV 3828.667 NV 0.3716659
NY 0.6915197 NY 3272.148 NY 0.5808707
OH 0.7446526 OH 2932.200 OH 0.3783350
OK 0.7243434 OK 3581.889 OK 0.5329230
OR 0.6345352 OR 3476.621 OR 0.3825794
PA 0.6933387 PA 2880.607 PA 0.4076703
RI 0.7180824 RI 3263.167 RI 0.4701529
SC 0.7729372 SC 3514.634 SC 0.5635219
SD 0.7910627 SD 3855.144 SD 0.3322975
TN 0.8101791 TN 3610.759 TN 0.4810667
TX 0.7150158 TX 3258.318 TX 0.4729199
UT 0.7170194 UT 3564.913 UT 0.2829244
VA 0.7599587 VA 3311.952 VA 0.4560145
VT 0.7168919 VT 3978.316 VT 0.3694382
WA 0.6363874 WA 3479.738 WA 0.2414028
WI 0.7885404 WI 3818.935 WI 0.2936748
WV 0.5972293 WV 3209.811 WV 0.3914106
WY 0.6660177 WY 3098.733 WY 0.6763376
PR* 0.7064134 PR* 3185.101 PR* 0.2911827
* The predictions for PR in all Youth models were estimated using PY 2018 Q2 rolling-4 quarter participant data. PR has not reported annual PY 2018 data. Also, PR did not have sufficient data to generate a state fixed effect in all Youth models. As a result, the following fixed effects were applied to PR for the following models to get a reasonable prediction for PY 2020: Q2ER - minimum fixed effect minus a constant of 0.2, ME - minimum fixed effect minus a constant of 7500, MSG - maximum fixed effect (fixed effect closest to zero).

Wagner-Peyser

Employment Rate 2nd Quarter after Exit
Median Earnings 2nd Quarter after Exit
State PY.2020.Prediction State PY.2020.Prediction
AK 0.5730471 AK 5652.127
AL 0.7240636 AL 4537.612
AR 0.7114929 AR 5216.546
AZ 0.6782213 AZ 5659.525
CA 0.6213485 CA 6688.863
CO 0.6490536 CO 5591.663
CT 0.6883512 CT 6673.140
DC 0.5833086 DC 5923.111
DE 0.6727038 DE 5541.858
FL 0.6678732 FL 5428.962
GA 0.6997761 GA 5259.565
HI 0.5185943 HI 5784.788
IA 0.7368773 IA 6290.172
ID 0.7243805 ID 6010.742
IL 0.6997339 IL 6056.718
IN 0.7669739 IN 6723.104
KS 0.7071987 KS 5570.034
KY 0.5088738 KY 6310.797
LA 0.6037290 LA 5267.321
MA 0.6527698 MA 7446.162
MD 0.6638574 MD 6379.129
ME 0.6637191 ME 5881.156
MI 0.7426081 MI 6488.197
MN 0.6959114 MN 7649.497
MO 0.7179970 MO 5448.711
MS 0.6959967 MS 4290.817
MT 0.6677525 MT 5848.017
NC 0.7280073 NC 5105.204
ND 0.6882037 ND 7128.957
NE 0.7382435 NE 6375.141
NH 0.6566797 NH 7210.855
NJ 0.5279984 NJ 5584.417
NM 0.6346405 NM 5167.622
NV 0.7359067 NV 5495.777
NY 0.6770521 NY 6401.276
OH 0.7251873 OH 7549.556
OK 0.6327638 OK 5834.820
OR 0.6918925 OR 6594.112
PA 0.6898713 PA 5971.141
RI 0.7047359 RI 6918.476
SC 0.7039770 SC 5359.758
SD 0.7043701 SD 5338.662
TN 0.7394089 TN 5500.371
TX 0.6870895 TX 5801.082
UT 0.7426627 UT 6337.516
VA 0.7616434 VA 5675.033
VT 0.6369132 VT 5081.508
WA 0.6939107 WA 7150.277
WI 0.7468480 WI 6661.939
WV 0.6097227 WV 5418.792
WY 0.6514563 WY 5394.571
PR* 0.5954038 PR* 5299.680
* The predictions for PR in all Wagner-Peyser models were estimated using PY 2018 Q2 rolling-4 quarter participant data from the Adult program. PR has not reported annual PY 2018 data. Also, PR did not have sufficient data to generate a state fixed effect in all Wagner-Peyser models. As a result, the following fixed effects were applied to PR for the following models to get a reasonable prediction for PY 2020: Q2ER - minimum fixed effect minus a constant of 0.2, ME - average fixed effect minus a constant of 6000.

Full Model Variable Table

The table below shows which variables are included in which models. It also includes both the variable names used in the modeling process and the full name of the variables.

Variable Names
Adult
Dislocated Worker
Youth
Wagner-Peyser
Model Variable Full Variable Name Q2ER ME MSG Q2ER ME MSG Q2ER ME MSG Q2ER ME
female Female x x x x x x x x x x x
age1415 Age 14 to 15 x x x
age1617 Age 16 to 17 x x x
age1819 Age 18 to 19 x x x
age2021 Age 20 to 21 x x x
age2544 Age 25 to 44 x x x x x x x x
age4554 Age 45 to 54 x x x x x x x x
age5559 Age 55 to 59 x x x x x x x x
age60 Age 60 or more x x x x x x x x
hispanic Hispanic Ethnicity x x x x x x x x x x x
raceasian Race: Asian x x x x x x x x x x x
raceblack Race: Black x x x x x x x x x x x
racehpi Race: Hawaiian or Pacific Islander x x x x x x x x
raceai Race: American Indian x x x x x x x x x x x
racemulti Race: Multiple x x x x x x x x x x x
hsgrad Highest Grade Completed: High School Equivalency x x x x x x x x x x x
collegedropout Highest Grade Completed: Some College x x x x x x x x x x x
certotherps Highest Grade Completed: Certificate or Other Post-Secondary Degree x x x x x x x x x x x
associate Highest Grade Completed: Associate Degree x x x x x x x x
ba Highest Grade Completed: Bachelor Degree x x x x x x x x
associateorba Highest Grade Completed: Associate or Bachelor Degree x x x
gradschool Highest Grade Completed: Graduate Degree x x x x x x x x
empentry Employed at Program Entry x x x x x x x x x x
edstatentry In School at Program Entry x x x x x x x x x x x
disabled Individual with a Disability x x x x x x x x x x
veteran Veteran x x x x x x x x
englearner Limited English Proficiency x x x x x x x x x x
singleparent Single Parent x x x x x x x x
lowinc Low Income x x x x x x x x
homeless Homeless x x x x x x x x
offender Individual who was Incarcerated x x x x x x x x x x
dishomemaker Displaced Homemaker x x x x x x
yfoster Foster Care Youth x x
yparent Youth Parent or Pregnant Youth x x
basiclitdeficient Skills/Literacy Deficient at Program Entry x x x
recwages2qprior Received Wages 2 Quarters Prior to Participation x x x x x x x x
wages2qprior Wages 2 Quarters Prior to Participation x x x
longtermunemp Long-Term Unemployed at Program Entry x x x x x x x x x x
uiclaimant UI Claimant x x x x x x x x x
uiexhaustee UI Exhaustee x x x x x x x x
recsuppserv Supportive Services Recipient x x x x x x x x x x x
recneeds Received Needs-related Payments x x x x x x x x
recotherasst Received Other Public Assistance x x x x x x x x x
recssi SSI or SSDI Recipient x x x x x x x x x x x
rectanf TANF Recipient x x x x x x x x x x
recpell Pell Grant Recipient x
ynaa Youth Needing Additional Assistance x x
wp Received Wagner-Peyser Act Services x x x x x x x x x
daysinprog Median Days in Program x x x x x x x x x x
daysenrolled Median Days Enrolled in Education or Training x x x
daysenrolled_under30 Percent Enrolled in Education or Training Under 30 Days x x x
natresources Natural Resources Employment x x x x x x x x x x x
construction Construction Employment x x x x x x x x x x x
manufacturing Manufacturing Employment x x x x x x x x x x x
information Information Services Employment x x x x x x x x x x x
financial Financial Services Employment x x x x x x x x x x x
business Professional and Business Services Employment x x x x x x x x x x x
edhealthcare Educational or Health Care Employment x x x x x x x x x x x
leisure Leisure, Hospitality, or Entertainment Employment x x x x x x x x x x x
otheremp Other Services Employment x x x x x x x x x x x
publicadmin Public Administration x x x x x x x x x x x
ur Unemployment Rate Not Seasonally Adjusted x x x x x x x x x x x