TVAAS

Misconception: Student growth is correlated with certain demographic variables, so TVAAS should control for demographics.

It is widely known that students with certain socioeconomic or demographic (SES/DEM) characteristics tend to score lower, on average, than students with other SES/DEM characteristics, and there is concern that educators serving those students could be systematically disadvantaged in the modeling, leading some to believe that TVAAS should control for demographics.

However, this adjustment is not statistically necessary for TVAAS because, like other sophisticated value-added models, it uses all available testing history for each individual student and does not exclude students who have missing test data. In essence, each student serves as their own control for SES/DEM influences, and, to the extent that SES/DEM influences persist over time, they are already represented in the student's data.

TVAAS in Theory

As a 2004 Ed Trust study stated, specifically with regard to the TVAAS modeling:

[I]f a student's family background, aptitude, motivation, or any other possible factor has resulted in low achievement and minimal learning growth in the past, all that is taken into account when the system calculates the teacher's contribution to student growth in the present.*

This approach has been confirmed through a variety of robust statistical analyses. In 2004, a SAS and Vanderbilt team published a study that closely examined SES/DEM adjustments and concluded:

SES and demographic covariates add little information beyond that contained in the covariance of test scores.#

This finding has been confirmed independently by prominent value-added experts who have replicated a variety of value-added models, including TVAAS models. More specifically, a 2007 paper by RAND researchers J.R. Lockwood and Dan McCaffrey explicitly verified the models used for TVAAS district, school, and teacher reporting when they wrote:

William Sanders, the developer of the TVAAS model, has claimed that jointly modeling 25 scores for individual students, along with other features of the approach is extremely effective at purging student heterogeneity bias from estimated teacher effects...The analytic and simulation results presented here largely support that claim.†

An economist-based perspective by UCLA researchers Kilchan Choi, Pete Goldschmidt, and Kyo Yamashiro provided a similar finding in their study comparing value-added models:

First, adding in an adjustment for student SES (as measured by eligibility for free- or reduced-price lunch) adds very little once a student's initial status is controlled...This indicates that student initial status captures many of the effects that SES is attempting to measure. In other words, by controlling for initial status, the model already captures the preceding effects that SES might have on students.‡

TVAAS in Practice

Although the statistical literature presents evidence that educators are not advantaged or disadvantaged by the type of students that they serve in sophisticated value-added reporting, actual data might be the most readily apparent evidence to support this belief.

The first figure below plots the percentage of tested students who are considered economically disadvantaged at each school in Tennessee against the school's growth index (the value-added estimate divided by its standard error) for TCAP Mathematics in grades 4–8 in 2019. Regardless of the school's student characteristics, there is little to no correlation to the growth index. In other words, the dots representing each school do not trend up or down as the percentage increases; the cluster of dots is fairly even across the spectrum.

TENNESSEE GROWTH INDEX VERSUS PERCENTAGE TESTED ECONOMICALLY DISADVANTAGED BY SCHOOL

Figure 2 provides similar information for the percentage of minority students. Again, there is little to no correlation to the growth index.

Figure 2: Tennessee Growth Index Versus Percentage Tested Minority by School