At the 28th annual New England Statistics Symposium (NESS) at Boston University on April 21st, mathematics majors Cortney Logan '12 and Kathleen Zarnitz '12 shared the top prize in the undergrad poster competition for their research project titled "Assessing Value-Added Models (VAM) for teacher effectiveness."
VAM models are being used to compute percentile rankings of schools and individual teachers based solely on the annual standardized test scores of students (for example, the recently published rankings of teachers in the New York City school system). Advocates claim that the models can isolate and accurately measure individual teacher effects. Critics question whether the estimates have enough precision to reliably estimate percentile rankings, and whether biasing factors like socioeconomic status have been properly accounted for.
Logan and Zarnitz focused on the precision issue. They simulated 10,000 pairs of cohorts (classes) with characteristics chosen to match published data from real schools.
Zarnitz examined the power of a commonly used model (i.e., the probability of detecting differences among teachers when differences are actually present) with sample sizes that are realistic for individual teachers. For example, 28 test scores might represent a full year of data for a typical elementary school teacher. Her results suggest that with these class sizes, only the largest differences can be reliably detected.
Logan investigated the effect of missing data, that is, what happens if the weakest students are systematically excluded. The results suggest that rankings are quite sensitive to censoring of lower performing students.
Both students were awarded copies of "The Elements of Statistical Learning - Data Mining, Inference, and Prediction" by Hastie, Tibshirani, and Friedman. Their research was conducted as a term project in the Mathematical Modeling course (Mathematics capstone) taught by Assistant Professor of Mathematics Eugene Quinn.
For more information, contact Communications and Media Relations at 508-565-1321.