Project Activities
Using administrative student panel data from NYCDOE the researchers will compare the longer-term (high school) success of two groups of students who were at-risk of being retained in their grade because of their scores on the spring district standardized tests. They will compare the high school outcomes of those who did not score high enough on a summer test to be promoted to those scored high enough on the summer test to be promoted. The main difference between the intervention and comparison groups will be retention status.
Structured Abstract
Setting
All non-charter New York City Department of Education schools serving general education students.
Sample
Students in grades 3 through 8 subject to the promotion policy in the 2003-04 through 2011-12 academic years and followed through the 2015-2016 school year.
The NYCDOE promotion policy ties grade retention decisions to standardized test performance. Students in grades 3 through 8 scoring in the lowest performance category (Level 1) on either the math or English Language Arts (ELA) spring assessments are at risk for being retained while those who score in the next highest category (Level 2) on both subjects are eligible for promotion. The policy provides students multiple attempts to demonstrate eligibility for promotion. Students with Level 1 scores on the mathematics or ELA spring assessment can demonstrate Level 2 proficiency through a portfolio review in June. Those who do not demonstrate Level 2 proficiency attend the City's summer instructional program, the Summer Success Academy (SSA). At the SSA's conclusion, students take a city summer assessment in the subjects in which they scored at Level 1 on the spring assessment. Those not demonstrating Level 2 performance on the summer assessment have the opportunity to do so through an August portfolio review. Those not demonstrating Level 2 or better performance by the end of this process (and not receiving an exemption by their principal or superintendent) are retained in grade.
Initial research
The researchers will use a regression discontinuity research design that exploits the fact that grade retention is largely determined by whether a student scores below the Level 2 cutoff on the summer assessment. Students scoring just above or just below the Level 2 cutoff are likely to be similar in all other dimensions so that comparisons between students who score just above and below the Level 1 summer assessment cutoff can be used to identify the effect of grade retention. As not all students who score below the Level 2 cutoff on the summer assessment are retained (some are promoted based on the August portfolio review or through exemptions), the researchers will use a fuzzy regression discontinuity design that uses the variation in the likelihood of being retained at the summer assessment Level 2 cutoff as an instrumental variable for retention. Provided that determinants of student outcomes other than the probability of retention do not change at the Level 2 cutoff, this approach will identify the impact of retention for students near the Level 2 cutoff whose retention status is affected by scoring above or below the cutoff. An implementation study will document the status of students who scored Level 1 in the summer (retained, promoted through portfolio review, or promoted through special exemption) and the distribution of these statuses across the 32 primary geographic school districts and by subgroups of interest.
Control condition
Students in the comparison group are those who are promoted to the next grade because of scoring Level 2 on the summer assessment who are nearest the threshold between Levels 1 and 2. The key difference between these two groups is that students in one are retained while those in the other are promoted.
Key measures
The impact analysis will address measures of high school persistence (e.g., dropout, high school graduation, credits accumulated) and behavioral problems (unexcused absences and disciplinary infractions). Exploratory analyses will address available high school achievement measures, including the New York Regents exams.
Data analytic strategy
Instrumental variables estimates of the effect of retention will be obtained using 2-stage least squares regression. Models will be estimated separately by grade in case effects vary by the grade of retention. The role of other moderating variables (e.g., race, gender, prior retention status) as well as mediating variables (e.g., future placement into special education, retention in subsequent grades) will be examined to obtain more detail on how retention affects student outcomes.
People and institutions involved
IES program contact(s)
Project contributors
Products and publications
Products: The products of this project will be evidence of the efficacy of NYCDOE's retention policy for grades 3 through 8. The evidence will be directly provided to the NYCDOE through the participation of department personnel on the project and to the research community and other education practitioners and policymakers through peer reviewed publications.
Publications:
Mariano, L. T., Martorell, P., & Berglund, T. (2024). The effects of grade retention on high school outcomes: Evidence from New York City schools. Journal of Research on Educational Effectiveness, 1-31.
Questions about this project?
To answer additional questions about this project or provide feedback, please contact the program officer.
