Project Activities
In this study, researchers carried out a randomized field trial in the state of West Virginia to test whether RM-CC5, a full-year digital core curriculum improves mathematics achievement for grade 5 students. In addition, the team examined how duration of intervention and intensity of use interacted with student outcomes. They also investigated the quality and fidelity of implementation.
Structured Abstract
Setting
This study was completed in 46 schools with fifth-grade classrooms in West Virginia. The state had already approved use of RM-CC5, established infrastructure for widespread adoption of a digital curriculum, and was in the process of adopting new standards and curricula.
Sample
A total of 56 schools were recruited and randomized to condition, and 23 treatment and 23 control schools completed the study. All fifth-grade classrooms in the treatment and control schools participated. Across these schools, achievement data was analyzed for 1,919 fifth-grade students. Student populations at participating West Virginia schools were predominantly White, rural and low-income.
Intervention
RM-CC5 focuses on key topics on the pathway to algebra, including place value system, fractions, rational numbers, geometric measurement, and graphing points in a coordinate plane. The intervention dramatically changes the role of the teacher. Rather than present instruction to the full class, teachers focus on intervening with specific students based on data from RM-CC5. Students receive instruction from a computer and work independently in RM-CC5 to solve mathematics problems. The intervention provides dynamic feedback (e.g., accuracy of responses, hints) and implements an adaptive instruction model based on each student's performance. The system incorporates motivational components, such as games. In addition, teachers receive up to 60 hours of professional development and implementation support, such as understanding progress reports on individual students and entire classroom.
Research design and methods
The study was a 2-year randomized controlled trial (RCT), implemented in SY2014–15 and SY2015–16. Schools were randomly assigned to a treatment or control condition and asked to participate for two full school years. Treatment teachers implemented RM-CC5 as their core grade 5 mathematics program; control teachers implemented their business-as-usual resources (see below). The research focused on findings from the second "measurement" year.
Control condition
Teachers of classrooms in the control condition followed their typical practice (e.g., they used existing curriculum resources and supplemental technologies), and teachers received professional development according to district policies and offerings. Control schools were asked to not adopt a blended or differentiated instruction-focused digital curriculum product during the study. For fairness across experimental groups, control schools were offered RM curricula for their second-grade classrooms (i.e., that were not participating in the research).
Key measures
Student achievement was measured with end-of-year grade 5 mathematics assessment, the West Virginia General Student Assessment (WVGSA). The WV Department of Education provided the data, as well as students' prior year grade 4 mathematics WVGSA. Students' productive dispositions toward mathematics were also measured using the Attitudes Toward Mathematics Inventory, and teachers' mathematical knowledge for teaching was measured using either the Pedagogical Content Knowledge Test or released items from the Learning Mathematics for Teaching Project. Treatment and control teachers completed electronic teacher logs. Data measuring student problem solving, use of remote tutoring, and teachers' use of reports was gathered directly from the RM system records. Measures of implementation were collected via classroom observations, stakeholder interviews, and teacher survey.
Data analytic strategy
Data analyses for the main treatment effect included a hierarchical linear regression model to examine mean differences in grade 5 achievement scores between students in treatment and control conditions, nested within schools, with prior (grade 4) achievement as a covariate. Secondary analyses examined differential impact for students at different strata of prior achievement, and across genders. Exploratory analyses examined differences within and across treatment classrooms of students' use of remote tutoring; teachers' use of RM reports to adjust instruction; students' productive time on task; and implementation compliance of both students and teachers. Regression models were conducted on achievement measures to examine potential associations between achievement and use metrics; these analyses were exploratory and did not afford inferences about causality.
Key outcomes
The main findings of this efficacy study were:
People and institutions involved
IES program contact(s)
Products and publications
Products:
Journal article, monograph, or newsletter
Roschelle, J., Gaudino, S., and Darling, S. (2016). Designing for Consistent Implementation of a 5th Grade Digital Math Curriculum. International Journal of Designs for Learning, 7(2).
Shechtman, N., Roschelle, J., Feng, M., and Singleton, C. (2019). An Efficacy Study of a Digital Core Curriculum for Grade 5 Mathematics. AERA Open, 5(2), 2332858419850482.
Proceeding
Herman, P., Bumgardner, K., Shechtman, N., and Feng, M. (2018). Teaching with a Fully Digital, Year-long Math Program: Learning Science Futures on the Front Line. In Kay, J. and R. Luckin (Eds.), Rethinking Learning in the Digital Age: Making the Learning Sciences Count,13th International Conference of the Learning Sciences (ICLS) 2018, Volume 3, (pp. 632-639). London, UK: International Society of the Learning Sciences.
Supplemental information
Co-Principal Investigators: Nicole Shechtman, Mingyu Feng, Corinne Singleton
- Treatment teachers implemented the RM-CC5 curriculum with adequate fidelity to the model. Across all treatment schools, students were logged into RM-CC5 for an overall estimated average of almost an hour a day.
- Mathematics achievement on the West Virginia end-of-year state assessment in the treatment group was no higher than achievement in the business-as-usual control group.
- Teachers were able to make dramatic changes in the organization of classrooms with a complete set of digital learning materials, a robust and well-designed program of teacher training and support, and enough time to implement them.
- Implementation of adaptive, digital curricula is complex and does not always reflect the needs of both the teachers and students.
- There were tensions between adaptive learning and coverage of all learning objectives measured on an end-of-year state assessment, between the degree of variation in student pace and a teacher's ability to interact with all students, and in balancing among the amount of time and attention teachers dedicated to full-group, small-group, and individualized instruction.
- The researchers produced information about the efficacy of Reasoning Mind compared to typical instructional practice, and about the implementation needs and challenges of digital curricula that emphasize adaptive, blended approaches
- These findings have been shared through peer-reviewed publications and professional conferences.
Questions about this project?
To answer additional questions about this project or provide feedback, please contact the program officer.
