Project Activities
During years 1–3 of the project, the research team will conduct user-testing and design-based research to create mobile tutors and teacher-parent nudges in an integrated design. Concurrently, during years 1–4, approximately 130 sixth to eighth grade students will participate in 5 experimental studies to evaluate the effects of the intervention on student math outcomes.
Structured Abstract
Setting
For experiments 1–4, the research will take place primarily in schools that are ethnically and racially diverse with a high percentage of low-SES students. For experiment 5, the research also will take place in schools that are primarily White and have a high percentage of high-SES students to provide an SES control.
Sample
During years 1-3 of the project, sixth to eighth grade students, parents, and teachers will participate in user-centered research and co-design to create mobile tutors and teacher-parent nudges in an integrated design. Concurrently, during Years 1-4, approximately 130 sixth to eighth grade students will participate in 5 experimental studies to evaluate the effects of the intervention on student math outcomes.
The project team will test the synergistic effect of three interventions intended to create more favorable conditions for homework: (1) data-optimized deliberate practice supported by AI-based tutoring software, (2) delivery of the tutoring software on smartphones, with an offline mode so that students can work on their phones even when having no connectivity for long periods of time, and (3) social/motivational support through teacher-parent nudges to help students stay motivated. As a platform for the research, the research team will use the free-access Mathtutor platform, which hosts AI-based tutoring software with comprehensive mathematics content for middle school.
Research design and methods
The research team will conduct five experiments to test the separate and joint effects of the interventions to create favorable homework conditions. The experiments will be interleaved with the design and development activities. For all five experiments, students will be randomly assigned within-class using a two-unit planned crossover design—Group A will receive the smartphone and tutor treatment for unit 1 and Group B will receive the treatment for unit 2—and counterbalanced order across subjects.
Control condition
The control condition teachers will deliver homework without technology support. Students in the control condition will be asked to solve fixed problem sets on paper, without immediate individualized feedback or adaptive mastery learning.
Key measures
The research team will measure student mastery of key skills in targeted mathematics areas using both researcher-developed items and standardized test items (including items from the Pennsylvania System of School Assessment (PSSA) and the National Assessment of Educational Progress (NAEP)). The research team will measure student engaged time with Mathtutor and the number of opportunities to practice knowledge components based on tutor log data.
Data analytic strategy
The research team will use hierarchical linear models with students nested within classes within teachers to compare learning gains between conditions.
Cost analysis strategy
The research team will conduct a cost analysis following the "ingredients" method to identify, quantify, and price (including time costs) each of the elements required for the intervention and the control conditions. The research team will calculate the cost-effectiveness of the intervention by calculating the costs associated with the intervention considering student gains.
People and institutions involved
IES program contact(s)
Project contributors
Products and publications
Products: The research team will generate new scientific knowledge regarding how best to use advanced learning technologies to support effective homework for middle school math. Researchers will disseminate project information and findings through reports and peer-reviewed publications to reach relevant educational and scientific communities. If successful, the project will result in a proof-of-concept of combining (1) data-optimized deliberate practice supported by AI-based tutoring software, (2) delivery of the tutoring software on smartphones, and (3) social/motivational support through teacher-parent nudges to improve student math outcomes.
ERIC Citations: Find available citations in ERIC for this award here.
Related projects
Questions about this project?
To answer additional questions about this project or provide feedback, please contact the program officer.
