Project Activities
Researchers will employ an iterative design-based process whereby the team will experiment with different forms of hardware interfaces to organize and present student data to teachers, including augmented reality, heads-up displays, and tablet and mobile apps. This research will inform prototyping of tools to support teacher instruction. After development is completed in year 4, the team will carry out a pilot study in middle-school mathematics classes to measure the effect of the tool on teacher-student interactions, and on student learning processes and outcomes.
Structured Abstract
Setting
The study will take place in middle schools in a large urban district in Florida.
Sample
The study will include 60 grade 7 to 8 mathematics classrooms, approximately 1,200 students, and the teachers in the classrooms.
Intervention
The tool will display mathematical data in real time and in new formats through the use of mixed reality, heads-up displays, and tablet and mobile apps as students solve complex problems using intelligent tutoring systems. The tool will generate information to enhance a teacher's awareness of who is struggling and aid decision making regarding who to help and how, and will improve teacher-student interactions by providing functionality and easy access to personalized resources to better focus and target such interactions. The team will also develop and publish an Application Programming Interface to allow any learning platform implementing to communicate with the tool. By the end of the project, the tool will be integrated with two commercially available ITS systems, potentially providing the opportunity for widespread use by thousands of math teachers and hundreds thousands of students.
Research design and methods
To develop the tool, the researchers will use a design-based implementation procedure where iterations of research and development will occur until feasibility, usability, and learning aims are met. The research team will experiment with multiple hardware interfaces to present data, including mixed reality, heads-up displays, and tablet and mobile apps, to create an orchestration tool. After development is complete, a classroom study with sixty grade 7 and 8 classrooms will examine how the tool affects teacher behaviors and students' learning processes, and whether it holds promise to improve student learning. Researchers will implement the tool within two existing commercially available tutoring systems which all treatment and control classes will use.
Control condition
Teachers in the control condition will be provided with a limited-functionality version of the orchestration tool that does not provide real-time recommendations or analytics.
Key measures
Researchers will gather data using student log data, teacher judgment data, and assessments. The assessments will include multiple-choice and fill-in-the-blank questions that align with Mathematics Florida Standards which are nearly identical to the Common Core State Standards in middle school grades.
People and institutions involved
IES program contact(s)
Products and publications
Products: Researchers will produce a tool for teachers that will present mathematical data in real time in new formats through the use of mixed reality, heads-up displays, and tablet and mobile apps as students solve complex problems using intelligent tutoring systems. Dissemination activities will include distribution of the tool to educational practitioners as well as producing peer reviewed publications with findings from the project.
ERIC Citations:Find available citations in ERIC for this award here.
Journal articles
Holstein, K., McLaren, B.M., & Aleven, V. (2019). Co-designing a real-time classroom orchestration tool to support teacher-AI complementarity. Journal of Learning Analytics, 6(2), 27-52. Full text
Proceedings
Fancsali, S. E., Holstein, K., Sandbothe, M., Ritter, S., McLaren, B. M., & Aleven, V. (2020). Towards practical detection of unproductive struggle. In International Conference on Artificial Intelligence in Education (pp. 92-97). Springer, Cham. Full text
Holstein, K., McLaren, B.M., & Aleven, V. (2019, June). Designing for Complementarity: teacher and student needs for orchestration support in ai-enhanced classrooms. In Proceedings of the 20th International Conference on Artificial Intelligence in Education (pp. 157-171). Springer, Cham. Full text
Zhang, C., Huang, Y., Wang, J., Lu, D., Fang, W., Stamper, J., ... & Aleven, V. (2019). Early detection of wheel spinning: comparison across tutors, models, features, and operationalizations. Proceedings of the 12th International Conference on Educational Data Mining Montreal, QC, Canada, July 2-5, 2019.Full text
Related projects
Supplemental information
Co-Principal Investigator: McLaren, Bruce
Data Analytic Strategies: During the iterative development period, the research team will use observational and log data to understand how expert teachers use results to inform instruction, as well as data mining studies to create recommender models to support awareness and decision making. In the pilot study the research team will use analysis of variance techniques to compare students in the treatment and control conditions. Additionally, researchers will use within-subjects comparisons for all students to test for learning gains by comparing performance at pre- and post-exposure to the intervention, and performance at near and far transfer.
Questions about this project?
To answer additional questions about this project or provide feedback, please contact the program officer.
