IES Staff
Allen Ruby
Associate Commissioner for Policy and Systems
NCER
About
Allen Ruby joined IES in 2007 and is the Associate Commissioner for Policy and Systems. He is also the Program Officer for the Evaluation of State and Local Education Programs and Policies topic and the Researcher–Practitioner Partnerships in Education topic. Before coming to IES, Dr. Ruby was at the Center for the Social Organization of Schools at Johns Hopkins University, where his research focused on school improvement and science education reform in low-performing middle schools. Dr. Ruby received his Ph.D. in policy analysis from the Rand Graduate School.
Associated IES Content
Grant
Equitable Rostering Solution: A Project on Evidence-Based Teacher-Student Assignment
Equitable Rostering Solution (ERS) is a computerized student-teacher rostering tool intended to optimize the match between student and teacher by using profiles of both student and teacher needs and strengths. The research team will further develop ERS and validate its ability to: (a) increase student equity in access to effective instruction; (b) reduce opportunity and achievement gaps between high and low-performing students; (c) improve all students' academic achievement and growth; and (...
Federal funding program:
Award number:
R305A210356
Grant
Using Machine Learning Methods to Improve Regression Discontinuity Designs
The purpose of this project is to develop an approach that incorporates machine learning methods into a regression discontinuity design to improve precision and reduce bias in the treatment effect estimates.
Federal funding program:
Award number:
R305D200008
Grant
Enhancing the Applicability of an Evidence-Based Intervention to Reduce Social Anxiety and Improve Educational Outcomes in Black American Students in Urban Public High Schools
In this project, the researchers will adapt Skills for Academic and Social Success (SASS), a school-based, group cognitive-behavioral intervention for students with social anxiety. Social anxiety is a debilitating fear that can arise in social situations (such as working with others) or while performing activities (speaking up in class). For a student with social anxiety, school settings can be particularly difficult, and he or she may be prone to increased school absences, lower academic pe...
Federal funding program:
Award number:
R305A200013
Grant
Developing Methodological Foundations for Replication Sciences
Using the Causal Replication Framework, the project team developed tools and methods to improve the design, implementation, and analysis of replication studies. The project team focused on three aims: (1) develop and improve research designs for evaluating the replicability of causal results, (2) create planning tools and diagnostic measures to improve the implementation of replication designs, and (3) build analytic tools for evaluating replication success from well-implemented designs.
Federal funding program:
Award number:
R305D190043
Grant
Assessing Generalizability and Variability of Single-Case Design Effect Sizes Using Multilevel Modeling Including Moderators
The increasing number of published single-case experimental design (SCED) studies in education sciences can be used to inform policy, research, and practice decisions. Multilevel meta-analysis is one technique to summarize the literature in the SCED field in a standardized, objective, reliable, and valid manner. A three-level meta-analysis is recommended to address the hierarchical structure of the SCED data with effect sizes clustered within cases and cases clustered within studies.
Federal funding program:
Award number:
R305D190022
Grant
Addressing Small Sample and Computational Issues in Mixture Models of Repeated Measures Data with Covariance Pattern Mixture Models
In this project, the research team applied a covariance pattern model approach to growth mixture models so that they can be applied more reliably within contexts in which they are already applied and to lower the data requirements needed to apply the method so that researchers with more modest samples (e.g., hard-to-reach populations) can use the method. The project team developed an alternative way to fit growth mixture models that is less demanding computationally, carried out simulations ...
Federal funding program:
Award number:
R305D190011
Grant
Bayesian Dynamic Borrowing: A Method for Utilizing Historical Data in Education Research
In education research, long-standing, large-scale educational assessments, such as the National Assessment of Educational Progress (NAEP ), the Programme for International Student Assessment (PISA), and the Early Childhood Longitudinal Program (ECLS), provide historical data that can be used to inform policy-relevant research. However, statistical analyses of current large-scale assessment data may not make use of this historical information, and it may be difficult to do so when applying co...
Federal funding program:
Award number:
R305D190053
Grant
Site Selection When Participation is Voluntary: Improving the External Validity of Randomized Trials
Randomized controlled trials (RCTs) in education are typically conducted in districts and schools that were not selected to represent broader populations of policy interest (e.g., all schools nationwide that could have implemented the education intervention). As a result, findings from these studies may not generalize to these populations. In this project, the research team developed products to help education researchers decide whether to use random or balanced sampling in selecting sites f...
Federal funding program:
Award number:
R305D190020
Grant
Does Incentivizing Value-Added Make it More or Less Meaningful?
The purpose of this project was to explore how the malleable incentives implicit in teacher tenure policy are related to teacher practice and student achievement by estimating whether the predictiveness of the incentivized measures changes after they become incentivized.
Federal funding program:
Award number:
R305A190148
Grant
Identifying Optimal Scoring Metrics and Prompt type for Written Expression Curriculum-Based Measurement
The purpose is to develop a revised assessment tool, Automated Written Expression Curriculum-Based Measurement (aWE-CBM), by combining advances in automated text evaluation with revised WE-CBM procedures for students in 3rd through 5th grade. Developing aWE-CBM will better align scoring with writing development theory and improve the technical adequacy and feasibility of screening for writing in upper elementary grades.
Federal funding program:
Award number:
R305A190100
Grant
Model-based Multiple Imputation for Multilevel Data: Methodological Extensions and Software Enhancements
Missing data are exceedingly common in educational research. Education research studies have missing data because students opt out of achievement testing, skip test items, or move to a different school district, among many other reasons. A previous IES award funded the development of a data analysis application, called Blimp, that addresses this issue by filling in missing values using sophisticated predictive models. The purpose of the current work was twofold: expand Blimp's missing data i...
Federal funding program:
Award number:
R305D190002
Grant
Planning Randomized Controlled Trials in Community Colleges
The project team developed products to support community college researchers with the same type of empirically based benchmarks, design parameters, and tools currently available for K-12 researchers planning and interpreting randomized controlled trials (RCTs).
Federal funding program:
Award number:
R305D190025
Grant
ASPIREPlus: A Researcher-Practitioner Partnership to Promote Self-Determination and More Positive Student Outcomes
The partnership team will examine whether two programs, the Active Student Participation Inspires Real Engagement (ASPIRE) program and the Self-Determined Learning Model of Instruction (SDLMI) are associated, on their own and in combination, with increased student involvement in transition planning (planning for post-high school) in middle and high school and improved transition outcomes (post-secondary education/training, employment, and community inclusion) for students with disabilities.
Federal funding program:
Award number:
R305H190017
Grant
The Language of Written Argumentation and Explanation: Individual Developmental Trajectories From 4th to 8th Grade
The purpose of this project is to examine how language skills change over time and how this supports academic writing in 4th through 8th grades. Language skills are necessary for writing complex texts as students get older, but research has not yet explored how these skills develop over time. The current study will explore how academic language (language features used in educational and scientific contexts) for writing, writing quality, and academic language for reading develop and change ov...
Federal funding program:
Award number:
R305A170185
Grant
An Efficacy Follow-Up Study of the Long-Term Effects of Reading Recovery Under the i3 Scale-Up
The purpose of this project is to test the sustained efficacy of Reading Recovery, an intensive one-on-one reading instruction program for the lowest-achieving first grade students, on state test scores in third and fourth grade. Many first grade students struggle with reading and for some of these students, low literacy achievement in first grade can set them up for continued difficulty in literacy throughout elementary school and beyond. The Reading Recovery program is based on the idea th...
Federal funding program:
Award number:
R305A170171
Grant
A User-Friendly Tool for Designing Cluster Randomized Trials with Power and Relevance
The purpose of this project was to develop a user-friendly webtool for planning cluster randomized trials (CRTs) based on generalizability and statistical power for testing moderator effects, not just average treatment effects. Funding institutions and statistical developments have started to shift the focus of causal impact studies beyond the detection of average treatment impact to understanding moderators of these impacts and the detection of effects for specific inference populations, bu...
Federal funding program:
Award number:
R305D180002
Grant
Cleveland Alliance for School Climate Research
CMSD has administered a school climate survey to secondary students for the last eight years. Under this project, researchers will use the data from that survey to explore the relationship between secondary school climate and student outcomes (social and behavioral competencies and academic ones) overall and for certain student subgroups within schools.
Federal funding program:
Award number:
R305H170068
Grant
Improving Student Learning and Engagement Through Gamified Instruction: Evaluation of iPersonalize
Research shows that 72 percent of youth are engaged in computer gaming after the school day ends. Because of its appeal to students, gaming has increasingly been incorporated into education technology used in classrooms. The present study investigates the impact of iPersonalize, a personalized gamified learning environment developed by the Fullerton School District, on student engagement and student achievement.
Federal funding program:
Award number:
R305L170006
Grant
Statistical Power When Adjusting for Multiple Hypothesis Tests: Methodology Expansions and Software Tools
This project extended and validated methods estimating power for different definitions of power and for several multiplicity adjustments commonly used in confirmatory analyses so that researchers who are planning randomized controlled trials (RCTs) with any number of common designs and who will adjust for multiple tests across multiple outcomes will be able to estimate power, minimum detectable effect sizes (MDESs), and sample sizes.
Federal funding program:
Award number:
R305D170030
Grant
Development of Statistically Sound Methods and User-friendly Software for Performing Data Forensics
Federal funding program:
Award number:
R305D170026
Grant
Uses of Posterior Distributions for Statistical Inference from Assessment Surveys
Federal funding program:
Award number:
R305D170013
Grant
An Experimental Evaluation of the Impact of HOME WORKS! The Teacher Home Visit Program
The increasing popularity of teacher home visit programs suggests that many educators believe home visits can make a difference in improving school attendance and student achievement outcomes. This study will evaluate the effects of HomeWorks! (HW!), a teacher home visit program that has operated in the greater St. Louis area and surrounding communities since 2007, using a cluster randomized control trial (RCT) design.
Federal funding program:
Award number:
R305L170005
Grant
Understanding the Effect of the North Carolina Read to Achieve Program on Student Outcomes
Elementary grade retention is a growing policy option nationwide for ensuring that students are prepared for upper elementary school and beyond. This research project will study the causal impacts of North Carolina's Read to Achieve (RtA) Program on student reading and grade retention outcomes in grades four and five.
Federal funding program:
Award number:
R305L160017
Grant
A Multisite Randomized Controlled Trial (RCT) of Descubriendo la Lectura (DLL) Topic Area: English learners
This study evaluated the efficacy of Descubriendo la Lectura (DLL) program, the Spanish reconstruction of Reading Recovery, an English literacy program. The population of Spanish-speaking ELs in U.S. schools is large and growing, and many of these students choose to enroll in bilingual instructional programs in early grades. As the number of students receiving initial literacy instruction in Spanish increases, the need for a Spanish language early-intervention program for students at risk of...
Federal funding program:
Award number:
R305A160060
Grant
Cognitive and Motivational Contributors to Reading Comprehension in English Learners (ELs) and English Monolinguals (EMs): Different or Similar Growth Patterns
The purpose of this project is to examine known cognitive factors (e.g., word level factors, oral language, strategy use), as well as novel cognitive malleable factors (e.g., EF skills: inhibitory control, working memory, and cognitive flexibility) and reading engagement components involved in the development of reading comprehension in English Learners (ELs) and English Monolinguals (EMs). Although ELs have adequate word reading skills in the early elementary grades, they perform relatively...
Federal funding program:
Award number:
R305A160280
Grant
A Clustered Randomized Trial of Function-based Check-In/Check Out
One approach to increasing student engagement and achievement is through the reduction of student disruptive behavior and positive reinforcement of behaviors that support academic success. Under the framework of School-Wide Positive Behavior Intervention and Supports (SWPBIS), schools use behavior analysis and positive reinforcement in a tiered system that increases in intensity for students who do not respond to the previous tier. Milwaukee Public Schools (MPS) uses a three-tiered system in...
Federal funding program:
Award number:
R305L160013
Grant
The Effects of Leveled Literacy Intervention on Secondary Students in the Oakland Unified School District
Struggling readers in secondary schools may end up graduating without the literacy skills needed for postsecondary education and the labor market. These students may be several years below the reading level for their grade due to lags that began in primary school or because they are English learners. The Leveled Literacy Intervention (LLI) is a short-term, intensive intervention system designed to help teachers provide daily, small-group instruction for students who are not achieving grade-...
Federal funding program:
Award number:
R305L160008
Grant
Response Ratio Effect Sizes for Single-Case Designs with Behavioral Outcome Measures
This project developed models and methods of estimating response ratio effect sizes in single-case designs. Current methods for response ratios make the assumption that outcomes are drawn independently from a process that is stationary in each phase, an assumption which is typically unrealistic when the observations are made repeatedly on the same individual. By elaborating an existing measurement model for behavioral observations, statistical models that can accommodate both serially-correl...
Federal funding program:
Award number:
R305D160002
Grant
Meta-analysis of Incorrectly Analyzed Studies
The purpose of this project was to develop a correction strategy for significance tests of incorrectly analyzed studies (i.e., single-level regression was conducted instead of multilevel modeling). The correction strategy currently used by the What Works Clearinghouse (WWC) frequently leads to very low statistical power, leading to the potentially inaccurate conclusion that from a statistical significance standpoint, the tested interventions are ineffective. The researcher developed robust m...
Federal funding program:
Award number:
R305D160016
Grant
Methods for Addressing Measurement Error Issues in Longitudinal Educational Studies
The purpose of this project was to investigate different methods of using a two-stage framework to address measurement error in the estimation of latent theta scores obtained from standardized tests through item response theory (IRT). In the two-stage approach, an appropriate measurement model is first fitted to the data, and the resulting theta scores are used in subsequent analysis. Potential benefits of this approach include clearer definition of factors, convenience for secondary data an...
Federal funding program:
Award number:
R305D170042
Grant
Estimating Population Effects: Incorporating Propensity Scores with Complex Survey Data
Federal funding program:
Award number:
R305D150001
Grant
Multilevel Modeling of Single-subject Experimental Data: Handling Data and Design Complexities
Federal funding program:
Award number:
R305D150007
Grant
How Does Implementation Mediate Program Impacts? Improving Current Evaluation Research Methods
The purpose of this project was to advance evaluation research methods that address questions about the mediational influences of education initiatives. The structure of education interventions makes investigating the relationship between implementation and impact particularly challenging, because education interventions frequently work through classroom teachers.
Federal funding program:
Award number:
R305D150016
Grant
Further Development of Effect Size Estimators for Single-case Designs: Extensions to Trend and Diverse Outcome Metrics
The purpose of this project is to extend previous work done through an IES-funded grant (A d-Estimator for Single Case Designs) by developing effect size measures that can account for trends when there is a repeated assessment of an outcome over time - and that can be used with outcome variables measured as count or rate data, which are common in single-case research. In addition, the research team will develop approaches to gauge the sensitivity of a planned study to detect effects and soft...
Federal funding program:
Award number:
R305D170041
Grant
Web-based Software to Estimate Power and Causal Effects of Various Regression Discontinuity Designs
Federal funding program:
Award number:
R305D150029
Grant
Attrition Benchmarks Across Students and School Contexts: Evidence from Student Mobility in National Longitudinal Survey Data
This research provided empirical benchmarks for the amount of attrition that can arise from natural rates of student mobility. Student-level, school-based longitudinal evaluations of education practices and interventions often encounter participant attrition when students move away from, or drop out of, study schools. In some cases, this mobility-induced attrition simply decreases sample size, weakening the study's power to detect a treatment effect. Attrition can also introduce bias into an...
Federal funding program:
Award number:
R305D150026
Grant
Multiple Imputation Procedures for Multilevel Data
Federal funding program:
Award number:
R305D150056
Grant
Understanding and Measuring Treatment Effect Heterogeneity in Large Scale Experiments and Pseudo-Experiments in Education
Federal funding program:
Award number:
R305D150040
Grant
Multilevel Item Bifactor Models with Semi-Nonparametric Latent Densities
The purpose of this project was to develop a better way to deal with measurement error in predictors and to address the impact of non-normality in latent variable distributions. Predictors that contain measurement error yield attenuated correlation or regression coefficient estimates, which results in bias in treatment effect estimates. This problem is exacerbated in multilevel models when level-1 values are simply aggregated to an upper level as group means. By introducing a multilevel and ...
Federal funding program:
Award number:
R305D150052
Grant
Hierarchical Network Models: Mediation and Influence
Federal funding program:
Award number:
R305D150045
Grant
Using Projective Unidimensional Models for Measuring Multidimensional Educational Data
Federal funding program:
Award number:
R305D150051
Grant
Statistical Methods for Using Rigorous Evaluation Results to Improve Local Education Policy Decisions
Federal funding program:
Award number:
R305D150003
Grant
Writing for English Language Learners (WELLS): Exploring the Relationship Between Writing Instruction and Student Outcomes
The ability to write well is a foundational skill necessary for success at school and beyond; however, English language learners (ELLs) consistently perform at lower levels in writing than proficient English speakers. Little is known about how best to improve the writing performance of ELLs. In this project, the research team aims to explore two key malleable factors that have the potential to improve ELLs writing outcomes - the quality of sheltered instruction which is designed to provide a...
Federal funding program:
Award number:
R305A160405
Grant
Psychometric Models for In-Classroom Observational Measures of Teaching with Applications to Value-Added Modeling
Value-added modeling is an approach to teacher evaluation that yields information about teacher performance as a function of student scores on standardized tests. While value-added modeling is based on student outcomes, this technique does not allow evaluators to focus on what teachers are doing or should be doing in their classrooms to influence those outcomes. Observational instruments offer the possibility of identifying in-classroom practices that are associated with student achievement....
Federal funding program:
Award number:
R305D140035
Grant
Robustness of Comparative Interrupted Time Series Designs in Practice
Much research in education is designed to address causal questions. By randomly assigning schools, classrooms, or students to treatment conditions, randomized controlled trials (RCTs) ensure that the treatment and control groups are equivalent in expectation, but RCTs are not always feasible to implement. When RCTs are not possible, education researchers rely on quasi-experimental research designs to address causal questions, but it is only appropriate to do so when these designs produce tru...
Federal funding program:
Award number:
R305D140030
Grant
An Empirical Analysis of Two Methodological Issues for Education Research Caused by Variation in Program Impacts
Federal funding program:
Award number:
R305D140012
Grant
Novel Models and Methods to Address Measurement Error Issues in Educational Assessment and Evaluation Studies
Measurement error issues adversely affect results obtained from typical modeling approaches used to analyze data from assessment and evaluation studies. In particular, measurement error can weaken the validity of inferences from student assessment data, such as the inferences made from the results of using value-added models. Measurement error also can reduce the statistical power of impact studies and can diminish the ability of researchers to identify the causal mechanisms that lead to an ...
Federal funding program:
Award number:
R305D140046
Grant
Measuring Oral Reading Fluency: Computerized Oral Reading Evaluation (CORE)
The University of Oregon sought to develop and validate a new computerized assessment system of oral reading fluency (ORF) for use with students in Grades 2 through 4, called Computerized Oral Reading Evaluation (CORE). CORE contains an automated scoring algorithm based on a speech recognition engine, shorter passages, and a latent variable psychometric model, collecting word-level data that are used in a model-based approach to scale ORF scores with increased reliability compared to trad...
Federal funding program:
Award number:
R305A140203
Grant
Solving Difficult Bayesian Computation Problems in Education Research Using STAN
Federal funding program:
Award number:
R305D140059
Grant
Estimating Statistical Power in Impact Evaluations when Making Adjustments for Multiple Hypotheses
Multiplicity adjustments to p-values protect against spurious statistically significant findings when there are multiple statistical tests (e.g., due to multiple outcomes, subgroups or time points). An important consequence of these adjustments is a change in statistical power. It is typically argued that multiplicity adjustments result in a loss of power, which can be substantial. Current practice for ensuring that impact evaluations in education have adequate statistical power does not tak...
Federal funding program:
Award number:
R305D140024
Grant
Development of Accessible IRT-Based Models and Methodologies for Improving the Breadth and Accuracy of Item Option-Scored Diagnostic Assessments
Federal funding program:
Award number:
R305D140023
Grant
A General Framework for Statistical Power Analysis with Non-normal and Missing Data through Monte Carlo Simulation
The increasing complexity of education research poses great challenges on existing techniques used to estimate statistical power. For example, education research often involves longitudinal and multilevel designs as well as advanced techniques such as structural equation and multilevel models. Furthermore, practical data in education are often not normally distributed and are incomplete. Without careful consideration of the complexity of study designs and the impact of non-normal data and mi...
Federal funding program:
Award number:
R305D140037
Grant
Improving Education Policy Analysis and Practice by Addressing Test Score Measurement Error
Federal funding program:
Award number:
R305D140032
Grant
Development of a Supplemental Instructional Course in Reading and Writing Arguments for Ninth Graders at Risk of Leaving School Before Graduating
The Common Core State Standards (CCSS) emphasize reading complex texts and writing argumentative and explanatory essays across content areas, including social studies. However, the National Assessment of Educational Progress shows that 70 percent of twelfth graders do not read and write proficiently. Additionally, research shows that students who are behind academically by the end of ninth grade are at greater risk for leaving school before graduating. Supplementary literary instruction in h...
Federal funding program:
Award number:
R305A140390
Grant
Beyond Triage: A Randomized Experiment in Sustained Pre-College Advising
This project evaluated a state-sponsored college access program known as Advise Texas (Advise TX) run by the Texas Higher Education Coordinating Board (THECB). The primary goal of Advise TX is to provide a uniform and readily available college access program that will raise the rates of college enrollment and completion among low-income, first-generation-college, and underrepresented high school students in Texas.
Federal funding program:
Award number:
R305E130009
Grant
Assessing the Fit of the Statistical Model Used in the National Assessment of Educational Progress
Federal funding program:
Award number:
R305D120006
Grant
Curricular Reform and Classroom Peer Ability: School-Specific and Citywide Effects
Focusing on mathematics education reform initiatives in Chicago Public Schools (Algebra for All starting in 1997 and Double-Dose Algebra starting in 2003), the researchers seek to gauge and start to account for the variability in the effects of the reform across different schools. Specifically, the research team seeks to gauge the effects of course taking, classroom peer composition, and their interaction on student mathematics performance, and to investigate the extent to which prior studen...
Federal funding program:
Award number:
R305A130703
Grant
Matching Strategies for Observational Studies with Multilevel Data in Educational Research
Federal funding program:
Award number:
R305D120005
Grant
Hierarchical Network Models for Education Research
Federal funding program:
Award number:
R305D120004
Grant
The Effect of Dual-Language Immersion on Student Achievement in the Portland Public Schools
Dual-language immersion programs seek to help the growing number of language minority students learn English and achieve academically, while giving language majority students the opportunity to develop proficiency in another language. These programs differ fundamentally from traditional foreign language programs in two important ways: 1) teachers deliver regular curricular content through a "partner language" (such as Spanish or Russian), but do not generally teach the partner language direc...
Federal funding program:
Award number:
R305E120003
Grant
Why are Some Charter Schools More Effective than Others? Econometric Methods and Empirical Evidence from Massachusetts, Michigan, and Texas
Enrollment in charter schools has tripled over the last decade, and more states are making it easier to open charter schools. These two factors make it likely that growth in charter school enrollment will continue in the coming years. The goal of the proposed research plan is to use a diverse sample of charter schools from three states to determine what makes some charter schools reliably succeed at increasing learning while others do not. A key feature of this study is that charter enrollme...
Federal funding program:
Award number:
R305A120269
Grant
Enhanced Value-Added Models for Estimating Teacher Effects
This research examined the performance of a class of value-added models for longitudinal student achievement data where we introduce and use latent student parameters to enrich other features of the model.
Federal funding program:
Award number:
R305U040005
Grant
Representation and Combination of the Results of Multi-site Randomized Experiments in Education
This research addressed two problems: (1) How to adjust (approximately) analyses that improperly ignored clustering inherent in the design without going through an entire re-analysis, and (2) How to represent and combine the results of several experiments, some of which yield several measures of the same outcome construct when the correlation structure of the outcomes is unknown.
Federal funding program:
Award number:
R305U040007
Improving Pandemic Recovery Efforts in Education Agencies
In FY 2022, IES established the Improving Pandemic Recovery Efforts in Education Agencies (Improving Pandemic Recovery) grant program to support research to counteract instructional and learning loss encountered by many learners during the COVID-19 pandemic.
Program Information
Welcome to the Institute of Education Sciences (IES) Annual Principal Investigators Meeting! This year’s theme, Relevance & Rigor: Creating the Future of Education Research, was selected to underscore IES’ commitment to supporting high-quality scientific research that will advance education practice and policy by sharing information in formats that are useful and accessible to educators, parents, policymakers, researchers, and the public.
Low-Cost, Short-Duration Evaluation of Education Interventions
The Low-Cost, Short-Duration Evaluation of Education Interventions (Low-Cost Evaluation) grant program (ALS 84.305L) was competed from FY 2016 – FY 2018. It was designed to support rigorous evaluations of education interventions (broadly defined as practices, programs, and policies) that state or local education agencies expect to produce meaningful improvements in student education outcomes within a short period (for example, within a single semester or academic year).
Minutes of Meeting: April 7-8, 2010
NBES meeting minutes on April 7-8, 2009
Minutes of Meeting: October 30-31, 2007
NBES meeting minutes on Minutes of Meeting: October 30-31, 2007
Minutes of Meeting: October 5, 2012
NBES meeting minutes on October 5, 2012
Minutes of Meeting: March 23, 2011
NBES meeting minutes on March 23, 2011
Minutes of Meeting: February 22, 2013
NBES meeting minutes on February 22, 2013
Minutes of Meeting: October 14, 2011
NBES meeting minutes on October 14, 2011
Minutes of Meeting: June 29, 2011
NBES meeting minutes on June 29, 2011
Minutes of Meeting: June 16, 2014
NBES meeting minutes on June 16, 2014
Minutes of Meeting: November 9, 2009
NBES meeting minutes on November 9, 2009
Minutes of Meeting: September 29, 2010
"NBES meeting minutes on September 29, 2010
Researcher-Practitioner Partnerships in Education Research
The Researcher-Practitioner Partnerships in Education Research (Research Partnerships) topic supports partnerships composed of research institutions and state or local education agencies that have identified an education issue or problem of high priority for the education agency
Prekindergarten Through Grade 12 Recovery Research Network
In FY 2022, IES established the Improving Pandemic Recovery Efforts in Education Agencies (Improving Pandemic Recovery) grant program to support research to counteract academic losses encountered by many learners during the COVID-19 pandemic
Evaluation of State and Local Education Programs and Policies
The Evaluation of State and Local Education Programs and Policies (State/Local Evaluation) topic supports the evaluation of fully-developed programs and policies implemented by state and local education agencies to determine whether they produce a beneficial impact on student education outcomes relative to a counterfactual when they are implemented under routine conditions in authentic education settings.
Using Longitudinal Data to Support State Education Policymaking
In FY 2021, IES began the Using Longitudinal Data to Support State Policymaking grant program (Using Data for Policymaking) to expand State agencies' use of their State Longitudinal Data Systems* (SLDS) for generating evidence in support of education policy decisions.
Grant
Resources to Standardize Cost Analysis and Cost-Effectiveness Analysis of Educational Interventions
The project team aims to improve the rigor, comparability, and usefulness of cost analyses and cost-effectiveness analyses (CEAs) of educational programs by facilitating the application of the recently established Standards for the Economic Evaluation of Educational and Social Programs (SEEESP). The team will develop an Economic Evaluation Standards Toolkit that includes a Standards Template that researchers can use to apply the Standards to planned or completed economic analyses.
Federal funding program:
Award number:
R305D240010
Grant
A Researcher-Practitioner Partnership to Improve Pandemic Recovery Efforts in California
The Public Policy Institute of California (PPIC) in collaboration with the California Department of Education, the University of California-Berkely, and the University of California-San Diego will examine California's $4.6 billion pandemic recovery grant program for K-12 school districts known as the Expanded Learning Opportunity (ELO) grant program. The project team will examine the ELO for California's more than 1000 school districts from 2021-22 through 2024-25 and look at how the ELO gra...
Federal funding program:
Award number:
R305X230002
Grant
Mitigating the Impact of the COVID-19 Pandemic Through Curriculum-Based Approaches to Learning Acceleration in Grades K-2 in Chicago
In school year (SY) 2020-21, Chicago Public Schools (CPS) sought to accelerate post-pandemic K-2 literacy outcomes by providing teachers with a comprehensive curriculum system with aligned professional learning (PL), called Skyline+PL. This network project team will examine the ongoing implementation of Skyline+PL and gather information regarding improvements in the quality of instruction, student engagement and student literacy achievement outcomes, especially for students whose learning wa...
Federal funding program:
Award number:
R305X220033
Grant
Accelerating Recovery in Community Colleges Network Lead
The Community College Recovery Research Network Lead provides the organizational structure needed to ensure that the Network runs smoothly and accomplishes its goals and objectives. The Network Lead coordinates bi-annual meetings of the entire Network as well as ongoing working group meetings focused on crucial pandemic recovery topics. The Lead will conduct independent research on pandemic recovery activities and enrollment patterns at community colleges across the United States to inform t...
Federal funding program:
Award number:
R305X220022
Grant
A Lottery-Based Efficacy Study of the District of Columbia Public Prekindergarten Program
While the evidence on short- and longer-term prekindergarten efficacy has grown in recent years, nearly all existing research comes from studies of programs for 4-year-olds. Increasing policy interest in programs for 3-year-olds, persistent disparities in access, and theories and evidence from economics, neuroscience, and developmental psychology highlight the need for new research so that policymakers, practitioners, and the public can make informed choices about investments in children's s...
Federal funding program:
Award number:
R305A210506
Grant
Equity in Virginia's Public Education System: A Longitudinal Examination Spanning the COVID-19 Shutdown
The Virginia Department of Education and the University of Virginia will examine student equity of access to instructional opportunities and educational outcomes and describe how Virginia's responses to COVID-19 may have influenced equity in student outcomes and differences in teacher outcomes across student characteristics.
Federal funding program:
Award number:
R305S210009
Grant
Understanding Pennsylvania's Educational Inequities in the time of COVID-19
The Pennsylvania Department of Education (PDE) and Mathematica examined educational inequities in Pennsylvania that have occurred since the disruptions of COVID-19 and evaluated strategies to reduce those inequities.
Federal funding program:
Award number:
R305S210026
Grant
Changing Policy and Practice in Developmental Education: Assessing the Evidence and Engaging the Field
The purpose of this project is to conduct a synthesis of research on developmental education reform, and to disseminate the findings to key stakeholders with the aim of supporting implementation and scaling of effective reform strategies. The research team will collect and analyze evaluation and implementation studies on developmental education from 2010 to 2020, a period during which policymakers and practitioners have worked to implement a host of reform strategies. The synthesis addresses...
Federal funding program:
Award number:
R305U200010
Grant
Using Student Achievement Data to Monitor Progress and Performance: Methodological Challenges Presented by COVID-19
The RAND Corporation and the Northwest Evaluation Association (NWEA) identified promising analytic methods for practitioners, policymakers, and researchers to adapt and respond to COVID-19-induced disruptions to student assessment programs. These methods are to help with decisions often based on student test scores regarding -
Federal funding program:
Award number:
R305U200006
Grant
Causal Mediation Analysis Under Partial Compliance in Single-Site and Multisite Randomized Trials
The purpose of this project is to develop analytic strategies that incorporate instrumental variables for identifying and estimating the average treatment effect (ATE), the moderated ATE, the mediated ATE, and the moderated mediation effects in a single-site randomized trial where the moderator of theoretical interest is individual predisposition for compliance. The research team will also develop approaches to causal moderation and mediation in cluster randomized trials to take into account...
Federal funding program:
Award number:
R305D200031
Grant
Building Financial Leadership in Education to Do More for Students
Georgetown University's Edunomics Lab will convert its existing training materials on district and school finance into self-contained modules and make them freely available on the internet to postsecondary faculty and professional associations who train district and school leaders.
Federal funding program:
Award number:
R305U200003
Grant
Assembling an Open-Access School-by-School Spending Data Archive
The purpose of this grant is to develop a school spending data archive of data from 10 states.
Federal funding program:
Award number:
R305D200032
Grant
Linking Data and Policy to Improve College Readiness in Delaware
The purpose of this project is to identify the mechanisms through which local curricula and course offerings, statewide assessments, and feedback on predicted college-readiness might be better leveraged to influence students' educational trajectories through high school and into college.
Federal funding program:
Award number:
R305A200181
Grant
The Effects of PACT on Social Studies Knowledge and Comprehension Among 8th Grade Students
The purpose of this project is to test the effectiveness of a variation of the Promoting Accelerated Reading Comprehension of Text (PACT) intervention. Previous research on PACT (including this WWC-reviewed study) has shown positive effects for specific social studies content knowledge and more generalized social studies reading comprehension among eighth graders. To build on the extant evidence and to address the need to build adolescents' comprehension skills in content courses, including ...
Federal funding program:
Award number:
R305R200002
Grant
Cost Analysis in Practice Project (CAP Project)
Teachers College, Columbia University collaborated with SRI International and NORC at the University of Chicago to develop and disseminate free, on-demand tools, guidance, and technical assistance to education researchers and practitioners planning or conducting cost analysis or cost-effectiveness analysis of education practices, programs, and policies.
Federal funding program:
Award number:
R305U200002
Grant
Evaluating Michigan's Early Literacy Law: Impacts, Implementation and Improving State Capacity
The purpose of this project is to assess the implementation, impact, and cost of the Michigan "Read by Grade 3" law which is intended to increase early literacy outcomes for Michigan students.
Federal funding program:
Award number:
R305H190004
Grant
Efficient and Flexible Tools for Complex Multilevel and Latent Variable Modeling in Education Research
The project team developed improved tools for fitting statistical models in complex settings, specifically hierarchical Bayesian models for handling multilevel data including nested and non-nested groupings, latent variables, and large numbers of parameters.
Federal funding program:
Award number:
R305D190048
Grant
SERP-DCPS Partnership to Promote Early Literacy Success
DCPS has worked to increase student reading proficiency in the early elementary grades but has found it difficult to help the lowest performing students in grade 1 and 2 reach proficiency by grade 3. The project carried out two studies seeking to identify the malleable factors linked to disparities in K-3 student reading success. Study 1 integrated multiple DCPS datasets for 77 elementary schools of which 58 serve primarily low-income students. Study 2 involved direct observation and intervi...
Federal funding program:
Award number:
R305H190057
Grant
The Strategic Data Project for Higher Education
The Strategic Data Project for Higher Education
Federal funding program:
Award number:
R305U190001
Grant
The National Center for Rural Education Research Networks (NCRERN)
The National Center for Rural Education Research Networks (NCRERN) will establish a continuous improvement network to build the capacity of rural school districts and supporting state agencies to use their own data to improve the education of their students. The specific network model (known as Proving Ground) has been used in urban and suburban districts but not rural districts.
Federal funding program:
Award number:
R305C190004
Grant
The National Center for Research on Education Access and Choice
Across the U.S., many students have opportunities to attend schools other than their own traditional neighborhood public schools. These opportunities are commonly referred to as “school choice” and include such practices as open enrollment (both intra- and inter-district), magnet schools, charter schools including virtual schools, and public financial supports for attending private schools (e.g., vouchers, tax credit funded scholarships, education savings accounts, personal tax credits a...
Federal funding program:
Award number:
R305C180025
Grant
WKI-Writing: Impact of Word Knowledge Instruction (WKI) on Writing Outcomes of 5th Grade Students
Writing achievement is an important aspect of language and literacy performance for ELs and students from low socioeconomic backgrounds. Writing and oral language skills are integrally intertwined, as writing relies on interconnected language skills including word knowledge. Academic language (referring to the language of schooling that occurs in content-area text) is thought to be central to language and literacy achievement. Academic language is particularly important for students from low...
Federal funding program:
Award number:
R305L180019
Grant
Evaluation of Lane ESD's Coder-in-Residence Program
The purpose of this project is to investigate the effects of an intervention focused on technology instruction on students' proximal and distal knowledge, perceptions, engagement and academic performance related to STEM disciplines and, secondarily, teachers' and computer coders' perceptions and confidence related to coding and teaching.
Federal funding program:
Award number:
R305L180016
Grant
Evaluating Content Area Reading Instruction Professional Development: A Partnership between Seminole County Public Schools and the Florida Center for Reading Research Targeting Underperforming Adolescent Students
The purpose of this project is to continue to build a partnership between SCPS and FCRR, and to begin to examine the effects of the Next Generation Content Area Reading Professional Development (CARPD) on the literacy outcomes of struggling high school students. The state of Florida requires districts to provide reading interventions to students who score the lowest on a statewide reading assessment, and SCPS began implementing CARPD to train teachers to infuse literacy strategies into the c...
Federal funding program:
Award number:
R305H180030
Grant
Cleveland Partnership for English Learner Success
For this project, the Cleveland Metropolitan School District (CMSD) will partner with the American Institutes for Research (AIR) to form the Cleveland Partnership for English Learner Success (CLE-PELS). CLE-PELS has two primary purposes. One is to explore and better understand how EL high school students in the district fare in terms of postsecondary readiness (meeting specific benchmarks during high school) and success (enrolling in any postsecondary institution). The other is to increase t...
Federal funding program:
Award number:
R305H180021
Grant
Training in Application of Economic Evaluation for State and Local Education Policy Employees
Training in Application of Economic Evaluation for State and Local Education Policy Employees
Federal funding program:
Award number:
R305U180001
Grant
Strengthening School Readiness through Pre-K for All: A University-District Partnership
In 2014, New York City's Department of Education (NYC DOE) launched PreK for All, a program to make high-quality pubic prekindergarten available to all 4-year olds in the city. NYC DOE assigns each prekindergarten site to one of four tracks, each with its own focus: (1) PreK Explore, which has a math focus; (2) PreK Thrive, which focuses on child behavior and family engagement; (3) PreK Create, which has an arts focus; and (4) PreK Inspire, which allows sites to choose their own focus. NYC D...
Federal funding program:
Award number:
R305H170042
Grant
Impact of an Orientation Course on Online Students' Completion Rates
The Education Development Center (EDC), Michigan Department of Education (MDE), and Michigan Virtual Schools (MVS) formed a partnership to evaluate the impact of a short online orientation course (called Strategies for Online Success or SOS) on students' engagement in and completion of full online courses offered by MVS. Using an experimental design, the project team examined the impact on student engagement and course completion of being assigned the SOS course for first-time online student...
Federal funding program:
Award number:
R305L170008
Grant
Application of Explanatory Item Response Models to Understand Influence of Reader-Text Interactions on Reading Comprehension and Reading Intervention Effects
Reading comprehension is viewed by researchers as being the product of complex interactions between a reader, a text, and an activity, but little research has been conducted to look at these specific interactions. At the same time, it is common for reading interventions to show positive effects on tests designed by researchers but not on standardized tests. The purpose of this project is to explore the interaction between readers and texts and how this interaction differs for different kinds...
Federal funding program:
Award number:
R305A170251
Grant
A Researcher-Practitioner Partnership Examining the Use of Automated Essay Evaluation Software for Improving Students Writing Achievement
This partnership project developed and strengthened the partnership between University of Delaware's School of Education and Red Clay School District, and explored the feasibility and promise of MI Write (formerly PEG Writing), an automated essay evaluation (AEE) software program. Red Clay, and other school districts, want to improve the writing achievement of students in elementary schools, Grades 3-5. However, teachers often do not assign enough writing or provide enough feedback because i...
Federal funding program:
Award number:
R305H170046
Grant
Developing a Research and Policy Agenda to Improve School Climate in Virginia
This project supported the Virginia Department of Education's (VDOE) efforts to systematically measure school climate and working conditions, provide targeted technical assistance to schools to improve their school climate and working conditions concerns, and incorporate school climate metrics into Virginia's school accountability system in order to help VDOE gain a better understanding of how school climate is related to student education outcomes.
Federal funding program:
Award number:
R305H170016
Grant
Study of Effects of Transition Planning Process (TPP) on Adult Basic Skills Learners' GED® Attainment and Enrollment in Postsecondary Education
A previous IES-funded project (R305H140108) found that Oregon's Adult Basic Skills (ABS) programs, run by community colleges, could better support students' GED® completion and transition to postsecondary programs. In response, Oregon created the ABS Transition Planning Process (TPP). Through this project, Oregon sought to evaluate the efficacy of the use of the TTP to determine its impact and whether it should be continued. TPP is a text-messaging supplement to ABS advising activiti...
Federal funding program:
Award number:
R305L170012
Grant
Partnership to Study Dual Language Immersion in Utah
The purpose of this two-year grant was to create partnership to assess the causal effects of immersion program participation on student achievement. Dual language immersion (DLI) refers to programs in which students receive general academic instruction in two languages beginning in the early grades. In Utah, DLI programs come in two forms: two-way programs, in which at least a third of the students in a classroom are native speakers of each of the two classroom languages (here, English and a...
Federal funding program:
Award number:
R305H170005
Grant
Diagnostic Inventories of Cognition in Education (DICE)
The Diagnostic Inventories of Cognition in Education (DICE) project team developed a freely available, web-based assessment system to provide teachers with feedback about student performance in probabilistic reasoning.
Federal funding program:
Award number:
R305A170441
Grant
Contexts Inside and Outside of School Walls as Predictors of Differential Effectiveness in Preschool Professional Development
The goal of this project was to identify variables that create differences in impacts of early childhood professional development interventions. Achieving this aim will support decision making around scaling and refining professional development programs in order to optimize classroom quality and child learning outcomes.
Federal funding program:
Award number:
R305A160013
Grant
Increasing the Capacity of Early Childhood Education Programs to Use Data to Improve Implementation and Evaluation
In this project, the partnership sought to identify approaches to support prekindergarten (pre-k) instruction that improve students' academic and social-emotional skills. Specifically, the partnership conducted research to examine how data use and enhanced coaching can improve teachers' instruction and, subsequently, students' school readiness for kindergarten.
Federal funding program:
Award number:
R305H160052
Grant
Testing the Efficacy of Embedded Social Skills within a Universal Classroom Management Program: Well-Managed Schools
The purpose of this project is to examine the efficacy of embedded social skills instruction within a universal classroom management program. The research team will conduct a randomized control trial with elementary school teachers and students to evaluate the efficacy of Well-Managed Schools (WMS), a classroom management program that focuses on preventive strategies that include teaching and reinforcing social skills within day-to-day classroom management.
Federal funding program:
Award number:
R305A160346
Grant
Research Partnership to Improve a Multi-tiered System of Supports in Early Childhood Programs in a Large Urban District
As Multi-Tiered System of Supports (MTSS) are extended to early childhood education, new issues arise regarding training teachers, identifying at-risk students, providing students with tiered services, monitoring student progress, and adjusting those services as needed. In this project, Florida's Hillsborough County Public Schools (HCPS) examined how MTSS has been implemented in inner-city early childhood settings in order to improve its implementation and prepare for its scale-up.
Federal funding program:
Award number:
R305H160034
Grant
California's College and Career Readiness Standards in the Era of Common Core Assessments
In this project, researchers will describe and evaluate the impact of the revised California Early Assessment Program (EAP) that signals 11th grade students about their readiness to take postsecondary for-credit courses (based on their 11th grade state test scores in English and math) and provides advice on improving their readiness during 12th grade. In a previously funded IES study, researchers found that the Early Assessment Program had modest effects of college readiness for students att...
Federal funding program:
Award number:
R305E150006
Grant
Evaluating Maryland State Policies to Improve School Climate
In this project, researchers will evaluate the impacts of Positive Behavioral Interventions and Supports (PBIS), a multi-tiered intervention to prevent student behavior problems and promote a positive school environment to support academic success. The Maryland State Department of Education has built the infrastructure for statewide implementation of PBIS through a long-standing partnership with Johns Hopkins University and Sheppard Pratt Health System. Maryland has provided training to almo...
Federal funding program:
Award number:
R305H150027
Grant
Strengthening Education Research through Professional Development and a Trial Registry
In this project, researchers will develop a set of online professional development modules addressing methodological issues in education research and develop a registry of randomized trials in education. Both will serve to strengthen education research by adding transparency to the planning and conduct of research.
Federal funding program:
Award number:
R305U150001
Grant
Advancing State-specific Design Parameters for Designing Better Evaluation Studies
Co-Principal Investigator: Eric Hedberg (NORC) In a multilevel model, the statistical power, precision of estimates of treatment effects, the most efficient allocation of sample between levels, and the minimum detectable effect size all depend on the intraclass correlation structure and the effectiveness of any covariates in explaining variation at each level where they are used.
Federal funding program:
Award number:
R305D140019
Grant
State Longitudinal Data Systems Public-Use Project Feasibility Study
Co-Principal Investigator: Eric Hedberg (NORC) In many states, the data from statewide longitudinal data systems (SLDS) are available to a small group of state education agencies and scholars.
Federal funding program:
Award number:
R305D140045
Grant
Dual-Credit Courses and the Road to College: Experimental Evidence from Tennessee
This project team evaluated the impact of Tennessee's dual credit course for Advanced Algebra/Trigonometry on students' high school and college outcomes. Dual-credit courses are courses taken in high school that can provide college credit. These courses are designed to align high school and college curricula by providing college-level instruction within the familiar context of high school. Curricular alignment between high school and college may better prepare students by exposing them to th...
Federal funding program:
Award number:
R305H140028
Grant
Multilevel Synthesis of Single-Case Experimental Data: Further Developments and Empirical Validation
The project team further developed the multilevel modeling approach for combining the results of studies that use single-subject experimental designs (SSED). The study empirically investigated the approach using both real data and simulation studies.
Federal funding program:
Award number:
R305D110024
Grant
Accessible Methodology and User-Friendly Software for Multivariate Hierarchical Models Given Incomplete Data
This project expanded the work completed in a previous IES grant (Development of Accessible Methodologies and Software in Hierarchical Models with Missing Data) on software to handle missing data in univariate two- and three-level multilevel models.
Federal funding program:
Award number:
R305D130033
Grant
Increased Accuracy in the Detection of Differential Item Functioning through Multilevel Analysis
The purpose of the research project was to create multilevel differential item functioning (DIF) methods and software to increase the accuracy of the detection of DIF. Detection of DIF is one step in the process of gathering score validity evidence. DIF occurs when examinees from different subgroups who have equal ability on the measured construct have different probabilities of responding correctly to an item. DIF detection is a common component of instrument development in testing programs...
Federal funding program:
Award number:
R305D110014
Grant
Bayesian Inference for Experimental and Observational Studies in Education
The purpose of this project was to develop, apply, and disseminate Bayesian statistical tools for designs and analytic strategies used in empirical education research. This included: (1) randomized experimental designs; (2) quasi-experimental/observational designs; and (3) longitudinal studies. In addition, the project compared different types of prior information that may be encountered in practical circumstances, and provided guidance into best practices in Bayesian statistical modeling fo...
Federal funding program:
Award number:
R305D110001
Grant
Psychometric Models for 21st Century Educational Survey Assessments
The project developed models for the statistical analysis of data from large-scale survey assessments including the National Assessment of Educational Progress (NAEP), Trends in Mathematics and Science Study (TIMSS), Progress in International Reading Literacy Study (PIRLS), and the Programme for International Student Achievement (PISA). These models reflect recent changes in assessment frameworks such as the increased utilization of technology and the increased integration of tasks. The psyc...
Federal funding program:
Award number:
R305D110027
Grant
Value-Added Models and Accountability: Next Steps
Value-added models and indicators are being considered for use and are being used to measure the performance of schools, teachers, programs, and policies on an ongoing basis. Most current value-added models fall into two somewhat distinct groups, those that treat individual effects as random and those that treat individual effects as fixed. Random-effects models have been found to yield relatively precise estimates of school and classroom effects but can suffer from an inability to control f...
Federal funding program:
Award number:
R305D100018
Grant
Evaluation of the Long-Term Effects of Retention under New York City's Student Promotion Policy
A growing number of states and school districts have established policies that link grade promotion decisions to standardized test performance. The assumption underlying such policies is that an additional year of instruction in the same grade will help struggling students acquire the skills needed to succeed in future grades. Grade retention remains highly controversial and considerable uncertainty exists regarding the causal effect of retention on student outcomes. The researchers will ex...
Federal funding program:
Award number:
R305E120006
Grant
Practical Tools for Multilevel Hierarchical Modeling in Education Research
In this project, the researchers developed and tested new approaches to support applied researchers' use of Bayesian modal estimation (also called maximum a posteriori estimation) for multilevel models. The goal of this methodological innovation was to make it easier for applied researchers to avoid nonsensical results when they use multilevel models.
Federal funding program:
Award number:
R305D100017
Grant
Assessment of the Florida College and Career Readiness Initiative
Students who receive a high school diploma may not have the requisite skills to successfully complete college-level work and may not recognize that they lack this preparation. As a result, when they enter college they may be required to enroll in remediation courses. To increase these students' college-readiness in high school and reduce their need for remediation at community colleges, Florida has implemented the Florida College and Career Readiness Initiative (FCCRI). The FCCRI uses test s...
Federal funding program:
Award number:
R305E120010
Grant
National Center for Analysis of Longitudinal Data in Education Research (CALDER)
Since 2006, the National Center for Analysis of Longitudinal Data in Education Research (CALDER) has conducted a focused program of research on current State or district education policies intended to improve student achievement and other education outcomes (e.g., high school graduation rates) in any grades from prekindergarten through Grade 12.
Federal funding program:
Award number:
R305C120008
Grant
Weighting Methods for Mediation Analysis in Experimental and Quasi-Experimental Multilevel Data
Education researchers often seek to ascertain whether changes in student or teacher experiences mediate the treatment effects of an intervention on student or teacher outcomes.
Federal funding program:
Award number:
R305D120020
Grant
Approaches for Weighting and Estimation of Public-release Education Data using Two-level Covariance Structure Models
This project identified best methods for estimating parameters and their sampling variances when using multilevel analyses with data collected via complex sampling designs typically used in education research.
Federal funding program:
Award number:
R305D110050
Grant
Constructing Value-Added Indicators of Teacher and School Effectiveness that We Can Trust
This project sought to improve value-added models by addressing two central issues involved in establishing the validity of inferences based on value-added models (VAMs). The first issue concerns the accuracy of measures of students' achievement growth. Educational tests that under-represent the full range of desired skills and knowledge, and that substantially shift emphasis on particular constructs from year to year, will underestimate students' growth in achievement and may lead to statis...
Federal funding program:
Award number:
R305D100028
Grant
A d-Estimator for Single Case Designs
Although a number of methods exist for analyzing data from single-case designs, none yields an effect size estimator that is comparable to the commonly used effect size statistics in between-groups designs like the standardized mean difference statistic. This project developed such a d-statistic (represented as d) for single-case designs that is comparable to and in the same metric as the d-statistic from a between-groups experiment. This statistic will allow researchers to assess effects fr...
Federal funding program:
Award number:
R305D100046
Grant
A Three Year Proposal to conduct Two Annual Workshops on Better Quasi-Experimental Design and Analysis in Education
Well executed randomized experiments provide the strongest evidence about causal effects of educational interventions, products, and services. But randomized experiments may not always be feasible or successful. As a result, quasi-experimental designs are often used in education research for causal purposes. However, the quality of quasi-experimental designs and their application varies widely. In addition, knowledge of the better quasi-experimental designs that provide stronger evidence for...
Federal funding program:
Award number:
R305U100001
Grant
Addressing Practical Problems in Achievement Gap Estimation: Nonparametric Methods for Censored Data
The project developed and assessed a set of methods for estimating achievement gaps and their standard errors in order to address two practical problems affecting the use of achievement gap statistics: (1) the dependence of traditional gap statistics-mean differences and effect sizes-upon scaling decisions, and (2) the reporting of achievement data in categorical terms (e.g., below basic, basic, proficient, and advanced) rather than as test score results and their distributional statistics (...
Federal funding program:
Award number:
R305D110018
Grant
Regression Discontinuity Designs with Assignment Based on Multiple Rating Scores: Statistical Properties and Issues in the Context of Education Evaluation
The purpose of this project was to provide practical guidance to education researchers on how to estimate program impacts using a regression discontinuity (RD) design with more than one rating variable. It is often the case that assignment to an education intervention is based on multiple criteria. For example, graduation from high school may depend on a student successfully passing standardized tests in several subjects and meeting Adequate Yearly Progress requires schools to meet a number ...
Federal funding program:
Award number:
R305D100027
Grant
Better Warranted Quasi-Experimental Practice for Evidence Based Practical Research
This project extended work on improving four quasi-experimental methods that have potential for providing unbiased or minimally biased causal inference when random assignment is not possible. These four methods include: regression discontinuity designs, propensity score matching methods, short interrupted time-series designs, and pattern matching.
Federal funding program:
Award number:
R305D100033
Grant
Non-Linear Multilevel Latent Variable Modeling with a Metropolis-Hastings Robbins-Monro Algorithm
The goal of this project was to bring together the benefits of multilevel modeling and latent variable modeling. To do so, the project proposed a flexible nonlinear multilevel latent variable modeling framework under which: (1) random effects and latent variables are treated synonymously because both represent unobserved heterogeneity; (2) a nonlinear random effect regression model permits the specification and testing of important structural relations (e.g. mediation or moderation effects) ...
Federal funding program:
Award number:
R305D100039
Grant
Methods for Parameter Inference, Model Comparison and Incomplete Data in Complex Psychometric Models for NAEP Survey Data
This project sought to extend the work done by the co-PIs on increasing the efficient analysis (both computational and statistical) of the large-scale, multi-level National Assessment of Educational Progress (NAEP). This project used data from the 2005 NAEP math survey in order to address two issues: (1) determining the best fitting model; and (2) missing data in the teacher questionnaire.
Federal funding program:
Award number:
R305D110008
Grant
Generalized Dimensionality Assessment for Multidimensional Psychometric Models
This project developed statistical procedures for conducting dimensionality analysis in the context of multidimensional item response theory that are generalizable to a broad class of related latent variable statistical models. Psychometric models are motivated by the fact that what is of most interest cannot be directly observed and so use latent variables to represent constructs. Observed variables then serve as indicators of the underlying constructs. Popular item response theory (IRT) an...
Federal funding program:
Award number:
R305D100021
Grant
Gifted Education Program Participation and Program Impacts
The goal of this project is to evaluate the impact of gifted education on student achievement in the Broward County Public Schools (BCPS) in Florida. The BCPS gifted program is district-wide and characterized by high-intensity models with separate gifted classrooms, specialized teacher training, and accelerated and differentiated instruction. The research team will also evaluate a BCPS universal screening program aimed at the under-representation of disadvantaged groups in the gifted program...
Federal funding program:
Award number:
R305E110019
Grant
Sensitivity Analysis—If We're Wrong, How Far Are We from Being Right?
This project extended existing and develop new methods for sensitivity analyses that can be used to quantify the uncertainty about causal inferences made when strong and often assumptions are required that are not testable. An example of this is when observational data is used or when randomized experiments suffer from missing data or non-compliance with assignment. Although reasonable strategies for sensitivity analyses already exist to address some of these assumptions, most remain severel...
Federal funding program:
Award number:
R305D110037
Grant
Testing Different Methods of Improving the External Validity of Impact Evaluations in Education
This study was motivated by the observation that most major, multi-site evaluations in education have chosen participating sites (e.g., districts, schools, or grantees) "purposively" and not randomly. This raises possible concerns about the generalizability of the findings from these studies. The goal of this project was to provide evidence regarding the external validity of evaluations that are based on purposive samples.
Federal funding program:
Award number:
R305D100041
Grant
Estimation and Inference in Education Research when Actions by Participants Impact Validity and Availability of Data
The project developed methods for estimation and inference in education research when actions by participants impact validity and availability of data. Specifically, the researchers addressed the impacts of participant actions in two research designs: differential attrition in lottery designs and manipulation of the assignment variable in regression discontinuity designs. The researchers used data from the Pittsburgh Public School’s magnet program to address attrition in lottery design...
Federal funding program:
Award number:
R305D090016
Grant
Using Instrumental Variables Analysis Coupled with Rigorous Multi-Site Impact Studies to Study the Causal Paths by which Educational Interventions Affect Student Outcomes
This project explored the theoretical properties and empirical implications of using instrumental variables analysis coupled with rigorous multi-site impact studies to estimate the causal paths by which educational interventions produce effects on students. To this end, it addressed five research questions:
Federal funding program:
Award number:
R305D090009
Grant
Cross-Classified Structural Equations Model: Development of an OpenMX Module and its Application to Multiyear Assessment and Intervention Data in Literacy Research
The proposed project aimed to: (1) develop a software library for fitting cross-classified structural equations models (CC-SEM); (2) analyze a number of large literacy-related datasets with common methodological issues; and (3) document and disseminate the software library.
Federal funding program:
Award number:
R305D090024
Grant
Reducing Bias and Improving Efficiency of Estimated Teacher Effects from Value-Added Models
Value-added models (VAMs) use longitudinal student-level test score data and complex statistical models to generate estimates of the effects of individual teachers on student learning. This holds the promise of providing fair comparisons of teachers teaching different types of students. These estimates have heightened interest in teachers' roles in student academic outcomes. There has been rapidly growing interest among policymakers to use these measures for high-stakes decisions about teach...
Federal funding program:
Award number:
R305D090011
Grant
Practical Solutions for Missing Data and Imputation
Missing data are ubiquitous in education research studies. The literature discusses the shortcoming of simple missing data approaches such as complete case analysis and inclusion of indicators for missing data; however, the use of these practices remains widespread. Multiple imputation is becoming an increasingly widely used approach to handling missing data but there are outstanding research questions regarding the most reliable methods for implementing it and when it is worthwhile to inves...
Federal funding program:
Award number:
R305D090006
Grant
State-specific Design Parameters for Designing Better Evaluation Studies
This project used five state data systems to compute design parameters (including intraclass correlations and variance accounted for by pretest data) at the state, district, school, and classroom levels. Additionally, meaningful subsets of the states defined geographically, in terms of achievement levels (e.g., low achieving schools or districts), and in terms of socioeconomic status (e.g., low SES schools or districts) were also collected.
Federal funding program:
Award number:
R305D110032
Grant
Middle School Intervention Project (MSIP)
This project will assess the implementation and impact of a targeted middle grades intervention to improve reading and school engagement among students with reading difficulties taking place within a universal intervention to improve reading and engagement among all students.
Federal funding program:
Award number:
R305E100043
Grant
Evaluation of the Effectiveness of the Ohio Department of Education's Literacy Core Curriculum for Early Childhood Educators
This project will evaluate the Ohio Department of Education-sponsored professional development course for early childhood educators known as the Literacy Core Curriculum for Early Childhood Educators ("Core") program. Nearly 1,200 Ohio pre-K educators have been trained in Core since 2008; however; the effectiveness of this program and its companion mentoring program, the Teacher Leader Program, to increase educators' literacy-related knowledge and practices has not been tested against other ...
Federal funding program:
Award number:
R305E100030
Grant
National Research and Development Center on Scaling Up Effective Schools
The National Center on Scaling up Effective Schools (NCSU) is a collaboration of researchers, developers, and practitioners from Vanderbilt University, Florida State University, the University of Wisconsin at Madison, the Education Development Center, the Broward County School District of Florida, and the Fort Worth Independent School District of Texas. NCSU's primary goal is to identify the combination of programs, practices, and policies that make some high schools in large urban districts...
Federal funding program:
Award number:
R305C100023
Grant
Intended and Unintended Consequences of State High-Stakes Testing: Evidence from Standards-Based Reform in Massachusetts
This project will evaluate the impact of high stakes (for students and schools or for schools only) testing in Massachusetts. Massachusetts began administering the Massachusetts Comprehensive Assessment System (MCAS) mathematics and English/language arts examinations in 1998. For the class of 2003, the 10th grade tests became high-stakes exit examinations. Students must pass both tests in order to receive a high school diploma.
Federal funding program:
Award number:
R305E100013
Grant
The Impact of the Michigan Merit Curriculum and Michigan Promise Scholarship on Student Outcomes
This project will assess the implementation and impact of two recent reforms in Michigan designed to work in tandem to promote college attendance and success: (1) the Michigan Merit Curriculum the requires a more rigorous high school curriculum; and (2) the Michigan Promise Scholarship that provides funding for college to students who meet academic criteria.
Federal funding program:
Award number:
R305E100008
Grant
The Effects of No Child Left Behind on Student Outcomes and School Services
The purpose of this project is to study whether the incentives built into the No Child Left Behind Act (NCLB) have a net positive impact on students' academic achievement, students' non-academic outcomes, and school resource allocation. Under NCLB, states annually determine whether public schools satisfy Adequate Yearly Progress (AYP) based on the fraction of students demonstrating proficiency on a statewide exam. Public schools have strong incentives to satisfy AYP: failure triggers sanctio...
Federal funding program:
Award number:
R305A090032
Grant
Evaluation of the New Jersey Preschool Expansion (NJPE) Program
This project will evaluate the effectiveness of the New Jersey Preschool Expansion (NJPE) Program in attaining its primary objectives of enhancing the school readiness of economically disadvantaged children. It will also examine the relationship between student outcomes and characteristics of students, teachers, and districts to determine which are associated with the largest effects and thus provide guidance for program improvement. Many states have undertaken pre-k initiatives with similar...
Federal funding program:
Award number:
R305E090010
Grant
A Proposal to Measure the Impact of Indiana's System of Diagnostic Assessments on Student Achievement Outcomes
This project will study the impact of Indiana's statewide Diagnostic Assessment Intervention on changes in teachers' instructional behaviors and on K-8 student performance on state accountability tests using a randomized control design. It will also examine whether variation in implementation of the Diagnostic Assessment Intervention is associated with differences in impacts and whether school, teacher or student characteristics modify the effects of the intervention on student test scores u...
Federal funding program:
Award number:
R305E090005
Grant
Evaluating the Effectiveness of Tennessee's Voluntary Pre-K Program
This project will evaluate the effectiveness of the Tennessee Voluntary Pre-k Program (TN-VPK) in attaining its primary objectives of enhancing the school readiness of economically disadvantaged children and improving their academic performance. It will also examine the relationship between student outcomes and selected policy-relevant characteristics of teachers, classrooms, and schools to determine which are associated with the largest effects and thus provide guidance for program improvem...
Federal funding program:
Award number:
R305E090009
Grant
Evaluation of Core Knowledge Charter Schools in Colorado
This project will evaluate the impact of Core Knowledge (CK) charter elementary schools in Colorado on student achievement primarily in kindergarten through third grade. It will also examine whether there are differential impacts by student and school characteristics, whether the level of implementation of the Core Knowledge program is associated with the impacts on student achievement, as well as the cost-effectiveness of the CK charter schools.
Federal funding program:
Award number:
R305E090003
Grant
Hierarchical Linear Modeling Under Multilevel Non-Ignorable Non-Responses with Applications to NAEP Data
The National Assessment of Educational Progress (NAEP) conducts surveys on the educational achievement of students in the United States. Multistage cluster sampling schemes are widely employed in those surveys, and multilevel non-responses at both the school and student levels are common in those surveys. The non-response adjusted Horvitz-Thompson (HT) estimator, which weights cases by the inverse of their selection probabilities adjusted by the non-response rate, has been regularly used to ...
Federal funding program:
Award number:
R305D090019
Grant
Developing Time-Indexed Effect Size Metrics for K–12 Reading and Math Educational Evaluation
The project developed academic growth references for K-12 reading and math achievement based on nationally representative longitudinal datasets and time-referenced effect size metrics, based on those national academic data, that can be used to assess the effectiveness of educational interventions.
Federal funding program:
Award number:
R305D090021
Grant
Using Imperfect Fidelity Measures to Improve Statistical Inferences about Educational Interventions
The purpose of this project was to develop statistical methods to conduct inference for impact studies of education interventions that take into account the variability of intervention fidelity in the experimental treatment group.
Federal funding program:
Award number:
R305D090020
Grant
Statistical Properties of Regression Discontinuity Analysis and Comparative Interrupted Time Series Analysis for Estimating Impacts
This project explored the circumstances under which two promising quasi-experimental designs can produce estimates of program impacts that are valid, precise and generalizable.
Federal funding program:
Award number:
R305D090008
Grant
Development of Accessible Methodologies and Software in Hierarchical Models with Missing Data
This project integrated the appropriate modeling of multilevel data with rigorous methods for modeling with missing data. The project developed new methods that integrate these, making currently available methods broadly accessible for the first time through user-friendly software, and trained educational researchers to use these methods and software.
Federal funding program:
Award number:
R305D090022
Grant
Representing and Combining the Results of Randomized Experiments in Education
The multi-site randomized experiment is an important tool in education science, but the use of evidence from such trials has sometimes posed technical problems. The results from this research project will make it easier to interpret, represent, and use the results of randomized experiments in education.
Federal funding program:
Award number:
R305U080002
Grant
Simultaneous Statistical Inference in Evaluating Teacher Performance
There is widespread interest in using student achievement data to evaluate the performance of individual schools for accountability. Such evaluations are the capstone of many federal and state education policies. Currently there is also increasing interest in using sophisticated value-added models (VAM) as a part of the evaluations for individual teachers as well as schools, and using the measures for a variety of purposes including merit pay. Evaluation systems based on value-added measures...
Federal funding program:
Award number:
R305U080003
Grant
A Study of the Missing Data Assumptions of the Non-equivalent Groups with Anchor Test Design and Their Implications for Test Equating
With the introduction of the No Child Left Behind Act of 2001, the number of tests taken by U.S. grade school students grew. Many of these tests administer different test forms on different dates so that students who take the test on different dates will not answer exactly the same test questions. To ensure fair scores on high-stakes exams, individual differences in scores taken on exams on different test dates must reflect differences in achievement levels of the students and not difference...
Federal funding program:
Award number:
R305U070009
Grant
Evaluating the Impact of the Choice of Test Score Scale on the Measurement of Individual Student Growth
State and federal educational policies are focusing increasingly on school accountability for individual student growth. As statistical models and policy approaches proliferate, little attention is paid to the serious dependency of growth statistics on the choice of the test score scale. A different score scale, defined by a nonlinear transformation of the original score scale, can theoretically alter growth statistics, reverse aggregate trends, and distort interpretations from so called val...
Federal funding program:
Award number:
R305U070008
Grant
Collaborative Research in Urban Education
The program funded by this grant is designed to put in place a unique structure for a high degree of collaboration between large urban school districts and senior researchers with established records of high quality research. Research is carried out by Senior Fellows in Urban Education selected on a competitive basis to carry out specific work described in their proposals. Under the grant, the Council of Great City Schools (CGCS) works with a Research Advisory Committee (RAC) composed of s...
Federal funding program:
Award number:
R305U060005
Grant
Assessing Intervention Fidelity in Randomized Field Experiments (RFTs)
This project developed a framework to identify all the components of an education intervention and gauge both the degree and extent to which each component is in place and operating as intended. To this end, the project developed, modified, tested and combined (a) intervention theory and structure with (b) measures of intervention delivery and receipt, and (c) methods for making use of the measures of intervention fidelity in analysis in order to better understand the effects of the interven...
Federal funding program:
Award number:
R305U060002
Grant
Estimation of Teacher Effects
year-to-year stability of teacher effects over a nine year period to determine how various measures of teacher performance vary over time, as well as how they vary across different teaching contexts.
Federal funding program:
Award number:
R305U060003
Grant
Psychometrics and Value Added Modeling
the research team is empirically examining the degree to which some psychometric decisions made in calibrating student response data on state assessment programs influence the teacher effects estimated under VAM
Federal funding program:
Award number:
R305U060004
Grant
National Research Center on Rural Education Support
The mission of the National Research Center on Rural Education (NRCRES) is to conduct research that addresses the diverse educational needs of rural schools, which face particular challenges that affect students' academic performance, such as: (1) high rates of child poverty; (2) limited resources for educational materials and professional development; (3) difficulty in attracting and retaining appropriately and highly qualified teachers; and (4) staffing issues stemming from sparse and remo...
Federal funding program:
Award number:
R305A040056
Grant
National Center for Performance Incentives (Policy-NCPI)
The purpose of the National Center on Performance Incentives (NCPI) was to conduct a focused program of education research on the impact of teacher incentives on student and teacher outcomes.
Federal funding program:
Award number:
R305A060034
Grant
Center for Research on Evaluation, Standards, and Student Testing (CRESST)
The National Center for Research on Evaluation, Standards, and Student Testing (CRESST) has been at the forefront of efforts to improve the quality of education and learning in America. Located within UCLA's Graduate School of Education & Information Studies, CRESST contributes to the development of scientifically based evaluation and testing techniques, encourages the development, validation and use of sound data for improved accountability and decision making, and explores technological ap...
Federal funding program:
Award number:
R305A050004
Grant
Technical Assistance Guidance for Cost Analysis at IES
The project team will identify and review currently available cost analysis models, compare them, and consider possible modifications. The team will use this information to develop technical guidance for applicants who are applying for IES funding. Applicants can then use this guidance as they develop and implement their cost analysis plans.
Federal funding program:
Award number:
R305U190002
View More