Skip to main content

Breadcrumb

Home arrow_forward_ios Information on IES-Funded Research arrow_forward_ios Consequences of Selective Reporting ...
Home arrow_forward_ios ... arrow_forward_ios Consequences of Selective Reporting ...
Information on IES-Funded Research
Grant Open

Consequences of Selective Reporting Bias in Education Research

NCER
Program: Statistical and Research Methodology in Education
Program topic(s): Core
Award amount: $896,931
Principal investigator: Martyna Citkowicz
Awardee:
American Institutes for Research (AIR)
Year: 2022
Award period: 3 years (09/01/2022 - 08/31/2025)
Project type:
Methodological Innovation
Award number: R305D220026

Purpose

The most popular meta-analytic methods have serious limitations in diagnosing and adjusting for selective reporting, especially when there are dependencies among multiple effects from primary studies, a widespread occurrence. For meta-analysis of independent effects, selection models have shown promise in flexibly capturing complex reporting patterns while providing adjusted meta-analytic estimates, but no existing model also simultaneously addresses effect size dependencies. The purpose of this project is to develop two selection models that will simultaneously account for selective reporting and effect size dependencies.

Project Activities

This model is based on the beta-density distribution and the other model is based on theoretically important p-value cut points. After these models are derived, their performance will be tested via Monte Carlo simulation studies under varying conditions, such as different sample sizes and different degrees of selection bias.

People and institutions involved

IES program contact(s)

Charles Laurin

Education Research Analyst
NCER

Project contributors

Joshua Polanin

Co-principal investigator

James Pustejovsky

Co-principal investigator

Ryan Williams

Co-principal investigator

Products and publications

The research team will then develop an R software package and a Shiny app for meta-analysis researchers to apply the new models to their own research. In order to help researchers use the new software, the project team will prepare several tutorials, prepare a webinar, and hold in-person trainings. They will also conduct and publish the results of a second-order meta-analysis.

Questions about this project?

To answer additional questions about this project or provide feedback, please contact the program officer.

 

Tags

Data and Assessments

Share

Icon to link to Facebook social media siteIcon to link to X social media siteIcon to link to LinkedIn social media siteIcon to copy link value

Questions about this project?

To answer additional questions about this project or provide feedback, please contact the program officer.

 

You may also like

Zoomed in IES logo
Workshop/Training

Data Science Methods for Digital Learning Platform...

August 18, 2025
Read More
Zoomed in IES logo
Workshop/Training

Meta-Analysis Training Institute (MATI)

July 28, 2025
Read More
Zoomed in Yellow IES Logo
Workshop/Training

Bayesian Longitudinal Data Modeling in Education S...

July 21, 2025
Read More
icon-dot-govicon-https icon-quote