Research
Research
Research by Jiayi Deng on AI fairness, psychometric validity, subgroup comparability, rapid guessing, and evidence synthesis.
Jiayi's research connects psychometric validity, subgroup comparability, behavioral evidence, and responsible AI evaluation. The throughline is practical: build evidence that high-impact assessment and selection systems are fair, valid, interpretable, and appropriately monitored.
Research themes
AI fairness and human-AI evaluation
Evaluation of model outputs, end-to-end workflows, human overrides, counterfactual behavior, monitoring, and responsible-use boundaries in high-impact decision contexts.
Psychometric validity and subgroup comparability
Item response theory, measurement invariance, differential item/distractor functioning, reliability, equating, calibration, and fairness-oriented validation.
Behavioral and process data
Response time, rapid guessing, aberrant response patterns, cognitive labs, think-aloud protocols, and structured behavioral coding.
Evidence synthesis
Systematic review and meta-analysis, including current work on how AI-enabled interventions affect student motivation and engagement.
Selected publications
Relevant publications and manuscripts
Under review
Can AI Motivate Students? A Systematic Review and Meta-analysis on AI's Role in Student Motivation and Engagement.
Applied Psychological Measurement · 2024
Gorney, K., & Deng, J. `aberrance`: An R package for detecting aberrant behavior in test data.
Educational and Psychological Measurement · 2024
Rios, J. A., & Deng, J. Is effort-moderated scoring robust to multidimensional rapid guessing?
Applied Psychological Measurement · 2022
Deng, J., & Rios, J. A. Investigating the effect of differential rapid guessing on population invariance in equating.
Large-scale Assessments in Education · 2021
Rios, J. A., & Deng, J. Does the choice of response time threshold procedure matter? Implications for identifying rapid guessing behavior.
Methods and software
Reproducible measurement and evaluation tools
Jiayi's work includes open-source statistical software, simulation studies, meta-analysis, response-time methods, and reproducible project artifacts such as model cards, dataset cards, monitoring plans, and decision logs.
Human-AI Fairness Audit Lab
A synthetic, non-hiring audit demonstration that evaluates model behavior, reviewer overrides, calibration, counterfactual sensitivity, subgroup outcomes, and monitoring regressions.
Academic archive
Historical publication, presentation, and teaching records remain available for continuity.