Projects
Projects
Selected reproducible research and evaluation projects by Jiayi Deng, including the Human-AI Fairness Audit Lab.
Selected public work focused on reproducible evaluation, psychometric evidence, behavioral data, and responsible AI assessment. The featured project is a synthetic, non-hiring audit demonstration designed to make fairness and workflow risks inspectable.
Featured
Human-AI Fairness Audit Lab
A public, reproducible audit of a synthetic AI-assisted candidate-review workflow. The project evaluates held-out model performance, subgroup and intersectional outcomes, calibration, counterfactual behavior, reviewer overrides, escalation policy behavior, and monitoring regressions.
Responsible-use boundary: synthetic applicants and simulated decisions only. This is an evaluation demonstration, not a hiring recommendation system.
- Training cases
- 6,000 synthetic
- Validation cases
- 2,000 synthetic
- Held-out audit
- 2,400 synthetic
- Monitoring
- 800 synthetic
Selected research and software work
`aberrance` R package
Open-source R package work for detecting aberrant behavior in test data, connected to peer-reviewed research in Applied Psychological Measurement.
Rapid guessing and effort-moderated scoring
Simulation and meta-analytic research on response-time thresholds, rapid guessing, reliability, test performance, and population invariance in equating.
AI effects evidence synthesis
Current systematic review and meta-analysis work on AI's role in student motivation and engagement, listed as under review in the supplied content source.