Name
Humans in the Driver's Seat: Effective Oversight of AI-based Assessment Tools
Speakers
Kimberly Swygert, National Board of Medical Examiners
John Kleeman, Learnosity
John Weiner, JW Advisers
John Kleeman, Learnosity
John Weiner, JW Advisers



Description
The inclusion of human judgment has always been crucial within assessment to ensure that content, predictions, decisions, and outcomes are accurate, unbiased and aligned with the intended purpose. AI use in assessment requires human oversight to help ensure accuracy, fairness, and compliance. But what does human oversight mean in practice?
This presentation from members of the ATP AI subcommittee explains what “human oversight” might mean in practice and covers:
Latest ATP guidance
Case studies/evidence from earlier adopters
Ethical and regulatory frameworks
Mitigating bias
Aligning with risk
Assessment examples (AI-generated items, score reports)
Implementation challenges (training, bias mitigation)
The session targets assessment professionals, developers, psychometricians, and AI implementers in educational, workplace, and certification testing. Attendees will gain practical takeaways to establish or strengthen human oversight in AI-enabled assessment programs.
Session Type
Panel Discussion
Session Area
Certification/Licensure, Education, Workforce Skills Credentialing
Primary Topic
Test Development and Psychometrics