Name
The Elusive Language Component of Assessment - Can AI Really Help Flag and Resolve Potential Issues?
Description

When item authors write test questions, they usually write them in their strongest language. They draw on their domain expertise and on their writing skills. The way they write items can add colour and flavour. It can also create construct-irrelevant variance: socially connotated words can introduce a cultural bias. The level of reading proficiency needed to fully understand the questions may put talented individuals at a disadvantage. Traditionally, this has been resolved by resorting to cultural review panels, by having culturally diverse item writers and subject matter experts. This is resource-intensive. In the age of AI, it is possible to use data from previous cultural review panels, translatability assessments and item health checks to generate a screening report of a mature draft of a test. In this case study, we demonstrate how the automated item health check was integrated into an assessment authoring platform and how the item writers could effectively process the reports.

Session Type
Presentation
Session Area
Education, Industrial/Organizational, Certification/Licensure, Workforce Skills Credentialing, Health Sector
Primary Topic
AI in Assessment