Psychometrician, Office of Assessment and Evaluation (PhD) (Phoenix)

Reporting to the assistant dean of Assessment and Evaluation, the psychometrician will have the following primary responsibilities:

• Perform item analysis, item response theory (IRT) calibrations, scoring, equating, standard setting and validity and reliability studies across multiple assessments. These include, but are not limited to, knowledge-based and clinical assessments in both the pre-clerkship and clerkship phases of the MD program.
• Conduct comprehensive assessment data analyses to support continuous quality improvement and ensure compliance with accreditation standards, including those of the Liaison Committee on Medical Education (LCME). Analyses include, but are not limited to, descriptive and inferential statistics; retrospective and prospective analyses; and cross-sectional and longitudinal analyses of assessment and program data.
• Develop, maintain and continuously improve operational psychometric processes to enhance efficiency, accuracy and scalability.
• Collaborate with subject matter experts and exam developers in item writing, item review and the design, development and evaluation of assessments.
• Design and build robust infrastructure for assessment data architecture, enabling efficient querying, rapid data retrieval and timely, accurate responses to data requests.
• Collaborate on research and development initiatives within OAE and across the institution and disseminate findings to the broader medical education community through presentations and peer-reviewed publications.
• Provide expert consultation on research design, measurement, statistical methodologies and data systems.
• Develop and maintain a comprehensive dashboarding system for student assessment data to support real-time insights and data-informed decision-making.
• Other duties as assigned.