Posting Title: Senior Psychometrician & Research Lead Department: Assessment Services Employment Type: Full-Time Minimum Experience: Experienced Work Environment: Remote eligible (with up to 10% travel)
Position Summary The Senior Psychometrician and Research Lead provides high-level technical expertise in psychometric analysis and applied research to support ABEM's high-stakes medical certification examinations. This dual-focus role manages psychometric services across the assessment lifecycle while simultaneously leading research initiatives-including study design, data analysis, and manuscript preparation-to ensure the validity, fairness, and reliability of physician certifications.
Essential Duties & Responsibilities
Assessment Strategy: Collaborate with the Principal Psychometrician to guide the design, implementation, and analysis of innovative examination processes. Technical Operations: Develop and execute computer programs for examination scoring, equating, and item analysis; review and interpret complex results. Quality Control: Conduct analytical studies to implement methodologies for improving examinations and measuring the effectiveness of process innovations. Equitable Assessment: Review and interpret Differential Item Functioning (DIF) analyses and lead standard-setting activities. Data Stewardship: Provide psychometric support for data exchanges with vendors and other medical boards, maintaining organized and secure data banks.
Research Responsibilities
Study Leadership: Design and execute research studies using ABEM's physician data, focusing on health equity, clinical performance metrics, and public health. Evidence Generation: Identify and access external datasets (e.g., CMS, ACGME) to complement internal resources. Scientific Contribution: Support the preparation and submission of research manuscripts for peer-reviewed journals as a co-author. Stakeholder Communication: Translate complex psychometric findings into accessible formats for the Board of Directors and internal stakeholders.
Minimum Qualifications
Education: Doctoral degree (Ph.D. or Ed.D.) in Psychometrics, Educational Measurement, Quantitative Social Science, or a related field. Experience: Minimum of 7 years of professional experience encompassing both psychometric practice and applied research. Technical Skills: Proficiency in R, or SAS software. Specialized Knowledge: Demonstrated experience with standard setting, equating, and DIF analyses. Communication: Proven record of peer-reviewed publications and the ability to present complex data to non-technical audiences.
Preferred Qualifications
Advanced expertise in assessment design and healthcare research. Familiarity with accreditation standards and medical specialty certification.