Artificial Intelligence (AI), Data Analysis, Data Quality, Leadership, Metrics, Performance Metrics, Quality Assurance, Team Lead/Manager, Test Data, Test Plan/Schedule, Test Strategy, Testing, Validation Testing
LOCATION
Bolingbrook, IL
POSTED
30+ days ago
E-commerce domain exp. is mandatory. Job Description:
The AI/ML Test Lead will own end to end quality assurance for AI driven solutions across Ulta's Enterprise AI.This role is responsible for defining and executing an Enterprise AI testing strategy, ensuring functional correctness, data accuracy, AI output quality, robustness, performance, and Responsible AI compliance.
The role blends hands on testing leadership, AI/ML specific validation, and onsite stakeholder collaboration, working closely with Product Owners, AI Engineering Leads and Data teams. Required Skills & Experience- Core QA & Testing Experience
10+ years of experience in QA / Testing roles, with leadership responsibility
Strong experience defining test strategy, test plans, and execution ownership
Experience testing data heavy and analytics driven systems
AI / ML / Data Testing Skills
Hands on experience testing AI/ML or LLM based solutions
Strong understanding of:
Data validation and KPI testing
AI output correctness and consistency
Prompt based and agent based systems
Familiarity with AI evaluation metrics (accuracy, error rate, hallucination rate, relevance)