AI QA Engineer

Track & report quality metrics such as accuracy, precision (and recall where applicable), clearly highlighting missing, incorrect, or hallucinated content. Reuse and expand the Golden Dataset for regression testing across builds, prompt/model changes, and releases.

Job Title: AI QA Engineer
Location: Rosemead, CA 91770 (Hybrid)
Duration: 06 Months

Must Have Skills:
QA/test planning, exploratory testing, and UAT experience
AI/ML fundamentals and evaluation metrics literacy
Data analysis and model behavior reasoning
Ability to test GenAI prompts and outputs (preferred)
Strong domain knowledge and user centric mindset
Clear documentation and communication skills

Nice to Have Skills:
3 7 years in QA, testing, or quality roles
Experience with AI/data driven products preferred
ISTQB, AI/Data, or Responsible AI training is a plus

Detailed Job Description:
Role Summary
Ensures AI products behave accurately, safely, fairly, and reliably across real world scenarios. Partners with Product, Engineering, and SMEs to validate model behavior, data quality, and trust risks across the AI lifecycle.

Key Responsibilities
Test AI outputs for accuracy, consistency, edge cases, and failure modes
Validate training and inference data quality; identify bias and data leakage risks
Define and execute model evaluation and regression tests
Reuse and expand the Golden Dataset for regression testing across builds, prompt/model changes, and releases.
Test AI behavior within end to end user workflows and integrations
Support Responsible AI testing (fairness, explainability, compliance)
Monitor model performance post release (drift, feedback, regressions)
Test automation coverage
Communicate risks and test findings clearly to stakeholders
Track & report quality metrics such as accuracy, precision (and recall where applicable), clearly highlighting missing, incorrect, or hallucinated content.

Minimum Years of Experience:
3-7 years