Data / AI QE Lead — Retail eCommerce

Data / AI QE Lead — Retail eCommerce

Location-San Ramon, California or Beverly Hills, CA (Onsite)
Job Type-Long Term Contract

Role Summary

The Data / AI QE Lead will define the quality engineering strategy for data pipelines, machine learning models, and AI-powered features across the retail eCommerce platform. This role bridges traditional data quality assurance and emerging AI/ML validation disciplines, ensuring that customer-facing capabilities — including product recommendations, personalization, search relevance, demand forecasting, and pricing intelligence — perform accurately, fairly, and reliably at scale.

Data Quality Engineering

• Define and own the QE strategy for data assets including customer, product, inventory, transaction, and behavioral event data
• Design and implement data validation frameworks covering completeness, accuracy, consistency, timeliness, and referential integrity
• Lead testing of ETL/ELT pipelines, data lake and warehouse layers (raw, curated, consumption), and real-time streaming pipelines
• Establish data contract testing practices between producing and consuming systems
• Build automated data quality monitors and alerting that operate continuously in production environments
• Partner with data governance and data stewardship teams to align QE standards with enterprise data policies

AI / ML Model Quality & Validation

• Lead quality validation for ML models powering eCommerce capabilities: product recommendations, personalized search, dynamic pricing, demand forecasting, propensity models, and generative AI features
• Define model evaluation frameworks including offline metrics and online business metrics (CTR, conversion rate, AOV, revenue lift)
• Design and execute A/B and shadow testing strategies to validate model performance before and during production rollout
• Assess and test for model fairness, bias, and regulatory compliance across customer segments and product categories
• Validate model monitoring and drift detection systems to ensure production models remain within acceptable performance thresholds
•  

eCommerce Platform Integration Testing

• Drive end-to-end quality of data flows from customer interaction events through to AI feature delivery on site, app, and email channels
• Test integrations between the eCommerce platform and downstream data consumers including CDP, CRM, marketing automation, and analytics tools
• Validate real-time personalization pipelines for homepage, PDP, cart, and post-purchase experiences
• Ensure data quality for key eCommerce events: product views, add-to-cart, checkout, order confirmation, returns, and search queries
• Test search and browse relevance improvements driven by ML rankers and query understanding models

Test Automation & Observability

• Build and scale automated data and AI testing frameworks integrated into CI/CD and model deployment pipelines
• Define and enforce data quality SLAs and embed automated gates into pipeline orchestration (Airflow, dbt, Spark, etc.)
• Implement observability tooling for data pipelines and AI model inputs/outputs in collaboration with data and ML engineering
• Drive adoption of synthetic data and data masking strategies to support safe,

Qualifications

Required

• 7+ years in data or quality engineering, with at least 2 years leading a team or technical discipline
• Proven experience testing data pipelines (batch and streaming) across modern data stack technologies (Spark, Kafka, Airflow, dbt, Snowflake, BigQuery, Databricks, or similar)
• Hands-on experience with ML model evaluation techniques, including offline metrics and online experimentation (A/B testing)
• Strong SQL skills and proficiency in Python for data validation scripting and test automation
• Familiarity with eCommerce data domains: customer behavior, product catalog, order management, inventory, and digital marketing