Senior ETL & AI/ML Test Automation Engineer
Must Have Technical/Functional Skills
ETL & Data Testing
- Validate end-to-end ETL/ELT pipelines across enterprise data platforms.
- Perform Source-to-Target validation and reconciliation testing.
- Validate data transformations, aggregations, business rules, and lineage.
- Execute data quality checks on large-scale structured and semi-structured datasets.
- Perform schema, metadata, and Delta Lake validations.
- Write complex SQL queries for backend validation and data integrity testing.
- Validate CDC (Change Data Capture) and incremental data loading processes.
- Test batch and streaming data pipelines.
Databricks Testing
- Validate Databricks notebooks, Workflows, Jobs, Delta Live Tables, and Unity Catalog implementations.
- Test Bronze, Silver, and Gold layer data transformations.
- Verify workflow orchestration and dependency handling across Databricks jobs.
- Perform performance validation of Spark and PySpark transformations.
- Validate data quality frameworks implemented in Databricks.
AI/ML Testing
- Validate AI/ML model integrations within enterprise applications.
- Verify model scoring results and prediction outputs.
- Perform data drift, schema drift, and feature validation testing.
- Validate feature engineering pipelines and training datasets.
- Execute model input/output verification against expected business outcomes.
- Validate explainability, traceability, and auditability requirements.
- Test model deployment workflows integrated through MLflow, Databricks ML, Azure ML, or similar platforms.
- Validate GenAI/RAG workflows, prompt-response accuracy, and grounding mechanisms where applicable.
Integration Testing
- Perform end-to-end integration testing across:
o Databricks
o Data warehouses
o APIs
o AI/ML services
o Cloud platforms
o Downstream reporting systems
- Validate data flow between source systems, data lakes, ML models, and consuming applications.
- Perform contract testing and API validation.
Test Automation
- Design and develop automation frameworks for:
o ETL testing
o API testing
o Data validation
o ML model validation
- Build reusable test utilities using Python.
- Integrate automated tests into CI/CD pipelines.
- Develop automated quality gates for deployment validation.
- Generate automated execution reports and test metrics.
required
6+ years of experience in ETL/Data Testing.
3+ years of experience testing Databricks or Spark-based platforms.
Hands-on experience in AI/ML Model Testing or MLOps validation.
Strong coding experience in Python and SQL. Education
Bachelor's degree
TCS Employee Benefits Summary
- Discretionary Annual Incentive.
- Comprehensive Medical Coverage: Medical & Health, Dental & Vision, Disability Planning & Insurance, Pet Insurance Plans.
- Family Support: Maternal & Parental Leaves.
- Insurance Options: Auto & Home Insurance, Identity Theft Protection.
- Convenience & Professional Growth: Commuter Benefits & Certification & Training Reimbursement.
- Time Off: Vacation, Time Off, Sick Leave & Holidays.
- Legal & Financial Assistance: Legal Assistance, 401K Plan, Performance Bonus, College Fund, Student Loan Refinancing.
Salary Range: $70000 - $85000 a year
#LI-JH1