| Location | TX, TX |
Location: Louisville, KY / Dallas, TX
Experience: 8–10 Years
Seeking a Senior Data QA / Automation Engineer with strong experience in ETL/Data Testing, Databricks, Spark/PySpark, Python, SQL, and AI/ML validation. The ideal candidate will be responsible for validating enterprise data pipelines, Databricks platforms, ML workflows, integrations, and automated quality processes.
Perform end-to-end ETL/ELT, data quality, and Source-to-Target validation.
Validate data transformations, aggregations, business rules, lineage, schemas, metadata, and Delta Lake.
Test CDC, incremental loads, batch and streaming pipelines.
Validate Databricks Notebooks, Workflows, Jobs, Delta Live Tables, Unity Catalog, and Bronze/Silver/Gold data layers.
Perform Spark/PySpark transformation and data quality testing.
Validate AI/ML models, predictions, feature engineering, model inputs/outputs, and MLOps workflows.
Test data drift, schema drift, model deployment, MLflow/Databricks ML/Azure ML integrations, and GenAI/RAG workflows where applicable.
Perform API, integration, contract, and end-to-end data flow testing.
Develop automated testing frameworks and reusable utilities using Python and SQL.
Integrate automated tests and quality gates into CI/CD pipelines.
Generate test reports, metrics, and quality validation results.
8–10 years of QA/Automation Testing experience.
6+ years of ETL/Data Testing experience.
3+ years of Databricks/Spark/PySpark testing.
Strong Python and SQL programming skills.
Hands-on AI/ML Model Testing or MLOps validation experience.
Strong experience with data warehouses, data lakes, APIs, and cloud data platforms.
Experience building and maintaining test automation frameworks.
Strong understanding of data quality, reconciliation, validation, and integration testing.
Experience with MLflow, Azure ML, Delta Lake, Unity Catalog, CI/CD, and GenAI/RAG testing.
Experience testing large-scale structured and semi-structured datasets.
Strong understanding of enterprise data architecture and data pipelines.