Piper Companies is seeking a Data Quality Engineer to design, implement, and scale enterprise data quality frameworks across modern data platforms. This role will focus on building automated validation pipelines, embedding quality into CI/CD processes, and ensuring data reliability across ingestion, transformation, and consumption layers. The ideal candidate brings strong SQL and programming expertise, deep knowledge of data quality dimensions, and hands-on experience with modern data stacks such as Fabric, Databricks, or Spark.
Responsibilities
- Design and implement automated data testing frameworks (e.g., PyTest, Great Expectations, dbt tests) to validate data pipelines at scale
- Develop advanced SQL queries for data profiling, reconciliation, validation, and anomaly detection
- Build and maintain data quality checks across ETL/ELT pipelines to ensure reliability across all stages of data movement
- Integrate data quality validations into CI/CD pipelines (e.g., Azure DevOps, GitHub Actions, Airflow) to enforce automated quality gates
- Implement observability and monitoring solutions to detect anomalies, schema drift, and data freshness issues in production
- Perform root-cause analysis on data defects and partner with engineering teams to implement sustainable fixes
- Collaborate with data engineers, analysts, and stakeholders to define data quality standards and translate business rules into testable checks
Qualifications
- 6+ years of experience in data engineering or data quality engineering roles
- Strong SQL expertise with the ability to write complex queries for data validation and analysis
- Proficiency in Python (preferred) or Scala/Java for building data validation frameworks and automation
- Hands-on experience with data testing tools such as Great Expectations, dbt, or custom frameworks using PyTest
- Solid understanding of ETL/ELT pipelines and common data quality failure points
- Experience implementing data quality checks within CI/CD environments (Azure DevOps, GitHub Actions, Airflow, etc.)
- Familiarity with data quality dimensions: accuracy, completeness, consistency, timeliness, validity, and uniqueness
- Experience working with modern data platforms such as Microsoft Fabric, Databricks, or Spark
- Strong analytical and debugging skills with a focus on root-cause resolution
- Excellent communication skills and experience working across cross-functional teams
Compensation
- Salary: Competitive based on experience
- Benefits: Comprehensive benefits package (details not specified)
Keywords
Data Quality Engineering, SQL, Python, PyTest, Great Expectations, dbt, ETL, ELT, Data Pipelines, Data Validation, Data Observability, Monte Carlo, Soda, CI/CD, Azure DevOps, GitHub Actions, Airflow, Databricks, Microsoft Fabric, Apache Spark, Data Testing Frameworks, Data Governance, Root Cause Analysis, Bethesda MD
Tags: #LI-MJ1 #LI-ONSITE