Title: Azure Databricks Data Engineer
Location: Saint Louis, MO-Day 1 Onsite
Duration: Full Time/Permanent Role
Salary Range: $110,000-$135,000 Per Year + Benefits
Must Have Technical/Functional Skills
- Significant data engineering experience, including substantial hands-on Databricks development experience.
- Azure Databricks - Strong hands-on experience building production-grade batch pipelines and lakehouse solutions.
- PySpark / Spark SQL - Advanced transformations, joins, window functions, reusable frameworks, debugging, and performance tuning.
- Delta Lake - Table design, schema evolution, merge/upsert patterns, Eligible to workimization, data reliability, and history management.
- Medallion architecture - Practical delivery experience across source-aligned, conformed, and consumption-ready layers.
- Data modeling - Fact/dimension design, conformed dimensions, slowly changing dimensions, aggregates, and semantic-consumption considerations.
- Data quality and testing - Profiling, rule implementation, reconciliation, unit/integration tests, controls, and defect analysis.
- Azure ecosystem - Working knowledge of Azure Data Lake Storage and relevant Azure integration, identity, monitoring, and DevOps services.
- DevOps - Git-based development, code reviews, CI/CD, environment promotion, release controls, and automated deployment.
- Security/governance - Experience implementing least-privilege access and working with catalog, lineage, classification, and row-level security requirements.
- SQL - Advanced SQL development and query Eligible to workimization for large analytical datasets.
- Hands-on exposure to Databricks Lakeflow Spark Declarative Pipelines or Delta Live Tables and the ability to adEligible to work evolving Databricks-native pipeline standards.
- Experience with Unity Catalog; familiarity with enterprise data-governance and data-quality tooling such as Informatica.
- Experience with HR data from Workday, Dayforce, PeopleSoft, recruiting, workforce management, or talent systems.
- Understanding of Power BI consumption patterns, star schemas, refresh needs, aggregates, and performance requirements.
- Experience creating accelerators, reusable engineering templates, metadata-driven pipelines, or automated test frameworks.
- Databricks Data Engineer certification or equivalent cloud/data engineering credential.
- Experience in a blended onshore/offshore Agile delivery model.
Roles & Responsibilities
- Design and develop scalable batch data pipelines in Azure Databricks using PySpark and Spark SQL.
- Build and maintain bronze, silver, and gold data structures, including certified fact and dimension data products.
- Translate business metric definitions, calculation rules, filters, granularity, history, and refresh requirements into technical designs.
- Create reusable transformations for HR domains such as retention, hiring, transfers, promotions, reviews, hours, and scheduling.
- Implement data-quality checks, reconciliation, exception handling, auditability, lineage, and technical documentation.
- Eligible to workimize Spark workloads, Delta tables, joins, partitions, file sizes, and cluster usage for performance and cost.
- Apply enterprise security and governance standards, including Unity Catalog integration, permissions, sensitive-data handling, and support for row-level access patterns.
- Coordinate with ingestion, data governance, modeling, DevOps, and Power BI teams to define clear data contracts and acceptance criteria.
- Contribute to CI/CD, source control, automated testing, deployment, operational monitoring, and production-readiness activities.
- Participate in requirement workshops and explain technical choices clearly to business, HR, analytics, and engineering stakeholders.
- Support knowledge transfer and mentor participating engineers or analysts
Disclaimer: Diverse Lynx LLC is an Equal Opportunity Employer. All applicants and employees are evaluated without discrimination, based solely on their qualifications, ability, competence and performance. This email and its attachments may contain confidential or proprietary information and is intended only for the recipient(s). If you received this message in error, please disregard it and notify the sender. If you no longer wish to receive our communications, you may unsubscribe at any time.
Security Notice: Our official website is www.diverselynx.com We do not operate or authorize any other websites representing Diverse Lynx LLC.