Data Engineer - DataBricks
Contech Systems
Newark, NJ
Apply
JOB DETAILS
SALARY
$85–$100 Per Hour
LOCATION
Newark, NJ
POSTED
30+ days ago
(Hybrid Newark, NJ — 2 days/month onsite, with flexibility to come in as needed)
Contract to Hire
We’re hiring a Senior Databricks Data Engineer to help build Horizon’s next-generation healthcare data platform. You’ll design large-scale pipelines, implement medallion architecture, optimize Spark performance, and shape the modeling patterns for analytics, clinical, and enterprise data.
If you're a true builder who knows the difference between "running Spark jobs" and actually engineering data systems, this is your playground.
Senior Databricks Data Engineer Responsibilities
- Architect, design, and develop large-scale ETL/ELT pipelines in Databricks, Spark, and Delta Lake.
- Implement medallion architecture and best practices for scalable data modeling.
- Build workflows, notebooks, jobs, and orchestration patterns within Databricks.
- Optimize Spark performance (clusters, caching, partitions, Z-ordering, autoscaling, shuffles).
- Apply data quality frameworks (expectations, validations, profiling).
- Work with cross-functional partners to translate requirements into actionable data solutions.
- Contribute to governance: lineage, cataloging, RBAC, compliance, metadata management.
- Troubleshoot and resolve issues across ETL pipelines, notebooks, and compute clusters.
- Mentor junior engineers as the team expands.
- Expert-level experience with Databricks, Spark, PySpark, and Delta Lake.
- Strong SQL and Python.
- Proven experience designing complex pipelines and distributed jobs.
- Experience with cloud platforms (Azure or AWS).
- Solid understanding of data modeling (conceptual, logical, physical).
- Experience with healthcare datasets (claims, eligibility, provider, clinical) is a major plus.
- Ability to work end-to-end: design → build → optimize → document.
- Knowledge of FHIR or other healthcare interoperability standards.
- Experience with orchestration tools (Airflow, ADF, Databricks Jobs).
- CI/CD for data engineering.
About the Company
C