Data Engineer (Azure Databricks) - Philadelphia, PA (Hybrid) Summary
Seeking a Data Engineer with 7+ years of experience to build and support cloud-based data platforms using Azure and Databricks. The role focuses on developing scalable data pipelines, supporting analytics and reporting, and enabling data-driven decision-making in a healthcare environment.
Key Responsibilities
- Build and maintain ETL/ELT data pipelines.
- Develop data solutions using Azure Databricks, ADF, and ADLS.
- Create and optimize PySpark and SQL-based data processing frameworks.
- Implement Delta Lake and Lakehouse architectures.
- Support data migration, modernization, and cloud transformation projects.
- Ensure data quality, reliability, and performance.
- Build data models and curated datasets for reporting and analytics.
- Implement CI/CD, monitoring, and DevOps best practices.
- Collaborate with business users, architects, and product teams in an Agile environment.
Required Skills
- Azure Databricks
- Azure Data Factory (ADF)
- Azure Data Lake Storage (ADLS Gen2)
- PySpark
- Python
- SQL
- Delta Lake
- Data Modeling
- ETL/ELT Development
- Azure Cloud Platform
- Git & CI/CD
- Agile/Scrum
Preferred Skills
- Azure Synapse Analytics
- Power BI
- Azure DevOps
- Infrastructure as Code (IaC)
- FHIR/HL7
- AI/ML Data Pipelines
Required Experience
- 7+ years in Data Engineering
- 5+ years with Azure Data Technologies
- 2+ years in Healthcare, Payer, Health Plan, or Managed Care domains
- Experience with large-scale data pipelines and integration solutions
Nice to Have
- Claims Data
- Membership Data
- Provider Data
- Care Management Data