Databricks Data engineer (Remote, USA)
Position Summary
We are seeking an experienced Databricks Solution Architect to join our growing healthcare analytics team. This is a full-time, remote position based in the United States. The ideal candidate will have strong hands-on experience designing and implementing scalable data solutions on Databricks and cloud platforms, with a deep understanding of healthcare data and modern data architecture.
This role is an individual contributor position responsible for solution design, technical leadership, and delivering high-quality data platforms that support analytics, reporting, and operational decision-making.
Key Responsibilities
- Design and implement scalable data solutions using Databricks Lakehouse architecture.
- Develop and optimize batch and real-time data pipelines for large-scale data processing.
- Lead technical solution design for data integration, transformation, and analytics workloads.
- Collaborate with business stakeholders, data engineers, analysts, and product teams to translate requirements into technical solutions.
- Implement data governance, security, and data quality best practices using Unity Catalog and related technologies.
- Optimize performance, reliability, and cost efficiency of Databricks workloads.
- Support data warehouse modernization and cloud migration initiatives.
- Create architecture documentation, technical standards, and design guidelines.
- Troubleshoot complex data platform issues and provide technical recommendations.
- Stay current with Databricks and cloud platform capabilities and drive adoption of best practices.
Required Qualifications
- Bachelor's degree in Computer Science, Information Systems, Engineering, or a related field.
- Minimum 5 years of experience in data engineering, data architecture, or related technical roles.
- Hands-on experience with Databricks in enterprise environments.
- Strong knowledge of PySpark, Spark SQL, Python, and SQL.
- Experience designing and implementing Lakehouse architectures.
- Experience building scalable ETL/ELT data pipelines.
- Strong understanding of data modeling, data warehousing, and analytics platforms.
- Experience working with structured and unstructured data.
- Strong problem-solving and communication skills.
- Ability to work independently in a remote environment.
- Experience implementing data governance and security frameworks.
Preferred Qualifications
- Experience in the healthcare industry, including claims, clinical, provider, member, or revenue cycle data.
- Experience with Azure, AWS, or GCP cloud platforms.
- Exposure to machine learning and AI/ML workloads on Databricks.
Certifications
Required:
- Databricks Certified Data Engineer Associate
- Databricks Certified Data Engineer Professional
Preferred:
- Cloud platform certifications (Azure, AWS, or GCP)