Role Technical Lead - Databricks Developer
Location Remote
Client BeOne Medicine
Position Overview
We are partnering with a world-class healthcare and biotechnology leader to find a highly skilled Technical Lead - Databricks Developer. In this critical role, you will design and implement robust, scalable data engineering solutions using the Databricks Lakehouse architecture.
This position is ideal for a hands-on technical leader who excels at bridging the gap between complex data processing and client business needs. You will collaborate closely with enterprise architects and lead engineers, taking ownership of end-to-end data pipelines, optimizing system performance, and driving modern data governance practices in a dynamic corporate environment.
Key Responsibilities
- Architecture & Solution Design: Collaborate with architects and lead engineers to design and build solutions that meet both functional and non-functional enterprise requirements.
- Pipeline Engineering: Architect, build, and manage complex, scalable data pipelines supporting both batch and streaming data processing.
- Databricks Orchestration & Governance: Implement and manage Databricks workflows orchestration, leveraging Unity Catalog for data governance and access control.
- Performance Optimization: Proactively identify bottlenecks and drive rigorous performance optimization initiatives to maximize efficiency and significantly reduce compute costs.
- Client Engagement: Work directly with customers and stakeholders, utilizing exceptional communication skills to translate business challenges into technical deliverables.
Required Qualifications
- Databricks Expertise: Must have successfully implemented at least one enterprise project end-to-end within Databricks. Deep understanding of the Databricks Lakehouse concept and its implementation in large-scale environments is mandatory.
- Core Data Components: Proven, hands-on experience with core ecosystem components including Delta Lake, Unity Catalog, Databricks Workflows, and basic data governance principles.
- Programming & Frameworks: Excellent, production-level coding skills in Python, strong proficiency in SQL and Spark-SQL, and extensive knowledge of Apache Spark and Hive data processing frameworks.
- Technical Foundations: Strong foundational knowledge of data structures, algorithms, and advanced analytical problem-solving skills.
- Communication Skills: Exceptional verbal and written communication skills with the proven ability to engage confidently and directly with enterprise clients.
Preferred Qualifications
- Industry Trends: Demonstrated knowledge of relevant industry standards, emerging data engineering trends, and modern cloud ecosystems.
- Continuous Growth: A strong, proactive attitude toward learning new skills, upskilling in cutting-edge data technologies, and mentoring peers.
Education
- Bachelors or Masters in Information Technology, Computer Science or relevant field.