Hands-on experience with AWS data services including S3, AWS Glue, Athena, Lambda, Step Functions, and EventBridge. Design and build enterprise ETL and ELT pipelines using Databricks and related cloud technologies.
Numbers & Facts
Location
Malvern, PA
Description
We are seeking a Senior Databricks Engineer for a contract position based in Malvern, Pennsylvania. This role is available for a minimum duration of six months.
Responsibilities
Design and build enterprise ETL and ELT pipelines using Databricks and related cloud technologies
Develop and optimize data processing solutions using Apache Spark and PySpark
Architect and implement large-scale cloud data platform solutions
Write and maintain Python and SQL code for data engineering workflows
Implement CI/CD pipelines for data engineering applications
Apply dimensional data modeling principles to support analytics and reporting
Collaborate with stakeholders to gather requirements and communicate technical solutions
Participate in Agile/Scrum development processes
Qualifications
Required
Bachelor's degree in Computer Science, Engineering, Information Systems, or related discipline
5-7+ years of enterprise Data Engineering experience
Strong Python development experience
Advanced SQL programming skills
Extensive experience with Apache Spark and PySpark
Strong experience with Databricks application
Hands-on experience with AWS data services including S3, AWS Glue, Athena, Lambda, Step Functions, and EventBridge
Experience building enterprise ETL and ELT pipelines
Strong understanding of distributed data processing
Experience working with large-scale cloud data platforms
Strong knowledge of dimensional data modeling
Experience implementing CI/CD pipelines
Familiarity with Agile/Scrum methodologies
Excellent communication and stakeholder management skills