Databricks Engineer - Remote, Washington, DC
Duration: 12 months
Seeking for a Databricks Engineer responsible for designing, developing, maintaining, and optimizing scalable data pipelines and ETL processes using Databricks, Python, PySpark, and SQL. This role will play a key part in modernizing data platforms and supporting enterprise data initiatives.
Day-to-day Responsibilities:
- Create and maintain ETL pipelines and/or migrate existing legacy pipelines (e.g., synapse) to Databricks
- Develop scalable data processing solutions using Python, PySpark, and SQL.
- Perform data transformation, cleansing, and validation to ensure data quality and consistency.
- Optimize data workflows and Spark jobs for performance and efficiency.
- Collaborate with business stakeholders, data architects, and development teams to understand requirements and deliver effective solutions.
- Monitor, troubleshoot, and resolve data pipeline issues.
Qualifications:- Hands-on experience with Databricks.
- Strong programming skills in Python and PySpark.
- Strong SQL development and query optimization experience.
- Experience designing, developing, and maintaining ETL pipelines.
- Experience working with large-scale data processing and analytics solutions.
- Strong analytical, troubleshooting, and problem-solving skills.
- Excellent communication and collaboration abilities.
Required Skills:
- Databricks
- Python
- PySpark
- SQL
Education: