Request ID: 107834-1
Title: Data Engineer with Databricks
Locations: Tampa, FL
Duration: 6 Months
Pay Range: $44 - $47/Hour on W2/C2C (All inclusive)
We are seeking an experienced Data Engineer with 4 6 years of experience and strong hands-on expertise in Databricks and Apache Spark. The ideal candidate will design, develop, and optimize scalable data pipelines to support analytics and data-driven business initiatives.
The candidate will work closely with data scientists, analysts, business stakeholders, and cross-functional teams to ensure data is reliable, secure, high-quality, and readily available.
Key Responsibilities
- Design, develop, and maintain scalable ETL/ELT data pipelines using Databricks and Apache Spark.
- Build and optimize batch and real-time/streaming data pipelines for large volumes of structured and unstructured data.
- Develop data engineering solutions using Databricks, Spark, and related data technologies.
- Collaborate with data scientists, analysts, and business stakeholders to understand data requirements and translate them into technical solutions.
- Monitor, troubleshoot, and optimize data processing workflows to ensure reliability and performance.
- Optimize Databricks clusters for performance, scalability, and cost efficiency.
- Implement data quality, governance, security, and compliance standards across data pipelines.
- Perform data validation and troubleshooting to identify and resolve pipeline issues.
- Maintain technical documentation related to data architecture, data pipelines, workflows, and processes.
- Support continuous improvement of data engineering processes and platform capabilities.
- Explore and integrate AI/Agentic AI capabilities into data engineering workflows where applicable.
Required Skills
- 4 6 years of experience in Data Engineering.
- Strong hands-on experience with Databricks.
- Strong experience with Apache Spark.
- Experience developing ETL/ELT pipelines.
- Experience with both batch and streaming data processing.
- Experience working with large volumes of structured and unstructured data.
- Strong understanding of data engineering and data processing concepts.
- Experience with data pipeline monitoring, troubleshooting, and performance optimization.
- Knowledge of data governance, security, and compliance.
- Experience optimizing Databricks clusters for cost and performance.
- Strong problem-solving and analytical skills.
- Good communication and collaboration skills.
Preferred Skills
- Experience with AI Agents / Agentic AI.
- Experience integrating AI capabilities into data platforms or data workflows.
- Knowledge of cloud data platforms such as AWS, Azure, or GCP.
- Experience with streaming technologies such as Kafka.
- Knowledge of data lake/lakehouse architecture.