Data Engineer

Must have atleast 8+ years of extensive Data engineering experience in Databricks, ETL/ELT using data pipelines, SQL/Procesurr experience. Must have 8 years of hands on development proficiency in implementing Databricks solutions using scala and spark with data frames and notebooks/sql.

Request ID: 110410-1
Title: Data Engineer
Locations: Quincy, MA - 100% ONSITE
Duration: 6 months
Pay Range: $45 - $50/Hour on W2/C2C (All inclusive)

JOB DESCRIPTION:

• Databricks, Scala- Data Engineer
• Must have atleast 8+ years of extensive Data engineering experience in Databricks, ETL/ELT using data pipelines, SQL/Procesurr experience
• Must have 8 years of hands on development proficiency in implementing Databricks solutions using scala and spark with data frames and notebooks/sql
• Must have 8+ years of hands on development and performance tuning/ enhancement experience in scala/pyspark and sql
• Must have strong experience with Databricks, including developing and optimizing spark jobs, data transformations, and data processing workflows

Must have experience and a good understanding on the below topics

• Databricks delta lake storage
• Unity catalog
• Autoloader
• Lakehouse architecture
• Medallion architecture
• Databricks serverless options
• Dataset and data frame
• Databricks clusters configuration and sizing
• Databricks job orchestration
• Spark structured streaming
• Must have experience with complex and large volume data streaming and data transformations using Azure Databricks
• Must have extensive experience in datawarehouse / data lakehouse implementation
• Must have experience with monitoring, debugging, and resolving issues in kive Databricks jobs
• Must have strong hands on expertise in troubleshooting devops pipelines, azure , and AKS services
• The candidate shd be able to provide data ingestion, streaming and transformations solutions and do development woek
• Experience with apache big data implementation with programming proficiency like pyspark, R or Java prior to Databricks is advantageous
• Good to have experience with containerization technologies such as docker and kubernetes

Company Benefits & Culture
• Opportunity to work with a dynamic team in a fast-paced environment
• Exposure to cutting-edge technologies and methodologies
• Supportive and collaborative work culture

For immediate consideration please click APPLY to begin the screening process with Alex.