Requisition ID: 109154-1
Title:Data Engineer (Azure Databricks)
Duration: 6-12 months with possible extension
Location:Cary NC
Salary Range: $50- $55 an hour on W2/C2C
Job Description:
" Design and build batch and streaming data pipelines using Azure, Databricks, and Kafka.
" Develop ETL/ELT solutions using Spark, SQL and Python/Pyspark, or Scala
" Implement and support Data Lake / Lakehouse architectures
" Integrate data from legacy and modern platforms. Preferred postgres DB migration to Azure Cloud.
" Expertise in handling Data migration using API s.
" Apply data quality checks and support data governance basics (metadata, lineage, access controls)
" Optimize Spark jobs for performance, reliability, and cost
" Collaborate with cross-functional teams, Product owners, Admin teams, BSAs, Scrum & Project managers
Required Skills & Experience:
" 8+ years of experience in data engineering in azure using Azure Databricks.
" Strong experience with Azure Data Lake, Databricks, ADF, Postgres.
" Strong Data Engineering experience in Big Data (Cloudera)
" Hands-on expertise in SQL, Python and PySpark or Scala
" Proven experience with batch & real-time ingestion
" Experience with Kafka and big data processing
" Added good to have experience with:
" Informatica PowerCenter
" Informatica IDMC or equivalent cloud data integration tools
" SQL Server, Oracle, Azure SQL
" Cosmos DB
" Unix/Linux
" Maestro (or similar enterprise schedulers)
" Data tools such as Precisely or equivalent
Added Advantage
" Insurance domain knowledge (Policy, Claims, Billing, Underwriting, Actuarial, Regulatory data)
" Experience with Azure DevOps (AzDO) and GitHub-based CI/CD
" Practical usage of Copilot for data engineering and development productivity
" Exposure to Generative AI / GenAI use cases in data platforms and analytics
Appreciate your quick response and please feel free to reach me out for any query you may have.
Thanks