Pay Rate Range: $ 51.42 - 52.94/hr.
Microsoft Azure (ADB/ADF)
Job Description:
• Design and build batch and streaming data pipelines using Azure, Databricks, and Kafka.
• Develop ETL/ELT solutions using Spark, SQL and Python/Pyspark, or Scala
• Implement and support Data Lake / Lakehouse architectures
• Integrate data from legacy and modern platforms. Preferred postgres DB migration to Azure Cloud.
• Expertise in handling Data migration using API's.
• Apply data quality checks and support data governance basics (metadata, lineage, access controls)
• Optimize Spark jobs for performance, reliability, and cost
• Collaborate with cross-functional teams, Product owners, Admin teams, BSAs, Scrum & Project managers
Required Skills & Experience:
• 8+ years of experience in data engineering in azure using Azure Databricks.
• Strong experience with Azure Data Lake, Databricks, ADF, Postgres.
• Strong Data Engineering experience in Big Data (Cloudera)
• Hands-on expertise in SQL, Python and PySpark or Scala
• Proven experience with batch & real-time ingestion
• Experience with Kafka and big data processing
• Added good to have experience with:
• Informatica PowerCenter
• Informatica IDMC or equivalent cloud data integration tools
• SQL Server, Oracle, Azure SQL
• Cosmos DB
• Unix/Linux
• Maestro (or similar enterprise schedulers)
• Data tools such as Precisely or equivalent
Added Advantage
• Insurance domain knowledge (Policy, Claims, Billing, Underwriting, Actuarial, Regulatory data)
• Experience with Azure DevOps (AzDO) and GitHub-based CI/CD
• Practical usage of Copilot for data engineering and development productivity
• Exposure to Generative AI / GenAI use cases in data platforms and analytics
Skills: Digital : Microsoft Azure
Experience Required: 6-8 years
| Skills: | | Category | Name | Required | Importance | Experience |
|---|
| SkillCategoryTest1_MN | Digital : Microsoft Azure | Yes | 1 | >7 years | |
|
|---|