Job Title: Azure Data Engineer (Azure to AWS Migration) Spark, Kafka & Databricks
Location: Dallas TX
Job Type: Contract
Work Arrangement: 5 Days Onsite
Domain: Telecom
Interview : 2-3 rounds, Final round in person required
Note : US Citizenship Required for this role
Job Summary
We are seeking an experienced Azure Data Engineer with strong expertise in Apache Spark, Apache Kafka, Azure Databricks , and cloud migration projects , specifically migrating data platforms from Microsoft Azure to AWS . The ideal candidate will have extensive experience building scalable data pipelines, streaming solutions, and modern data lake architectures while driving cloud transformation initiatives.
This role requires hands-on experience with Azure services, Databricks, Spark optimization, Kafka event streaming, and AWS data services to support enterprise-scale migration efforts.
Key Responsibilities- Design, develop, and maintain scalable batch and real-time data pipelines.
- Lead migration of enterprise data platforms from Microsoft Azure to Amazon Web Services (AWS) .
- Develop and optimize ETL/ELT pipelines using Azure Databricks and Apache Spark .
- Build and maintain real-time streaming applications using Apache Kafka .
- Migrate data lakes, data warehouses, and analytics workloads from Azure to AWS.
- Optimize Spark jobs for performance, scalability, and cost efficiency.
- Work closely with solution architects to design cloud-native AWS data architectures.
- Implement CI/CD pipelines for data engineering deployments.
- Ensure data quality, governance, monitoring, and security throughout migration activities.
- Collaborate with business stakeholders, data scientists, and application teams to deliver reliable data solutions.
- Troubleshoot production issues and optimize distributed data processing environments.
Required Skills Cloud Platforms- Microsoft Azure
- Amazon Web Services (AWS)
- Azure to AWS Migration experience (Mandatory)
Mandatory Qualifications- 8+ years of Data Engineering experience.
- 4+ years of hands-on experience with Azure Databricks .
- Strong expertise in Apache Spark and PySpark .
- Hands-on experience with Apache Kafka and streaming architectures.
- Proven experience in Azure to AWS cloud migration projects .
- Experience designing enterprise-scale data lake solutions.
- Strong SQL and Python programming skills.
- Experience with Delta Lake and modern Lakehouse architecture.
- Knowledge of CI/CD and Infrastructure as Code.
- Excellent communication and stakeholder management skills.
Preferred Qualifications- AWS Certified Data Engineer Associate or AWS Certified Solutions Architect.
- Microsoft Azure Data Engineer Associate (DP-203).
- Databricks Certified Data Engineer Associate/Professional.
- Experience with Snowflake, Iceberg, or Apache Hudi.
- Experience with Airflow or other orchestration tools.