Mandatory Skills: Azure Databricks (ADB), Azure Data Factory (ADF), Azure Analysis Service (AAS), SQL
Job Summary
We are seeking a skilled Azure Data Engineer with strong hands-on expertise in Azure Databricks (ADB), Azure Data Factory (ADF), and SQL. The ideal candidate will design and build scalable data pipelines and transformations on the Azure platform, with solid experience in Spark (PySpark), the Databricks Lakehouse, and ETL/ELT orchestration. Exposure to Delta Lake, data modeling, and CI/CD for data workloads is a plus.
Key Responsibilities
Design, develop, and maintain data pipelines using Azure Databricks (PySpark/Spark SQL) and Delta Lake.
Build and orchestrate ETL/ELT workflows in Azure Data Factory, including linked services, datasets, and triggers.
Write and optimize complex SQL for data transformation, aggregation, and performance tuning.
Collaborate with cross-functional teams to deliver reliable, high-quality data solutions.
Implement data quality checks, monitoring, and follow engineering best practices.
Work in an Agile environment and contribute to sprint planning and retrospectives.
Skill Requirements
Databricks: Strong proficiency in Azure Databricks, PySpark, Spark SQL, and notebook development.
Data Integration: Hands-on experience with Azure Data Factory (pipelines, mapping data flows, triggers).
Database: Expert-level SQL (mandatory) with T-SQL/Spark SQL; query optimization and stored procedures.
Storage & Format: Experience with Delta Lake, Parquet, and Azure Data Lake Storage (ADLS Gen2).
Data Modeling: Understanding of dimensional modeling and data warehousing concepts.
Version Control: Git/Azure Repos or similar tools.
CI/CD: Familiarity with Azure DevOps pipelines for data deployments.
Cloud: Working knowledge of the broader Azure ecosystem (Key Vault, Synapse is a plus).