Azure Data Engineer

Must Have Skills: - Azure Data Factory Pipelines, Data Flows (Mapping/Wrangling), Triggers, Integration Runtimes, Parameterization, Error Handling, and Retries. - Azure Databricks: Databricks Notebooks, Jobs/Workflows, PySpark, Scala, Delta Lake, Cluster Configuration, and Optimization.

Job Title: Azure Data Engineer
Location: Houston, TX 77079 (Onsite)
Duration: 06 Months

Interview Process (Is face to face required?) Yes (2nd round)

Must Have Skills:
- Azure Data Factory Pipelines, Data Flows (Mapping/Wrangling), Triggers, Integration Runtimes, Parameterization, Error Handling, and Retries.
- Azure Databricks, Databricks Notebooks, Jobs/Workflows, PySpark/Scala, Delta Lake, Cluster Configuration, and Optimization.
- SQL: Expert-level SQL (CTEs, Window Functions, Query Tuning, and working with large datasets).

Nice to Have Skills:
- Agile.
- Infrastructure as Code (IaC) exposure is a plus.

Detailed Job Description:
Experience: 5 8 years in Data Engineering, with at least 3 years on Azure.
- Azure Data Factory: Pipelines, Data Flows (Mapping/Wrangling), Triggers, Integration Runtimes, Parameterization, Error Handling, and Retries.
- Azure Databricks: Databricks Notebooks, Jobs/Workflows, PySpark, Scala, Delta Lake, Cluster Configuration, and Optimization.
- Programming: Advanced Python or Scala (at least one), strong coding standards, modularization, and unit testing (e.g., pytest).
- SQL: Expert-level SQL (CTEs, Window Functions, Query Tuning, and working with large datasets).
- Data Storage Formats: ADLS Gen2, Delta Lake, Parquet, Partitioning, Z-Ordering, OPTIMIZE, and VACUUM.
- Ops: CI/CD, Git, Branching Strategies, and CI/CD for ADF/Databricks Environment Promotion.
- Infrastructure as Code (IaC) exposure is a plus.

Minimum years of experience
8-10 years

Top 3 responsibilities:
Coding and Development
UI testing
Stakeholder management