Azure Data Engineer (ADF & Databricks)

Job Title: Azure Data Engineer (ADF & Databricks)

Work Location & Reporting Address: Houston, TX 77079

Vendor Rate: $50/hr

Contract duration 6 Months

Interview Process (Is face to face required?) Yes

Must Have Skills

Azure Data Factory Pipelines, data flows (mapping wrangling), triggers, integration runtimes, parameterization, error handling & retries.

Azure Databricks Databricks Notebooks, Jobs Workflows, PySpark Scala, Delta Lake, cluster configuration and Eligible to workimization.

SQL Expert level SQL (CTEs, window functions, query tuning), working with large datasets.

Nice to have skills

Agile

Infrastructure as Code exposure is a plus.

Detailed Job Description

Experience 6-8 years in data engineering with at least 3 years on Azure. Azure Data Factory Pipelines, data flows mapping wrangling, triggers, integration runtimes, parameterization, error handling retries. Azure Databricks Databricks Notebooks, Jobs Workflows, PySpark Scala, Delta Lake, cluster configuration and Eligible to workimization. Programming Advanced Python or Scala at least one, strong coding standards, modularization, and unit testing e.g., pytest. SQL Expert level SQL CTEs, window functions, query tuning, working with large datasets. Data Storage Formats ADLS Gen2, Delta Lake, Parquet, partitioning, ZOrdering, Eligible to workIMIZE VACUUM. Ops CICD Git, branching strategies, CICD for ADF Databricks environment promotion Infrastructure as Code exposure is a plus.

Minimum years of experience: 8-10 years

Certifications Needed :No

Top 3 responsibilities you would expect the Subcon to shoulder and execute

Coding and Development

UI testing

Stakeholder management

Disclaimer: Diverse Lynx LLC is an Equal Opportunity Employer. All applicants and employees are evaluated without discrimination, based solely on their qualifications, ability, competence and performance. This email and its attachments may contain confidential or proprietary information and is intended only for the recipient(s). If you received this message in error, please disregard it and notify the sender. If you no longer wish to receive our communications, you may unsubscribe at any time.

Security Notice: Our official website is www.diverselynx.com We do not operate or authorize any other websites representing Diverse Lynx LLC.