Data Engineer

The ideal candidate must have production-level experience building and managing data pipelines using Databricks, Apache Spark, PySpark, Delta Lake, and Databricks Jobs/Workflows. We are looking for an experienced Databricks Engineer with strong hands-on experience in designing, developing, and maintaining scalable data pipelines using the Databricks platform.

Job Role : Data Engineer

Location : Irvine, CA

Job Summary

We are looking for an experienced Databricks Engineer with strong hands-on experience in designing, developing, and maintaining scalable data pipelines using the Databricks platform.

The ideal candidate must have production-level experience building and managing data pipelines using Databricks, Apache Spark, PySpark, Delta Lake, and Databricks Jobs/Workflows.

Key Responsibilities

Design, develop, and maintain scalable ETL/ELT data pipelines using Databricks.

Build and optimize data processing solutions using Apache Spark and PySpark.

Work extensively with Delta Lake and Delta Tables for reliable data storage and processing.

Develop, schedule, monitor, and troubleshoot Databricks Jobs and Workflows.

Implement batch and incremental data processing pipelines.

Perform data transformations, cleansing, validation, and aggregation using PySpark.

Required Skills:

Hands-on Databricks experience is mandatory.

Production experience building and deploying data pipelines on Databricks.

Strong experience with Spark/PySpark.

Practical experience working with Delta Tables and Delta Lake.

Experience developing and managing Databricks Jobs/Workflows.