Data Engineer

Request ID: 100167-1
Title: Data Engineer
Locations: San Jose, CA
Duration: 06+ Months with Possible Extension.
Pay Range: $55-$60/Hour on W2/C2C (All inclusive)

We are seeking a skilled Data Engineer. The ideal candidate will have strong expertise in SQL, Azure data services, Databricks, and modern data engineering practices. You will be responsible for designing, building, and maintaining scalable data pipelines that support analytics, reporting, and operational initiatives. Experience with workflow orchestration tools such as Airflow or Astronomer is highly desirable.

Key Responsibilities

• Design, develop, and maintain scalable and reliable data pipelines for batch and real-time data processing.
• Build and optimize ETL/ELT workflows to ingest, transform, and load data from multiple source systems.
• Develop and optimize complex SQL queries, stored procedures, and database objects for high-performance data processing.
• Work with Azure data services, including Azure Data Factory, Azure Data Lake Storage, Azure SQL Database, and related cloud technologies.
• Develop data transformation workflows using Databricks and Apache Spark.
• Orchestrate and monitor data pipelines using Apache Airflow or Astronomer.
• Process and manage large-scale structured and unstructured datasets efficiently.
• Collaborate with data architects, analysts, and business stakeholders to understand data requirements.
• Ensure data quality, integrity, security, and governance across data platforms.
• Troubleshoot pipeline failures and optimize performance for scalable data processing.
• Participate in code reviews and follow best practices for CI/CD and version control.

Required Skills

• Strong proficiency in SQL with excellent database design and query optimization skills.
• Hands-on experience with Azure Data Services, including:
  • Azure Data Factory (ADF)
  • Azure Data Lake Storage (ADLS)
  • Azure SQL Database
  • Azure Synapse Analytics (preferred)
• Working experience with Azure Databricks and Apache Spark.
• Experience building and maintaining ETL/ELT data pipelines.
• Knowledge of Apache Airflow or Astronomer for workflow orchestration.
• Experience working with both structured and unstructured datasets.
• Understanding of data warehousing concepts and dimensional data modeling.
• Familiarity with Git and CI/CD deployment practices.
• Strong analytical and problem-solving skills.

Preferred Skills

• Experience with Delta Lake and Lakehouse architecture.
• Knowledge of Python or PySpark for data engineering tasks.
• Familiarity with Azure DevOps.
• Experience with data governance and metadata management.
• Exposure to streaming technologies such as Kafka or Event Hubs is a plus.
• Experience working in Agile/Scrum environments.

Qualifications

• Bachelor's degree in Computer Science, Information Technology, Engineering, or a related field.
• 5+ years of experience in Data Engineering or related roles.
• Experience building cloud-native data solutions on Client Azure.
• Strong communication and collaboration skills.

Nice to Have

• Azure Data Engineer Associate certification.
• Experience with enterprise-scale analytics platforms.
• Exposure to monitoring and observability tools for data pipelines.

This role is ideal for a Data Engineer with expertise in Azure, Databricks, SQL, and modern ETL/ELT pipeline development who enjoys building scalable, cloud-based data solutions.

Company Benefits & Culture

• Inclusive and diverse work environment
• Opportunities for professional growth and development
• Comprehensive health and wellness benefits

Appreciate your quick response and please feel free to reach me out for any query you may have.

Thanks