Lead Data Engineer Snowflake, DBT & IBM DataStage

Requirement ID: 91760-1
Title: Lead Data Engineer Snowflake, DBT & IBM DataStage
Pay range: $53- $60/hr
Location: :San Antonio, TX
Duration:6 months

We are seeking an experienced Lead Data Engineer with 10+ years of experience in designing, developing, and optimizing enterprise data platforms using Snowflake, DBT, IBM DataStage, SQL, and Python. The ideal candidate will lead the development of scalable ETL/ELT pipelines, modern cloud data warehouse solutions, and data transformation frameworks that enable analytics, reporting, and business intelligence.

This role requires expertise in both traditional ETL (IBM DataStage) and modern ELT (DBT + Snowflake), along with strong experience in data modeling, performance optimization, orchestration, and cloud-based data engineering.

Key Responsibilities

Data Engineering & Development

• Design, develop, and maintain scalable ETL/ELT pipelines for structured and semi-structured data.
• Build modern ELT pipelines using DBT and Snowflake.
• Develop and maintain enterprise ETL workflows using IBM DataStage.
• Design scalable dimensional data models including Star and Snowflake Schemas.
• Develop optimized SQL transformations for large-scale datasets.
• Implement efficient data loading strategies leveraging Snowflake architecture.
• Process structured, semi-structured, and JSON/Parquet data within Snowflake.
• Develop reusable and modular DBT models, macros, and tests.

Data Architecture & Optimization

• Optimize Snowflake performance through clustering, partitioning, and warehouse tuning.
• Improve ETL/ELT pipeline performance and scalability.
• Ensure high availability, reliability, and maintainability of data pipelines.
• Implement best practices for metadata management and documentation.
• Monitor data quality, reconciliation, and validation processes.

Cloud & Automation

• Work with cloud platforms including AWS, Azure, or GCP.
• Schedule and monitor workflows using Control-M, Airflow, or equivalent orchestration tools.
• Develop Python scripts for automation, data processing, and utilities.
• Utilize UNIX/Linux shell scripting for operational support and automation.

Leadership & Collaboration

• Lead technical design discussions and mentor junior data engineers.
• Collaborate with Data Architects, Business Analysts, Data Scientists, and business stakeholders.
• Translate business requirements into scalable technical solutions.
• Participate in code reviews and establish engineering best practices.
• Drive continuous improvement initiatives across the data platform.

• Inclusive and diverse work environment
• Opportunities for professional growth and development
• Comprehensive health and wellness benefits