Lead Data Engineer
Qcells
San Francisco, CA
Description
POSITION DESCRIPTION:
We are seeking a Lead Data Engineer to architect, build, and lead the development of scalable, cloud-based data platforms that support enterprise analytics, operational reporting, and advanced data use cases. This role provides technical leadership in designing and optimizing ETL/ELT frameworks using Azure data services (Fabric, Data Lake, Data Factory), integrating data from ERP, CRM, and operational systems, and establishing robust data models within a modern lakehouse architecture.
The ideal candidate brings deep SQL and Python expertise, extensive experience with distributed data platforms, and strong knowledge of data architecture, governance, and performance optimization. This individual will serve as a technical leader and mentor, partnering closely with architects, analysts, application teams, and business stakeholders to deliver reliable, scalable, and well-governed enterprise data solutions.
RESPONSIBILITIES
- Lead the design, development, and optimization of scalable data pipelines supporting ingestion, transformation, and enterprise-wide data consumption.
- Architect and implement enterprise-grade ETL/ELT frameworks using Azure Fabric or comparable cloud data platforms.
- Oversee and optimize data integrations from ERP (NetSuite/SAP), CRM (Salesforce), internal systems, APIs, and third-party data sources.
- Design and govern high-quality, scalable data models supporting analytics, reporting, operational systems, and advanced use cases.
- Partner with Data Architects to define and implement lakehouse patterns, Delta Lake strategies, medallion architecture, and domain-driven design principles.
- Establish and enforce data quality frameworks, validation standards, lineage tracking, and observability practices.
- Drive performance optimization, scalability, reliability, and cost governance across cloud environments.
- Provide technical leadership and mentorship to data engineers; conduct design reviews and enforce engineering best practices.
- Collaborate cross-functionally with analysts, application teams, and business stakeholders to translate requirements into scalable data solutions.
- Lead MDM, metadata management, governance, and data standardization initiatives.
- Oversee CI/CD automation, DevOps integration, testing frameworks, and monitoring strategies for data workflows.
- Evaluate emerging technologies and recommend platform improvements aligned with enterprise strategy.
MINIMUM QUALIFICATIONS
- 10+ years of experience in data engineering, data architecture, or related roles.
- Proven experience leading large-scale data platform initiatives in cloud environments.
- Extensive hands-on experience with Azure data services (Data Lake, Data Factory, Fabric, Synapse, or similar).
- Advanced proficiency in SQL and Python; experience with Spark or distributed processing frameworks.
- Deep experience designing and implementing enterprise ETL/ELT frameworks.
- Strong expertise in data modeling (dimensional modeling, star schema, lakehouse/Delta modeling).
- Experience integrating complex enterprise systems (ERP, CRM, operational platforms).
- Strong understanding of data governance, metadata management, MDM, and data quality frameworks.
- Experience with performance tuning, workload optimization, and cloud cost management.
- Demonstrated ability to lead technical teams, conduct architecture reviews, and mentor engineers.
- Strong problem-solving, debugging, and system design skills.
- Travel may be required up to 5%, depending on business needs.
PREFERRED QUALIFICATIONS
- Experience with Delta Tables, Snowflake, Synapse, or comparable cloud data platforms.
- Experience with event-driven and streaming architectures (Kafka, Event Hub, streaming pipelines).
- Familiarity with finance, operations, energy, or ERP-driven data domains.
- Experience designing API-based data integrations and modern integration patterns.
- Azure certifications (Data Engineer Associate, Solutions Architect, or equivalent).
- Experience enabling analytics teams, data science workflows, or ML pipelines.
- Experience implementing enterprise data security and compliance frameworks.
PHYSICAL, MENTAL & ENVIRONMENTAL DEMANDS: To comply with the Rehabilitation Act of 1973 the essential physical, mental and environmental requirements for this job are listed below. These are requirements normally expected to perform regular job duties. Incumbent must be able to successfully perform all of the functions of the job with or without reasonable accommodation. | |||||||||||||||||||||||||||||||||||||||||||||||||||||
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||
The salary range is required by the California Pay Transparency Act and may differ depending on the location of those candidates hired nationwide. Actual compensation is influenced by a wide array of factors including but not limited to, skill set, education, licenses and certifications, essential job duties and requirements, and the necessary experience relative to the job’s minimum qualifications.
*This target salary range is for CA positions only and should not be interpreted as an offer of compensation.
You may view your privacy rights by reviewing Qcells' Privacy Policy or by contacting ourhqca-recruiting@qcells.com
for a copy.