Lead Data Engineer

The engineer will be responsible for preparing and validating data in BigQuery, moving and mirroring data into Microsoft Fabric, developing Fabric-based data pipelines and Lakehouse data models, and performing detailed source-to-target reconciliation. The ideal candidate should be strong in SQL and PySpark, understand dimensional data modeling, and be comfortable troubleshooting large-volume customer datasets across GCP and Microsoft Fabric.

Job Title: Lead Data Engineer
Work Location & Reporting Address: Duluth, GA 30097 (Onsite--client's office on Day 1)
Contract duration: 6 Months
Must Have Skills:
Strong hands-on experience in Google BigQuery, Microsoft Fabric/Azure, SQL, and PySpark
Solid understanding of:
1.BigQuery architecture (partitioning, clustering, query optimization)
2.Microsoft Fabric components (Lakehouse, OneLake, Data Factory pipelines, Notebooks)
3.Azure data services (Data Factory, Synapse, Data Lake Storage)
4.PySpark-based transformation and data processing frameworks
5.ETL/ELT and data integration concepts across GCP and Azure

Nice to Have Skills:
Experience with workflow orchestration tools such as Cloud Composer (Apache Airflow) or Azure Data Factory
Familiarity with:
1.Power BI or other reporting and BI tools
2.CI/CD and version control (Git, Azure DevOps or GitHub)
3.Data modeling, data quality checks, and performance tuning
4.Data governance and security concepts (IAM, access control)
Experience migrating or integrating data between GCP and Azure environments
Retail or enterprise client domain experience (a plus for the client)
Strong communication skills, with the ability to work with client tech leads and onsite teams

Detailed Job Description:
Data Engineer to support the development of a cloud analytics pipeline spanning Google BigQuery and Microsoft Fabric/Azure.
The engineer will be responsible for preparing and validating data in BigQuery, moving and mirroring data into Microsoft Fabric, developing Fabric-based data pipelines and Lakehouse data models, and performing detailed source-to-target reconciliation.
The ideal candidate should be strong in SQL and PySpark, understand dimensional data modeling, and be comfortable troubleshooting large-volume customer datasets across GCP and Microsoft Fabric.
The candidate will need to be able to work with Microsoft and come up with solutions

Minimum Years of Experience:
6 years

Certifications Needed:
None