Databricks Data Engineer

Skip to content

menu

• About Us

• GSG Difference

• Our Purpose

• Our Values

• Contact Us

• Our Consultants

• Expertise

• Skillsets

• Careers

• Case Studies

Databricks Data Engineer

Job Category: Data Analytics

Job Location:Charlotte, NC; Irving, TX

Consultant, Data Engineering

About Our Firm:

Granite Solutions Groupe provides financial services decision makers with people and solutions that deliver. Our clients count on us to deliver the right talent at the right time to achieve critical business results. We leverage our industry knowledge and passion for client priorities to deliver human capital solutions.

GSG was founded in 1998 as a consulting firm delivering project management and technology solutions for the financial services industry. GSG now has a global presence, with team members deployed across the U.S. at Fortune 1000 companies and high-performing Fintech firms.

High-caliber consultants are the face of GSG at our client organizations and offices. We value the deep relationships we have with our consultants that enable us to place them in roles where their skills will have an immediate impact. The GSG consulting team is comprised of diverse, experienced, driven, and dynamic contributors who excel at getting things done.

About the Opportunity:

We are seeking a Senior Azure Databricks Data Engineer to support a large-scale migration of enterprise data pipelines and platforms from Azure Synapse and a custom data-platform environment to Databricks on Azure.

The ideal candidate will have hands-on experience delivering enterprise Databricks migrations and building scalable, resilient data pipelines in complex financial services environments. This role will be highly focused on data engineering, migration execution, data validation, reconciliation, data quality, observability, and data governance.

The successful candidate will be a proactive technical contributor who can independently drive work forward, identify potential issues, and recommend solutions with minimal oversight.

Ideal Candidate Profile

The ideal consultant is a senior, hands-on Azure Databricks Data Engineer who combines deep technical expertise with strong business and communication skills. They should be comfortable working independently, taking ownership of complex migration work, and proactively identifying and resolving issues.

Candidates should bring practical experience migrating enterprise data platforms to Databricks-not simply greenfield Databricks development-and should understand the importance of data quality, reconciliation, observability, governance, and controlled data sharing in a large financial services environment.

Responsibilities:

• Design, develop, and migrate enterprise data pipelines to Databricks on Microsoft Azure.
• Migrate existing pipelines and data workloads from Azure Synapse and comparable legacy data platforms into Databricks.
• Develop scalable and resilient data pipelines using PySpark and Python.
• Perform detailed source-to-target data validation and reconciliation during migration and parallel-run activities.
• Analyze complex enterprise data and identify discrepancies between legacy and target environments.
• Build and implement automated data-quality controls to support migration accuracy and production readiness.
• Develop data pipelines with observability and operational monitoring embedded from the beginning of the engineering lifecycle.
• Implement monitoring, alerting, and operational visibility for pipeline execution, health, failures, performance, reliability, and resiliency.
• Integrate Databricks data pipelines and workloads with Datadog and other observability tools.
• Work within multi-tenant Databricks environments, including data access, governance, and controlled data-sharing practices.
• Publish data to appropriate catalogs and support catalog-sharing requirements.
• Help reduce data duplication and data sprawl through effective data architecture, governance, and platform practices.
• Build and support API-level integrations with internal platforms and external services.
• Evaluate and implement integrations involving Databricks native AI capabilities and external AI agents.
• Collaborate with data engineers, architects, technical leads, and business stakeholders throughout migration and implementation activities.
• Support production deployment, verification, stabilization, and transition activities as required.
• Proactively identify risks, technical issues, and opportunities for improvement and recommend solutions.
• Take ownership of assigned workstreams and drive deliverables forward with minimal day-to-day direction.

Requirements:

• 5+ years of experience in data engineering, with significant experience working in enterprise environments.
• Strong hands-on experience with Azure Databricks.
• Strong programming skills in Python and PySpark.
• Demonstrated experience with enterprise data platform or pipeline migrations, preferably from Azure Synapse or similar legacy platforms to Databricks.
• Experience designing and developing scalable, reliable, and resilient data pipelines.
• Strong experience with data analysis, data validation, source-to-target reconciliation, and data quality.
• Experience with data observability, including pipeline monitoring, alerting, performance monitoring, and operational health.
• Experience integrating data platforms or pipelines with Datadog or comparable observability solutions.
• Experience working in multi-tenant Databricks environments.
• Knowledge of Databricks catalogs, catalog publishing, and data-sharing patterns.
• Understanding of data governance and strategies for reducing data duplication and data sprawl.
• Experience with API-based integrations.
• Ability to work with complex enterprise data and support migration validation and parallel-run activities.
• Strong communication, collaboration, ownership, and accountability.
• Ability to proactively identify problems, anticipate next steps, and drive solutions independently.

Preferred:

• Experience working in banking or financial services.
• Experience supporting large-scale enterprise data migrations.
• Experience with Azure data services such as Azure Data Lake Storage, Azure Data Factory, and Azure Synapse.
• Experience with Apache Spark and data pipeline orchestration tools such as Apache Airflow.
• Experience with Databricks-native AI capabilities and/or integration with external AI agents.
• Experience working with financial, customer, investment, compliance, business banking, or other regulated enterprise data.
• Experience operating in highly governed, enterprise-scale data environments.

Employee Benefits:

GSG is excited to offer its consultants a robust option of benefits which include comprehensive medical, dental, vision and prescription coverage. You will receive company-paid life insurance. In addition, we hope you will take advantage of eligible commuter benefits-like paying for work-related public transit and parking with pre-tax dollars. Save for your retirement with pre-tax contributions that go directly into your 401K. We also care about your pets with several options for pet insurance, discount card and prescriptions.

GSG's Commitment to Diversity:

As a diverse-owned business, GSG is committed to creating a diverse workforce, and we are proud to be an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, gender, gender identity or expression, sexual orientation, national origin, genetics, disability, age, or veteran status.

Your Name:

Email Address:

Phone Number:

• I consent to receive SMS and MMS messages from Granite Solutions Groupe about job opportunities and updates. Message and data rates may apply. Message frequency may vary. You can opt out at any time by replying STOP, or get help by replying HELP. Providing consent is not a condition of submitting an application.

Upload Resume File:

Upload Resume File:…

Attach a Resume file. Accepted file types are DOC, DOCX, PDF, HTML, and TXT.

• By submitting this application, you give us permission to store your personal information, and use it in the consideration of your fitness for the position, including sharing it with the hiring firm.

Review our Privacy Policy for more information.

• By submitting this application, you agree to our Terms of Service.

Review Terms of Service

People looking for jobs should not put anything here.

We are uploading your application. It may take a few moments to read your resume. Please wait!

• View all opportunities

California Headquarters

(415) 963-3999

26565 Agoura Road, #58

Calabasas, CA 91302

• GSG Difference
• Supplier Diversity Program
• Commitment to Community
• Careers
• Contact Us
• LinkedIn

2026 Granite Solutions Groupe. NGLCC Certified Diversity Supplier.

Terms of Service | Privacy Policy

This website use cookies to improve your experience. You accept the use of cookies by closing or dismissing this banner, by clicking a link or button, or by continuing to browse otherwise. Cookie settingsACCEPT

Privacy & Cookies Policy

Close

Privacy Overview

This website uses cookies to improve your experience while you navigate through the website. Out of these cookies, the cookies that are categorized as necessary are stored on your browser as they are as essential for the working of basic functionalities of the website. We also use third-party cookies that help us analyze and understand how you use this website. These cookies will be stored in your browser only with your consent. You also have the option to opt-out of these cookies. But opting out of some of these cookies may have an effect on your browsing experience.

Necessary

Necessary

Always Enabled

Necessary cookies are absolutely essential for the website to function properly. This category only includes cookies that ensures basic functionalities and security features of the website. These cookies do not store any personal information.

SAVE & ACCEPT