Senior Data Engineer

As a senior Data engineer, you will develop reusable frameworks, delivery patterns, and engineering practices that improve quality, consistency, performance, and maintainability across data products, pipelines, and medallion-layer models. As a Senior Data Engineer, you will play a critical role in Insight Global's data modernization initiative , helping lead design and hands-on implementation of our next-generation data platform in Databricks.

Overview:

The Data & Insights organization at Insight Global is responsible for delivering Trusted Data & Verified Insights to power decision‑making, AI, BI, and self‑service analytics across the enterprise.

As a Senior Data Engineer, you will play a critical role in Insight Global's data modernization initiative, helping lead design and hands-on implementation of our next-generation data platform in Databricks. You will help design and build scalable, governed, and performant data models across multiple business domains, ensuring they enable Business Intelligence, AI/ML, and self-service analytics at enterprise scale.

 

This role combines senior-level technical leadership with execution—setting the standard for how data engineering work is designed, built, tested, deployed, and supported. As a senior Data engineer, you will develop reusable frameworks, delivery patterns, and engineering practices that improve quality, consistency, performance, and maintainability across data products, pipelines, and medallion-layer models. You will mentor other engineers, drive delivery with AI, and help turn strategy and solution direction into reliable, production-ready data engineering solutions.

This role reports into the Data Engineering and works in close partnership with Data Architecture, Operations, and Data Strategy & Governance teams based out of the Atlanta office (HQ).

Responsibilities:

Key Responsibilities

• Design, build, test, deploy, and support scalable data engineering solutions primarily on Azure Databricks.
• Set the engineering standard for reliable, maintainable, performant, and well-governed data pipelines, data products, and medallion-layer models.
• Develop reusable frameworks, tooling, delivery patterns, and engineering practices that improve quality, consistency, efficiency, performance, and maintainability.
• Translate solution direction and business requirements into production-ready data engineering implementations.
• Build well-structured, reusable data models and pipelines that support Power BI reporting, AI/ML use cases, GenAI enablement, and self-service analytics.
• Modernize legacy data pipelines and processes as part of the migration to Databricks, supporting both greenfield development and legacy platform modernization.
• Apply automated testing, CI/CD, code review, and release management practices to improve delivery quality and execution.
• Optimize data structures, batch and streaming patterns, compute usage, pre-aggregation, and performance strategies for cost-effective Databricks delivery.
• Partner with Data Architecture, Data Strategy & Governance, BI, and Data Operations teams to align engineering implementations with enterprise standards, governance requirements, and supportability expectations.
• Implement governance-aware engineering practices including lineage, metadata, data ownership, policy-driven access, auditability, and standardized data contracts.
• Drive semantic consistency across domains by supporting standardized business definitions, KPI logic, metric calculations, dimensions, grain, and conformance rules.
• Leverage AI-assisted development practices and tools, including GitHub Copilot and Databricks capabilities, to improve delivery speed and engineering effectiveness.
• Provide technical guidance, mentoring, feedback, and code/design review for Data Engineers while encouraging innovation, collaboration, and best practices.

Technical Focus

• Databricks‑first architecture (Lakehouse, Medallion layers: Bronze / Silver / Gold)
• Enterprise and domain‑driven data modeling
• Data structures optimized for AI / ML and self‑service consumption
• Governance‑aware design (lineage, catalogs, metadata, ownership)
• Performance‑aware modeling and pre‑aggregation strategies, and Development
• Supporting both greenfield and legacy modernization use cases
• Architectural definitions intended to be operationalized through shared platforms and engineering automation

Qualifications:

Required Qualifications

• 5+ years of experience in Data Engineering or Data Architecture
• 2+ years of experience with developing with Databricks
• Strong expertise in data modeling concepts (conceptual, logical, physical; dimensional and domain‑oriented models).
• Hands‑on experience designing solutions on Databricks or modern cloud data platforms using the Medalion architecture.
• Deep knowledge of SQL and strong understanding of how data models impact performance and usability.
• Experience designing data structures that support BI tools (Power BI) and advanced analytics.
• Proven experience implementing enterprise data governance controls through architecture across multiple systems of record (authoritative source designations, canonical models, data contracts, and standard integration patterns) for production use.
• Strong communication skills and ability to influence across technical and business stakeholders.

Preferred Qualifications

• Experience with Databricks Unity Catalog, lineage, and governance tooling.
• Familiarity with AI / ML‑driven analytics and GenAI data requirements.
• Experience working in a global, distributed team model.
• Exposure to data mesh, domain‑oriented ownership, or product‑based data architectures.
• Background in enterprise modernization or large‑scale legacy platform migrations.