Databricks Platform Architect

Data Bricks Role Overview

We are seeking an experienced Databricks Platform Architect who can work with our client(s) in defining and delivering end to end analytics and data science solutions on the Databricks Lakehouse Platform.
Has good experience in designing SQL and Spark based Transformations on Databricks, the person needs to be experienced enough to showcase the technical maturity on various aspects of data architecture.
Has experience and worked with Iceberg for open table format workloads and cross-platform interoperability between Databricks, Snowflake and Microsoft Fabric.
Key Responsibilities

• Solution Architecture & Design
• Lead workshops with technical stakeholders to gather requirements and translate them into a comprehensive Databricks Lakehouse architecture and has experience with interoperability with snowflake, Databricks on another tenant and Microsoft Fabric.
• Leverage Photon execution engine to accelerate SQL-based transformations, queries, and high-performance ELT pipelines.
• Understanding of the DBT based ETL processing is advantage is added advantage.

Performance & Optimization

• Tune SQL and Spark jobs for optimal performance, including partitioning, Adaptive Query Execution (AQE), broadcast joins, and cluster-level optimization.
• Improve pipeline performance and cost efficiency through Photon-optimized SQL workloads, cluster sizing, autoscaling, and effective use of SQL Warehouses and Job Clusters.
• Align workloads to the appropriate Databricks compute types (SQL Warehouses, Photon, All-Purpose Clusters) to balance performance, concurrency, and cost
• Proven track record of designing and deploying large scale data platforms on Azure and Databricks
• Strong understanding of data modeling (star/snowflake schemas), ELT/ETL patterns, and data governance.

Excellent communication skills: ability to lead technical discussions and present to executive stakeholders.

Preferred Certifications
• Databricks Certified Data Engineer Associate (or higher)
• Databricks Certified Data Architect Professional (preferred)
• Certification in at least one major cloud provider (AWS/Azure/GCP)
 
“All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, disability, or status as a protected veteran”