Databricks SME Lead / Principal
Location: Dallas, TX (Onsite, 5 days/week)
Role Overview
Seeking a highly experienced Databricks Subject Matter Expert to lead data platform architecture, engineering, and transformation initiatives. The role involves designing scalable Lakehouse solutions, advising senior stakeholders, leading technical teams, and supporting consulting and pre-sales activities.
Key Responsibilities
Design enterprise-scale Databricks Lakehouse architectures using Delta Lake, Unity Catalog, and medallion architecture.
Lead large-scale ETL/ELT pipelines using Apache Spark, PySpark/Scala, Delta Live Tables, and Databricks Workflows.
Establish data governance, security, quality, lineage, and performance optimization standards.
Guide ML workflows and MLOps implementation using MLflow, Feature Store, and Model Serving.
Advise clients on data strategy, architecture, cloud adoption, and technical roadmaps.
Lead architecture reviews, technical assessments, proposals, and client workshops.
Mentor engineering teams and contribute to best practices and reusable frameworks.
Required Qualifications
10+ years of experience in data engineering, analytics, or platform architecture.
5+ years of hands-on Databricks experience with enterprise production deployments.
Expert knowledge of Apache Spark, PySpark/Scala, Delta Lake, and SQL.
Experience with cloud-based Lakehouse architectures on Azure, AWS, or GCP.
Strong expertise in Unity Catalog, data governance, and lineage.
Proven client-facing leadership and stakeholder communication skills.
Consulting or professional services experience managing multiple engagements.
Bachelor's degree in a relevant technical field or equivalent practical experience.
Preferred Qualifications
Databricks Certified Data Engineer Professional or Machine Learning Professional.
Experience with Delta Live Tables, Databricks Workflows, and Photon optimization.
Knowledge of dbt, Terraform/Pulumi, and CI/CD pipelines.
Production experience with MLflow, Feature Store, and Model Serving.
Experience in Databricks consulting or partner practices.