Databricks Platform Engineer

Are you passionate about enabling advanced analytics, data science, and AI? Do you thrive at the intersection of cloud engineering and transformative business initiatives? If so, the Data Platform Engineering team is seeking an experienced Databricks Platform Engineer to help drive our enterprise data modernization strategy. This unique opportunity positions you at the forefront of our mission to accelerate advanced analytics, data science, and machine learning across the enterprise by building, managing, and evolving our Databricks platform on Azure.

As a Databricks Platform Engineer, you will architect, implement, and optimize our Azure Databricks environment, supporting mission-critical analytics projects across multiple business units. You’ll directly enable high-value initiatives while enhancing platform reliability, scalability, and governance. You will collaborate with data engineering, data science, analytics, and application teams to empower business innovation, drive technical excellence, and support key transformation initiatives.

What you will do:

• Platform Architecture & Management: Design, implement, and manage scalable, secure Databricks platforms including workspace/cluster provisioning, lifecycle management, and resource optimization using Delta Lake and Photon to ensure performance, reliability, and cost efficiency.

• Infrastructure Automation & Monitoring: Automate provisioning and deployment using Terraform, GitLab CI/CD, and scripting (Python, Bash, PowerShell) while building observability dashboards to monitor platform health, performance, and costs.

• Security & Governance: Implement and enforce security policies, access controls, and compliance requirements using Unity Catalog and Databricks security features to maintain enterprise-grade data protection.

• Collaboration & Troubleshooting: Partner with data scientists, ML engineers, and product teams to deploy data pipelines and analytical services while resolving cluster configuration issues, job failures, and performance bottlenecks.

• Platform Innovation & Support: Evaluate and adopt emerging Databricks capabilities, document architecture and best practices, mentor junior engineers, and participate in on-call rotation for rapid incident response and platform stability.

• You will also participate in the Databricks platform on-call support rotation, providing expert assistance and rapid response for high-priority incidents, escalations, and critical service interruptions. This is a vital component of our commitment to platform stability and end-user success.

Position Requirements:

• Bachelor’s degree in Computer Science, Information Systems or another applicable field is preferred

• 5+ years of experience in Systems Administration

• Minimum 3 years hands-on experience designing, deploying, managing, and supporting Azure Databricks, Azure Data Lake Storage (ADLS), and related data solutions as a Platform Engineer/Administrator, with deep knowledge of Azure cloud infrastructure, resource management, security, and networking.

• Proficiency in automation and scripting tools including Terraform (IaC), Azure CLI, Databricks CLI, PowerShell, Python, REST API, and SQL for infrastructure provisioning, deployment, and platform management.

• CI/CD and version control expertise with Git, GitHub Actions, and related tools to automate data and infrastructure workflows, ensuring reliable and repeatable deployment processes.

• Preferred experience in performance tuning and optimization of Azure ADLS and Databricks clusters, cost optimization strategies, and familiarity with data ingestion tools (Qlik Replicate, Apache Kafka, Matillion ELT, Snowflake).

• Agile development experience working collaboratively in cross-functional product teams with exposure to iterative development practices and modern collaboration frameworks.

• Excellent communication skills with the ability to articulate complex system designs, architectural patterns, and technical solutions clearly to stakeholders across all levels of leadership, from technical teams to executives.

**Candidates that live within a commutable distance from our Tempe, AZ and St. Louis, MO home office locations are expected to work in the office four days per week effective June 1, 2026. Before June 1, 2026, candidates that live within a commutable distance from our Tempe, AZ and St. Louis, MO home office locations are expected to work in the office three days per week, with preference for Tuesday through Thursday.**