Platform Administration: Own the administration, security, optimization, and support of Databricks accounts, workspaces, Unity Catalog metastores, clusters, SQL warehouses, jobs, instance pools, repositories, libraries, secrets, and access controls on AWS.
Architecture & Governance: Design and maintain secure, scalable, and highly available Databricks platform architectures aligned with enterprise cloud standards. Implement robust data governance utilizing Unity Catalog (catalogs, schemas, external locations, storage credentials, privileges, lineage, and auditability).
AWS & Infrastructure Integration: Configure and troubleshoot core AWS services, including S3, IAM, VPC, PrivateLink, KMS, CloudWatch, CloudTrail, Route 53, and Secrets Manager.
FinOps & Cost Management: Drive FinOps practices by analyzing DBU and AWS consumption, enforcing tagging strategies, establishing budgets, optimizing compute configurations, and reducing idle or inefficient resources.
Automation & CI/CD: Automate workspace provisioning, configuration, policy enforcement, and operational tasks utilizing Terraform, Databricks Asset Bundles, Databricks CLI/SDK, Python, and CI/CD pipelines.
Operations & Incident Management: Lead incident, problem, change, release, vulnerability, and service-request management while meeting established SLAs and operational KPIs. Plan and execute runtime upgrades, disaster recovery tests, and backup validations.
Collaboration: Partner closely with data engineering, analytics, security, networking, and cloud engineering teams to onboard workloads and resolve complex technical issues.
Overall Experience: 9+ years of overall IT experience in cloud infrastructure, data platforms, DevOps, platform engineering, or production administration.
Databricks Expertise: 5+ years of hands-on Databricks administration experience with enterprise deployments on AWS.
Core Technical Competencies: Strong command of Databricks account console workspace administration, Unity Catalog, Delta Lake, SQL warehouses, cluster policies, and system audit logs.
AWS Competencies: Deep knowledge of AWS networking, security, storage encryption, monitoring, and identity services (VPC, S3, IAM, KMS, PrivateLink, CloudWatch, CloudTrail).
Automation & Scripting: Hands-on experience with Infrastructure as Code (Terraform), Python, shell scripting, REST APIs, and CI/CD tooling.
Foundational Knowledge: Solid working knowledge of Linux administration, networking, DNS, firewalls, certificates, proxies, and secure connectivity patterns, alongside a good understanding of Apache Spark architecture and distributed data workloads.
Education: Bachelor's degree in Computer Science, Information Technology, Engineering, or a related discipline (or equivalent practical experience).