Sr. Software Engineer

BI Admin Databricks Job Description

Key Responsibilities

• Administer and support the Databricks workspace environment across development, test, and production.
• Manage user provisioning, role based access controls, service principles, groups, and entitlements.
• Configure and maintain cluster policies, instance pools, job compute, and all purpose compute resources.
• Monitor platform health, job performance, usage trends, and operational incidents.
• Implement security controls for secrets management, data access, encryption, private connectivity, and audit logging.
• Support Unity Catalog, data governance standards, and workspace level administration.
• Develop and maintain automation for workspace setup, policy enforcement, monitoring, and administrative tasks.
• Support CI/CD and infrastructure as code practices for Databricks deployments and configuration management.
• Troubleshoot platform, connectivity, identity federation, and job execution issues.
• Create and maintain operational documentation, runbooks, standards, and support procedures.
• Understanding data engineering workflows using Spark, notebooks, Delta Lake, and job scheduling.

Required Qualifications

• Experience with Azure Cloud.
• Strong understanding of identity and access management, including SSO, SCIM, RBAC, and service principals.
• Experience with Databricks cluster management, workspace administration, job orchestration, and governance controls.
• Knowledge of networking concepts such as VNet/VPC, private endpoints, firewalls, DNS, and secure connectivity.
• Experience with scripting or automation using Python, PowerShell, Bash, Terraform, or similar tools.
• Familiarity with monitoring, logging, alerting, and incident response processes.
• Strong troubleshooting, communication, and cross functional collaboration skills.

Key Skills

• Databricks Workspace Administration
• Unity Catalog and Governance
• Cluster Policies and Performance Tuning
• Cloud Platform Integration
• Security and Access Management
• Automation and Infrastructure as Code
• Cost Optimization
• Troubleshooting and Operational Support
• Documentation and Cross Functional Collaboration
• Logic Apps, APIM, App Gateways