Overview :
Job Title: Sr. Data Architect – Azure Databricks
Location: REMOTE Hybrid
Key Responsibilities
Required Experience
Required Databricks Expertise
Should have worked with the latest Databricks capabilities including:
Unity Catalog:
- Catalog and schema design
- Permissions and security
- Managed and external tables
- Lineage and governance
Delta Live Tables (DLT):
- Reliable pipeline development
- CDC implementation
- Data transformations
- Data quality controls
- Scaling and performance tuning
- Other modern Databricks capabilities and best practices
Data Ingestion & ELT Experience
Requirements
Technical Skills
- Strong expertise in data ingestion, transformation, and orchestration frameworks
- Hands-on experience building insurance-specific data models
- Proficiency in Python / PySpark and Spark SQL
- Strong understanding of streaming vs batch processing
- Experience with Spark job performance tuning
- Ability to optimize:
- Data storage formats
- Partitioning strategies
- Schema design
- Experience working with:
- Object stores
- Data lakes
- Cloud compute scaling
- Databricks cluster management
Governance, Security & Operations
- Deep understanding of:
- Data governance
- Security and compliance
- RBAC and attribute-based access control
- Encryption and audit logs
- Data privacy requirements
- Experience with operational excellence practices:
- Monitoring and observability
- Reliability and metrics
- Backup, failover, and disaster recovery strategies
Preferred Qualifications
- Prior experience working on Insurance P&C cloud-based data and analytics projects
- Strong analytical and problem-solving skills
- Experience defining and enforcing best practices for Databricks workloads and Spark-based pipelines
- Solid understanding of distributed data processing concepts
- Strong communication and stakeholder management skills
- Ability to document architecture decisions, trade-offs, and technical recommendations
Benefits
Diversity Inclusion: Exavalu also promotes flexibility, adapting to the needs of employees, customers, and the business. It might be part-time work, working outside normal 9-5 business hours or working remotely. We also offer a welcome back program to help individuals return to the mainstream after a prolonged absence due to health or family reasons. At Exavalu, we are committed to building a diverse and inclusive workforce. We welcome applications for employment from all qualified candidates, regardless of race, colour, gender, national or ethnic origin, age, disability, religion, sexual orientation, gender identity or any other status protected by applicable law. We nurture a culture that embraces all individuals and promotes diverse perspectives, where you can make an impact and grow your career