Role Overview This role is part of a multiyear enterprise initiative to modernize data platforms by migrating from legacy and onprem environments to
cloudnative, governed, and scalable architectures .
The role focuses on
migrating enterprise data workloads to Databricks on strategic cloud platforms , enabling standardized data engineering, analytics, centralized reporting, reconciliation utilities, and AI/ML use caseswhile adhering to
controls, security, resilience, and regulatory compliance .
Key Responsibilities- Lead and execute migration of legacy data platforms (onprem / nonstandard tools) to Databricks on cloud under the Olympus program
- Perform application, data, and pipeline refactoring to cloudnative Databricks patterns
- Drive migration planning including dependency analysis, sequencing, and cutover strategy
- Support coexistence models and transition from dualrun to cloudonly execution
Databricks Lakehouse Engineering- Design and implement Databricks Lakehouse architecture (Bronze / Silver / Gold)
- Build scalable batch and streaming pipelines using PySpark, Spark SQL
- Leverage Delta Lake for reliability, versioning, and performance
- Optimize compute usage and cost in line with enterprise cloud efficiency goals
Enterprise Data Controls & Governance- Embed data quality, reconciliation, and completeness controls as part of migration
- Ensure migrated workloads meet EDO governance, MCA, and audit requirements
- Maintain lineage, traceability, and explainability across migrated assets
- Support riskcritical use cases (Finance, Ops, Recon, Reporting)
Cloud Security & Resilience- Implement cloudaligned RBAC, identity controls, and secure access patterns
- Enforce data encryption, masking, and classification standards
- Ensure workloads meet operational resilience and recovery expectations
- Partner with cloud platform and security teams for certification and signoff
Reporting, Analytics & AI Enablement- Enable downstream BI, regulatory reporting, and MI workloads on Databricks
- Support centralized reporting programs (e.g., ARA, GRUrelated use cases)
- Prepare data foundations for AI / ML and Agentic workflows postmigration
Required Qualifications- 812+ years in data engineering / platform modernization
- Strong handson experience with Databricks in largescale enterprises
- Proven experience delivering cloud migration programs (onprem cloud)
- Deep expertise in Apache Spark, PySpark, Spark SQL
- Experience embedding controls, reconciliation, and data quality in migrations
- Experience in regulated environments (banking / financial services preferred)
Preferred Qualifications- Experience with Client Olympus or equivalent enterprise cloud programs
- Knowledge of legacy data platforms and modernization patterns
- Familiarity with Finance, Ops, Recon, or BalanceSheet data domains
- Exposure to MLflow, AI pipelines, or GenAI enablement on cloud
- Strong understanding of runthebank vs changethebank execution
Behavioral & Delivery Expectations- Strong ownership and execution mindset
- Comfortable operating in large, multivendor transformation programs
- Ability to engage with Technology, Operations, Risk, and Audit stakeholders
- Disciplined approach to migration risk, controls, and documentation