div>
Responsibilities/Duties:
Build Scalable Data & ML Infrastructure
· Design and implement medallion architecture (Bronze/Silver/Gold) using Databricks for reliable data processing and ML model training
· Develop automated data pipelines that process structured and unstructured data from multiple sources into analytics-ready formats
· Create robust ETL/ELT workflows using Apache Spark and modern data engineering practices for both batch and streaming data
· Build and maintain data quality monitoring and validation systems across the entire data and ML lifecycle
Drive ML Platform Excellence
· Implement MLOps best practices including automated model training, validation, deployment, and monitoring using MLflow and Databricks workflows
· Design scalable ML inference systems that handle high-volume, low-latency predictions in production environments
· Create comprehensive monitoring and alerting systems for model performance, data drift, and system health
· Build self-service ML capabilities that enable data scientists to deploy and monitor their own models efficiently
.
Integrate with Enterprise Systems
· Build integrations with MAVEN Smart Systems (Palantir Foundry) environments to support operational and predictive analytics
· Connect Databricks-based systems with enterprise data warehouses, streaming platforms, and business applications
· Implement security and compliance controls that meet enterprise requirements while enabling self-service capabilities
· Collaborate with platform engineers to integrate ML systems with broader application architecture and infrastructure
Required Skills – What You’ll Bring:
· 5+ years of technical experience, including 3+ years building production data pipelines and ML infrastructure using distributed computing platforms like Databricks.
·