Data Engineer

About the Role

This role focuses on designing, building, and maintaining scalable Azure Databricks-based data pipelines and architectures that enable analytics, AI/ML, and reporting across commercial functions. You'll lead data engineering initiatives, ensure governance and compliance, optimize performance and costs, and collaborate across teams to advance the company's data and AI strategy.

Responsibilities

• Design, build, and maintain scalable, reliable, and cost-efficient data pipelines using Azure Databricks in support of analytics, machine learning, data science, and operational use cases
• Lead data engineering initiatives that enable AI/ML model development, LLM integrations, and AI-driven applications
• Architect and implement data integration frameworks across Adtech and Martech ecosystems, incorporating Google Analytics, media campaign data, and third-party marketing APIs like Salesforce
• Manage and optimize data ingestion, transformation, and storage processes using SQL, Python, and PySpark
• Design and maintain API integrations with internal and external systems, including AI/LLM-powered APIs for advanced analytics and automation
• Administer and maintain Azure-based data tools and platforms (Databricks, ADF)
• Ensure data quality, integrity, and consistency through robust validation, monitoring, and alerting within Azure Databricks
• Implement and enforce data governance, security, and compliance standards in collaboration with IT and InfoSec
• Partner with analytics, marketing, commercial operations, and technology teams to deliver commercial data products
• Monitor and optimize data infrastructure costs, performance, and scalability across Azure cloud environments
• Participate in architecture discussions, design reviews, and CI/CD workflows as part of an Agile engineering team

Required Qualifications

• Bachelor's degree required; 5+ years in Data Engineering, Data Architecture, or Cloud Platforms
• Pharma or biotech industry experience strongly preferred, specifically within commercial data (sales, market access/payer, marketing)
• Hands-on experience with Azure Cloud (Databricks, DevOps, DataFactory)
• Strong Python, PySpark, and SQL skills
• Experience building and leveraging API integrations in ETL pipeline development
• Experience integrating AI models and APIs (OpenAI API, Databricks AI models, etc.)
• Self-driven; able to independently design end-to-end data pipelines while ensuring architectural best practices

Preferred Qualifications

• Databricks Unity Catalog or Databricks One experience
• Familiarity with Martech and Adtech tools and practices
• Knowledge of Consumer, Patient, or HCP data ecosystems
• Experience with identity providers such as LiveRamp, Acxiom, or Experian
• Experience with Customer Data Platforms (CDP)
• Experience with marketing automation tools

Working Conditions

General office environment. Travel up to 20%.