Detailed Job Description :
As a Consulting Data Engineer, you will architect, build, and optimize the next generation of our real-time healthcare data platforms and event-driven architectures. You will play a foundational role in moving our clinical, operational, and financial data away from legacy batch processing toward high-throughput, near-zero-latency streaming pipelines.
In this role, you will champion modern data engineering practices, design highly scalable cloud-native pipelines, and mentor a growing team of engineers. You will bridge the gap between complex electronic health records (EHR) and data consumer-enabling data scientists, software engineers, and clinical analysts to securely access, stream, and query compliant data at scale to improve patient outcomes and operational efficiency.
Core Responsibilities
- Architect Real-Time Pipelines: Design, implement, and monitor high-throughput, fault-tolerant streaming pipelines that ingest and process millions of concurrent healthcare data events (e.g., live telemetry, HL7 feeds, ADT messages).
- Scale Google Cloud Infrastructure: Build and maintain scalable, secure, and cost-effective cloud data infrastructure using Google Cloud healthcare-native tools and Infrastructure as Code (IaC).
- Enforce Healthcare Compliance & Security: Uphold strict data security protocols (HIPAA, HITRUST, SOC2) by building robust data lakehouse architectures that enforce PHI/PII masking, role-based access control (RBAC), and encryption at rest and in transit.
- Enable Self-Service Data & Interoperability: Build out streaming data ecosystems that support modern healthcare interoperability, managing real-time data catalogs and schema registries.
- Promote Code Excellence: Write modular, highly reusable, and performance-tested code; lead code reviews and establish standardized CI/CD automated deployment workflows.
- Optimize Data Reliability: Set and champion strict SLAs for clinical data delivery, establish automated data-quality testing, and implement proactive system monitoring and alerting.
- Collaborate & Mentor: Work closely with Data Architects, Clinical Product Managers, and AI/ML teams to translate healthcare requirements into technical blueprints while mentoring mid-level engineers.
Required Qualifications
Experience & Education
- 13+ years of experience in dedicated data engineering, data architecture, or distributed systems software engineering roles.
- 4+ years of experience specifically within the healthcare domain (payer, provider, digital health, or health tech).
- Google Cloud Certified Professional Data Engineer (or equivalent proven expert-level architecture experience on GCP).
True Streaming Focus
- Advanced Event-Driven Architecture: Mastery of high-volume streaming technologies including Apache Kafka and Google Cloud Pub/Sub for decoupled, real-time message routing.
- Stream Processing: Deep hands-on experience with Apache Flink or Google Cloud Dataflow (Apache Beam) for real-time streaming, complex event processing (CEP), windowing, and streaming data transformations.
Healthcare-Specific Tech Stack
- Interoperability Standards: Deep expertise working with FHIR (Fast Healthcare Interoperability Resources) JSON structures, HL7 v2/v3 messaging, and DICOM imaging standard streams.
- GCP Healthcare Tools: Hands-on experience implementing and scaling the Google Cloud Healthcare API (specifically FHIR stores and HL7v2 stores).
- EHR Integration: Experience integrating streaming pipelines with major EHR systems (such as Epic, Cerner, or Meditech).
Cloud Ecosystem & Modern Tooling
- Google Cloud Ecosystem: Technical mastery of BigQuery (including partitioning, clustering, and streaming inserts), Cloud Storage, Dataproc, and Cloud Composer (Apache Airflow).
- Programming Languages: Proficiency in Python, Scala, or Java, with a strong focus on writing optimized code for distributed computing platforms (like Spark or Beam).
- Modern Data Stack & Modeling: Mastery of modern analytics engineering frameworks (such as dbt), dimensional data modeling, and lakehouse architectures (e.g., Iceberg, Delta Lake).
- DevOps & Automation: Robust experience building automated CI/CD pipelines (GitHub Actions, GitLab CI) and managing infrastructure using Terraform.