Dallas, Texas30+ days ago
div> Data Ingestion & Orchestration
· Experience building batch and streaming ingestion pipelines using GCP-native services
· Knowledge of Pub/Sub-based streaming architectures, event schema design, and versioning
· Strong understanding of incremental ingestion and CDC patterns, including idempotency and deduplication
· Hands-on experience with workflow orchestration tools (Cloud Composer / Airflow)
· Ability to design robust error handling, replay, and backfill mechanisms
Data Processing & Transformation
· Experience developing scalable batch and streaming pipelines using Dataflow (Apache Beam) and/or Spark (Dataproc)
· Strong proficiency in BigQuery SQL, including query optimization, partitioning, clustering, and cost control.
· About Position:
Identity & Access Management (IAM) Data Modernization – migration of an on‑premises SQL data warehouse to a target‑state Data Lake on Google Cloud (GCP), enabling metrics & reporting, advanced analytics, and GenAI use cases (natural language querying, accelerated summarization, cross‑domain trend analysis) leveraging PySpark‑based processing, cloud‑native DevOps CI/CD pipelines, and containerized deployments on OpenShift (OCP) to deliver scalable, secure, and high‑performance data solutions.
.