Role :: GCP Senior Data Engineer
Location :: Remote
Rate :: On C2C
8 12 years data engineering, including 4+ years hands-on GCP. Reports to the Platform Architect.
Leads the build of one or more source domains and acts as technical lead for a pod of two to four engineers. Owns delivery within the architect's frame; does not own platform-wide design or the DataOps layer.
Responsibilities
- Own a source domain end to end: Pub/Sub consumption, Dataflow ingestion, bronze landing, Dataform conformance, and the resulting data mart.
- Confirm the source-side publishing contract with system owners and third-party integrators, applying the defined onboarding pattern.
- Build streaming pipelines handling ordering, idempotency, deduplication, and late-arriving events; implement and prove DLQ, archival, and replay.
- Build conformed and mart-layer Dataform models with assertions covering agreed data quality rules; conform shared dimensions rather than forking them.
- Build and operate reconciliation against the system of record and produce the evidence package for sign-off.
- Apply Dataplex registration, policy tags, and row-level security across the domain.
- Lead the pod: assign work, review code, hold the quality bar, and mentor on streaming concepts.
- Translate the target-state design into an executable build plan; escalate architectural conflicts early rather than coding around them.
- Ensure every pipeline emits structured logs and metrics so the platform's operations layer can monitor it; write runbooks and lead hypercare for the domain.
Required
- Production streaming experience - Pub/Sub and Dataflow, or Kafka / Flink / Kinesis - including deduplication, ordering, and replay.
- Strong Python, advanced SQL, and Apache Beam.
- Deep hands-on BigQuery: partitioning, clustering, incremental merge patterns, cost-aware design.
- Dimensional modeling built in a real warehouse, including conformed dimensions.
- Dataform or dbt at production scale with tests or assertions and dependency management.
- Terraform, Git workflow, and CI/CD for data pipelines.
- Experience integrating a major SaaS platform as a data source.
- Track record leading a small team or owning a workstream, with judgment on which decisions are theirs and which belong to the architect.
Preferred
GCP Professional Data Engineer certification; Dataplex and DLP; Analytics Hub or Looker; public sector delivery experience