Core Engineer

Job Description: The role

Senior hands-on engineer to own a data and event backbone - how events flow, and how live and historical data are separated, stored, secured, and served. Accountable for delivering their scope, built correctly, securely, and on time. A hands-on expert engagement: building, not advising, requiring real streaming and data-modeling depth.

This is core software development and transaction-processing work - building the operational data backbone of a live system. It is not an analytics, data-warehousing, or business-intelligence role; the data modeling here is transactional (OLTP-style), not dimensional / reporting modeling.

Must-have summary - the hard bar

A candidate must clear all of these to be a fit:

• 6 8 years hands-on, currently building - recent, personally-built and delivered work they can speak to in depth.
• Streaming / event-processing depth - partitioning, ordering, consumer semantics, replay (not batch-ETL-only).
• Strong SQL and relational data modeling / projection design - the read layer is relational; SQL-light is not a fit.
• Object storage and lakehouse depth, including its security posture - not just basic file-store usage.
• Streaming-to-object-store integration - moving the event log to archival storage with idempotent / exactly-once delivery.
• Data-plane security - tenant / row-level isolation, object-storage security (encryption, immutability, public-access blocking), and encryption in transit.
• Test judgment for data properties - event replay, projection-versus-log correctness, idempotency, cross-tenant isolation.
• AI-assisted-development fluency and verification - directs and verifies AI-generated code and tests, catching plausible-but-wrong output.
• Strong Python; serverless-first, container fluency, local cloud emulation.
• Ownership and delivery discipline; clear communication; can hold core consistency under direction when the lead is unavailable.

Core skills - full detail

• 6 8 years hands-on, with recent work you personally built and delivered.
• Depth in streaming / event-processing platforms - partitioning, ordering, consumer semantics, replay.
• Streaming-to-object-store integration - moving the event log into archival object / lakehouse storage reliably, with idempotent / exactly-once delivery and schema handling.
• Strong relational database proficiency - the read / projection layer is relational and central to the role.
• Strong read-model / projection design and data modeling.
• Object storage and lakehouse depth - bucket / prefix design, partitioning, schema evolution, snapshot / lifecycle management, and columnar query over it.

• Data-plane security across all three stores:
  • Tenant / party data isolation - row / tenant-level isolation, enforced server-side and per request.
  • Object-storage security - public-access blocking, bucket / prefix isolation, encryption with managed keys, immutability where retention requires it, and lifecycle / tiering.
  • Encryption at rest and in transit across the relational store, the event log, and object / lakehouse storage.
• Sound data-lifecycle judgment - what to cache, project, or archive, and why; hot vs. cold separation.
• Security verification of AI-generated code - catches data-exposure and access-control flaws in generated output before they land.
• Testing the hard data-plane properties - event ordering / replay, projection-versus-log correctness, idempotency / exactly-once, and cross-tenant isolation (proving data can't leak).
• Test strategy and verification judgment - reviews generated tests for genuine coverage rather than green-but-hollow passing.
• Hands-on with automated testing frameworks - unit / integration testing, service mocking / stubbing, and test-data generation, alongside local cloud emulation.
• Ownership and delivery discipline - accountable for getting work to done under time pressure.
• Clear communicator - surfaces risk and status clearly.
• Strong Python proficiency.
• Serverless-first cloud-native build - object storage and serverless compute as primary building blocks.
• Container fluency - containerized local development and container-image packaging of compute.
• Local cloud emulation for development and testing.
• Fluent with modern AI-assisted development tooling - directs and verifies AI-generated code with rigor.
• Comfortable applying an established architectural decision framework under direction - able to hold core consistency when the lead is unavailable.

Advantageous

• Cloud data services.
• Key management / secrets handling for data stores.
• Data retention / records-lifecycle and immutability experience.
• Container orchestration - good to know, not required.
• High-volume IoT / telemetry data.
• Observability / distributed-tracing tooling.
• Domain exposure in a data-intensive, operationally complex industry.

Assessment

A deep-dive on a data / event system you personally built - including how you moved the event stream into archival storage, isolated tenants, secured object storage, and protected data at rest and in transit - plus how you'd verify a data-layer implementation is correct and judge whether its test suite proves the hard properties (replay, projection correctness, isolation).