Senior Data Engineer

Peyton Resource Group

Houston, TX

Apply

JOB DETAILS

SKILLS

Amazon Simple Storage Service (S3), Apache Cassandra, Apache Kafka, Apache Spark, Architectural Analysis, Artificial Intelligence (AI), Autoscaling, Best Practices, Capacity Management, Cataloguing, Centers for Disease Control and Prevention (CDC), Cloud Architecture, Cloud Computing, Coaching, Code Reviews, Communication Skills, Computer Science, Continuous Deployment/Delivery, Continuous Integration, Contract Requirements, Cross-Functional, Cryptography, Data Lake, Data Management, Data Science, Database Extract Transform and Load (ETL), Docker, Electronic Medical Records, Emerging Technology, Enterprise Protection, GCP (Good Clinical Practices), Hubs, Information/Data Security (InfoSec), Internet of Things, Java, Machine Learning, Microservices, Microsoft Windows Azure, MongoDB, NoSQL, On Call, Operations Processes, PostgreSQL, Privacy Controls, Proof of Concept, Python Programming/Scripting Language, REST (Representational State Transfer), Redis, Risk Management, SQL (Structured Query Language), SQL Databases, Scala Programming Language, Service Level Agreement (SLA), Software Development, Software Engineering, Software as a Service (SaaS), Team Player, Technical Leadership, Use Cases

LOCATION

Houston, TX

POSTED

30+ days ago

Position Overview
We are seeking a Senior Data Engineer to design, build, and operate cloud-scale data platforms and pipelines that support analytics, machine learning, and data products across the organization. This role collaborates with Product, Architecture, and Data Science teams to deliver secure, reliable, and cost-efficient batch and streaming data services. The ideal candidate brings deep technical expertise in cloud data engineering, data platform architecture, and best practices for governance, reliability, and observability.

Key Responsibilities

Lead the design, development, and operation of enterprise data platforms, including lakehouse, streaming, and centralized data hubs, while managing dependencies and integrations with enterprise systems.
Provide technical leadership and guidance to engineering teams, including planning, scoping, risk management, and delivery accountability.
Coach engineers on standards, observability, operational excellence, and best practices;conduct design and code reviews.
Define and evolve cloud data architectures for ingestion, storage, compute, cataloging, and access across real-time and batch use cases.
Build and maintain robust ELT/ETL and streaming pipelines using orchestration frameworks (Airflow, Dagster), event platforms (Kafka, Kinesis, Event Hub), and distributed processing frameworks (Spark, Databricks, Flink, Beam).
Implement lakehouse/lake patterns with medallion layering, CDC, and schema evolution;ensure quality gates, testing, and lineage.
Establish and monitor service-level objectives for data freshness, availability, and quality;implement observability and runbooks for operations.
Optimize storage and compute costs and performance;manage capacity, autoscaling, and governance.
Ensure privacy, security, and compliance (IAM, tokenization, row/column security, data retention);work closely with enterprise security teams.
Translate product and stakeholder requirements into data contracts and SLAs;document architecture and operational procedures.
Evaluate emerging data technologies and provide proof-of-concepts and reference implementations.
Anticipate future platform needs, including multi-cloud, IoT/edge ingestion, and AI/ML workloads.
Modernize existing data architectures using current best practices: lakehouse, CDC, medallion layering, data mesh principles, and streaming pipelines.
Develop software engineering solutions in Python/SQL and one additional language (Scala/Java);familiarity with REST/gRPC and microservices patterns.
Utilize cloud data services (Azure Data Lake/Databricks/Synapse;AWS S3/EMR/Glue/Redshift;GCP BigQuery/Dataflow) and distributed stores (Delta/Iceberg/Hudi, Kafka, Redis).
Work with SQL/NoSQL databases (Postgres, DynamoDB, MongoDB, Cassandra) and CI/CD pipelines, containerization (Docker/Kubernetes), and infrastructure as code (Terraform).
Ensure proper data governance and security across catalogs, lineage, RBAC/ABAC, encryption, and key management.
Communicate effectively with stakeholders, translating business needs into scalable, reliable data solutions.

Required Qualifications

Bachelor’s degree in Computer Science, Engineering, or related field;Master’s preferred.
5–10+ years of experience in data engineering or platform roles delivering enterprise-grade systems (SaaS or large-scale internal platforms).
2+ years leading or architecting cloud data platforms at scale.
Proven experience with batch and streaming pipelines, lakehouse implementations, and production ML data flows.
Experience operating and supporting data platforms with SLAs/SLOs, cost governance, and on-call responsibilities.
Hands-on experience with cloud-native data services, orchestration frameworks, and distributed processing.

Preferred / Additional Experience

Experience with real-time data systems and IoT ingestion.
Background in large-scale SaaS or multi-cloud platforms.
Strong collaboration skills with cross-functional teams, including product, architecture, and analytics.

About the Company

Peyton Resource Group

Established in 2001, Peyton Resource Group is a solution-based staffing company that matches businesses with top talent for short-term, long-term or permanent needs. People are a business’s most valuable asset. Peyton Resource Group is dedicated to helping companies find the best talent, matching professionals with jobs where they will thrive. With locations in Dallas/Fort Worth, San Antonio and Austin, we are available to serve your staffing needs throughout Texas and across the country.

COMPANY SIZE

100 to 499 employees

INDUSTRY

Staffing/Employment Agencies

WEBSITE

https://www.prg-usa.com/

Senior Data Engineer

Peyton Resource Group

Houston, TX

About the Company

Peyton Resource Group

Similar Job Searches