Build and improve monitoring, dashboards, and alerting across a distributed microservices platform, using APM, cloud-native metrics, and log analytics tools. Strong hands-on AWS experience: ECS/Fargate, load balancers, Route 53, Lambda, Step Functions,S3, IAM, KMS, Secrets Manager, CloudWatch.
Numbers & Facts
Location
Palo Alto, CA
Description
Our client, an IT Services and Consulting company, is looking for a DevOps/SRE Engineer for their Palo Alto, CA/Plano, TX location.
Responsibilities:
Observability and incident response
Build and improve monitoring, dashboards, and alerting across a distributed microservices platform, using APM, cloud-native metrics, and log analytics tools.
Define service level indicators and objectives (SLIs/SLOs) for key customer journeys, and set up error-budget tracking.
Support incident response and post-incident reviews. Write and maintain operational runbooks.
Resilience and disaster recovery
Review, test, and improve multi-region disaster recovery and failover processes, including running failover exercises.
Validate resilience patterns such as circuit breakers, retries, bulkheads, and timeouts against how dependencies really behave.
Plan and run chaos experiments and game days.
Performance and capacity
Set up a performance and load testing practice: baselines, repeatable test suites, and regression checks in the delivery pipeline.
Analyze capacity and right-size container workloads, JVM settings, and autoscaling. Find savings incloud costs.
Infrastructure, delivery, and automation
Contribute to and review infrastructure-as-code (Terraform) across multiple environments.
Improve CI/CD and deployment safety: blue/green and canary releases, post-deploy verification, and automated rollback signals.
Automate operational toil, such as event-reprocessing workflows, health reporting, and drift detection.
Operational security
Strengthen operational security hygiene: certificate lifecycle, secrets management, least-privilege access, and keeping sensitive data out of logs.
Work with application engineers, QA, and dependent teams to embed reliability into design and deliver
Requirements:
5+ years in SRE, production engineering, or DevOps roles supporting customer-facing distributed systems.
Production Terraform experience: modules, remote state, cross-workspace dependencies, and safeplan/apply workflows.
Deep observability skills with at least one APM tool and one log analytics platform (e.g., Dynatrace,Splunk). Experience defining SLOs and actionable alerting.
Able to read and troubleshoot Java/Spring Boot services, including reactive frameworks, JVM tuning in containers, and resilience libraries.
Working knowledge of Kafka in production: consumer groups, lag, and retry/dead-letter patterns.
Experience with blue/green or canary deployments and CI/CD tooling (e.g., Jenkins, Spinnaker).
Proven incident-handling experience and strong documentation skills.