We are looking for an experienced Unix/Linux System Administrator with strong systems automation experience to administer, operate, and automate Linux-based server and edge computing environments, including Azure IoT Edge devices, at scale across a fleet of approximately 11,000 virtual machines. This is a contract position based in Bentonville, Arizona, with a duration of 6+ months. The role encompasses day-to-day system administration, patching, automation, container operations, and production-grade observability across on-premises and cloud-based disaster recovery systems. The ideal candidate is a hands-on operator who is comfortable scripting, troubleshooting, and maintaining CI/CD pipelines in a fast-paced, always-on environment.
Responsibilities
- Administer and support Azure IoT Edge devices and modules (edgeAgent/edgeHub) in production, including provisioning, patching, updates, and day-to-day troubleshooting.
- Plan and execute quarterly Azure IoT Edge upgrade and maintenance activities - upgrading the IoT Edge runtime across the fleet within scheduled maintenance windows, with minimal disruption to operations.
- Perform L3-level Linux system administration: user/access management, kernel and OS-level troubleshooting, performance/capacity tuning, and root-cause analysis for recurring or high-severity incidents.
- Operate and maintain the observability stack (OpenObserve for logs, Prometheus/Grafana for metrics and dashboards) across edge, application, and cloud systems.
- Support disaster recovery execution between primary edge/on-premises infrastructure and cloud-based DR environments, including running failover/failback procedures and DR drills.
- Build, maintain, and troubleshoot CI/CD pipelines for application and infrastructure deployment.
- Write and maintain automation scripts/playbooks (Ansible, Puppet, Chef, shell, Python) to reduce manual operational work.
- Respond to alerts, triage incidents, and escalate per runbook/on-call procedures; keep runbooks and documentation current.
- Collaborate with cross-functional teams (security, networking, application) on day-to-day operational issues and change requests.
Qualifications
- Required
- 10+ years of hands-on experience as a Unix/Linux System Administrator, with a strong track record of operating production Linux systems, containers, and CI/CD-driven automation workflows at scale.
- Strong hands-on, L3-level administration experience on RHEL/Ubuntu: advanced troubleshooting and root-cause analysis across kernel, filesystem, storage, and networking issues, plus performance/capacity tuning, security hardening, and user/permission management in production.
- Proven ability to independently diagnose and resolve complex, escalated (L3) production issues - performing deep root-cause analysis rather than surface-level fixes, and driving permanent remediation for recurring incidents.
- Hands-on experience operating and supporting Azure IoT Edge runtime (edgeAgent/edgeHub), edge modules, and device provisioning/management in production.
- Strong knowledge and hands-on experience running and troubleshooting Docker containers in production, including on resource-constrained edge hardware.
- Working experience with CI/CD pipelines, building and troubleshooting automated build/deploy workflows.
- Mandatory hands-on experience building and maintaining log aggregation, search, and dashboards using OpenObserve for production infrastructure telemetry.
- Mandatory hands-on experience instrumenting metrics pipelines with Prometheus and building actionable dashboards and alerts in Grafana across on-premises and cloud environments.
- Working knowledge of configuration management tools (Ansible, Puppet, or Chef) for automating repeatable system administration tasks.
- Proficiency in shell scripting and at least one general-purpose language (Python, Java, C#, or Go) for automation and tooling.
- Working knowledge of the full IoT lifecycle: device provisioning, commissioning, remote device management, module/firmware updates, and decommissioning.
- Preferred