Highlights
Success is demonstrated through service objective attainment, reduced recurring incidents and manual toil, completed corrective actions, and validated recovery capability. Required experience and skills- 7+ years in SRE, production operations, or reliability-focused infrastructure engineering, with hands on responsibility for distributed services.
Numbers & Facts
| Location | San Jose, CA |
| Industry | Computer Software |
| Company Size | 20 to 49 employees |
| Year Founded | 2014 |
| Website | http://www.tekfortune.com/ |
Description
| Job Title: | AI Site Reliability Engineer |
| Location: | Bay Area, CA [ONSITE] |
Job Description:
The engineer will improve the availability, performance, and operability of AI platform services and their
supporting infrastructure. This senior role combines incident response with reliability engineering across
Linux, Kubernetes, GPU systems, storage, and networks. The engineer will turn production evidence
into automation, safer changes, and sustained reductions in operational toil.
Key responsibilities-
- Define service-level indicators and objectives for inference, platform APIs, orchestration, and relevant
- batch services. Establish actionable alerts and use error budgets to guide reliability work.
- Build observability across service, workload, GPU, host, and dependency layers using metrics, logs,
- traces, events, and health signals. Correlate symptoms across systems.
- Participate in the agreed on-call rotation and lead or support incident triage. Restore service, maintain
- incident timelines, coordinate escalations, and communicate impact clearly.
- Diagnose service degradation involving Kubernetes, Linux, scheduling, storage, networking, model
- endpoints, or job orchestration. Distinguish application faults from infrastructure failures.
- Automate routine operations and remediation with Python, Bash, or Go. Introduce preventive checks
- for recurring failures and validate automation before production use.
- Conduct capacity and performance analysis for GPU memory, utilization, queues, throughput, and
- service latency. Work with inference and platform engineers on scaling and bottlenecks.
- Review production readiness and change plans, including observability coverage, rollback, recovery,
- and dependency health. Execute resilience and recovery tests appropriate to the service.
- Lead post-incident reviews, track corrective actions, and maintain runbooks and escalation paths.
- Use operational reviews to prioritize recurring risks and reliability improvements.
- 7+ years in SRE, production operations, or reliability-focused infrastructure engineering, with hands on responsibility for distributed services.
- Strong troubleshooting across Linux, Kubernetes, networking, storage dependencies, and distributed systems under production pressure.
- Experience defining SLIs and SLOs, building alerting and response workflows, and using service evidence to prioritize engineering work.
- Strong scripting and automation skills, including tested remediation tooling, configuration management, and integration with delivery pipelines.
- Experience with incident management, root cause analysis, post-incident follow-through, and clear stakeholder communication.
- Ability to balance urgent service restoration with longer-term reliability improvements and to document actionable operational procedures.
- Experience operating AI platforms, ML infrastructure, GPU-backed inference, distributed training, or large HPC-like service environments.
- Familiarity with Prometheus, Grafana, ELK/OpenSearch, Loki, PagerDuty, DCGM, and GPU failure diagnostics.
- Experience with error budgets, safe deployment strategies, disaster recovery, and capacity planning across mixed batch and online services.
- Preferred: NVIDIA-Certified Professional AI Operations (NCP-AIO) and Certified Kubernetes
- Administrator (CKA). Observability specialization: Prometheus Certified Associate (PCA). Foundational:
- NVIDIA-Certified Associate AI Infrastructure and Operations (NCA-AIIO).
- Certifications are preferred credentials. Candidates should demonstrate the corresponding skills
- through practical delivery and troubleshooting experience.
- Deliver service dashboards, SLO definitions, actionable alerting, incident runbooks, and tested operational automation. Success is demonstrated through service objective attainment, reduced recurring incidents and manual toil, completed corrective actions, and validated recovery capability.
About Company
Tekfortune is a software solutions company providing project development and staff augmentation services to companies in various industries including Healthcare, Banking, Finance, Telecommunication and Aviation.
Tekfortune Inc delivers solutions to a wide variety of organizations of various sizes and industries. Our clients include mid-market to Fortune 1000 companies spanning across industries such as: Insurance, Financial, Pharmaceutical, Manufacturing, Retail and many others.
In Tekfortune Inc, we are offering selection of services and solutions at very reasonable rates. With years of experience in the placement and recruitment Industry and over time we have acquired the knowledge essential to comprehend the exact need and dependence on customers. Therefore we provide the customers the best and also the brightest candidates for vacancies. Moreover, by understanding the working interest of customers who is able to place them within the companies where they are able to get maximum opportunity for professional growth.
Tekfortune Inc has a strong emphasis on quality. Its goal would be to achieve client satisfaction by looking into making deliveries satisfaction of customer's business needs. Our ability to quickly identify and effectively employ proven fundamental principles, techniques and resources yields highly desirable results for our clients.
We understand that each industry and organization share a level of uniqueness that requires a tailored approach in order to deliver specific solutions. Such factors as client culture, industry regulations, organizational maturity, financial health and many more items may impact our approach to delivering client solutions.
Having served a wide variety of clients has provided Tekfortune the ability to build subject matter expertise across numerous industries. Our industry expertise combined with our specialized knowledge of Business Process Improvement, Requirements Management and Program & Project Management provides a powerful combination and a tremendous value to our clients.
We are run by a group of industry experts who have decades of experience in delivering custom solutions to our clients.
Tekfortune Inc delivers solutions to a wide variety of organizations of various sizes and industries. Our clients include mid-market to Fortune 1000 companies spanning across industries such as: Insurance, Financial, Pharmaceutical, Manufacturing, Retail and many others.
In Tekfortune Inc, we are offering selection of services and solutions at very reasonable rates. With years of experience in the placement and recruitment Industry and over time we have acquired the knowledge essential to comprehend the exact need and dependence on customers. Therefore we provide the customers the best and also the brightest candidates for vacancies. Moreover, by understanding the working interest of customers who is able to place them within the companies where they are able to get maximum opportunity for professional growth.
Tekfortune Inc has a strong emphasis on quality. Its goal would be to achieve client satisfaction by looking into making deliveries satisfaction of customer's business needs. Our ability to quickly identify and effectively employ proven fundamental principles, techniques and resources yields highly desirable results for our clients.
We understand that each industry and organization share a level of uniqueness that requires a tailored approach in order to deliver specific solutions. Such factors as client culture, industry regulations, organizational maturity, financial health and many more items may impact our approach to delivering client solutions.
Having served a wide variety of clients has provided Tekfortune the ability to build subject matter expertise across numerous industries. Our industry expertise combined with our specialized knowledge of Business Process Improvement, Requirements Management and Program & Project Management provides a powerful combination and a tremendous value to our clients.
We are run by a group of industry experts who have decades of experience in delivering custom solutions to our clients.
Similar Job Searches
Systems Engineer Jobs in San Jose, CADesign Engineer Jobs in San Jose, CAProduct Engineer Jobs in San Jose, CAApplications Engineer Jobs in San Jose, CATooling Engineer Jobs in San Jose, CAProcess Engineer Jobs in San Jose, CAReliability Engineer Jobs in Milpitas, CAReliability Engineer Jobs in Cupertino, CAReliability Engineer Jobs in Santa Clara, CAReliability Engineer Jobs in Sunnyvale, CA