AI Inference Optimization Engineer

Tekfortune Inc.

San Jose, CA
4 days ago

Highlights

Required experience and skills- 7+ years in performance engineering, ML systems, GPU software, or related systems engineering, including hands-on model serving and optimization. Expected deliverables and success measures: Deliver reproducible benchmark suites, profiling evidence, quality comparisons, and recommended serving configurations.

Numbers & Facts

Location	San Jose, CA
Industry	Computer Software
Company Size	20 to 49 employees
Year Founded	2014
Website	http://www.tekfortune.com/

Description

Job Title:	AI Inference Optimization Engineer

Location:	Bay Area, CA [ONSITE]

Job Description:
The engineer will optimize LLM inference across GPU hardware, serving engines, and distributed deployments. This is a senior performance engineering role focused on achieving agreed latency, throughput, model quality, and cost targets. The engineer will use controlled benchmarks and profiling to distinguish model, runtime, memory, and communication bottlenecks.
Key responsibilities-

Establish reproducible inference baselines using representative models, input and output lengths, concurrency levels, and traffic patterns. Record hardware, software, and configuration versions.
Evaluate and tune vLLM, SGLang, TensorRT-LLM, Triton Inference Server, and NVIDIA Dynamo where appropriate to the deployment architecture.
Measure time to first token, inter-token latency, end-to-end latency, tail latency, throughput, GPU memory use, and resource efficiency under sustained and burst load.
Optimize batching, KV-cache allocation and reuse, scheduling, model loading, and parallelism strategies. Evaluate distributed and disaggregated serving where supported and beneficial.
Assess quantization approaches such as FP8 and NVFP4 on compatible hardware and software. Use representative quality evaluations and document accuracy and performance tradeoffs.
Profile CPU and GPU execution, memory transfers, kernel behavior, and communication with appropriate tools. Diagnose out-of-memory conditions and poor utilization.
Investigate multi-node serving issues with platform and network teams, including NCCL, RDMA paths, topology, and transport dependencies where used by the serving stack.
Create performance regression and soak tests, recommend production settings, and provide validated deployment artifacts, benchmark reports, and rollback guidance to delivery teams.

Required experience and skills-

7+ years in performance engineering, ML systems, GPU software, or related systems engineering, including hands-on model serving and optimization.
Strong Python and practical C++ skills, with knowledge of GPU execution, memory hierarchy, numerical precision, and parallel computing.
Hands-on experience with at least one production LLM serving stack such as vLLM, SGLang, TensorRT-LLM, or Triton.
Understanding of transformer inference, prefill and decode phases, KV-cache behavior, batching, quantization, and model parallelism.
Ability to design fair benchmarks, control variables, interpret latency distributions, and verify that performance changes preserve required output quality.
Strong Linux, container, and debugging skills, with the ability to isolate runtime, model, driver, hardware, and communication issues.
Preferred experience
Experience with NVIDIA Nsight Systems or Nsight Compute, CUDA profiling, and targeted kernel or memory-transfer optimization.
Experience with NVIDIA Dynamo, multi-GPU or multi-node inference, speculative decoding, and prefix caching where supported.
Experience validating new GPU architectures, low-precision formats, and compatibility across model, driver, runtime, and serving engine versions.

Recommended certifications Preferred:

NVIDIA-Certified Professional Generative AI LLMs (NCP-GENL). Additional infrastructure or operations credentials: NVIDIA-Certified Professional AI Infrastructure (NCP-AII) or AI Operations (NCP-AIO). Foundational: NCA-AIIO.
Certifications are preferred credentials. Candidates should demonstrate the corresponding skills through practical delivery and troubleshooting experience.

Expected deliverables and success measures:

Deliver reproducible benchmark suites, profiling evidence, quality comparisons, and recommended serving configurations. Success is demonstrated by improvements against agreed baselines, stable sustained-load behavior, and clearly documented tradeoffs without unvalidated quality regression.

About Company

Tekfortune is a software solutions company providing project development and staff augmentation services to companies in various industries including Healthcare, Banking, Finance, Telecommunication and Aviation.

Tekfortune Inc delivers solutions to a wide variety of organizations of various sizes and industries. Our clients include mid-market to Fortune 1000 companies spanning across industries such as: Insurance, Financial, Pharmaceutical, Manufacturing, Retail and many others.

In Tekfortune Inc, we are offering selection of services and solutions at very reasonable rates. With years of experience in the placement and recruitment Industry and over time we have acquired the knowledge essential to comprehend the exact need and dependence on customers. Therefore we provide the customers the best and also the brightest candidates for vacancies. Moreover, by understanding the working interest of customers who is able to place them within the companies where they are able to get maximum opportunity for professional growth.

Tekfortune Inc has a strong emphasis on quality. Its goal would be to achieve client satisfaction by looking into making deliveries satisfaction of customer's business needs. Our ability to quickly identify and effectively employ proven fundamental principles, techniques and resources yields highly desirable results for our clients.

We understand that each industry and organization share a level of uniqueness that requires a tailored approach in order to deliver specific solutions. Such factors as client culture, industry regulations, organizational maturity, financial health and many more items may impact our approach to delivering client solutions.

Having served a wide variety of clients has provided Tekfortune the ability to build subject matter expertise across numerous industries. Our industry expertise combined with our specialized knowledge of Business Process Improvement, Requirements Management and Program & Project Management provides a powerful combination and a tremendous value to our clients.

We are run by a group of industry experts who have decades of experience in delivering custom solutions to our clients.