Back-end Engineer

A fast-growing open-source platform is building a cutting-edge feedback loop for optimizing large language model (LLM) applications turning production data into smarter, faster, and more cost-effective models.

What were building

• A unified model gateway
  
  

• Real-time feedback and metric collection
  
  

• Tools to optimize prompts, models, and inference strategies
  
  

• Built-in A/B testing and experimentation frameworks
  
  

• Seamless observability and optimization pipelines
  
  

Our platform creates a continuous learning flywheel for LLMs, combining:

• Inference: One API for all LLMs with near-zero overhead
  
  

• Observability: Feed inference data and feedback directly into your database
  
  

• Optimization: Improve from prompt engineering all the way to fine-tuning and reinforcement learning
  
  

• Experimentation: Native support for routing, fallbacks, and advanced testing setups
  
  

We began building quietly in early 2024, ran a successful technical pilot by mid-year, and launched our open-source project in the fall. Backed by world-class investors with experience supporting industry-defining AI and database platforms, we have the resources and runway to pursue an ambitious, long-term vision.

The Role

Were hiring a Founding Member of Technical Staff with deep systems engineering experience (especially in Rust).

Why not just engineer? Because we want cross-functional builders people who can bridge research and production, infrastructure and experimentation. Our core systems are written in Rust, and the scope of work includes features like inference-time optimizations and advanced experimentation frameworks.

What We Offer

• Competitive compensation (salary, equity, benefits)
  
  

• The chance to have most of your work be open-source and highly visible
  
  

• Rapid personal growth across both systems and machine learning domains
  
  

• A collaborative, technical, in-person environment where craftsmanship and ambition meet
  
  

• Backing from leading venture funds with experience in AI, infrastructure, and developer tools
  
  

What Were Looking For

• Strong technical foundation and the ability to drive large projects from start to finish
  
  

• Background in back-end systems and infrastructure (especially with Rust)
  
  

• Hunger for personal and professional growth there are no limits here
  
  

• Willingness to work on-site in NYC with a small, highly focused team