Amazon.com Inc logo

Senior Technical Program Manager, RL Gyms, Frontier AI RL Gym Assets

Amazon.com Inc

  • Bellevue, WA
  • 4 days ago

    Highlights

    Technical Depth: Understand gym harness architecture, evaluation frameworks, and agent benchmarking methodologies (e.g., SWE-bench style evaluations) to make informed tradeoffs and unblock teams. In this role, you will define program structure, build mechanisms, and drive accountability to deliver RL Gyms at scale - partnering with scientists, engineers, and domain experts across multiple organizations.

    Numbers & Facts

    LocationBellevue, WA
    IndustryRetail
    Company Size10,000 employees or more
    Year Founded1994
    Websitehttp://Amazon.com/militaryroles

    Description

    We are seeking an experienced Senior Technical Program Manager to design and manage the program that drives execution of our Reinforcement Learning (RL) Gym development initiative. RL Gyms are standardized evaluation environments that measure AI agent performance across enterprise domains. In this role, you will define program structure, build mechanisms, and drive accountability to deliver RL Gyms at scale - partnering with scientists, engineers, and domain experts across multiple organizations.

    This is a high-ownership role where you will shape how the program operates end-to-end - from partner engagement and requirements definition through certification and delivery - rather than building gyms yourself.

    Key job responsibilities

    Program Execution: Own end-to-end delivery of RL Gym development programs - from requirements gathering and design through implementation, certification, and launch readiness

    Cross-functional Coordination: Drive alignment across engineering, science, and domain teams to define gym specifications, collection standards, and certification criteria

    Technical Depth: Understand gym harness architecture, evaluation frameworks, and agent benchmarking methodologies (e.g., SWE-bench style evaluations) to make informed tradeoffs and unblock teams

    Stakeholder Management: Partner with domain stakeholders across the company to identify high-value workflows suitable for gym development

    Certification & Quality: Define and enforce certification standards for RL Gyms, ensuring they meet bar for agent evaluation fidelity and reproducibility

    Capacity Planning: Track gym development targets, manage pipeline throughput, and forecast delivery timelines across multiple concurrent gym workstreams

    Tooling & Process: Drive improvements to gym tracking tools, development workflows, and operational processes (e.g., DPM tracking, standup cadences)

    Risk Management: Identify blockers early, escalate appropriately, and develop mitigation plans to keep programs on track

    About the team

    The RL Gym Program develops standardized evaluation environments ("gyms") that test AI agents on realistic enterprise tasks. Our gyms span multiple business domains and are used to certify agent readiness for production deployment. We operate at the frontier of AI evaluation, working closely with model teams, infrastructure teams, and business stakeholders to define what "good" looks like for AI agents.

    About Company

    At Amazon, we don’t wait for the next big idea to present itself. We envision the shape of impossible things and then we boldly make them reality. So far, this mindset has helped us achieve some incredible things. Let’s build new systems, challenge the status quo, and design the world we want to live in. We believe the work you do here will be the best work of your life.

    Wherever you are in your career exploration, Amazon likely has an opportunity for you. Our research scientists and engineers shape the future of natural language understanding with Alexa. Fulfillment center associates around the globe send customer orders from our warehouses to doorsteps. Product managers set feature requirements, strategy, and marketing messages for brand new customer experiences. And as we grow, we’ll add jobs that haven’t been invented yet.

    It’s Always Day 1
    At Amazon, it’s always “Day 1.” Now, what does this mean and why does it matter? It means that our approach remains the same as it was on Amazon’s very first day – to make smart, fast decisions, stay nimble, invent, and stay focused on delighting our customers. In our 2016 shareholder letter, Amazon CEO Jeff Bezos shared his thoughts on how to keep up a Day 1 company mindset. “Staying in Day 1 requires you to experiment patiently, accept failures, plant seeds, protect saplings, and double down when you see customer delight,” he wrote. “A customer-obsessed culture best creates the conditions where all of that can happen.” You can read the full letter here

    Our Leadership Principles
    Our Leadership Principles help us keep a Day 1 mentality. They aren’t just a pretty inspirational wall hanging. Amazonians use them, every day, whether they’re discussing ideas for new projects, deciding on the best solution for a customer’s problem, or interviewing candidates. To read through our Leadership Principles from Customer Obsession to Bias for Action, visit https://www.amazon.jobs/principles

    Similar Jobs

    See more jobs