Senior Data Scientist - AI Red Teaming & Model Risk

About the Role

As AI systems-particularly LLMs and agentic AI-become core to our products and internal platforms understanding how these systems fail is just as important as improving their performance. We are looking for a Senior Data Scientist to join our AI Red Teaming efforts and focus on adversarial evaluation failure analysis and risk discovery in AI models and AI agents.

In this role you will systematically probe AI systems to uncover unsafe unintended or harmful behaviors including prompt injection jailbreaks behavioral drift tool misuse and context or memory poisoning. You will design experiments build evaluation frameworks and analyze outcomes to surface risks that traditional ML metrics do not capture.

This role is ideal for a data scientist who enjoys working at the edge of model behavior cares deeply about safety and robustness and wants to apply scientific rigor to securing real-world AI systems.

What the Candidate Will Need

Bonus Points -------------

• Design and execute AI red-teaming experiments against LLMs and AI agents to identify prompt injection direct & indirect jailbreaking and policy bypass model and tool poisoning context and memory poisoning behavioral drift and unsafe autonomy
• Develop adversarial datasets probes and test harnesses to systematically evaluate model and agent behavior under attack
• Define and track AI risk metrics beyond accuracy e.g. failure rates drift indicators unsafe action likelihood confidence miscalibration
• Analyze agent workflows and decision traces to understand how failures emerge across multi-step reasoning and tool use
• Collaborate with security engineers and AI platform teams to translate findings into guardrails mitigations and design improvements
• Build reusable evaluation pipelines to support continuous red teaming and regression testing as models and agents evolve

Basic Qualifications -------------------

• 5 years of experience as a Data Scientist Applied Scientist or ML Scientist
• Hands-on experience working with LLMs or generative AI systems
• Direct experience with AI red teaming model safety or adversarial evaluation
• Direct experience with prompt injection jailbreaks and LLM failure modes
• Strong background in experimental design evaluation and statistical analysis
• Experience analyzing complex model behavior and failure cases beyond standard metrics
• Proficiency in Python and common DSML tooling

Preferred Qualifications ----------------------

• Experience evaluating agentic systems including tool use memory or multi-step workflows
• Knowledge of GenAI architectures transformers embeddings RAG agent frameworks
• Experience building custom evaluation datasets or simulation environments
• Background or strong interest in security privacy or trust & safety
• Familiarity with AI evaluation tools e.g. custom judges LLM-as-judge simulation frameworks

Compensation ------------

For New York NY-based roles The base salary range for this role is USD171000 per year - USD190000 per year.

For San Francisco CA-based roles The base salary range for this role is USD171000 per year - USD190000 per year.

For Seattle WA-based roles The base salary range for this role is USD171000 per year - USD190000 per year.

For Sunnyvale CA-based roles The base salary range for this role is USD171000 per year - USD190000 per year.

For all US locations you will be eligible to participate in Ubers bonus program and may be offered an equity award & other types of comp. You will also be eligible for various benefits. More details can be found at the following link https://www.uber.com/careers/benefitshttps://www.uber.com/careers/benefits.

Ubers mission is to reimagine the way the world moves for the better. Here bold ideas create real-world impact challenges drive growth and speed fuels progress. What moves us moves the world - lets move it forward together.

Uber is proud to be an Equal Opportunity employer. All qualified applicants will receive consideration for employment without regard to sex gender identity sexual orientation race color religion national origin disability protected Veteran status age or any other characteristic protected by law. We also consider qualified applicants regardless of criminal histories consistent with legal requirements. If you have a disability or special need that requires accommodation please let us know by completing this form https://forms.gle/DWTk9k6xtMU25Y5A.

Offices continue to be central to collaboration and Ubers cultural identity. Unless formally approved to work fully remotely Uber expects employees to spend at least half of their work time in their assigned office. For certain roles such as those based at green-light hubs employees are expected to be in-office for 100 of their time. Please speak with your recruiter to better understand in-office expectations for this role.