Internship | Remote | LLM Evaluation | Reports to CTO or Safety Lead
About Elloe
Elloe is the immune system for AI.
We don’t train models — we protect their outputs. We trace every hallucination, enforce every policy boundary, and create an audit trail for every critical LLM interaction.
Our modules (TruthChecker, AutoRAG, Autopsy) are embedded in hospitals, banks, and regulatory sandboxes. Our job is to make sure these systems are safe before anything hits production.
This role will help us break, stress-test, and harden the models used by governments and enterprises alike.
About the Role
You’ll red team real-world LLM deployments, design eval harnesses, and help scale Elloe’s output-level safety layer. This isn’t just prompt tuning — it’s forensic risk mapping.
You’ll work directly with product and safety leads to uncover failure patterns and codify guardrails for GenAI systems under real-world scrutiny.
What You’ll Own
1. Red Teaming & Risk Testing
- Create prompts to trigger hallucinations, policy violations, or failure scenarios
- Stress test Elloe-protected deployments using open and proprietary models
- Document behavioral exploits across use cases (healthcare, compliance, gov)
2. Evaluation Design
- Build truthsets and scoring rubrics tied to factuality, policy, or ethical standards
- Benchmark Elloe’s modules across model types (Claude, GPT-4, Gemini, open models)
- Collaborate with product to refine and expand our eval harnesses
3. Safety Intelligence
- Identify blind spots in current detection logic
- Recommend scoring methods or red flag thresholds for deployment
- Support internal model comparison reports or customer safety audits
Who You Are
- ML/AI researcher or engineer (undergrad, grad, or early career)
- Experience working with LLMs, eval sets, and prompt design
- Strong attention to detail, grounded in safety and adversarial thinking
- Bonus: exposure to safety benchmarks like TruthfulQA, MMLU, or red teaming tools
Why This Matters
This is real-world alignment, not research theater.
You’ll be helping define how AI gets deployed responsibly — with traceability, transparency, and real-time protection.
You’ll leave this role with:
- Exposure to high-stakes LLM safety deployments
- Published frameworks or scoring methods used by enterprises
- Mentorship from technical founders operating at the bleeding edge of AI safety
Logistics & Application
- Start Date: Rolling
- Duration: 12–16 weeks
- Compensation: Research stipend
- Location: Remote-first; flexible for global candidates
- To Apply: Share a jailbreak or eval idea you’d love to run against GPT-4.