Remote | AI Safety Evaluation Specialist — $55–$65/hour

We are sharing a specialised part-time consulting opportunity for experienced AI safety, trust and safety, public policy, journalism, scientific research, security, and content-evaluation professionals with strong judgment across complex and policy-sensitive subject matter.

This role supports a frontier AI initiative focused on evaluating the safety, quality, factual accuracy, and alignment of advanced models. Selected professionals will review AI-generated responses across sensitive and ambiguous scenarios, apply structured safety policies and rubrics, identify behavioural failures, and provide detailed feedback that supports safer and more reliable model performance.

Key Responsibilities

AI Safety & Quality Evaluation

• Evaluate AI-generated responses for safety, factual accuracy, policy compliance, relevance, and overall quality
• Assess whether outputs demonstrate appropriate judgment across nuanced and ambiguous scenarios
• Identify unsafe, misleading, incomplete, or poorly reasoned responses
• Compare alternative outputs and determine which response better satisfies safety and quality standards

Sensitive-Domain Content Review

• Review content involving misinformation, political persuasion, self-harm, violence, cybersecurity, biosecurity, fraud, and other sensitive areas
• Apply appropriate evaluation standards across high-risk and grey-area scenarios
• Distinguish between legitimate informational requests, potentially harmful content, and clear policy violations
• Evaluate whether model responses remain useful while handling sensitive subject matter responsibly

Rubric Application & Development

• Apply structured rubrics used in AI safety benchmarking, RLHF, and supervised fine-tuning workflows
• Assess outputs against defined criteria covering safety, accuracy, reasoning, and instruction adherence
• Identify ambiguity, gaps, or inconsistencies within evaluation guidelines
• Contribute to the refinement of scoring standards, policy interpretations, and reviewer instructions

Failure Analysis & Structured Feedback

• Identify hallucinations, unsafe outputs, reasoning failures, and policy-compliance issues
• Classify recurring model weaknesses and behavioural patterns
• Provide clear written explanations supporting each evaluation decision
• Collaborate with researchers and safety teams on calibration and ongoing evaluation initiatives

Ideal Profile

Strong candidates may have:

• At least 5 years of professional experience in AI safety, trust and safety, journalism, public policy, scientific research, security, content integrity, or a related field
• Strong analytical reasoning and the ability to assess nuanced, policy-sensitive scenarios consistently
• Excellent written English and the ability to explain complex evaluation decisions clearly
• Experience reviewing sensitive, high-risk, or ambiguous content
• Strong attention to factual accuracy, context, and policy interpretation
• Ability to work independently while applying detailed evaluation standards
• Professional residence in one of the eligible countries listed below

Educational Background

• A bachelor's degree or higher in journalism, communications, psychology, sociology, public policy, law, biology, chemistry, computer science, or a related discipline is highly relevant
• Graduate-level education in policy, behavioural science, security, law, life sciences, or artificial intelligence may be valuable
• Equivalent specialist experience in safety evaluation, content integrity, scientific review, or risk analysis may also be considered
• Relevant research, policy, moderation, or AI evaluation work may strengthen an application

Nice to Have

• Experience with AI safety, reinforcement learning from human feedback, supervised fine-tuning, trust and safety, or model evaluation
• Familiarity with content policies, safety standards, moderation frameworks, or rubric development
• Experience evaluating frontier AI models or language-model outputs
• Background in misinformation, political content, cybersecurity, biosecurity, scientific safety, or behavioural risk
• Experience participating in reviewer calibration or quality-assurance programmes
• Familiarity with structured annotation, safety benchmarking, or human-feedback workflows
• Previous collaboration with researchers, engineers, policy specialists, or safety teams

Why This Opportunity

• Help shape the safety and behaviour of advanced AI systems
• Work across challenging real-world scenarios involving complex and sensitive topics
• Apply professional judgment to improve model alignment, factual quality, and policy compliance
• Collaborate with experienced AI researchers and safety specialists
• Participate in flexible remote work with competitive hourly compensation

Contract Details

• Independent contractor role
• Fully remote with flexible scheduling
• Competitive rates between $55–$65 per hour depending on expertise and project scope
• Weekly payments via Stripe or Wise
• Eligible locations include Albania, Austria, Belgium, Bosnia and Herzegovina, Bulgaria, Croatia, the Czech Republic, Denmark, Estonia, Finland, France, Germany, Greece, Hungary, Iceland, Ireland, Italy, Kosovo, Latvia, Liechtenstein, Lithuania, Luxembourg, Malta, Moldova, Monaco, the Netherlands, North Macedonia, Norway, Poland, Portugal, Romania, San Marino, Serbia, Slovakia, Slovenia, Spain, Sweden, Switzerland, the United Kingdom, and the United States
• Projects may be extended, shortened, or adjusted depending on scope and performance
• Work will not involve access to confidential or proprietary information from any employer, client, or institution

About the Platform

This opportunity is available through 24-MAG LLC. We connect experienced professionals with remote consulting opportunities across technical, evaluation, and project-based workstreams.

By submitting this application, you acknowledge that your information may be processed by 24-MAG LLC for recruitment and opportunity matching in accordance with our Privacy Policy: https://www.24-mag.com/privacy-policy.