AI Evaluation Engineer

AI Evaluation Engineer

Management

R & D

$ 43 - $ 44 / Hour

location_onTampa, Florida

acuteContractor

Apply now

Copy Linklink

Your Role

Akkodis is seeking an AI Evaluation Engineer for a Contract with a client in Tampa, FL (Hybrid). The ideal candidate will drive the assessment and continuous improvement of AI agent performance by building automated evaluation pipelines and identifying failure patterns across workflows.

Rate Range: $43/Hour to $44/Hour; The rate may be negotiable based on experience, education, geographic location, and other factors.

AI Evaluation Engineer Responsibilities include:

• Define and execute evaluation strategies, success metrics, and quality benchmarks for AI agents and GenAI-powered solutions.

• Develop comprehensive test cases, including edge, adversarial, and real-world scenarios, to validate agent performance and reliability.

• Build and maintain automated evaluation pipelines using Python, APIs, JSON datasets, and AI evaluation frameworks.

• Analyze agent outputs, prompts, retrieval results, tool usage, and execution traces to identify defects and improvement opportunities.

• Perform regression testing across model, prompt, workflow, and tool changes to ensure consistent performance and stability.

• Collaborate with AI engineering, product, and business teams to enhance agent accuracy, safety, user experience, and overall effectiveness.

Required Qualifications:

• Bachelor's degree in computer science, Information Technology, Data Science, Artificial Intelligence, or a related field.

• 2-4 years of experience in AI/ML, software testing, evaluation engineering, or a related technical domain.

• Strong understanding of Generative AI, Large Language Models (LLMs), RAG frameworks, prompt engineering, and AI agent evaluation methodologies.

• Proficiency in Python with hands-on experience working with APIs, JSON datasets, automated testing, and evaluation frameworks.

If you are interested in this role, then please click APPLY NOW. For other opportunities available at Akkodis, or any questions, feel free to contact me at Shashank.Tewari@akkodisgroup.com

Pay Details: $43.00 to $44.00 per hour

Benefit offerings available for our associates include medical, dental, vision, life insurance, short-term disability, additional voluntary benefits, EAP program, commuter benefits and a 401K plan. Our benefit offerings provide employees the flexibility to choose the type of coverage that meets their individual needs. In addition, our associates may be eligible for paid leave including Paid Sick Leave or any other paid leave required by Federal, State, or local law, as well as Holiday pay where applicable.

Equal Opportunity Employer/Veterans/Disabled

Military connected talent encouraged to apply

To read our Candidate Privacy Information Statement, which explains how we will use your information, please navigate to https://www.akkodis.com/en/us/candidate-privacy-policy

The Company will consider qualified applicants with arrest and conviction records in accordance with federal, state, and local laws and/or security clearance requirements, including, as applicable:

• The California Fair Chance Act
• Los Angeles City Fair Chance Ordinance
• Los Angeles County Fair Chance Ordinance for Employers
• San Francisco Fair Chance Ordinance

Massachusetts Candidates Only: It is unlawful in Massachusetts to require or administer a lie detector test as a condition of employment or continued employment. An employer who violates this law shall be subject to criminal penalties and civil liability.

Ref: 1647202

Apply now