Senior AI/QA Engineer (519)

Johnson Service Group

Boston, MA

Apply

JOB DETAILS

SKILLS

Artificial Intelligence (AI), Cross-Functional, Microsoft Windows Azure, Python Programming/Scripting Language, Quality Assurance, Quality Metrics, Safety/Work Safety, Support Documentation, Test Automation, Testing

LOCATION

Boston, MA

POSTED

30+ days ago

Our client is seeking a Senior AI Engineer with strong AI quality experience to build, evaluate, and improve enterprise AI solutions, with a focus on GenAI, RAG, and agent-based workflows in an Azure environment. (remote US only)

Must Have skills

Experience building and supporting GenAI and LLM-based solutions
Strong skills in Python
Experience with Azure AI services, including Azure AI Foundry or related tools
Experience with RAG systems and retrieval-based AI workflows
Familiarity with LangChain, LangGraph, or agent-based orchestration
Experience validating LLM outputs for quality, accuracy, and consistency
Exposure to evaluation frameworks, testing, or quality processes for AI systems

Nice to Have skills

Experience with Azure AI Search
Exposure to AI test automation or monitoring workflows
Familiarity with Responsible AI, governance, or compliance practices
Experience with knowledge graphs, GraphRAG, Pinecone, FAISS, OpenSearch, or similar tools
Experience in enterprise or regulated environments
Experience supporting human-in-the-loop or audit-focused AI processes

Responsibilities

Build and improve enterprise AI solutions across GenAI, RAG, and agent-based systems
Support development of LLM-powered applications in an Azure-based environment
Design and enhance retrieval pipelines and grounded response workflows
Contribute to evaluation and validation of AI outputs for quality, safety, and consistency
Help develop structured approaches for testing and monitoring AI solutions
Work with cross-functional teams to deliver reliable, governed, and production-ready AI systems
Support documentation, quality standards, and Responsible AI practices

#LI-SB1
#D480

About the Company

Johnson Service Group

At JSG, We Understand People

It is our mission to empower people, help our clients reach their goals, and hire motivated individuals.

Since our start in 1984, we’ve worked hard to establish our present position as the most effective cross-industry staffing and consulting solution available.

Headquartered in Chicago, IL with local offices throughout the United States and Canada, JSG proudly serves a diverse range of industries with the right solutions to help clients reach their company goals. Our client-centric approach enables us to create personalized solutions for businesses in need of top-tier talent. Our dedication doesn’t stop at our clients. We strive to connect candidates with opportunities that match their skills and goals. More than just a go-between, we are a dependable bridge, uniting top talent with suitable opportunities, facilitating shared opportunities and growth.

COMPANY SIZE

1,000 to 1,499 employees

INDUSTRY

Staffing/Employment Agencies

FOUNDED

1984

WEBSITE

https://www.jsginc.com/

Senior AI/QA Engineer (519)

Johnson Service Group

Boston, MA

About the Company

Johnson Service Group

At JSG, We Understand People

Similar Job Searches