Title: AI Engineer (Agentic AI / AWS Bedrock)
Location: Los Angeles, CA - locals only
Work Arrangement: Hybrid - 2 days onsite in LA, CA
Duration: 6+ month contract-to-hire
Experience Range: 6-10 years
Our client is seeking an
AI Engineer with 6-10 years of experience to design, develop, and deploy intelligent AI agents using Claude Agent SDK and Amazon Bedrock, building production-ready agentic systems with comprehensive observability and evaluation frameworks.
Responsibilities & Qualifications- Design, develop, and deploy AI agents using Claude Agent SDK and Python; build intelligent, autonomous workflows capable of executing complex tasks and interacting with external tools, APIs, and knowledge bases.
- Develop agent orchestration workflows on Amazon Bedrock AgentCore Runtime with error handling, retry mechanisms, and guardrails; integrate agents with enterprise applications, data sources, and foundation models.
- Implement comprehensive observability and monitoring using Langfuse for LLM and agent tracing, capturing prompts, responses, tool calls, execution metrics, and production performance analytics.
- Design and implement continuous evaluation frameworks for production AI agent traffic using LLM-as-a-Judge, RAGAS, DeepEval, and custom metrics to detect regressions and ensure quality benchmarks.
- Build scalable RAG pipelines and vector database solutions (OpenSearch, Pinecone, FAISS, pgvector) for document ingestion, chunking, embedding generation, and knowledge base retrieval optimization.
- Develop AWS Step Functions workflows integrating Amazon Textract and Amazon Comprehend for intelligent document processing and data transformation at scale.
- Build and maintain CI/CD pipelines using GitHub Actions, automating agent code builds, testing, deployment, and integrated evaluation workflows for Amazon Bedrock AgentCore Runtime.
- Implement AWS security best practices including IAM policies, access controls, authentication, CloudWatch monitoring, and cost optimization for production agent systems.
Requirements- 6-10 years of professional software engineering or AI/ML experience with demonstrated expertise in Python and LLM-based systems.
- Advanced proficiency with Claude Agent SDK, agent orchestration, tool calling, and agentic AI frameworks (LangChain, LangGraph); proven experience deploying AI agents in production environments.
- Deep expertise with Amazon Bedrock, Bedrock AgentCore Runtime, and AWS services (Step Functions, Textract, Comprehend, OpenSearch, IAM, CloudWatch).
- Hands-on experience with LLM observability and evaluation tools: Langfuse, LLM-as-a-Judge frameworks, RAGAS, DeepEval, and custom evaluation metrics.
- Strong proficiency with vector databases and RAG implementation (Pinecone, OpenSearch, FAISS, pgvector); experience with embeddings, semantic search, and retrieval optimization.
- Expertise in CI/CD and DevOps: GitHub Actions, Git, Docker, Terraform, Infrastructure as Code, and automated deployment pipelines.
- Proficiency in prompt engineering, Model Context Protocol (MCP), AI governance principles, and AWS architecture best practices for secure, scalable systems.
- W2 employment only; must be available for hybrid work arrangement.