| Location | Tysons, VA |
Must Have Qualifications: Must have 5+ years of experience and a strong AI development
background. Must have hands on experience building agentic workflows, automating testing using
AI, and working with tools such as GitHub, Copilot, and Claude.
Schedule: Standard
Shortlisting Deadline: July 23rd
Interview Information:
Rounds: 2 rounds
Duration: 30-60 mins
Interview Type: 1
1st round virtual | 2nd round onsite
Interview Placeholders: Targeting Jul 28th Aug 4th
Interview Debrief: TBD Will be scheduled with suppliers once all interview rounds are completed.
JOB DESCRIPTION
1) Agentic test automation foundation (reusable patterns + reference implementations)
Design and implement agentic testing patterns that can be adopted by multiple
Underwriting teams (and later other domains).
Create reference implementations (sample repos / templates) demonstrating:
o Test generation assistance (from requirements, APIs, contracts, schemas)
o Test maintenance assistance (auto-updating selectors/contracts, flaky test triage)
o Failure analysis assistance (root cause suggestions, log correlation, defect drafting)
Establish a standard architecture for test code organization, tagging, data management,
and execution across UI + API + service layers.
2) Coverage standards, templates, and governance
Define and publish coverage standards (what "good" looks like) including:
o Minimum coverage expectations by service/component
o Test type mix (unit vs API vs UI vs contract vs integration)
o Risk-based prioritization and traceability to requirements
Provide templates usable across teams:
o Test plan templates
o Test case/spec templates (Gherkin-style or equivalent)
o Definition of Ready / Definition of Done quality checklists
Create a scalable tagging/metadata strategy (e.g., feature, service, risk, priority, data
sensitivity) to support reporting and quality gates.
3) GenAI-assisted reporting and quality insights across microservices
Build automated reporting that aggregates test + service data across multiple
microservices, such as:
o Test execution results (Karate/Playwright + CI runs)
o Service health signals (logs/metrics/traces if available)
o Defect signals (issue tracker metadata if available)
Generate GenAI-driven summaries:
o Release readiness narratives
o Failure clustering and trend analysis
o "What changed?" insights (commit/PR correlation)
Produce outputs consumable by engineering leadership and teams (dashboards,
markdown summaries in PRs, artifacts in CI).
4) "Quality gates" via agents
Build automated review agents that evaluate user stories/requirements for minimum
required clarity and data before development/testing starts:
o Required fields present (acceptance criteria, testable outcomes, data needs,
dependencies)
o Ambiguity detection and missing edge cases
o Data/privacy considerations and environment needs
Integrate gates into workflow (PR checks, issue templates, GitHub Actions) to reduce churn
and rework.
Required Technical Skills (must-have)
GenAI / LLM + agentic development
Hands-on experience building LLM-powered agents (tool-using, multi-step reasoning,
guardrails).
Experience with prompting patterns, structured outputs (JSON schemas), evaluation, and
reducing hallucinations.
Ability to design agent workflows for:
o Test generation/augmentation
o Requirements review and completeness validation
o Report generation and summarization
GitHub platform + GHCP (Copilot) for engineering workflows
Strong proficiency with GitHub Copilot in day-to-day development.
Deep experience with GitHub platform capabilities:
o GitHub Actions (CI/CD pipelines, reusable workflows, composite actions)
o PR checks, branch protections, CODEOWNERS, templates
o Automation via GitHub APIs/webhooks (as needed)
Test automation engineering (framework expertise)
Advanced experience designing and implementing automation with:
o Karate (API testing, contract-like checks, data-driven testing, mocks)
o Playwright (UI automation, selectors strategy, parallelization, trace/video
artifacts)
Strong understanding of test design and coverage:
o Happy path scenarios
o Negative/validation scenarios
o Edge/boundary scenarios
o Data setup/teardown strategies and test isolation
Cross-service reporting and data aggregation
Proven ability to aggregate and normalize results from multiple microservices and multiple
pipelines.
Experience producing actionable automated reports (trend analysis, failure clustering,
service correlation).
Automated requirements review agents
Experience implementing automated checks that validate:
o Acceptance criteria completeness
o Required test data and environment dependencies
o Non-functional requirements (performance, security, observability) when
applicable
Deliverables / What success looks like (for the posting)
A reusable agentic testing automation kit adopted by multiple teams.
Published coverage standards + templates and onboarding documentation.
A working GenAI-assisted reporting pipeline aggregating results across microservices.
Automated quality gates integrated into GitHub workflows that measurably reduce story
churn.
Required Skills :
Basic Qualification :
Additional Skills :
Background Check : No
Drug Screen : No