Principal AI SOC Engineer

The ideal candidate is a builder and a leader: an engineer who writes production Python daily, is deeply fluent in Splunk, and has carried AI/ML and GenAI capabilities through a real software development lifecycle — versioned, tested, evaluated, monitored, and governed — inside a federal agency's accreditation and oversight regime. Build evaluation and test methodology for AI-enabled workflows: golden datasets, regression suites, precision/recall and false-positive measurement, and human-in-the-loop review gates before any capability influences analyst action or containment.

Description:

Quantum Sky is searching for a Principal AI SOC Engineer to drive efficiency, velocity, and operational effectiveness across Security Operations Centers supporting federal missions. This is a hands-on engineering role for someone who has worked inside a federal watch floor, has built and integrated SOC tooling at enterprise scale, and knows how to turn analyst pain points into production-grade automated capability.

 

The ideal candidate is a builder and a leader: an engineer who writes production Python daily, is deeply fluent in Splunk, and has carried AI/ML and GenAI capabilities through a real software development lifecycle — versioned, tested, evaluated, monitored, and governed — inside a federal agency's accreditation and oversight regime. The role blends technical execution with engineering leadership, including backlog ownership in Jira, design documentation in Confluence, delivery oversight, code review, and mentoring.

 

Responsibilities:

AI Capability Engineering

• Design and engineer AI-powered SOC capabilities that improve analyst efficiency, reduce alert fatigue, and accelerate detection and response.
• Apply AI/ML and GenAI techniques to concrete SOC problems: alert enrichment, triage and prioritization, entity and campaign correlation, investigation summarization, phishing and insider-threat triage, and automated response recommendation.
• Own the full AI SDLC for delivered capabilities — problem framing, data curation and labeling, model or prompt development, evaluation harness design, CI/CD integration, ATO-compatible deployment, monitoring for drift and regression, and rollback.
• Build evaluation and test methodology for AI-enabled workflows: golden datasets, regression suites, precision/recall and false-positive measurement, and human-in-the-loop review gates before any capability influences analyst action or containment.
• Implement guardrails, output validation, prompt and response logging, and decision traceability so AI-assisted findings are auditable and defensible to agency leadership, oversight bodies, and assessors.
• Engineer AI capabilities consistent with federal AI governance expectations — current OMB AI guidance, agency Chief AI Officer requirements, AI use case inventory reporting, and NIST AI RMF — including the additional practices that apply when a capability is designated high-impact.
• Ensure AI capabilities handle sensitive government data appropriately, including PII and law enforcement sensitive material, with data minimization, retention controls, and model-training exclusions.

SOC Engineering and Automation

• Build and evolve detection and response pipelines across SIEM, SOAR, EDR, email security, identity, and cloud security platforms, with Splunk as the primary analytic platform.
• Engineer Splunk content and infrastructure: advanced SPL, data models and CIM normalization, correlation searches and notable event tuning in Enterprise Security, ingest and index architecture, and performance tuning for high-volume telemetry.
• Build and maintain SOC automation in Python — SOAR playbooks, custom microservices, API-driven integrations, and AI-driven decisioning across security and infrastructure controls.
• Engineer onboarding and normalization pipelines for telemetry from organizationally distinct components with heterogeneous tooling, ownership models, and data-sharing constraints.
• Support enterprise event logging maturity requirements, including log source coverage, retention tiering, and log integrity.
• Build and maintain integrations supporting federal reporting and directive compliance, including CISA sensor and CDM data flows, Binding Operational and Emergency Directive response, and incident notification timelines.
• Improve SOC velocity and throughput by automating repetitive analyst tasks and standardizing response patterns into reusable, tested components.
• Support design of scalable SOC architectures for high-volume telemetry and real-time workflows in a 24/7 operations environment.
• Maintain a tool-agnostic engineering mindset; integrate Elastic, Microsoft Sentinel and Defender, or cloud-native services where the mission calls for it.

Delivery and Technical Leadership

• Own and groom the engineering backlog in Jira; prioritize, decompose, estimate, and ship production-ready increments on a predictable cadence against contract deliverables.
• Translate analyst user stories and operational requirements into concrete technical designs; document architecture decisions, runbooks, SOPs, and capability documentation in Confluence.
• Set engineering standards and patterns for AI-enabled SOC capabilities — code review expectations, testing requirements, repository structure, CI/CD pipelines, and secure development practices aligned to NIST SSDF.
• Partner with government stakeholders, analysts, and fellow engineers to deliver solutions, setting a high technical bar through hands-on contribution and shared ownership.
• Mentor engineers on Python, Splunk, and AI engineering practice.

Qualifications:

Required:

• 8–12 years of hands-on enterprise IT and cybersecurity engineering experience spanning security operations, cloud platforms, automation, and AI/ML.
• 3–5 years of direct SOC engineering experience designing, building, and optimizing SOC tooling, with at least 2 years supporting federal SOC environments (civilian agencies, DoD, Intelligence Community, or federal law enforcement).
• Expert-level Python: production services, automation frameworks, API integrations, unit and integration testing, packaging, and code review of others' work.
• Deep hands-on Splunk expertise: advanced SPL, data model and CIM work, Splunk Enterprise Security content development, ingest pipeline and architecture design, and Splunk SOAR playbook development.
• 3+ years applying AI/ML techniques to cybersecurity or operational systems, including AI-enabled workflows for alert enrichment, triage, detection engineering, or automated response.
• 2+ years working with LLMs or GenAI systems in production or near-production environments, including RAG pipelines and LLM-integrated automation for SOC use cases.
• Demonstrated AI SDLC ownership: taking an AI/ML or GenAI capability from requirement through evaluation, deployment, and sustained operation, with versioning, reproducibility, testing, and monitoring in place.
• Strong experience with SOC automation, orchestration, and playbook design, including API-driven integrations and detection, correlation, and response pipeline engineering.
• Proficiency with Git-based workflows, CI/CD pipelines, and containerized deployment.
• Working fluency with Jira and Confluence as engineering delivery and documentation tools — backlog ownership, sprint execution, and written design and decision records.
• Working knowledge of the federal cybersecurity compliance stack: FISMA, NIST 800-53, RMF and the ATO process, FedRAMP for cloud services, and CISA directive-driven operations.
• Demonstrated ability to set technical strategy, review designs, and foster best practices across a team of engineers.

Desired:

• Prior engineering support to a large, federated federal civilian agency SOC or a federal law enforcement environment.
• Familiarity with current OMB AI governance requirements, NIST AI RMF, and federal AI use case inventory and impact-assessment processes.
• Experience with TIC 3.0, CDM, and CISA incident reporting workflows.
• Experience deploying AI/ML workloads in FedRAMP-authorized environments, and experience self-hosting open-weight models where data sensitivity precludes commercial API use.
• Familiarity with CJIS Security Policy and handling of law enforcement sensitive data.
• Infrastructure-as-code and configuration management experience (Terraform, Ansible, Kubernetes).
• Experience with Azure and Microsoft Sentinel, including Azure OpenAI or Azure ML for GenAI use cases.
• Certifications: Splunk Enterprise Certified Architect, Splunk SOAR Certified Automation Developer; CISSP; GCIA, GCDA, or GCFA; Microsoft Certified: Azure Security Engineer Associate.

Clearance

• Must have a current TS/SCI

Location

• Hybrid remote at HQ in Reston, VA
• ~2 day work from home flexibility.

About Quantum Sky:

Compensation:

• Compensation is unique to each candidate and relative to the skills and experience they bring to the position. The salary range for this position is typically between $200,000-$225,000. This does not guarantee a specific salary as compensation is based upon multiple factors such as education, experience, certifications, and other requirements, and may fall outside of the above-stated range.

Benefits: