Technical - SQL Automation Engineer

The SQL Always On Patch Automation Engineer is a contract role on the mission-critical Platform Operations team at HCHB and will build the automation that patches the production SQL Server Always On Availability Group estate today the largest remaining block of enterprise patching still performed by hand under coordinated downtime. The work runs in parallel with the wider replatform of Windows patch automation from legacy SCORCH runbooks onto Ansible Automation Platform (AAP), and is delivered against the same change, approval, and evidence controls as the monthly cycle.

Description: SQL Always On Patch Automation Engineer (Contract)
Position Summary
The SQL Always On Patch Automation Engineer is a contract role on the mission-critical Platform Operations team at HCHB and will build the automation that patches the production SQL Server Always On Availability Group estate today the largest remaining block of enterprise patching still performed by hand under coordinated downtime.
This role sits inside HCHB s Server and Endpoint Vulnerability Management program, which is establishing remediation timelines driven by vulnerability severity and asset criticality. Automated patching of Always On Availability Group infrastructure is a named deliverable of that program. The work runs in parallel with the wider replatform of Windows patch automation from legacy SCORCH runbooks onto Ansible Automation Platform (AAP), and is delivered against the same change, approval, and evidence controls as the monthly cycle.
The ideal candidate is equally comfortable in SQL Server high availability and in infrastructure automation: someone who understands what an availability group will and will not tolerate during a rolling patch, and who can express that safely and repeatably as code.
Essential Duties and Responsibilities
"Design, build, and maintain automation that patches production SQL Server Always On Availability Group clusters without unplanned loss of service.
"Automate availability group discovery so each patch wave is driven from the live state of the estate replicas, roles, synchronization health, and routing rather than a static host list.
"Automate the pre-patch sequence, including removal of secondary replicas from read-only routing, placement of nodes into monitoring maintenance mode, and controlled stop of services on DR availability group instances.
"Orchestrate wave ordering across secondary replicas, DR replicas, and primaries, including failover and failback, so that each customer s availability group remains online throughout the cycle.
"Automate suppression and restoration of SQL Sentry (SentryOne) monitoring alerts, including instances not represented in the configuration database.
"Integrate patch execution with existing JAMS scheduled jobs and the established maintenance-mode job chain.
"Deliver the automation in Ansible Automation Platform as reusable, parameterized job templates, playbooks, and roles, replacing hand-run scripts and legacy SCORCH runbooks.
"Integrate with ServiceNow for change record creation, approval gating, execution status updates, and automated closure.
"Build verification and rollback into every stage, so that a failed node halts the wave rather than propagating, and so that any step can be safely re-run.
"Produce post-patch evidence replica health, synchronization state, and patch level in a form that supports vulnerability SLA and audit reporting.
"Support out-of-cycle and emergency patching of the database tier where a high-severity or CISA Known Exploited Vulnerability finding carries a remediation deadline shorter than the next scheduled cycle.
"Partner with the Database Administration, Security, Infrastructure Operations, and Application teams on maintenance window design and cutover planning.
"Support remediation of database-tier findings raised through Tenable and related vulnerability platforms.
"Follow HCHB change management practice, raising a change record before any production change and validating in non-production ahead of production.
"Maintain source-controlled automation in Azure DevOps, participate in code review, and leave behind runbooks, SOPs, and documentation the permanent team can support.
Required Skills
"Deep hands-on experience with SQL Server Always On Availability Groups, including replica roles, synchronization modes, read-only routing, listeners, failover and failback, and quorum behavior.
"Strong Windows Server Failover Clustering (WSFC) administration and troubleshooting.
"Proven experience patching or maintaining highly available database infrastructure in production, with a working understanding of what breaks an availability group during a rolling change.
"Advanced PowerShell scripting, including the SqlServer module, dbatools, or equivalent.
"T-SQL sufficient to interrogate availability group state through dynamic management views.
"Strong Ansible experience, ideally Ansible Automation Platform (AAP) playbooks, roles, inventories, credentials, and job templates.
"Experience with enterprise Windows patch management, including MECM/SCCM, deployment rings, and maintenance windows.
"Experience integrating ServiceNow APIs and change workflows into automation.
"Experience with enterprise job scheduling platforms such as JAMS, Control-M, or similar.
"Experience integrating with monitoring and maintenance-mode tooling such as SQL Sentry/SentryOne, SCOM, or equivalent.
"Working knowledge of Git and Azure DevOps Repos and Pipelines, with a habit of source-controlled, peer-reviewed automation.
"Ability to design idempotent, restartable automation with explicit verification, halt conditions, and rollback.
"Understanding of the vulnerability remediation lifecycle and severity-based remediation SLAs.
"Strong analytical and troubleshooting skills, and sound judgment about operational risk.
"Strong written and verbal communication skills, and the ability to work with database administrators and application owners on shared maintenance windows.
Experience
"6+ years of enterprise Windows Server infrastructure engineering experience.
"4+ years of hands-on SQL Server Always On Availability Group operations in production high-availability environments.
"3+ years building infrastructure automation with Ansible and/or PowerShell.
"Demonstrated experience automating patching or maintenance of clustered or highly available database infrastructure.
"Experience delivering automation into an existing enterprise operations practice, working to that team s change and documentation standards.
"Experience working in regulated environments (HIPAA, PCI, SOX) preferred.
Education / Certification / Training
"Bachelor s degree in Computer Science, Engineering, or related field (equivalent experience considered).
"Preferred certifications include:
oClient certifications (Azure Database Administrator Associate, Windows Server)
oRed Hat Certified Specialist in Ansible Automation
oServiceNow certifications
oITIL Foundations