HPC Storage and Data Management Engineer

Pay Range: $50.78hr - $55.78hr

Job Overview:

The HPC Storage & Data Management Engineer is a dedicated infrastructure specialist responsible for the design, administration, and ongoing optimization of storage environments within an enterprise-scale, AWS-based HPC platform. This role exists within ADT but operates in close collaboration with the Digital Technology (DT) team in storage operations and data lifecycle management. The engineer serves approximately 1,500 global users supporting engineering simulation workloads across Gas Power and broader business units.

Key Responsibilities:

• Manage and optimize the full HPC storage topology including home directories, high-performance local scratch/compute, project storage, and archive tiers, ensuring appropriate data placement aligned with workload performance requirements.
• Administer and enforce per-user and per-project storage quotas, monitor utilization trends across the user base, and provide capacity forecasting inputs to the HPC Principal Engineer and DT team.
• Manage file permissions and access control lists (ACLs) across multi-protocol (NFS/SMB) environments, ensuring consistent access governance for both Linux HPC and Windows VDI users.
• Contribute to storage cost benchmarking and reporting, providing data-driven insight into storage spend within the AWS environment.
• Define and enforce data lifecycle policies governing the progression of data through active, scratch, cold, and archive states, while implementing and maintaining automated tiering and expiration workflows.
• Lead the definition and implementation of an engineering simulation data archival standard to ensure alignment with the broader engineering simulation data retention strategy.
• Partner with DT to define and maintain collaborative team and project folder structures, establishing and communicating standard work for shared HPC storage use.
• Support the organization's initiative to define engineering simulation data standards that ensure data is structured and accessible for AI/ML workloads.

Technical Skills:

• Demonstrated ability to develop and document standard operating procedures.
• Experience in HPC storage administration in a production environment.
• Experience with quota management systems and storage capacity planning.
• Expertise in high performance parallel/distributed file systems (e.g., Lustre, GPFS/IBM Spectrum Scale, BeeGFS, WekaIO, or similar).
• Proficiency in storage tiering, data lifecycle management, and archival tooling.
• Solid Linux systems administration skills.
• Strong experience with NFS and SMB/CIFS dual permission/ACL management.
• Experience in an AWS or cloud hybrid storage environment (S3, FSx for Lustre, EFS).
• Experience working within a DevOps or collaborative Infrastructure as Code model.
• Familiarity with engineering simulation data (CFD, FEA, or similar) and associated file size/storage patterns.
• Familiarity with Slurm job scheduler in the context of scratch and burst storage.