| Location | Harrisburg, PA (Remote) |
The Data Engineer & Insights Analyst will design, develop, and maintain data infrastructure and analytical solutions that transform complex, multi-source datasets into reliable, actionable information. This role supports program delivery, operational improvements, research initiatives, and executive decision-making through the development of scalable data pipelines, cloud-based data solutions, and analytical products. The ideal candidate combines strong technical expertise in data engineering with analytical judgment and the ability to support data modernization, system integration, and data-driven initiatives across multiple teams.
Design, develop, and maintain scalable data pipeline architectures to support enterprise data and analytics initiatives.
Implement cloud-based data solutions using AWS services, including S3, Glue, Redshift, EMR, Athena, Kinesis, and Lambda.
Develop and maintain data orchestration workflows using Apache Airflow, AWS Step Functions, or comparable technologies.
Integrate, extract, transform, and load large, complex datasets from multiple sources using SQL and big data technologies.
Build and optimize analytical data environments that support statistical analysis, predictive modeling, machine learning, and research initiatives.
Collaborate with data scientists, research analysts, and policy analysts to develop reliable data assets that support reproducible research and statistical modeling.
Identify and implement process improvements through automation, infrastructure optimization, and more efficient data delivery methods.
Develop analytics tools and data products that provide actionable insights into education and workforce outcomes, operational efficiency, and key performance metrics.
Establish and maintain data pipeline monitoring, alerting, logging, data quality checks, and validation frameworks.
Develop dashboards, reports, analytical models, and other decision-support products for technical and non-technical stakeholders.
Perform data analysis and troubleshooting to identify data quality issues, investigate root causes, and improve the reliability of data processes.
Support data conversions and integrations associated with system modernization and replacement initiatives.
Partner with executive, data, design, and support teams to address technical issues and support enterprise data infrastructure needs.
Develop tools and resources that help analytics and data science teams build, maintain, and optimize data products.
Ensure data solutions comply with applicable data governance standards, privacy requirements, security controls, organizational policies, and interagency data-sharing agreements.
Continuously evaluate and improve data systems, architecture, and processes to support evolving business and analytical needs.
Required Skills & Experience
5 years of hands-on experience with cloud-based data services such as AWS (e.g., S3, EC2, Glue, EMR, RDS, Redshift, Athena, Lambda, DynamoDB).
5 years of proficiency in object-oriented/object-function scripting languages, such as Python, R, Java, C++, or Scala.
5 years of proficiency in relational SQL and NoSQL databases, including Oracle, MS SQL Server, PostgreSQL, and Cassandra.
5 years of experience with version control systems such as Git and CI/CD pipelines.
5 years of knowledge of data modeling techniques, including dimensional modeling, data vault, and star/snowflake schemas.
5 years of familiarity with data cataloging and metadata management tools, such as AWS Glue Data Catalog.
5 years of understanding of data governance, data lineage, and data quality frameworks.
5 years of knowledge of REST API design and integration patterns.
5 years of experience with big data tools, such as Hadoop, Spark, and Kafka.
5 years of experience with data pipeline and workflow management tools, such as Azkaban, Luigi, and Airflow.
5 years of experience with data integration solutions from vendors such as Informatica, MuleSoft, Talend, and TIBCO.
5 years of experience with stream-processing systems, such as Storm and Spark Streaming.