Data Engineer

Location: Charlotte, NC
Experience: 10+ Years
Work Arrangement: Day 1 Onsite (Local)
Position IDs: 114068-20-1 and 114068-21-1

In-Person is mandatory

Key Responsibilities

• Optimize data pipelines that process data from Amazon Kinesis to Oracle, improving exception handling for schema and data type mismatches and implementing data quality alerts.

• Develop services to read and write data to NetApp Object Storage using S3 APIs.

• Create analytical datasets in Parquet and Apache Iceberg formats.

• Develop housekeeping scripts for log cleanup, archiving, compression, and object storage file management.

• Required Skills

• Strong expertise in Python, Java, and Data Engineering.

• Hands-on experience building and deploying data pipelines using Apache Spark, Apache Flink, and distributed data processing frameworks.

• Experience with real-time streaming architectures, REST APIs, and database connectors.

• Proficiency in relational databases, particularly Oracle.

• Experience with Amazon S3 APIs for data storage and integration.

• Preferred Skills

• Experience with Apache Iceberg.