Python Platform Engineer

This role focuses on building platform capabilities, not simply consuming existing data tools enabling consistent, secure, and scalable access to Spark-based data pipelines. We are seeking a Python-based platform engineer to design and build a containerized API layer that abstracts and governs interactions with Apache Spark through a well-defined API contract.

Job Title: Python Platform Engineer
Work Location & Reporting Address: Charlotte, NC 28202 (Onsite-Hybrid. Local Candidates only)

Must Have Skills:
Python
Pyspark
API
Microservices CI/CD
GCP
ETL

Nice to Have Skills:
ETL

Detailed Job Description:
At least 10 years of experience in IT industry.
We are seeking a Python-based platform engineer to design and build a containerized API layer that abstracts and governs interactions with Apache Spark through a well-defined API contract. This role focuses on building platform capabilities, not simply consuming existing data tools enabling consistent, secure, and scalable access to Spark-based data pipelines.
Should have good hands-on experience in GCP skills. Should be able to interact, coordinate with the client.
Should be able to manage offshore resources also.
Should be able to work with other onsite team members and client.

Tech Skills
Python,
Pyspark
Experience building control planes
Design patterns
1.Modular containerized services
2.Asynchronous processing
3.Job-based work queue execution
4.Immutability and idempotency
5.Metadata management
6.Policy enforcement
7.Semantic modeling
8.Observability and telemetry
9.Side effect capability injection
GCP

Testing
Functional testing
Behavioral testing

Languages
Python

Frameworks and Platforms
FastAPI
ARQ
Kubernetes
Pytest

Minimum Years of Experience:
8 years

Certifications Needed:
None