SeniorData Engineer – Python, PySpark & AWS
Location:Plano, TX
Relocation: Open
Open for W2 only- No c2c/1099 or third party resumes
Due to the nature of the roles client is onlyaccepting USC/GC /GC EAD only no other visas accepted.
RoleOverview
We areseeking a hands-on Senior Data Engineer with strong Python, PySpark, and AWSexperience to develop and maintain scalable data pipelines and distributed dataprocessing solutions. The ideal candidate will have experience supportingproduction data engineering workloads, cloud infrastructure, and dataorchestration.
KeyResponsibilities
- Develop, test, and maintain ETL/ELT data pipelines using Python andPySpark.
- Build and support big-data processing solutions on AWS using S3,EMR, and Lambda.
- Implement infrastructure automation using Terraform.
- Develop data integrations using REST APIs, Kafka, and MQ messaging.
- Support data storage and retrieval using Cassandra.
- Orchestrate and monitor workflows using Airflow, Control-M, and native schedulers.
- Troubleshoot pipeline failures and optimize performance, reliability, and data quality.
RequiredSkills
- Strong hands-on Python and PySpark experience.
- Experience with big-data processing and production data pipelines.
- AWS experience with S3, EMR, and Lambda.
- Working knowledge of Terraform.
- Familiarity with Kafka, MQ messaging, REST APIs, Java, andCassandra.
- Experience with Airflow, Control-M, or other workflow scheduling tools.
Preferred: Databricks experience and exposureto machine-learning workflows.