| Job Description: | Data engineer Remote 1. Design, develop, and optimize Spark applications using Scala and Python. 2. Build scalable and high performance ETL/ELT pipelines for batch and streaming data. 3. Work with large datasets in distributed environments. 4. Develop and deploy data pipelines using AWS services such as: 5. S3, EMR, Glue, Lambda, Athena, Redshift, Kinesis 6. Implement CI/CD pipelines for data workflows using AWS-native tools or DevOps frameworks. 7. Manage infrastructure-as-code using Terraform or CloudFormation (preferred). 8. Work with structured, semi structured, and unstructured data. 9. Implement data quality checks, validation frameworks, and metadata management. 10. Optimize Spark jobs for performance, memory usage, and cost efficiency. 11. Partner with data scientists, analysts, and business teams to deliver data solutions. 12. Troubleshoot production issues and ensure high availability of data pipelines. 13. Document technical designs, workflows, and best practices. |
|---|---|
| Years of Experience: | 8.00 Years of Experience |
Since 1996, RJT has provided successful SAP, Oracle, and IT consulting solutions and staffing services to clients around the world. The new Apolis brings you the same personalized service fortified with a greater array of IT solutions, global expertise, and cost-management strategies.
We are a global IT consultancy that seamlessly integrates experts and leading-edge solutions into your organization so you can focus on what really matters.