Senior Data Engineer - AWS

Senior Data Engineer - AWS

Charlotte, NC - Onsite

6 Months+

Required Skills & Qualifications

• Data engineering, backend development, or software engineering, with demonstrated hands-on development experience.

• Strong hands-on experience with AWS data and cloud services, including services such as S3, Glue, EMR, Lambda, and Redshift.

• Proven experience developing and maintaining production-grade APIs and backend services using Java, with a strong understanding of RESTful API design, integration patterns, and error handling.

• Strong programming skills in Python, with hands-on experience building reusable modules, backend components, or API integrations.

• Hands-on experience with PySpark and distributed data processing, including developing scalable data transformations and optimising data pipelines.

• Proficiency in SQL, including complex queries, data transformations, and query performance optimisation.

• Strong understanding of object-oriented programming, software design principles, data structures, and maintainable application architecture.

• Practical exposure to MCP, LLMs, generative AI, or AI application integration, with an understanding of how AI capabilities can be integrated into APIs and enterprise workflows.

• Experience with API testing, debugging, authentication and authorisation, and application integration.

• Familiarity with Git, CI/CD pipelines, automated testing, and Agile development practices.

• Strong problem-solving, communication, and collaboration skills, with the ability to work independently on complex technical requirements.

Preferred Qualifications (Nice to Have)

• Experience building AI-enabled applications, LLM-powered workflows, or MCP-based integrations.

• Exposure to LLM orchestration frameworks, tool calling, retrieval-augmented generation (RAG), or enterprise AI integration patterns.

• Experience with AWS-native AI services and deployment of AI-enabled applications in cloud environments.

• Familiarity with API gateways, containerisation, microservices architecture, and application observability.

• Experience with data warehousing, distributed computing, Spark performance tuning, and data quality frameworks.

Job Responsibilities

Key Responsibilities

• Design, develop, and optimise scalable data pipelines and data processing workflows using Python, PySpark, and SQL.

• Develop, enhance, and maintain production-grade APIs and backend services using Java and Python, with a strong focus on API design, integration, performance, and reliability.

• Build and integrate RESTful APIs and microservices to support data-driven applications and enterprise integrations.

• Leverage AWS services such as S3, Glue, EMR, Lambda, Redshift, and other relevant cloud-native services to build and manage scalable data engineering solutions.

• Work with Model Context Protocol (MCP), Large Language Models (LLMs), and generative AI technologies to develop or integrate AI-enabled applications and workflows.

• Implement reusable, maintainable, and scalable code using object-oriented programming principles, design patterns, and established engineering best practices.

• Write and optimise complex SQL queries and implement efficient ETL/ELT processes for large-scale data transformations.

• Collaborate with data engineers, application developers, architects, data scientists, and business stakeholders to translate requirements into robust technical solutions.

• Troubleshoot and resolve issues across APIs, data pipelines, integrations, and cloud-based applications.

• Participate in code reviews, automated testing, CI/CD implementation, performance tuning, and production support while adhering to engineering standards.