Client is seeking a Sr. Data Engineer to build and optimize data pipelines, ETL processes, and cloud-based data architectures. This role focuses on Python development, cloud data integration, and supporting advanced analytics and BI initiatives. You'll collaborate with cross-functional teams to enhance data flows and drive data product innovation.
Develop Python modules using Numpy, Pandas, and dynamic programming techniques
Design, build, and optimize ELT/ETL pipelines using AWS and Snowflake
Manage data orchestration, transformation, and job automation
Troubleshoot complex SQL queries and optimize data queries
Collaborate with Data Architects, Analysts, BI Engineers, and Product Owners to define data transformation needs
Utilize cloud integration tools (Matillion, Informatica Cloud, AWS Glue, etc.)
Develop data validation processes to ensure data quality
Support continuous improvement and issue resolution of data processes
Bachelor's degree in Computer Science, Engineering, or related field
8 years of experience in data engineering, ETL, data warehouse, and data lake development
Strong SQL expertise and experience with relational databases
Experience with Snowflake, Amazon Redshift, or Google BigQuery
Proficiency in AWS services: EC2, S3, Lambda, SQS, SNS, etc.
Experience with cloud integration tools (Matillion, Dell Boomi, Informatica Cloud, Talend, AWS Glue)
GitHub version control knowledge and its integration with ETL pipelines
Familiarity with Spark, Hadoop, NoSQL, APIs, and streaming data platforms
Python (preferred), Java, or Scala scripting experience
Agile/Scrum development experience
Excellent communication and collaboration skills
Strong problem-solving abilities and attention to detail
Ability to multitask and meet tight deadlines
Highly motivated and self-directed
Comfortable working with technical and business stakeholders
Experience with data quality and metadata management tools
Exposure to BI platforms and data visualization tools
Experience in building event-driven data architectures