Lead Data Engineer

Key Skills Required

Data Architecture Platform and Technology Experience using and implementing database technologies, storage layers, modeling frameworks, and data integration tooling. Able to follow strategy and established guidelines, making recommendations for enhancements or changes, and able to participate in discussion on future strategic design.

Experience in optimization and performance tuning in all aspects of the platform and technology, database pipelines, SQL statements, resource scaling.

Application Architecture System Integration and Orchestration Understanding of possible integration options between systems and applications. Understanding of orchestration for applications, experience with Airflow would be a bonus.

Security Understanding of security models and controls. Able to implement and troubleshoot controls in platform and technology.

Data Engineering Use of azure data factory, especially with metadata driven pipelines. Strong understanding of ETL patterns, especially within a data warehouse environment. Items like merge and upsert handling, slowly changing dimensions, snapshot fact tables, etc.

Data Quality Good understanding of testing lifecycle from unit testing to system integration testing to user acceptance testing. Able to review and critique test plans and recorded results. Able to direct developers on necessary test scenarios if plans are lacking. Able to execute data quality checks for data products within the data warehouse. May include technical checks such as duplication checks, orphan record checks, and invalid surrogate key checks. May also include business rule checks.

Soft Skills Collaboration Driving sessions for requirement gathering and review for both functional and technical requirements.

Documentation Able to produce clear and concise documentation, appropriately tailored for audience, for policies, standards, and procedures.

Job Description and Responsibilities

The Lead Data Engineer role has a national salary range of $85,000 - $150,000. DHL Supply Chain offers multiple benefits, including Medical, Dental, Vision, Prescription, Discounted Stock Purchase, General Bonus Plan, and a generous PTO policy. Would you like to join the Logistics Company for the World? DHL Supply Chain is just that.

Become an essential part of everyday life by contributing to an organization that is Connecting People and Improving Lives. If you have a passion for people, a desire to problem-solve, and eagerness to pursue continuous improvement opportunities… we look forward to exploring career possibilities with you.

We are seeking a skilled Data Engineering Lead to oversee technical data projects and develop efficient data pipelines within the organization. This role is essential for optimizing data architecture to meet the needs of cross-functional teams, ensuring rapid data ingestion into global platforms while upholding data integrity and accessibility. The successful candidate will collaborate with stakeholders to promote data-driven decision-making and enhance data initiatives. As a subject matter expert, the Data Engineering Lead will mentor team members at all levels, fostering their development and ensuring adherence to best practices in data engineering.

Responsibilities:

• Lead technical data engineering projects, ensuring rapid data ingestion into regional and global platforms.
• Guide the development, testing, and maintenance of large-scale data pipelines while adhering to best practices and quality standards.
• Support software developers, database architects, data analysts, and data scientists in enhancing data accessibility and usability.
• Maintain consistency in data delivery architecture across projects by following established reference architecture.
• Recommend and implement optimized data pipeline architectures for cross-functional teams.
• Design and deploy data flow processes that meet the needs of various systems and products.
• Assemble complex data sets to meet business requirements and support strategic initiatives.
• Identify and implement internal process improvements for data reliability, efficiency, and quality.
• Mentor team members to foster development and ensure adherence to data engineering best practices.
• Collaborate with the data analytics community to integrate new data sources and develop operational procedures with operations teams.

Required Education and Experience

• Undergraduate Degree in Computer Science, Logistics, Mathematics, Statistics, or related field.
• 3 years of experience in data engineering or equivalent role.
• Strong understanding of SQL and database management systems, e.g. MySQL, PostgreSQL, SQL Server.
• Proven experience in data pipeline development and management in Azure environments.
• Proficient with Azure Data Factory, Azure Databricks, or Azure Synapse Analytics.
• Familiar with data warehousing solutions and large datasets.
• Strong understanding of SQL and database management systems, e.g. MySQL, PostgreSQL, SQL Server.
• Proficient in programming languages such as Python, Java, or Scala.
• Knowledge of data modeling, ETL processes, and data warehousing concepts in Azure.
• Experience with data visualization tools, e.g. Power BI.
• Understanding of data governance and quality principles.
• Experience leading data initiatives.
• Ability to mentor and support data engineers and guide project direction.
• Knowledge of machine learning concepts and tools for advanced analytics.
• Understanding of containerization and orchestration technologies, e.g. Docker, Kubernetes.

Our Organization

We have a business casual environment and focus on teamwork, associate development, training, and continuous improvement. We offer competitive wages, excellent affordable insurance benefits, including health, dental, vision, and life, 401K plan, paid vacation, and holidays. Our Organization is an equal opportunity employer.