Icon hamburger
Engineering in Newark Valley,New York
Apply to this job.
Think you're the perfect candidate?
Apply Now

You’re being taken to an external site to apply.

Enter your email below to receive job recommendations for similar positions.

Lead Data Engineer

Apex Systems Long Island Full-Time
Apply Now

Our Client is looking for a Lead Data Engineer who will help us build their next generation core data platform using the latest open-source software, frameworks, tools, and cloud computing services. The Lead Data Engineer will be responsible for the management, design, architecture, and development of our next generation core data platform. As Lead Data engineer you’ll be an integral member of our technology team and report directly to the CIO. Please send resumes to technical recruiter Tom Whelan at URL blocked - click to apply
Roles / Responsibilities

• Lead transformation of core data platform from Enterprise MSSQL stack to modern cloud-based analytics stack leveraging the latest open-source software, frameworks, tools, and cloud computing services
• Formulate best practices for the data team codebase
• Set quarterly and annual priorities for the Data Engineering team
• Coach, lead, and motivate Data Engineering team members to deliver on time and according to spec
• Hire for data engineers when necessary
• Help analyze performance throughout the stack to optimize data pipelines and workflows
• Work with Product, Marketing, Finance, Customer Service, and other teams to understand their needs and continually optimize BI stack at the data warehouse level to accommodate those needs
• Write high-quality code that is self-explanatory, tested, and meets objectives as described in functional specifications
• Collaborate with other engineering teams and participate in data team code reviews
• Drive projects to completion while providing guidance to team members
About You
• Degree in Statistics, Economics, Applied Mathematics, Computer Science, Engineering, or other Quantitative field of study. Master’s degree desired
• Over 5 years of engineering experience in a fast-paced environment, including at least 3 years experience in scalable data architecture, fault-tolerant ETL, and monitoring data quality in the cloud
• At least 1 year of experience leading data engineering teams
• Experience in setting up, maintaining, and optimizing data infrastructure (preferably on AWS)
• Willing to roll up your sleeves and hit the ground running with a commit on day one. You will be spending 70% of your time developing and 30% of the time managing
• Ability to lead by example and clearly communicate opinions on best practices
• You know how to ask the right questions, evaluate tradeoffs fairly and holistically, and apply analytic reasoning skills to get the right answer most of the time
• You are deeply curious and driven to understand our business, product, and technology, and you apply that knowledge to create better data capabilities
• You are exceptional at building a data engineering team and painting a compelling vision for the team
• Always excited to share ideas
• You communicate clearly and can own complex projects end-to-end while coordinating with other, non-engineering teams as necessary

Required skills

• Fluency in SQL (we use MSSQL, MySQL, and Redshift)
• Expert knowledge of Python, Spark, Scala
• Expert knowledge of Hadoop and Spark
• Experienced with AWS and Infrastructure as Code
• Experienced with ETL pipelining and data warehousing
• Strong knowledge of Git workflows, particularly in a collaborative environment
• Streaming, processing, and queueing with NSQ, Celery, Kafka, Kinesis (or similar)
Preferred skills and tools:
• Analytics tools such as Tableau
• Data pipelining tools such as Airflow
• NoSQL databases/datastores such as MongoDB, Redis, and DynamoDB
• Big data tools such as Hadoop and Spark
• Familiarity with Docker and/or Kubernetes a plus
• Familiarity with Microsoft Analytics stack (SSIS, SSRS, MSSQL) a plus
• Linux

Objectives in the first three months

• Learn our systems and take ownership thereof and develop a plan to migrate them to our next generation core data platform
• Define technology stack for next generation core data platform leveraging the latest open-source software, frameworks, tools, and cloud computing services
• Develop Data Engineering team hiring strategy
• Establish Data Engineering team policies
• Partner with Product, Marketing, Finance, Customer Service, and other  teams to deliver a Data Engineering roadmap that supports business objectives
• Review each data team Pull Request (Bitbucket)
• Plan and help estimate team’s work in weekly Sprint Planning meetings.

EEO Employer

Apex is an Equal Employment Opportunity/Affirmative Action Employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, age, sexual orientation, gender identity, national origin, disability, protected veteran status, or any other characteristic protected by law. Apex will consider qualified applicants with criminal histories in a manner consistent with the requirements of applicable law. If you have visited our website in search of information on employment opportunities or to apply for a position, and you require an accommodation in using our website for a search or application, please contact our Employee Services Department at Phone number blocked - click to apply

Apply to this job.
Think you're the perfect candidate?
Apply Now


For your privacy and protection, when applying to a job online, never give your social security number to a prospective employer, provide credit card or bank account information, or perform any sort of monetary transaction. Learn more.

By applying to a job using CareerBuilder you are agreeing to comply with and be subject to the CareerBuilder Terms and Conditions for use of our website. To use our website, you must agree with the Terms and Conditions and both meet and comply with their provisions.