We are seeking a highly skilled and experienced Data Engineering Lead/Architect to join our dynamic team. The ideal candidate will have a proven track record of designing, building, and maintaining scalable data pipelines, with strong expertise in Python programming, cloud technologies, and large-scale data systems. If you have a passion for working with data and enabling AI/ML capabilities in products, we want to hear from you.
Key Responsibilities:
• Design, develop, and maintain robust and scalable data pipelines to support analytics and machine learning applications.
• Collaborate with cross-functional teams, including data scientists and software engineers, to implement data-driven solutions.
• Optimize and manage data storage systems and ensure high availability, reliability, and performance.
• Design, develop, and maintain robust and scalable ETL (Extract, Transform, Load) and ELT (Extract, Load, Transform) data pipelines to support analytics and machine learning applications.
• Ensure data pipelines are optimized for efficiency, reliability, and scalability, handling both structured and unstructured data seamlessly.
• Handle large-scale datasets, ensuring data integrity and consistency across platforms.
• Provide technical expertise and mentorship to junior engineers and stakeholders.
• Implement best practices in data engineering, including version control, testing, and deployment.
• Stay updated with emerging technologies and tools in data engineering, AI/ML, and cloud ecosystems.
Requirements:
• Education: Bachelor’s or Master’s degree in Computer Science, Engineering, or a related field.
• Minimum 5+ years of hands-on experience in data engineering or related roles.
• Proficiency in Python programming and its data-processing libraries (e.g., Pandas, PySpark).
• Proven expertise in handling large-scale data systems such as distributed databases, data warehouses, and data lakes.
• Strong experience with cloud platforms (AWS, Azure, or GCP) and associated tools for data storage, processing, and orchestration.
• Practical knowledge of data pipeline frameworks like Apache Airflow, Kafka, or Spark.
• Hands-on technical expertise in designing and implementing end-to-end data solutions.
• Familiarity with Generative AI (GenAI) and AI/ML technologies.
What We Offer:
• Enjoy the flexibility to work from the comfort of your home, with no commute hassles.
• Work directly with the CXO team, gaining valuable insights and contributing to strategic decisions.
• Take the opportunity to initiate, own, and drive impactful data engineering projects across the organization.
• Become a key member of the engineering leadership team, driving innovation and excellence within the data domain.
• Work with state-of-the-art technologies in AI, ML, and data engineering.
• Competitive compensation and ample opportunities for career growth.