Big Data Engineer

Job Title: Big Data Engineer

Location: Basking Ridge, NJ (5-day onsite/week)

Duration : 6 Months

Job Responsibilities:

• Design and lead end-to-end data integration
• Perform performance tuning, diagnose, and resolve complex big data and streaming technical issues.
• Design and implement scalable solutions across the technology stack, including data processing, database, web services, network, and security layers.
• Perform integration testing and provide support to QA teams on large-scale projects.
• Contribute to requirements analysis, project estimation, system design, coding, and testing.
• Evaluate and implement appropriate third-party libraries and software to enhance product capabilities.
• Stay current with emerging techniques and trends in big data and AI/ML.

Job Requirement:

• 5+ years of hands-on experience designing and implementing data applications in production using Java, Python, or Scala on big data platforms, with deep expertise in big data technologies and data optimization techniques.
• 5+ years of hands-on experience with streaming technologies, specifically with a strong focus on Kafka, and various open-source software platforms (e.g., Apache ecosystem, Linux).
• Proven ability to optimize data flow and design efficient, scalable data pipelines, including data lakes and warehouses on containerized clusters.
• Strong understanding of distributed data architecture concepts (e.g., caching, MapReduce, stream processing).
• Solid knowledge of data security, governance, and compliance.
• Proficiency in programming languages: Java, Python, Scala, and SQL.
• Strong analytical capabilities, creativity, and critical thinking skills.
• Effective written and verbal communication, with the ability to facilitate team collaboration.