Big Data Engineer - W2 Only

This role involves designing, developing, and maintaining robust data pipelines and applications, supporting real-time business functions, and contributing to architectural decisions in a fast-paced, data-driven environment. We are seeking a highly skilled Big Data Engineer to join a dynamic team at a leading financial services firm.

Job Title: Big Data Engineer
Location: Pittsburgh, PA or Cleveland, OH (Hybrid – 3 days onsite per week)
Employment Type: W2 Only
Overview:
We are seeking a highly skilled Big Data Engineer to join a dynamic team at a leading financial services firm. This role involves designing, developing, and maintaining robust data pipelines and applications, supporting real-time business functions, and contributing to architectural decisions in a fast-paced, data-driven environment.Key Responsibilities:

• Design and implement new data applications and maintain existing systems.
• Support real-time business operations and ensure high availability of data services.
• Collaborate with cross-functional teams to define and implement scalable data solutions.
• Optimize data workflows and ensure data quality, integrity, and security.
• Participate in architectural planning and provide input on data modeling and system design.

Technical Skills (Required – 4+ years):

• Apache Spark 3
• Python for data processing and automation
• Shell Scripting for system-level tasks
• Hadoop (Cloudera) – including Hadoop administration, HDFS file processing, and Hadoop table management
• Data Architecture and System Design experience
• Data Modeling using tools like Erwin
• Familiarity with Alation or similar data cataloging/mining/reporting tools

Additional/Preferred Skills:

• Experience with Agile methodologies
• Tools: JIRA, GIT Bash, Jenkins, uDeploy, CA7
• Strong understanding of CI/CD pipelines and deployment automation

Soft Skills:

• Strong analytical and problem-solving abilities
• Excellent communication and collaboration skills
• Proven ability to work effectively in a team environment
• Experience in the banking or financial services industry
• Minimum 3 years of hands-on development experience

Education & Certifications:

• Bachelor’s degree in computer science, Engineering, or a related field
• Relevant certifications are a plus (e.g., Cloudera, Spark, or Python-related)

#INDGEN