Sign upLog in

Lead Data Engineer

Artech LLC

Malvern, PA

Apply
JOB DETAILS
SALARY
$60–$65 Per Hour
SKILLS
Amazon Simple Storage Service (S3), Amazon Web Services (AWS), Analysis Skills, Apache, Apache Cassandra, Apache Spark, Application Programming Interface (API), Artificial Intelligence (AI), Cloud Computing, Code Reviews, Coding Standards, Concrete, Continuous Deployment/Delivery, Continuous Integration, Cost Control, Data Processing, Data Storage, Database Administration, DevOps, Distributed Computing, Docker, Leadership, Mentoring, Metrics, NoSQL, Parallel Computing, Performance Tuning/Optimization, Project Estimates, Public Cloud, Python Programming/Scripting Language, Relational Databases (RDBMS), SQL (Structured Query Language), Scala Programming Language, Scripting (Scripting Languages), Snowflake Schema, Software Engineering, Style Guide, Team Player, Technical Leadership
LOCATION
Malvern, PA
POSTED
8 days ago
Request ID: 86258-1
Title: Lead Data Engineer
Locations:  Malvern, PA
Duration: 6 Months
Pay Range: $60 - $65/Hour on W2/C2C (All inclusive)
 
Role Descriptions:
 
Technical skill sets: Python, AWS- S3, Lambdas, Glue, Gen-Ai , LLMs , SQL, DynamoDB , Kafka/ Kinesis, pySpark

Responsibilities :
1. Advanced Architecture & System Design
A Tech Lead is primarily responsible for the overall platform vision and ensuring systems do not break under scale.
Distributed Computing: Mastery of frameworks like Apache Spark or Ray for massive-scale parallel data processing.
Streaming & Event-Driven Architecture: Deep understanding of real-time pipeline design using Kafka, Kinesis, or Flink.
Cloud Infrastructure: Expertise in at least one major public cloud (AWS), specifically understanding storage/compute decoupling and cost optimization.
2. Core Programming & Database Management
Leads set coding standards and review code, requiring complete fluency in the fundamentals. [1]
SQL: Advanced mastery for metrics computation, window functions, and query performance tuning across relational and columnar databases (e.g., Snowflake, Redshift, BigQuery).
Scripting Languages: High proficiency in Python or Scala for writing reusable pipeline code and interacting with APIs.
Data Storage: Deep familiarity with both columnar/analytical stores and NoSQL databases (e.g., DynamoDb, Cassandra).
3. Pipeline Orchestration & DevOps
Ensuring pipelines run smoothly, idempotently, and securely in production. [1, 2]
Workflow Orchestration: Ability to architect Directed Acyclic Graphs (DAGs) in tools like Apache Airflow or Prefect.
CI/CD & Infrastructure as Code (IaC): Applying software engineering principles to data by using Docker, Kubernetes, and Terraform.
Data Governance & Security: Implementing Role-Based Access Control (RBAC), data masking, and compliance frameworks.
4. Leadership & Soft Skills
Tech leads also mentor junior engineers, estimate project timelines, and translate ambiguous business needs into concrete technical specifications.
Mentorship & Code Review: Fostering a collaborative development environment and enforcing style guidelines.
System Observability: Building logging, monitoring, and alerting mechanisms so the team knows exactly when and why pipelines fail.

Skills: Digital : Python~ Digital : Amazon Web Service(AWS) Cloud Computing~ Advanced Java Concepts~ Core Java
Experience Required: 8-10
 
Company Benefits & Culture
  • Inclusive and diverse work environment
  • Opportunities for professional growth and development
  • Comprehensive health and wellness benefits
 
 
Appreciate your quick response and please feel free to reach me out for any query you may have.
 
Thanks
 

About the Company

A

Artech LLC