Sign upLog in

Big Data Engineer

ConsultNet

Rockville, MD

Apply
JOB DETAILS
SKILLS
AWS Lambda, Amazon Simple Storage Service (S3), Amazon Web Services (AWS), Apache Hadoop, Apache Hive, Apache Spark, Artificial Intelligence (AI), Automation, Autoscaling, Best Practices, Big Data, Cloud Computing, Communication Skills, Computer Science, Concurrency, Consulting, Continuous Deployment/Delivery, Continuous Improvement, Continuous Integration, Cross-Functional, Data Analysis, Data Management, Data Processing, Data Science, Data Sets, Data Structures, Distributed Computing, Docker, Electronic Medical Records, Emerging Technology, GitHub, Identify Issues, Jenkins, Lambda Architecture, Large-Scale Systems, Network Monitoring, Performance Tuning/Optimization, Problem Solving Skills, Process Improvement, Programming Tools, Python Programming/Scripting Language, Query Optimization, SQL (Structured Query Language), Scala Programming Language, Scalable System Development, Team Player, Test Automation, Test Harness, Unit Test
LOCATION
Rockville, MD
POSTED
30+ days ago

Title: Big Data Engineer
Location : Rockville, MD or McLean. VA
Target Start Date : ASAP
Type: contract
Pay Rate : DOE

We are seeking an experienced Big Data Engineer to design, build, and optimize large-scale data processing systems. This role partners closely with engineering, data, and business teams to deliver scalable, high-performance data solutions across modern cloud environments.

Key Responsibilities

  • Design, develop, and maintain scalable data pipelines using technologies such as Spark, Hadoop, Python, and Scala
  • Architect and deploy big data workloads using Amazon EMR on EKS (Kubernetes)
  • Build and manage Kubernetes-based infrastructure for distributed data processing
  • Implement data ingestion, transformation, and storage solutions for large-scale datasets
  • Optimize pipelines for performance, scalability, and reliability
  • Monitor, troubleshoot, and resolve production data issues
  • Develop and execute automated testing frameworks (unit, integration, E2E)
  • Collaborate with data scientists, analysts, and cross-functional teams to support data-driven initiatives
  • Continuously evaluate and adopt emerging technologies and best practices

Core Technical Skills

Big Data & Distributed Systems

  • Strong experience with Spark, Hadoop, Hive, Trino
  • Expertise handling large-scale (petabyte-level) datasets
  • Deep understanding of performance challenges (e.g., data skew, resource constraints)

Kubernetes & Containerization

  • Hands-on experience with Kubernetes (EKS) including pods, services, deployments
  • Experience running Spark on Kubernetes (EMR on EKS)
  • Knowledge of Helm, YAML manifests, autoscaling, and cluster troubleshooting

Cloud (AWS)

  • Experience with S3, EMR, Glue, Lambda, Athena
  • Strong understanding of EKS architecture, IAM roles (IRSA), networking, and monitoring (CloudWatch/CloudTrail)
  • Familiarity with serverless architectures (Lambda, Fargate)

Programming

  • Proficiency in Python or Scala
  • Strong understanding of data structures, concurrency, and performance optimization

SQL

  • Advanced SQL skills including window functions, joins, aggregations, and query optimization

AI & Modern Development Practices

  • Experience with AI-assisted development tools (e.g., ChatGPT, GitHub Copilot)
  • Prompt engineering and AI-driven workflow optimization
  • Ability to incorporate AI insights into engineering practices and team productivity

Preferred / Nice-to-Have

  • CI/CD tools (Jenkins, GitLab CI, GitHub Actions, ArgoCD)
  • Infrastructure as Code (Terraform, CloudFormation)
  • Docker and container optimization
  • Observability tools (Prometheus, Grafana, ELK)
  • Service mesh technologies (Istio, Linkerd)
  • AWS or Kubernetes certifications

Qualifications

  • Bachelor's degree in Computer Science or related field (Master's preferred)
  • 5+ years of experience in big data engineering or related roles
  • Strong background in distributed systems, cloud platforms, and data architecture
  • Excellent communication and collaboration skills
  • Ability to operate effectively in a fast-paced environment

What Success Looks Like

  • Reliable, scalable data pipelines powering business insights
  • Strong partnership across engineering and business teams
  • Continuous improvement of data platforms and development workflows
  • Effective use of automation and AI to enhance productivity and outcomes


Welcome to ConsultNet, a premier national provider of technology talent and solutions. Our expertise spans across project services, contract-to-hire, direct search, and managed services onshore, nearshore, and hybrid. For over 25 years, we have connected thousands of consultants with meaningful roles through a personal, communication-driven approach, partnering with a diverse client base to build high-performing teams and create lasting impact. Our comprehensive service offerings cover a wide range of technology and engineering positions across key markets nationwide. Learn more at www.consultnet.com .

We champion equality and inclusivity, proudly supporting an Equal Opportunity Employer policy. We welcome applicants regardless of Race, Color, Religion, Sex, Sexual Orientation, Gender Identity, National Origin, Age, Genetic Information, Disability, Protected Veteran Status, or any other status protected by law.




About the Company

C

ConsultNet