Data Engineer ( Databricks) Big Impact Tech (BIT)
Washington, DC17 days ago
- Full-time
Highlights
Your Toolkit Category Tools & Skills Big Data PySpark, Spark SQL, Databricks (plus) Cloud Data Warehouse Snowflake (SnowSQL, Snowpipe, Streams, Tasks, Iceberg Tables, Dynamic Tables, Materialized Views) AWS Services S3, Glue, EMR, Lambda, Step Functions, Redshift, IAM, CloudWatch Storage Formats Parquet, Delta Lake, Apache Iceberg Programming Python, SQL, Shell Scripting Orchestration Airflow, Step Functions, Control-M DevOps Git, Jenkins, Terraform, CI/CD pipelines Legacy ETL (plus) IBM DataStage, Informatica Governance Data lineage, quality frameworks, audit readiness . Title: Data Engineer – PySpark, Snowflake & AWS Location: McLean, VA Contract for 6 months About the Role We're looking for a hands-on, delivery-focused Data Engineer to help build and scale our cloud data platform.
Numbers & Facts
| Location | McLean, VA |
Description
Title: Data Engineer – PySpark, Snowflake & AWS
Location: McLean, VA
Contract for 6 months
About the Role
We're looking for a hands-on, delivery-focused Data Engineer to help build and scale our cloud data platform. You'll design and develop modern data pipelines using PySpark, Snowflake, and AWS to optimize cloud data workloads.
This is a contractor, hands-on role requiring the candidate to be local to McLean, VA and available for on-site engagement.
The ideal candidate combines strong engineering fundamentals with cloud-native data expertise — able to translate business needs into robust, performant, and well-documented data solutions. Experience with Client, Freddie Mac, or equivalent GSE/mortgage enterprise environments is strongly preferred.
What You'll Do
- Design & Build Scalable Data Pipelines using PySpark for large-scale batch and streaming data processing.
- Develop PySpark Solutions — write production-grade PySpark code to read data from S3 (Parquet/Delta files), perform complex transformations, and process large-scale datasets efficiently.
- Implement Deduplication Logic — design and implement robust deduplication strategies for large datasets using PySpark.
- Performance Tuning (PySpark) — optimize PySpark jobs for reading and processing very large datasets, including partitioning, caching, broadcast joins, and shuffle optimization.
- Develop Cloud-Native Data Solutions on AWS — leveraging services like S3, Glue, EMR, Lambda, Step Functions, and Redshift.
- Engineer Snowflake Data Platforms — build warehouses, schemas, and data models optimized for analytics and reporting.
- Build Snowflake Iceberg Tables — design and implement Apache Iceberg tables in Snowflake for open lakehouse architectures and interoperability.
- Develop Dynamic Tables & Materialized Views — build and maintain Snowflake Dynamic Tables and Materialized Views to support near real-time analytics and query acceleration.
- Snowflake Performance Tuning — optimize Snowflake workloads through clustering, micro-partition pruning, query profiling, warehouse sizing, result caching, and materialization strategies.
- Modernize Legacy ETL — migrate on-prem ETL workloads (e.g., DataStage, Informatica) to PySpark and Snowflake-based cloud pipelines.
- Optimize Performance — tune Spark jobs, Snowflake queries, and AWS resource utilization for speed and cost.
- Ensure Data Quality & Governance — implement validation, lineage, and monitoring across every pipeline.
- Collaborate Across Teams — partner with data architects, analysts, TPMs, and business stakeholders to deliver trustworthy data products.
- Document Everything — from technical designs to runbooks, ensuring every pipeline is maintainable and audit-ready.
What We're Looking For
Must-Haves
- 6+ years of hands-on data engineering experience in enterprise environments.
- Strong expertise in PySpark — building distributed data processing pipelines at scale.
- Hands-on experience writing PySpark code to read data from S3 Parquet/Delta files, perform transformations, and handle large datasets.
- Proven experience in performance tuning of PySpark jobs when reading and processing very large datasets.
- Experience implementing deduplication logic in PySpark for high-volume data pipelines.
- Deep hands-on experience with Snowflake — data modeling, SnowSQL, Snowpipe, Streams, Tasks, and role-based access.
- Hands-on experience creating Snowflake Iceberg Tables for open table format and lakehouse use cases.
- Experience building Snowflake Dynamic Tables and Materialized Views for incremental data processing and query acceleration.
- Strong Snowflake performance tuning skills — clustering keys, micro-partition pruning, query profiling, warehouse right-sizing, caching strategies, and cost optimization.
- Proven AWS experience — S3, Glue, EMR, Lambda, IAM, Step Functions, CloudWatch, and Redshift.
- Advanced SQL skills — complex joins, subqueries, window functions, CTEs, and performance tuning.
- Strong Python programming skills beyond PySpark — for utilities, automation, and orchestration.
- Experience with orchestration tools — Airflow, AWS Step Functions, or equivalent.
- Solid understanding of data warehousing, ELT/ETL patterns, data lakes, and lakehouse architectures.
- Excellent communication skills — able to articulate technical decisions to both engineers and business stakeholders.
Nice to Have
- Experience with IBM DataStage or other legacy ETL tools (for modernization contexts).
- Familiarity with CI/CD for data pipelines (Git, Jenkins, GitHub Actions, Terraform).
- Exposure to data quality frameworks (Great Expectations, dbt tests).
- Knowledge of streaming platforms (Kafka, Kinesis).
- Experience in regulated environments — financial services, mortgage, or GSE programs (Client / Freddie Mac).
- Certifications: AWS Certified Data Analytics / Solutions Architect, SnowPro Core/Advanced.
Your Toolkit
Category | Tools & Skills |
Big Data | PySpark, Spark SQL, Databricks (plus) |
Cloud Data Warehouse | Snowflake (SnowSQL, Snowpipe, Streams, Tasks, Iceberg Tables, Dynamic Tables, Materialized Views) |
AWS Services | S3, Glue, EMR, Lambda, Step Functions, Redshift, IAM, CloudWatch |
Storage Formats | Parquet, Delta Lake, Apache Iceberg |
Programming | Python, SQL, Shell Scripting |
Orchestration | Airflow, Step Functions, Control-M |
DevOps | Git, Jenkins, Terraform, CI/CD pipelines |
Legacy ETL (plus) | IBM DataStage, Informatica |
Governance | Data lineage, quality frameworks, audit readiness |
Similar Job Searches
Data Manager Jobs in Mclean, VAData Analyst Jobs in Mclean, VAData Specialist Jobs in Mclean, VABusiness Intelligence Manager Jobs in Mclean, VADatabase Engineer Jobs in Mclean, VABusiness Intelligence Analyst Jobs in Mclean, VAData Engineer Jobs in Tysons, VAData Engineer Jobs in Falls Church, VAData Engineer Jobs in Arlington, VAData Engineer Jobs in Reston, VA
Similar Jobs
- New!
Mechanical Commissioning Engineer / Project Manager (Data Centers) Jobot
Sterling, VA3 days agoRemote- $165,000–$180,000 Per Year
- New!
Lead Specialist, Federal Data Engineer KPMG
Washington, DC3 days ago 
Lead Data Scientist Calibre Systems
Alexandria, VA15 days ago- New!
Commissioning Project Engineer (Data Centers) Jobot
Sterling, VA3 days agoRemote- $90,000–$160,000 Per Year
Data Storage Engineer Leidos
Poolesville, MD30+ days ago- $107,900–$195,050