Data Engineer with GCP and Pyspark

As an African American-owned and Veteran-owned company, ApTask offers a comprehensive suite of services, including staffing and recruitment solutions, managed services, IT consulting, and project management. " Engineer data transformations primarily using PySpark on GCP (Dataproc/Sparkflow) and BigQuery as the main query engine.

Data Engineer with GCP and Pyspark
Location - Irving, TX/Charlotte, NC
Contract

Responsibilities

" Design, build, and harden production-grade data pipelines for ingest, curation, reconciliation, DQ monitoring, lineage, and notifications.
" Contribute to and extend the generic pipeline framework so federated teams can onboard data consistently and at speed.
" Engineer data transformations primarily using PySpark on GCP (Dataproc/Sparkflow) and BigQuery as the main query engine.
" Implement best practices for schema management, performance, reliability, and cost efficiency across large-scale datasets (Parquet, Iceberg).
" Integrate with centralized data cataloging and governance (e.g., Dataplex/Knowledge Catalog or equivalent), and support automated lineage harvesting.
" Collaborate with platform, security, and central cloud teams; support hybrid access patterns using Starburst where needed.
" Operate with an ownership mindset in an on-time, production SLAs environment, ensuring resilient, observable, and recoverable data flows.

Key Qualifications Required (for all levels):
" 3+ years (L3) / 6+ years (L4) hands-on data engineering in production environments.
" Strong SQL skills and proficiency building data transformations at scale.
" Practical experience with Spark (preferably PySpark) and big data file formats (Parquet; familiarity with Iceberg concepts).
" Experience delivering reliable, scheduled pipelines with monitoring, alerting, and incident response.
" Understanding of data cataloging, metadata, and lineage concepts, and why they matter for governance and reuse.
" Exposure to GCP data services (e.g., Dataproc/Spark, BigQuery). Deep infra setup knowledge is not required; collaboration with central cloud teams is expected.
" Solid software engineering practices: version control, CI/CD, testing, code reviews, and documentation.

Preferred:
" Experience building reusable pipeline frameworks or platform components adopted by multiple teams.
" Familiarity with Starburst/Trino for federated query across hybrid environments.
" Knowledge of data quality frameworks, reconciliation, and auditability in regulated or enterprise settings.
" Python proficiency; Java experience a plus (willingness to work primarily in Python).
" Background in operational data environments with time-bound SLAs and prod support rotations.
" Exposure to AI/ML model lifecycle platforms or MLOps concepts (platform enablement rather than model authoring).

Soft Skills
" High ownership, bias for action, and clear communication with technical and non-technical partners.

" Comfort operating in a fast-scaling organization with evolving scope. has context menu

About ApTask:
ApTask is a leading global provider of workforce solutions and talent acquisition services, dedicated to shaping the future of work. As an African American-owned and Veteran-owned company, ApTask offers a comprehensive suite of services, including staffing and recruitment solutions, managed services, IT consulting, and project management. With a focus on excellence, collaboration, and innovation, ApTask provides unparalleled opportunities for professional growth and development. As a member of the ApTask team, you will have the chance to connect businesses with top-tier professionals, optimize workforce performance, and drive success across diverse industries. Join us at ApTask and be part of our mission to empower organizations to thrive while fostering a diverse and inclusive work environment.

Applicants may be required to attend interviews in person or by video conference. In addition, candidates may be required to present their current state or government issued ID during each interview.

Candidate Data Collection Disclaimer:
At ApTask, we prioritize safeguarding your privacy. As part of our recruitment process, certain Personally Identifiable Information (PII) may be requested by our clients for verification and application purposes. Rest assured, we strictly adhere to confidentiality standards and comply with all relevant data protection laws. Please note that we only collect the necessary information as specified by each client and do not request sensitive details during the initial stages of recruitment.

If you have any concerns or queries about your personal information, please feel free to contact our compliance team at

businessexcellence@aptask.com

ompliance@aptask.com

.

Applicant Consent:
By submitting your application, you agree to ApTask's (www.aptask.com) Terms of Use and Privacy Policy, and provide your consent to receive SMS and voice call communications regarding employment opportunities that match your resume and qualifications. You understand that your personal information will be used solely for recruitment purposes and that you can withdraw your consent at any time by contacting us at 732-355-8000 or help@aptask.com. Message frequency may vary. Msg & data rates may apply.