Sign upLog in

Data Engineering (Palantir, Spark, PySpark, Python)- W2 Role

CLOUDSCOUTS SOFTWARE SOLUTIONS LLC

Frisco, TX

Apply
JOB DETAILS
SKILLS
Amazon Web Services (AWS), Apache Spark, Best Practices, Cloud Computing, Code Reviews, Communication Skills, Computer Science, Continuous Deployment/Delivery, Continuous Integration, Cross-Functional, Data Management, Data Modeling, Data Processing, Data Quality, Data Science, Data Sets, Data Warehousing, Database Extract Transform and Load (ETL), DevOps, Distributed Computing, Emerging Technology, GCP (Good Clinical Practices), Identify Issues, Leadership, Mentoring, Merchandising, Microsoft Windows Azure, Multiplatform/Cross-Platform, Ontology, Performance Tuning/Optimization, Python Programming/Scripting Language, Release Management/Engineering, Requirements Management, Retail, SQL (Structured Query Language), Software Engineering, Supply Chain, Team Lead/Manager, Technical Leadership
LOCATION
Frisco, TX
POSTED
30+ days ago
Job Title: Technical Lead – Data Engineering (Palantir, Spark, PySpark, Python)

Please dont apply if you dont have PALANTIR experience

Onsite work :One or two days in a month
W2 role
OPT with 7 years exp is fine


Job Summary


We are looking for a hands-on Technical Lead in Data Engineering to drive the design, development, and delivery of scalable data solutions in a large retail enterprise. This role requires strong expertise in Palantir Foundry, Spark/PySpark, SQL, and Python, along with the ability to lead engineering teams and partner with business stakeholders across supply chain, merchandising, and store operations.


Key Responsibilities


  •  Lead the design and implementation of scalable data platforms and pipelines using PySpark, Spark, and Python
  •  Drive adoption and best practices for Palantir Foundry (data pipelines, ontology, workflows, and operational applications) 
  •  Architect and optimize high-performance data processing solutions for large-scale datasets 
  •  Provide technical leadership and mentorship to data engineers, ensuring code quality and best practices 
  •  Collaborate with cross-functional teams (business, analytics, data science) to translate requirements into scalable solutions 
  •  Design robust data models, ETL/ELT frameworks, and data integration strategies 
  •  Ensure data quality, governance, security, and compliance across enterprise data platforms 
  •  Lead performance tuning, troubleshooting, and optimization of data pipelines 
  •  Drive CI/CD implementation, code reviews, and release management 
  •  Stay current with emerging data engineering technologies and recommend improvements 
Required Qualifications


  •  Bachelor’s or Master’s degree in Computer Science, Engineering, or related field 
  •  8+ years of experience in data engineering, with at least 2–3 years in a technical leadership role
  •  Strong hands-on experience with: 
    • Python & PySpark
    • Apache Spark
    •  Advanced SQL
  •  Experience with Palantir Foundry or similar modern data platforms 
  •  Deep understanding of data engineering principles (ETL/ELT, data modeling, distributed systems) 
  •  Experience designing and managing large-scale data architectures
  •  Strong leadership, communication, and stakeholder management skills 
Preferred Qualifications


  •  Experience in retail domain (supply chain, inventory, merchandising, store analytics) 
  •  Experience with cloud platforms (Azure, AWS, or GCP) 
  •  Familiarity with orchestration tools (Airflow, Azure Data Factory, etc.) 
  •  Experience with real-time/streaming data pipelines
  •  Exposure to DevSecOps practices 
Key Skills


  •  Technical Leadership 
  •  Data Engineering Architecture 
  •  PySpark & Spark 
  •  Python Programming 
  •  SQL Optimization 
  •  Palantir Foundry 
  •  Data Modeling & Warehousing 
  •  CI/CD & DevOps

Flexible work from home options available.

About the Company

C

CLOUDSCOUTS SOFTWARE SOLUTIONS LLC