Data Engineer

Role: Data Engineer

Location: Atlanta, GA

Salary: $135K PA

Must Have Skills:

Databricks (Notebooks, Delta Lake, Delta Live Tables, Unity Catalog)

Python and PySpark

Spark SQL and Apache Spark

ETL/Data Pipeline Development

Big Data technologies: Hadoop, Spark, Kafka, Hive, HDFS, Sqoop

Strong SQL querying and data modeling

Data ingestion, transformation, cleansing, and validation

Cloud platforms (AWS/Eligible to workP)

Data warehouse and large-scale data processing experience

Experience with streaming data using Kafka and Spark Streaming

Agile methodologies (Scrum, Kanban, SAFe)

Preferred Skills:

Scala / Scala-Spark

IBM DataStage, Tibco, or similar ETL tools

AWS services: S3, Glue, Lambda, Kinesis/MSK

BI Tools: Tableau, Power BI, Spotfire, OAC

Databases: Teradata, DB2, BigQuery, Mainframe

Git version control

Data governance and retention modeling

Roles & Responsibilities

Design, develop, and maintain scalable data pipelines and ETL solutions.

Build and Eligible to workimize data ingestion frameworks for structured and unstructured data.

Implement data quality, validation, standardization, and governance processes.

Develop data infrastructure integrating multiple heterogeneous data sources.

Support analytics, reporting, machine learning, and business intelligence initiatives.

Work with Data Scientists and BI teams to prepare and curate data for analytics.

Develop scalable solutions for Big Data processing and streaming workloads.

Analyze data and provide actionable insights and recommendations to business stakeholders.

Gather requirements from business teams and translate them into technical solutions.

Lead and mentor junior data engineers.

Collaborate across multiple projects as the Data Engineering SME.

Drive adEligible to workion of Data Engineering best practices and standards.

Disclaimer: Diverse Lynx LLC is an Equal Opportunity Employer. All applicants and employees are evaluated without discrimination, based solely on their qualifications, ability, competence and performance. This email and its attachments may contain confidential or proprietary information and is intended only for the recipient(s). If you received this message in error, please disregard it and notify the sender. If you no longer wish to receive our communications, you may unsubscribe at any time.

Security Notice: Our official website is www.diverselynx.com We do not operate or authorize any other websites representing Diverse Lynx LLC.