Position Overview

We are seeking an experienced Senior Data Engineer to help design, develop, and enhance a modern cloud-based data platform supporting enterprise analytics and business intelligence initiatives. This individual will play a key role in building reliable data pipelines, transforming complex datasets, and enabling data-driven decision-making across the organization.

The ideal candidate is a hands-on engineer with strong experience in Databricks, AWS, Python, PySpark, and SQL, along with a solid understanding of modern data architectures and engineering best practices. This position requires someone comfortable working across the full data engineering lifecycle, from solution design and development through deployment, optimization, and production support.

Key Responsibilities

• Design, develop, and maintain scalable data pipelines to ingest, process, and transform information from diverse data sources.

• Build and optimize data solutions using Databricks and AWS cloud technologies.

• Develop reusable ETL/ELT frameworks to support enterprise reporting, analytics, and downstream applications.

• Create and maintain high-quality, structured datasets that enable business intelligence and advanced analytics.

• Improve data processing efficiency through performance tuning, query optimization, and effective data modeling.

• Establish and maintain data quality controls, validation procedures, and monitoring capabilities to ensure data accuracy and reliability.

• Collaborate with cross-functional engineering, architecture, and analytics teams to translate technical and business requirements into scalable solutions.

• Participate in technical design discussions and contribute to the ongoing evolution of the organization's data architecture.

• Support deployment, troubleshooting, and maintenance of data engineering solutions in production environments.

• Explore opportunities to support emerging analytics, machine learning, and AI-driven capabilities.

Required Qualifications

• 8+ years of professional experience in data engineering, data integration, or related software development disciplines.

• Strong hands-on experience developing data solutions within Databricks, including Delta Lake, Databricks Workflows, and Databricks SQL.

• Advanced proficiency in Python, PySpark, and SQL for large-scale data processing and transformation.

• Proven experience designing and implementing complex ETL/ELT pipelines in enterprise environments.

• Practical experience working with AWS services and cloud-based data infrastructure.

• Solid understanding of data warehousing concepts, dimensional modeling, and modern lakehouse architectures.

• Experience optimizing distributed data processing workloads for scalability, reliability, and performance.

• Familiarity with version control, Agile development methodologies, and collaborative engineering practices.

• Strong analytical and problem-solving skills with the ability to work independently and navigate complex technical challenges.

Preferred Qualifications

• Experience working with NoSQL databases, graph databases, or vector-based data storage technologies.

• Exposure to machine learning data pipelines, generative AI applications, or retrieval-augmented generation (RAG) solutions.

• Familiarity with infrastructure-as-code and automated deployment practices using tools such as Terraform or Jenkins.

• Experience with application and infrastructure monitoring solutions such as Splunk or Dynatrace.

• Understanding of CI/CD practices and operational best practices for enterprise data platforms.

Ideal Candidate

The successful candidate will be a technically strong, self-sufficient engineer who enjoys solving complex data challenges and building scalable solutions. They should be comfortable working in an evolving technology environment, collaborating with multiple teams, and taking ownership of engineering deliverables from initial design through production implementation.