Senior Data Engineer (Databricks & Cloud Analytics)

Job Title: Senior Data Engineer (Databricks & Cloud Analytics)
Location: Salt Lake City, Utah
Work Model: Hybrid – onsite and remote

Responsibilities
• Design, develop, and support enterprise data platforms, cloud analytics solutions, or large-scale data engineering initiatives.
• Work hands-on with Databricks, Apache Spark (PySpark), Python, SQL, and Scala to build scalable data pipelines and analytics solutions.
• Manage and implement Databricks ecosystem components, including Workspaces, Delta Lake, Unity Catalog, Delta Live Tables, Databricks SQL, MLflow, and Databricks Jobs.
• Optimize SQL queries and improve query performance through tuning and best practices.
• Design and implement ETL/ELT processes utilizing cloud platforms such as Azure Data Factory.
• Work with event streaming technologies like Apache Kafka and integrate enterprise applications via REST APIs and web services.
• Utilize Azure Data Lake Storage (ADLS Gen2) and other cloud storage solutions.
• Collaborate within Linux environments, including shell scripting and command-line utilities.
• Use source control tools like Git for collaborative development.
• Participate in Agile teams practicing Scrum, Kanban, or SAFe methodologies.
• Provide technical mentoring and leadership within data engineering teams.

Requirements
• Bachelor’s degree in Computer Science, Information Systems, Engineering, or related field or equivalent professional experience.
• At least 6 years of experience designing and supporting enterprise data platforms or cloud data solutions.
• Extensive hands-on experience with Databricks, Apache Spark, Python, SQL, and Scala.
• Proven SQL development skills, including query optimization and performance tuning.
• Experience designing ETL/ELT solutions with cloud platforms such as Azure Data Factory.
• Knowledge of Kafka or similar event streaming systems.
• Familiarity with cloud data storage (Azure Data Lake Storage or equivalents).
• Proficiency using Git for version control.
• Experience working in Linux environments and scripting.
• Preferred: Infrastructure as Code (Terraform), CI/CD pipelines (Azure DevOps, GitHub Actions).
• Strong analytical and troubleshooting skills.
• Prior experience working in Agile methodologies.
• Ability to prioritize multiple tasks independently and mentor team members.

#M-
#LI-
Ref: #404-IT Pittsburgh