Databricks Data Analyst

Job Overview

We are seeking a Databricks Data Analyst with hands-on experience in Databricks, PySpark, Spark SQL, and data visualization tools to develop data solutions that support business needs. This role involves analyzing complex datasets, developing data mappings and dashboards, ensuring data quality, and collaborating with technical teams and business stakeholders.

Job Duties

• Conduct comprehensive data analysis to identify trends, patterns, and actionable insights while ensuring data quality and integrity.

• Use Spark SQL within Databricks to query and analyze large datasets, developing efficient and complex SQL queries.

• Collaborate with data engineers to design, develop, and deliver high-quality data products.

• Develop detailed data mappings to integrate and transform data from diverse sources into structured, analysis-ready formats.

• Design and build intuitive dashboards using Power BI and Databricks' native visualization tools to communicate insights effectively.

• Build Gold-layer data products to support custom dashboards, APIs, and user interface tools.

• Develop and maintain documentation for data pipelines, mappings, data models, and dashboard functionality.

• Perform statistical analysis, data mining, and machine learning to identify patterns, trends, and opportunities for improvement.

• Gather and develop business requirements to ensure data solutions align with stakeholder needs.

• Support change management documentation and collaborate with stakeholders to maintain alignment between business requirements and technical solutions.

• Participate in design discussions and project planning with technical leads and the broader development team.

Required Skills and Experience

• 5 years of hands-on experience with the Databricks platform and PySpark for data analysis and transformation.

• 5 years of experience writing efficient, complex Spark SQL queries to analyze large datasets within Databricks.

• 5 years of experience developing clear, intuitive dashboards using Power BI and Databricks' native visualization tools.

• 5 years of experience developing detailed data mappings to integrate and transform data.

• 5 years of experience gathering and developing business requirements and collaborating with stakeholders.

• 5 years of experience conducting thorough data analysis.