Highlights
You will design and implement native operators and expressions using Apache DataFusion and Apache Arrow, extend Comets integration across Spark versions, and accelerate Apache Iceberg and Parquet reads and writes, all while guaranteeing strict behavioral and numerical compatibility with vanilla Spark. You will join a collaborative team where your code directly shapes the efficiency, speed, and architectural future of Apples data ecosystem3+ years of professional software engineering experience.
Numbers & Facts
| Location | Cupertino, CA |
| Industry | Computer/IT Services |
| Company Size | 10,000 employees or more |
| Year Founded | 1976 |
| Website | https://www.apple.com/jobs |
Description
We are seeking exceptional software engineers to work on Apache DataFusion Comet, Apache Spark, and related open-source technologies. Comet, the native accelerator for Apache Spark, was created at Apple and donated to the Apache Software Foundation. If you love pushing distributed query engines to their performance limits without compromising correctness or stability, this role is for you. We are looking for engineers with deep expertise in Spark internals and in related data formats and query engines, such as Apache Iceberg, Parquet, Arrow, and DataFusion. In this role, youll collaborate with teams across Apple operating Spark at massive scale and actively contribute upstream to these open-source communities. As an engineer on our team, you will advance Apache Spark to meet Apples scale while driving significant efficiency gains through Apache DataFusion Comet. You will design and implement native operators and expressions using Apache DataFusion and Apache Arrow, extend Comets integration across Spark versions, and accelerate Apache Iceberg and Parquet reads and writes, all while guaranteeing strict behavioral and numerical compatibility with vanilla Spark. Working alongside a team of open-source experts, including Apache Spark, DataFusion, and Iceberg committers, you will actively contribute that work back upstream. You will also profile and benchmark production workloads, optimizing memory, shuffle, and I/O to deploy Spark and Comet reliably across on-premises data centers and cloud environments. Your contributions will directly enhance the efficiency, resilience, and scale of systems powering services used by millions of people worldwide.What you will do and contribute to: Design, test, and ship high-performance engine features and platform components that power Apples big data workloads. Translate Apple-scale challenges into robust designs and contribute them upstream to projects such as Apache Spark and Apache DataFusion Comet. Actively participate in these open-source communities through upstream code contributions, code reviews, and architectural discussions. Partner across teams to build automation and monitoring that streamline deployment and triage. Optimize runtime efficiency, memory usage, and I/O so services scale reliably across data centers and cloud environments under growing load. Build clean, well-defined interfaces and abstractions that integrate native query execution seamlessly into broader data workflows and platform services.
You will join a collaborative team where your code directly shapes the efficiency, speed, and architectural future of Apples data ecosystem3+ years of professional software engineering experience. A proven track record of building distributed systems. Contributor or committer to Apache Spark and/or Apache DataFusion, Comet. BS, MS, or PhD in Computer Science or a related field, or equivalent practical experience.Experience with columnar or vectorized query execution, or with performance profiling and optimization of query engines. Good knowledge of Apache Iceberg and Apache Parquet internals. Experience working on or building Spark connectors to external data sources. Knowledge of Kubernetes, or other compute platforms.
About Company
We bring amazing people together to make amazing things happen.
We’re a diverse collection of thinkers and doers, continually reimagining what’s possible to help us all do what we love in new ways. The people who work here have reinvented entire industries with the Mac, iPhone, iPad, and Apple Watch, as well as with services, including iTunes, the App Store, Apple Music, and Apple Pay. And the same passion for innovation that goes into our products also applies to our practices — strengthening our commitment to leave the world better than we found it.
About Apple
There’s a place here for every kind of brilliant. Everyone here is an innovator, or an innovator-to-be, no matter what your team or your role. So bring your passion, courage, and original thinking and get ready to share it, because every new product, service, or feature we invent is the result of people working together to make each others’ ideas stronger. Innovation at this level depends on people who represent the variety of the human experience and inspire us with their own fresh perspectives. Together, we’ll do amazing work that can make a difference in people’s lives. Including your own. Learn more about working at Apple.