Senior Kafka Platform Engineer

Your Opportunity

At Schwab, you're empowered to make an impact on your career. Here, innovative thought meets creative problem solving, helping us challenge the status quo and transform the finance industry together. We believe in the importance of in-office collaboration and fully intend for the selected candidate for this role to work on site in the specified location(s).

Event Streaming team in Enterprise Middleware organization is seeking a Senior Kafka Platform Engineer with experience to lead the evolution of its enterprise event-streaming platform and drive enterprise-scale event streaming, Confluent Cloud adoption, platform efficiency and AI-enabled SDLC transformation. This role will provide technical leadership across Confluent Platform and Confluent Cloud, drive cloud modernization and platform resiliency, and build automation to enhance scalability, security, reliability, performance, stability, and developer productivity. The ideal candidate combines deep Kafka administration expertise with a strong Java development background, hands-on experience in cloud and Kubernetes environments, and the ability to apply AI-assisted engineering to accelerate delivery, improve operational efficiency, and support secure, business-critical solutions at scale.

Key Responsibilities:

• Design and build AI agents, custom skills, or equivalent frameworks to automate engineering and operational workflows at scale
• Leverage AI-powered tools (e.g., Copilot, AI agents) to accelerate development, automate testing, improve code quality, and enable intelligent SDLC workflows aligned to AI-DLC vision.
• Design agentic workflows and multi-agent orchestration patterns to solve complex distributed systems, integration, and operational challenges
• Act as the senior technical authority for Confluent Platform and Confluent Cloud, providing architecture, design, and implementation guidance to solve complex enterprise integration challenges at scale.
• Lead solution architecture across cross-functional initiatives, ensuring alignment with cloud modernization, resiliency, security, and AI-driven engineering priorities.
• Perform deep-dive analysis of complex production issues, drive root cause identification, and implement sustainable remediation strategies with full documentation.
• Design and build automation frameworks and self-service tools to streamline Kafka operations, reduce manual effort, and improve developer productivity and platform scalability.
• Lead seamless migration of on-prem Kafka workloads to Confluent Cloud using cluster linking, schema portability, and hybrid streaming patterns-ensuring zero business disruption.
• Drive execution across Kafka, infrastructure, security, and application teams to remove dependencies, mitigate risks, and ensure on-time delivery of committed milestones.
• Contribute to enterprise architecture forums, defining and enforcing best practices for event streaming, integration patterns, and platform standardization.
• Establish best practices for testing, reliability, observability, and operational readiness, ensuring high availability and scalable platform performance.
• Identify systemic and architectural risks early, leading mitigation efforts to prevent client impact and ensure platform stability.
• Implement enterprise-grade security controls (mTLS, RBAC, encryption) and drive continuous remediation of vulnerabilities, ensuring compliance with regulatory and audit requirements.
• Interpret technical requirements to build and enhance Kafka configurations, message flows, and integration solutions supporting business-critical applications.
• Partner with distributed teams (including offshore engineering) to ensure consistent standards, efficient delivery, and 24x7 operational support across time zones.

What you have

Required Qualifications: Technical Expertise and Educational Requirements

• Bachelor's degree in computer science, Engineering, or related field
• 10+ years of hands-on software engineering experience with proven technical leadership
• 5+ years of expertise in large-scale distributed systems and mission-critical infrastructure
• Strong hands-on experience with GitHub Copilot and other AI-assisted development tools, including advanced prompt engineering and workflow integration to accelerate engineering
• Proven ability to leverage AI-assisted development (e.g., Copilot, code generation) and build AI-driven agents/workflows to improve team productivity and engineering efficiency
• Deep expertise in Confluent Kafka (capacity planning, administration, internals, performance tuning)
• Strong experience with Kafka clusters, replication, disaster recovery, and data pipelines
• Hands-on experience with Kubernetes, Confluent Operator, and cloud-native modernization (GCP)
• Strong development background on Linux/Unix platforms
• Experience with data modeling and streaming technologies (Avro, Protobuf, JSON Schema)
• Strong automation mindset with hands-on expertise in Python, Ansible, and CI/CD pipelines (Jenkins, DevOps tooling) to drive operational efficiency and reduce manual effort
• Solid foundation in security engineering (encryption, TLS, certificate and key management)
• Strong focus on observability, reliability, automation, and production readiness with proactive risk mitigation
• Experience leading architecture and execution across cross-functional, large-scale programs
• Strong understanding of networking, load balancing, OS fundamentals, and performance optimization
• Experience in regulated environments (financial services preferred)
• Strong collaboration skills across engineering, architecture, infrastructure, and business teams
• Experience working with globally distributed/offshore teams
• Strong Agile delivery experience (Scrum/Kanban/Scaled Agile)
• Strong leadership and soft skills including strategic thinking, decision-making, mentoring/coaching, and driving engineering excellence
• Excellent communication, influencing, and stakeholder management skills, with ability to align diverse teams toward common goals
• High ownership mindset with ability to prioritize, multitask, and lead through high-pressure production situations
• Strong problem-solving skills with a proactive, innovation-driven approach
• Self-driven, adaptable, and customer-focused with strong relationship management and team-building capabilities

Preferred Qualifications: What experience and skills are DESIRED, but are not REQUIRED to perform this role today?

• Experience applying AI-Driven Development Lifecycle (AI-DLC) practices to modernize SDLC pipelines through intelligent automation, improving development, testing, and release efficiency
• Demonstrated contributions to architecture standards, reusable frameworks, or open-source initiatives in distributed systems or event-driven platforms
• Deep expertise in Kafka platform administration and development, including large-scale event streaming systems and platform modernization in enterprise environments
• Certifications in Confluent Kafka, Confluent Cloud, and major cloud platforms (GCP/AWS/Azure) are a strong plus.