Kafka Admin - Automation
Role Descriptions
Kafka Administrator with automation focus manages, secures, and scales high-availability Kafka clusters on-prem/cloud using infrastructure-as-code (IaC) tools like Ansible, Terraform, and GitOps to minimize manual work. They specialize in deploying Kafka on Kubernetes (CFK), automating topic/partition management, monitoring performance with Prometheus/Grafana, and implementing security ACLs/RBAC.
Techfetch
Techfetch
6
Core Responsibilities
Automation Tasks
- Automation Infrastructure-as-Code (IaC)
Use Ansible playbooks and Python scripts to automate Kafka installation, broker configuration, upgrades, and patching.
Implement Terraform for infrastructure provisioning and GitOps workflows (ArgoCD/Jenkins) for continuous deployment.
Cluster Management
- Deployment
Deploy and manage Confluent Platform using Confluent for Kubernetes (CFK) on OpenShift/Kubernetes.
Manage Kraft Kafka Raft mode, topics, partitions, and replication factors to ensure high availability.
Monitoring Performance Tuning
- Utilize Prometheus, Grafana, and Confluent Control Center for proactive monitoring of broker health, consumer lag, and producer latency.
Security Compliance
- Implement and automate security protocols including SASL/SSL, RBAC, ACLs, and certificate management.
Kafka Connect Schema Registry
- Configure, manage, and automate the deployment of Kafka Connectors (e.g., S3, JDBC, Snowflake) and manage data governance via Confluent Schema Registry.
Disaster Recovery
- Set up and manage Active-Passive clusters and ClusterLink for multi-region replication.
Techfetch
Techfetch
8
Required Technical Skills Expertise
- 10 years in Apache Kafka/Confluent Kafka administration.
- Independent lead Performer
- Automation Tools
- Ansible
- Terraform
- GitOps
- Shell scripting
- Python
- Infrastructure
- Kubernetes
- OpenShift
- Linux
- Monitoring
- Prometheus
- Grafana
- JMX
- Networking/Security
- TLS
- Kerberos
- SASL
- RBAC