Kafka Admin

We are seeking an experienced Kafka Administrator to manage and maintain enterprise-level Apache Kafka clusters. The ideal candidate will be responsible for installation, configuration, monitoring, performance tuning, and ensuring high availability of Kafka infrastructure in distributed environments.

Key Responsibilities

Kafka Cluster Administration

Install, configure, and maintain Apache Kafka clusters Manage Kafka brokers, topics, partitions, and replication Perform cluster upgrades, scaling, and capacity planning

Monitoring & Performance Tuning

Monitor Kafka cluster health using tools like:

Prometheus, Grafana, Confluent Control Center

Optimize broker performance, throughput, and latency Tune configurations for producers and consumers

Data Streaming & Integration

Manage real-time data pipelines using Kafka

Handle integration with:

Spark, Hadoop, Flink Databases & APIs

Support streaming use cases and event-driven architecture

Security & Governance

Implement authentication and authorization (SSL, SASL, Kerberos) Configure ACLs (Access Control Lists) Ensure secure data transmission and compliance

Replication & High Availability

Ensure fault tolerance and data replication across brokers Configure multi-cluster setups (MirrorMaker, Cluster Linking) Handle disaster recovery strategies

Troubleshooting & Support

Diagnose and fix issues like:

Broker failures Consumer lag Performance bottlenecks

Analyze logs and perform root cause analysis

Automation & DevOps

Automate deployments using Ansible, Terraform, or scripts Manage CI/CD pipelines for Kafka environments Version control using Git