<em>Kafdrop is a web UI for viewing Kafka topics and browsing consumer groups.</em> The tool displays information such as brokers, topics, partitions, consumers, and lets you view messages.
This project is a reboot of Kafdrop 2.x, dragged kicking and screaming into the world of JDK 11+, Kafka 2.x, Helm and Kubernetes. It's a lightweight application that runs on Spring Boot and is dead-easy to configure, supporting SASL and TLS-secured brokers.
# Features
***View Kafka brokers** — topic and partition assignments, and controller status
***View topics** — partition count, replication status, and custom configuration
***Browse messages** — JSON, plain text and Avro encoding
***View consumer groups** — per-partition parked offsets, combined and per-partition lag
***Create new topics**
***View ACLs**
***Support for Azure Event Hubs**
# Requirements
* Java 11 or newer
* Kafka (version 0.11.0 or newer) or Azure Event Hubs
Optional, additional integration:
* Schema Registry
# Getting Started
You can run the Kafdrop JAR directly, via Docker, or in Kubernetes.
Optionally, configure a schema registry connection with:
```
--schemaregistry.connect=http://localhost:8081
```
Finally, a default message format (e.g. to deserialize Avro messages) can optionally be configured as follows:
```
--message.format=AVRO
```
Valid format values are `DEFAULT`, `AVRO`, `PROTOBUF`. This can also be configured at the topic level via dropdown when viewing messages.
## Configure Protobuf message type
In case of protobuf message type, the definition of a message could be compiled and transmitted using a descriptor file. Thus, in order for kafdrop to recognize the message, the application will need to access to the descriptor file(s). Kafdrop will allow user to select descriptor and well as specifying name of one of the message type provided by the descriptor at runtime.
To configure a folder with protobuf descriptor file(s) (.desc), follow:
```
--protobufdesc.directory=/var/protobuf_desc
```
If preferred the message type could be set to default as follows:
```
--message.format=PROTOBUF
```
## Running with Docker
Images are hosted at [hub.docker.com/r/obsidiandynamics/kafdrop](https://hub.docker.com/r/obsidiandynamics/kafdrop).
Launch container in background:
```sh
docker run -d--rm-p 9000:9000 \
-eKAFKA_BROKERCONNECT=<host:port,host:port> \
-eJVM_OPTS="-Xms32M -Xmx64M"\
-eSERVER_SERVLET_CONTEXTPATH="/"\
obsidiandynamics/kafdrop
```
Launch container in background with protobuff definitions:
```sh
docker run -d--rm-v <path_to_protobuff_descriptor_files>:/var/protobuf_desc -p 9000:9000 \
Then access the web UI at [http://localhost:9000](http://localhost:9000).
> **Hey there!** We hope you really like Kafdrop! Please take a moment to [⭐](https://github.com/obsidiandynamics/kafdrop)the repo or [Tweet](https://twitter.com/intent/tweet?url=https%3A%2F%2Fgithub.com%2Fobsidiandynamics%2Fkafdrop&text=Get%20Kafdrop%20%E2%80%94%20a%20web-based%20UI%20for%20viewing%20%23ApacheKafka%20topics%20and%20browsing%20consumers%20) about it.
For all Helm configuration options, have a peek into [chart/values.yaml](chart/values.yaml).
Replace `3.x.x` with the image tag of [obsidiandynamics/kafdrop](https://hub.docker.com/r/obsidiandynamics/kafdrop). Services will be bound on port 9000 by default (node port 30900).
**Note:** The context path _must_ begin with a slash.
Proxy to the Kubernetes cluster:
```sh
kubectl proxy
```
Navigate to [http://localhost:8001/api/v1/namespaces/default/services/http:kafdrop:9000/proxy](http://localhost:8001/api/v1/namespaces/default/services/http:kafdrop:9000/proxy).
### Protobuf support via helm chart:
To install with protobuf support, a "facility" option is provided for the deployment, to mount the descriptor files folder, as well as passing the required CMD arguments, via option _mountProtoDesc_.
After cloning the repository, building is just a matter of running a standard Maven build:
```sh
$ mvn clean package
```
The following command will generate a Docker image:
```sh
mvn assembly:single docker:build
```
## Docker Compose
There is a `docker-compose.yaml` file that bundles a Kafka/ZooKeeper instance with Kafdrop:
```sh
cd docker-compose/kafka-kafdrop
docker-compose up
```
# APIs
## JSON endpoints
Starting with version 2.0.0, Kafdrop offers a set of Kafka APIs that mirror the existing HTML views. Any existing endpoint can be returned as JSON by simply setting the `Accept: application/json` header. Some endpoints are JSON only:
*`/topic`: Returns a list of all topics.
## Swagger
To help document the Kafka APIs, Swagger has been included. The Swagger output is available by default at the following Kafdrop URL:
```
/v2/api-docs
```
This can be overridden with the following configuration:
Currently only the JSON endpoints are included in the Swagger output; the HTML views and Spring Boot debug endpoints are excluded.
You can disable Swagger output with the following configuration:
```
swagger.enabled=false
```
## CORS Headers
Starting in version 2.0.0, Kafdrop sets CORS headers for all endpoints. You can control the CORS header values with the following configurations:
```
cors.allowOrigins (default is *)
cors.allowMethods (default is GET,POST,PUT,DELETE)
cors.maxAge (default is 3600)
cors.allowCredentials (default is true)
cors.allowHeaders (default is Origin,Accept,X-Requested-With,Content-Type,Access-Control-Request-Method,Access-Control-Request-Headers,Authorization)
```
You can also disable CORS entirely with the following configuration:
```
cors.enabled=false
```
## Topic Configuration
By default, you could delete a topic. If you don't want this feature, you could disable it with:
```
--topic.deleteEnabled=false
```
By default, you could create a topic. If you don't want this feature, you could disable it with:
```
--topic.createEnabled=false
```
## Actuator
Health and info endpoints are available at the following path: `/actuator`
This can be overridden with the following configuration:
```
management.endpoints.web.base-path=<path>
```
# Guides
## Connecting to a Secure Broker
Kafdrop supports TLS (SSL) and SASL connections for [encryption and authentication](http://kafka.apache.org/090/documentation.html#security). This can be configured by providing a combination of the following files (placed into the Kafka root directory):
*`kafka.truststore.jks`: specifying the certificate for authenticating brokers, if TLS is enabled.
*`kafka.keystore.jks`: specifying the private key to authenticate the client to the broker, if mutual TLS authentication is required.
*`kafka.properties`: specifying the necessary configuration, including key/truststore passwords, cipher suites, enabled TLS protocol versions, username/password pairs, etc. When supplying the truststore and/or keystore files, the `ssl.truststore.location` and `ssl.keystore.location` properties will be assigned automatically.
### Using Docker
The three files above can be supplied to a Docker instance in base-64-encoded form via environment variables:
