Commit 060e6eff authored by Prashant Chitta's avatar Prashant Chitta
Browse files

Added apps and helm charts folders

parent 514731a7
apiVersion: helm.toolkit.fluxcd.io/v2beta1
kind: HelmRelease
metadata:
name: kafka-zookeeper-helmrelease
spec:
releaseName: kafka-zookeeper
chart:
spec:
chart: kafka-zookeeper
sourceRef:
kind: HelmRepository
name: bitnami
namespace: testing
version: "6.4.0"
interval: 1h0m0s
install:
remediation:
retries: 3
apiVersion: source.toolkit.fluxcd.io/v1beta1
kind: HelmRepository
metadata:
name: bitnami
namespace: testing
spec:
interval: 30m
url: https://charts.bitnami.com/bitnami
apiVersion: helm.toolkit.fluxcd.io/v2beta1
kind: HelmRelease
metadata:
name: kafka-helmrelease
spec:
releaseName: kafka
chart:
spec:
chart: kafka
sourceRef:
kind: HelmRepository
name: bitnami
namespace: testing
version: "12.7.3"
interval: 1h0m0s
install:
remediation:
retries: 3
apiVersion: source.toolkit.fluxcd.io/v1beta1
kind: HelmRepository
metadata:
name: bitnami
namespace: testing
spec:
interval: 30m
url: https://charts.bitnami.com/bitnami
# Patterns to ignore when building packages.
# This supports shell glob matching, relative path matching, and
# negation (prefixed with !). Only one pattern per line.
.DS_Store
# Common VCS dirs
.git/
.gitignore
.bzr/
.bzrignore
.hg/
.hgignore
.svn/
# Common backup files
*.swp
*.bak
*.tmp
*~
# Various IDEs
.project
.idea/
*.tmproj
apiVersion: v1
appVersion: "1.0"
description: A Helm chart for Confluent Schema Registry on Kubernetes
name: cp-schema-registry
version: 0.1.0
\ No newline at end of file
# CP-Schema Registry Helm Chart
This chart bootstraps a deployment of a Confluent Schema Registry
## Prerequisites
* Kubernetes 1.9.2+
* Helm 2.8.2+
* A healthy and accessible Kafka Cluster
## Developing Environment
* [Pivotal Container Service (PKS)](https://pivotal.io/platform/pivotal-container-service)
* [Google Kubernetes Engine (GKE)](https://cloud.google.com/kubernetes-engine/)
## Docker Image Source
* [DockerHub -> ConfluentInc](https://hub.docker.com/u/confluentinc/)
## Installing the Chart
### Install along with cp-helm-charts
```console
git clone https://github.com/confluentinc/cp-helm-charts.git
helm install cp-helm-charts
```
To install with a specific name, you can do:
```console
helm install --name my-confluent cp-helm-charts
```
### Install with a existing cp-kafka release
```console
helm install --set kafka.bootstrapServers="PLAINTEXT://unhinged-robin-cp-kafka-headless:9092" cp-helm-charts/charts/cp-schema-registry
```
### Installed Components
You can use `helm status <release name>` to view all of the installed components.
For example:
```console{%raw}
$ helm status lolling-chinchilla
NAMESPACE: default
STATUS: DEPLOYED
RESOURCES:
==> v1beta2/Deployment
NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE
lolling-chinchilla-cp-schema-registry 1 1 1 0 1s
==> v1/Pod(related)
NAME READY STATUS RESTARTS AGE
lolling-chinchilla-cp-schema-registry-58f854bd47-jxrcj 0/1 ContainerCreating 0 1s
==> v1/Service
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
lolling-chinchilla-cp-schema-registry ClusterIP 10.19.245.11 <none> 8081/TCP 1s
==> v1/ConfigMap
NAME DATA AGE
lolling-chinchilla-cp-schema-registry-jmx-configmap 1 1s
```
There are
1. A [Deployment](https://kubernetes.io/docs/concepts/workloads/controllers/deployment/) `lolling-chinchilla-cp-schema-registry` which contains 1 Schema Registry [Pod](https://kubernetes.io/docs/concepts/workloads/pods/pod-overview/): `lolling-chinchilla-cp-schema-registry-58f854bd47-jxrcj`.
1. A [Service](https://kubernetes.io/docs/concepts/services-networking/service/) `lolling-chinchilla-cp-schema-registry` for clients to connect to Schema Registry.
1. A [ConfigMap](https://kubernetes.io/docs/tasks/configure-pod-container/configure-pod-configmap/) which contains configuration for Prometheus JMX Exporter.
## Configuration
You can specify each parameter using the `--set key=value[,key=value]` argument to `helm install`.
Alternatively, a YAML file that specifies the values for the parameters can be provided while installing the chart. For example,
```console
helm install --name my-schema-registry -f my-values.yaml ./cp-schema-registry
```
> **Tip**: A default [values.yaml](values.yaml) is provided
### Schema Registry Deployment
The configuration parameters in this section control the resources requested and utilized by the cp-schema-registry chart.
| Parameter | Description | Default |
| --------- | ----------- | ------- |
| `replicaCount` | The number of Schema Registry Servers. | `1` |
### Image
| Parameter | Description | Default |
| --------- | ----------- | ------- |
| `image` | Docker Image of Confluent Schema Registry. | `confluentinc/cp-schema-registry` |
| `imageTag` | Docker Image Tag of Confluent Schema Registry. | `6.0.1` |
| `imagePullPolicy` | Docker Image Tag of Confluent Schema Registry. | `IfNotPresent` |
| `imagePullSecrets` | Secrets to be used for private registries. | see [values.yaml](values.yaml) for details |
### Confluent Schema Registry Configuration
| Parameter | Description | Default |
| --------- | ----------- | ------- |
| `configurationOverrides` | Schema Registry [configuration](https://docs.confluent.io/current/schema-registry/docs/config.html) overrides in the dictionary format. | `{}` |
| `customEnv` | Custom environmental variables | `{}` |
### Port
| Parameter | Description | Default |
| --------- | ----------- | ------- |
| `servicePort` | The port on which the Schema Registry will be available and serving requests. | `8081` |
### Kafka
| Parameter | Description | Default |
| --------- | ----------- | ------- |
| `kafka.bootstrapServers` | Bootstrap Servers for Schema Registry | `""` |
### Confluent Schema Registry JVM Heap Options
| Parameter | Description | Default |
| --------- | ----------- | ------- |
| `heapOptions` | The JVM Heap Options for Confluent Schema Registry | `"-Xms512M -Xmx512M"` |
### Resources
| Parameter | Description | Default |
| --------- | ----------- | ------- |
| `resources.requests.cpu` | The amount of CPU to request. | see [values.yaml](values.yaml) for details |
| `resources.requests.memory` | The amount of memory to request. | see [values.yaml](values.yaml) for details |
| `resources.limits.cpu` | The upper limit CPU usage for a Schema Registry Pod. | see [values.yaml](values.yaml) for details |
| `resources.limits.memory` | The upper limit memory usage for a Schema Registry Pod. | see [values.yaml](values.yaml) for details |
### Annotations
| Parameter | Description | Default |
| --------- | ----------- | ------- |
| `podAnnotations` | Map of custom annotations to attach to the pod spec. | `{}` |
### JMX Configuration
| Parameter | Description | Default |
| --------- | ----------- | ------- |
| `jmx.port` | The jmx port which JMX style metrics are exposed. | `5555` |
### Prometheus JMX Exporter Configuration
| Parameter | Description | Default |
| --------- | ----------- | ------- |
| `prometheus.jmx.enabled` | Whether or not to install Prometheus JMX Exporter as a sidecar container and expose JMX metrics to Prometheus. | `true` |
| `prometheus.jmx.image` | Docker Image for Prometheus JMX Exporter container. | `solsson/kafka-prometheus-jmx-exporter@sha256` |
| `prometheus.jmx.imageTag` | Docker Image Tag for Prometheus JMX Exporter container. | `6f82e2b0464f50da8104acd7363fb9b995001ddff77d248379f8788e78946143` |
| `prometheus.jmx.imagePullPolicy` | Docker Image Pull Policy for Prometheus JMX Exporter container. | `IfNotPresent` |
| `prometheus.jmx.port` | JMX Exporter Port which exposes metrics in Prometheus format for scraping. | `5556` |
| `prometheus.jmx.resources` | JMX Exporter resources configuration. | see [values.yaml](values.yaml) for details |
### Deployment Topology
| Parameter | Description | Default |
| --------- | ----------- | ------- |
| `nodeSelector` | Dictionary containing key-value-pairs to match labels on nodes. When defined pods will only be scheduled on nodes, that have each of the indicated key-value pairs as labels. Further information can be found in the [Kubernetes documentation](https://kubernetes.io/docs/concepts/configuration/assign-pod-node/) | `{}`
| `tolerations`| Array containing taint references. When defined, pods can run on nodes, which would otherwise deny scheduling. Further information can be found in the [Kubernetes documentation](https://kubernetes.io/docs/concepts/configuration/taint-and-toleration/) | `{}`
This chart installs a Confluent Kafka Schema Registry
https://github.com/confluentinc/schema-registry
{{/* vim: set filetype=mustache: */}}
{{/*
Expand the name of the chart.
*/}}
{{- define "cp-schema-registry.name" -}}
{{- default .Chart.Name .Values.nameOverride | trunc 63 | trimSuffix "-" -}}
{{- end -}}
{{/*
Create a default fully qualified app name.
We truncate at 63 chars because some Kubernetes name fields are limited to this (by the DNS naming spec).
If release name contains chart name it will be used as a full name.
*/}}
{{- define "cp-schema-registry.fullname" -}}
{{- if .Values.fullnameOverride -}}
{{- .Values.fullnameOverride | trunc 63 | trimSuffix "-" -}}
{{- else -}}
{{- $name := default .Chart.Name .Values.nameOverride -}}
{{- if contains $name .Release.Name -}}
{{- .Release.Name | trunc 63 | trimSuffix "-" -}}
{{- else -}}
{{- printf "%s-%s" .Release.Name $name | trunc 63 | trimSuffix "-" -}}
{{- end -}}
{{- end -}}
{{- end -}}
{{/*
Create chart name and version as used by the chart label.
*/}}
{{- define "cp-schema-registry.chart" -}}
{{- printf "%s-%s" .Chart.Name .Chart.Version | replace "+" "_" | trunc 63 | trimSuffix "-" -}}
{{- end -}}
{{/*
Create a default fully qualified kafka headless name.
We truncate at 63 chars because some Kubernetes name fields are limited to this (by the DNS naming spec).
*/}}
{{- define "cp-kafka-rest.cp-kafka-headless.fullname" -}}
{{- $name := "cp-kafka-headless" -}}
{{- printf "%s-%s" .Release.Name $name | trunc 63 | trimSuffix "-" -}}
{{- end -}}
{{/*
Form the Kafka URL. If Kafka is installed as part of this chart, use k8s service discovery,
else use user-provided URL
*/}}
{{- define "cp-schema-registry.kafka.bootstrapServers" -}}
{{- if .Values.kafka.bootstrapServers -}}
{{- .Values.kafka.bootstrapServers -}}
{{- else -}}
{{- printf "PLAINTEXT://%s:9092" (include "cp-kafka-rest.cp-kafka-headless.fullname" .) -}}
{{- end -}}
{{- end -}}
{{/*
Default GroupId to Release Name but allow it to be overridden
*/}}
{{- define "cp-schema-registry.groupId" -}}
{{- if .Values.overrideGroupId -}}
{{- .Values.overrideGroupId -}}
{{- else -}}
{{- .Release.Name -}}
{{- end -}}
{{- end -}}
\ No newline at end of file
{{- if .Capabilities.APIVersions.Has "apps/v1" }}
apiVersion: apps/v1
{{- else }}
apiVersion: apps/v1beta2
{{- end }}
kind: Deployment
metadata:
name: {{ template "cp-schema-registry.fullname" . }}
labels:
app: {{ template "cp-schema-registry.name" . }}
chart: {{ template "cp-schema-registry.chart" . }}
release: {{ .Release.Name }}
heritage: {{ .Release.Service }}
spec:
replicas: {{ .Values.replicaCount }}
selector:
matchLabels:
app: {{ template "cp-schema-registry.name" . }}
release: {{ .Release.Name }}
template:
metadata:
labels:
app: {{ template "cp-schema-registry.name" . }}
release: {{ .Release.Name }}
{{- if or .Values.podAnnotations .Values.prometheus.jmx.enabled }}
annotations:
{{- range $key, $value := .Values.podAnnotations }}
{{ $key }}: {{ $value | quote }}
{{- end }}
{{- if .Values.prometheus.jmx.enabled }}
prometheus.io/scrape: "true"
prometheus.io/port: {{ .Values.prometheus.jmx.port | quote }}
{{- end }}
{{- end }}
spec:
containers:
{{- if .Values.prometheus.jmx.enabled }}
- name: prometheus-jmx-exporter
image: "{{ .Values.prometheus.jmx.image }}:{{ .Values.prometheus.jmx.imageTag }}"
imagePullPolicy: "{{ .Values.prometheus.jmx.imagePullPolicy }}"
command:
- java
- -XX:+UnlockExperimentalVMOptions
- -XX:+UseCGroupMemoryLimitForHeap
- -XX:MaxRAMFraction=1
- -XshowSettings:vm
- -jar
- jmx_prometheus_httpserver.jar
- {{ .Values.prometheus.jmx.port | quote }}
- /etc/jmx-schema-registry/jmx-schema-registry-prometheus.yml
ports:
- containerPort: {{ .Values.prometheus.jmx.port }}
resources:
{{ toYaml .Values.prometheus.jmx.resources | indent 12 }}
volumeMounts:
- name: jmx-config
mountPath: /etc/jmx-schema-registry
{{- end }}
- name: {{ template "cp-schema-registry.name" . }}-server
image: "{{ .Values.image }}:{{ .Values.imageTag }}"
imagePullPolicy: "{{ .Values.imagePullPolicy }}"
ports:
- name: schema-registry
containerPort: {{ .Values.servicePort }}
protocol: TCP
{{- if .Values.prometheus.jmx.enabled }}
- containerPort: {{ .Values.jmx.port }}
name: jmx
{{- end }}
resources:
{{ toYaml .Values.resources | indent 12 }}
env:
- name: SCHEMA_REGISTRY_HOST_NAME
valueFrom:
fieldRef:
fieldPath: status.podIP
- name: SCHEMA_REGISTRY_LISTENERS
value: http://0.0.0.0:{{ .Values.servicePort }}
- name: SCHEMA_REGISTRY_KAFKASTORE_BOOTSTRAP_SERVERS
value: {{ template "cp-schema-registry.kafka.bootstrapServers" . }}
- name: SCHEMA_REGISTRY_KAFKASTORE_GROUP_ID
value: {{ template "cp-schema-registry.groupId" . }}
- name: SCHEMA_REGISTRY_MASTER_ELIGIBILITY
value: "true"
- name: SCHEMA_REGISTRY_HEAP_OPTS
value: "{{ .Values.heapOptions }}"
{{ range $configName, $configValue := .Values.configurationOverrides }}
- name: SCHEMA_REGISTRY_{{ $configName | replace "." "_" | upper }}
value: {{ $configValue | quote }}
{{ end }}
{{- range $key, $value := .Values.customEnv }}
- name: {{ $key | quote }}
value: {{ $value | quote }}
{{- end }}
{{- if .Values.schemaRegistryOpts }}
- name: SCHEMA_REGISTRY_OPTS
value: "{{ .Values.schemaRegistryOpts }}"
{{- end }}
{{- if .Values.jmx.port }}
- name: JMX_PORT
value: "{{ .Values.jmx.port }}"
{{- end }}
{{- if .Values.imagePullSecrets }}
imagePullSecrets:
{{ toYaml .Values.imagePullSecrets | indent 8 }}
{{- end }}
volumes:
{{- if .Values.prometheus.jmx.enabled }}
- name: jmx-config
configMap:
name: {{ template "cp-schema-registry.fullname" . }}-jmx-configmap
{{- end }}
{{- if .Values.nodeSelector }}
nodeSelector:
{{ toYaml .Values.nodeSelector | indent 8 }}
{{- end }}
{{- if .Values.tolerations }}
tolerations:
{{ toYaml .Values.tolerations | indent 8 }}
{{- end }}
{{- if .Values.affinity }}
affinity:
{{ toYaml .Values.affinity | indent 8 }}
{{- end }}
{{- if and .Values.prometheus.jmx.enabled }}
apiVersion: v1
kind: ConfigMap
metadata:
name: {{ template "cp-schema-registry.fullname" . }}-jmx-configmap
labels:
app: {{ template "cp-schema-registry.name" . }}
chart: {{ template "cp-schema-registry.chart" . }}
release: {{ .Release.Name }}
heritage: {{ .Release.Service }}
data:
jmx-schema-registry-prometheus.yml: |+
jmxUrl: service:jmx:rmi:///jndi/rmi://localhost:{{ .Values.jmx.port }}/jmxrmi
lowercaseOutputName: true
lowercaseOutputLabelNames: true
ssl: false
whitelistObjectNames:
- kafka.schema.registry:type=jetty-metrics
- kafka.schema.registry:type=master-slave-role
- kafka.schema.registry:type=jersey-metrics
rules:
- pattern : 'kafka.schema.registry<type=jetty-metrics>([^:]+):'
name: "cp_kafka_schema_registry_jetty_metrics_$1"
- pattern : 'kafka.schema.registry<type=master-slave-role>([^:]+):'
name: "cp_kafka_schema_registry_master_slave_role"
- pattern : 'kafka.schema.registry<type=jersey-metrics>([^:]+):'
name: "cp_kafka_schema_registry_jersey_metrics_$1"
{{- end }}
\ No newline at end of file
apiVersion: v1
kind: Service
metadata:
name: {{ template "cp-schema-registry.fullname" . }}
labels:
app: {{ template "cp-schema-registry.name" . }}
chart: {{ template "cp-schema-registry.chart" . }}
release: {{ .Release.Name }}
heritage: {{ .Release.Service }}
spec:
ports:
- name: schema-registry
port: {{ .Values.servicePort }}
{{- if .Values.prometheus.jmx.enabled }}
- name: metrics
port: {{ .Values.prometheus.jmx.port }}
{{- end }}
selector:
app: {{ template "cp-schema-registry.name" . }}
release: {{ .Release.Name }}
# Default values for cp-schema-registry.
# This is a YAML-formatted file.
# Declare variables to be passed into your templates.
## ------------------------------------------------------
## Schema Registry
## ------------------------------------------------------
## Number of Scheme Registry Pod
replicaCount: 1
## Image Info
## ref: https://hub.docker.com/r/confluentinc/cp-schema-registry/
image: confluentinc/cp-schema-registry
imageTag: 6.0.1
## Specify a imagePullPolicy
## ref: http://kubernetes.io/docs/user-guide/images/#pre-pulling-images
imagePullPolicy: IfNotPresent
## Specify an array of imagePullSecrets.
## Secrets must be manually created in the namespace.
## ref: https://kubernetes.io/docs/concepts/containers/images/#specifying-imagepullsecrets-on-a-pod
imagePullSecrets:
## Schema Registry Settings Overrides
## Configuration Options can be found here: https://docs.confluent.io/current/schema-registry/docs/config.html
configurationOverrides: {}
## Additional env variables
customEnv: {}
## The port on which the Schema Registry will be available and serving requests
servicePort: 8081
## Schema registry JVM Heap Option
heapOptions: "-Xms512M -Xmx512M"
## You can list load balanced service endpoint, or list of all brokers (which is hard in K8s). e.g.:
## bootstrapServers: "PLAINTEXT://dozing-prawn-kafka-headless:9092"
## Charts uses Kafka Coordinator Master Election: https://docs.confluent.io/current/schema-registry/docs/design.html#kafka-coordinator-master-election
kafka:
bootstrapServers: ""
resources: {}
# We usually recommend not to specify default resources and to leave this as a conscious
# choice for the user. This also increases chances charts run on environments with little
# resources, such as Minikube. If you do want to specify resources, uncomment the following
# lines, adjust them as necessary, and remove the curly braces after 'resources:'.
# limits:
# cpu: 100m
# memory: 128Mi
# requests:
# cpu: 100m
# memory: 128Mi
## Custom pod annotations
podAnnotations: {}
## Node labels for pod assignment
## Ref: https://kubernetes.io/docs/concepts/configuration/assign-pod-node/
nodeSelector: {}
## Taints to tolerate on node assignment:
## Ref: https://kubernetes.io/docs/concepts/configuration/taint-and-toleration/
tolerations: []
## Pod scheduling constraints
## Ref: https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#affinity-and-anti-affinity
affinity: {}
## Monitoring
## Schema Registry JMX Settings
## ref: https://docs.confluent.io/current/schema-registry/docs/monitoring.html
jmx:
port: 5555
## Prometheus Exporter Configuration
## ref: https://prometheus.io/docs/instrumenting/exporters/
prometheus:
## JMX Exporter Configuration
## ref: https://github.com/prometheus/jmx_exporter
jmx:
enabled: true
image: solsson/kafka-prometheus-jmx-exporter@sha256
imageTag: 6f82e2b0464f50da8104acd7363fb9b995001ddff77d248379f8788e78946143
imagePullPolicy: IfNotPresent
port: 5556
resources: {}
Supports Markdown
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment