Control Plane

ControlPlane resource exposes a set of configurations necessary to automatically install the Service Bridge control plane on a cluster. The installation API is an override API so any unset fields that aren't required will use sensible defaults.

Prior to creating the ControlPlane resource, a cluster needs to be created in the management plane. Control plane install scripts would create the following secrets in the Kubernetes namespace the control plane is deployed into. Make sure they exist:

• tsbd-jwt-token
• oap-token
• zipkin-token
• otel-token

If your Elasticsearch backend requires authentication, ensure you create the following secret:

• elastic-credentials

A minimal resource must have the container registry hub, telemetryStore, and managementPlane fields set.

apiVersion: install.tetrate.io/v1alpha1
kind: ControlPlane
metadata:
  name: controlplane
  namespace: istio-system
spec:
  hub: docker.io/tetrate
  telemetryStore:
    elastic:
      host: elastic
      port: 5678
  managementPlane:
    host: tsb.tetrate.io
    port: 8443
    clusterName: cluster
    tenant: default

To configure infrastructure specific settings such as resource limits in Kubernetes, set the relevant field in a component. Remember that the installation API is an override API so if these fields are unset the operator will use sensible defaults. Only a subset of Kubernetes configuration is available and only for individual components.

apiVersion: install.tetrate.io/v1alpha1
kind: ControlPlane
metadata:
  name: controlplane
  namespace: istio-system
spec:
  hub: docker.io/tetrate
  telemetryStore:
    elastic:
      host: elastic
      port: 5678
  managementPlane:
    host: tsb.tetrate.io
    port: 8443
    clusterName: cluster
    tenant: default
  components:
    collector:
      kubeSpec:
        resources:
          limits:
            memory: 500Mi
          requests:
            memory: 750Mi

ControlPlaneComponentSet

The set of components that make up the control plane. Use this to override application settings or Kubernetes settings for each individual component.

Field	Description	Validation Rule
collector	tetrateio.api.install.controlplane.v1alpha1.OpenTelemetryCollector	–
oap	tetrateio.api.install.controlplane.v1alpha1.Oap	–
zipkin	tetrateio.api.install.controlplane.v1alpha1.Zipkin	–
xcp	tetrateio.api.install.controlplane.v1alpha1.XCP	–
istio	tetrateio.api.install.controlplane.v1alpha1.Istio	–
rateLimitServer	tetrateio.api.install.controlplane.v1alpha1.RateLimitServer	–

ControlPlaneSpec

ControlPlaneSpec defines the desired installed state of control plane components. Specifying a minimal ControlPlaneSpec with hub, clusterName, and managementPlane set will create an installation with sensible defaults.

Field	Description	Validation Rule
hub	string REQUIRED TSB container hub path e.g. docker.io/tetrate.	string = {   min_len: 1 }
components	tetrateio.api.install.controlplane.v1alpha1.ControlPlaneComponentSet The set of components that make up the control plane. Use this to override settings for individual components. These components assume the following secrets are present: tsbd-token, oap-token, zipkin-token and otel-token.	–
managementPlane	tetrateio.api.install.controlplane.v1alpha1.ManagementPlaneSettings REQUIRED Configure the management plane to retrieve configuration from.	message = {   required: true }
meshExpansion	tetrateio.api.install.controlplane.v1alpha1.MeshExpansionSettings Configure mesh expansion to connect workloads external to Kubernetes to the mesh.	–
telemetryStore	tetrateio.api.install.controlplane.v1alpha1.ControlPlaneSpec.TelemetryStore REQUIRED Configure the store that TSB will use to persist application telemetry data.	message = {   required: true }
tier1Cluster	bool DEPRECATED**: This should not be set through Control plane API Instead use XCP's Cluster API. Indicates that this cluster is used for tier1 gateways. Tier one clusters can only contain tier 1 gateways. Non-tier1 clusters contain tier2 gateways but not tier 1.	–

TelemetryStore

Configure the store that TSB will use to persist application telemetry data. Select one of the TelemetryStore settings to see complete examples.

Field	Description	Validation Rule
elastic	tetrateio.api.install.controlplane.v1alpha1.ElasticSearchSettings	–

ElasticSearchSettings

Configure an Elasticsearch connection.

apiVersion: install.tetrate.io/v1alpha1
kind: ControlPlane
metadata:
  name: controlplane
  namespace: istio-system
spec:
  telemetryStore:
    elastic:
      host: elastic
      port: 5678
      protocol: https
      selfSigned: true
      version: 7

Field	Description	Validation Rule
host	string REQUIRED Elasticsearch host address (can be hostname or IP address).	string = {   address: true }
port	int32 REQUIRED Port Elasticsearch is listening on.	int32 = {   lte: 65535   gte: 1 }
protocol	tetrateio.api.install.controlplane.v1alpha1.ElasticSearchSettings.Protocol Protocol to communicate with Elasticsearch, defaults to https.	–
selfSigned	bool Use Self-Signed certificates. The Self-signed CA bundle and key must be in a secret called es-certs.	–
version	int32 Major version of the Elasticsearch cluster. Currently supported Elasticsearch major versions are 6 and 7	int32 = {   lte: 7   gte: 6 }

Protocol

The list of supported protocols to communicate with Elasticsearch.

Name	Number	Description
https	0
http	1

Istio

Mesh and Kubernetes settings for Istio.

Field	Description	Validation Rule
kubeSpec	tetrateio.api.install.kubernetes.KubernetesIstioComponentSpec Configure Kubernetes specific settings.	–
traceSamplingRate	double The percentage of traces Envoy will sample.	–
defaultWorkloadCertTTL	google.protobuf.Duration The default TTL of issued workload certificates. This sets both the default client-side CSR TTL and the default server-side issued certificate TTL.	–
maxWorkloadCertTTL	google.protobuf.Duration The maximum TTL that can be set in issued workload certificates.	–

ManagementPlaneSettings

Configure the management plane connection.

apiVersion: install.tetrate.io/v1alpha1
kind: ControlPlane
metadata:
  name: controlplane
  namespace: istio-system
spec:
  managementPlane:
    host: tsb.tetrate.io
    port: 8443
    clusterName: cluster

Field	Description	Validation Rule
host	string REQUIRED Management plane host address (can be hostname or IP address).	string = {   address: true }
port	int32 REQUIRED Port management plane is listening on.	int32 = {   lte: 65535   gte: 1 }
clusterName	string REQUIRED The unique identifier for this cluster that was created in the management plane.	string = {   min_len: 1 }

MeshExpansionSettings

Configure mesh expansion to connect workloads external to Kubernetes to the mesh.

To enable mesh expansion set it to an empty object:

apiVersion: install.tetrate.io/v1alpha1
kind: ControlPlane
metadata:
  name: controlplane
  namespace: istio-system
spec:
  meshExpansion: {}

If external workloads are unable to communicate with the default mesh expansion gateway via external IPs or hostnames, then you must specify the gateway that enables them to do so. This custom gateway must be configured to forward this communcation to the VM gateway service:

apiVersion: install.tetrate.io/v1alpha1
kind: ControlPlane
metadata:
  name: controlplane
  namespace: istio-system
spec:
  meshExpansion:
    customGateway:
      host: customgateway.tetrate.io
      port: 15443

Field	Description	Validation Rule
customGateway	tetrateio.api.install.controlplane.v1alpha1.MeshExpansionSettings.Gateway A custom mesh expansion gateway. This is required when the workload can't access the default gateway directly via the external IP or hostname.	–

Gateway

A custom mesh expansion gateway. This is required when the workload can't access the default gateway directly via the external IP or hostname.

Field	Description	Validation Rule
host	string REQUIRED Mesh expansion gateway host address (can be hostname or IP address).	string = {   address: true }
port	int32 REQUIRED Port mesh expansion gateway is listening on.	int32 = {   lte: 65535   gte: 1 }

Oap

Kubernetes settings for the OAP (SkyWalking) component.

Field	Description	Validation Rule
kubeSpec	tetrateio.api.install.kubernetes.KubernetesComponentSpec Configure Kubernetes specific settings	–

OpenTelemetryCollector

Kubernetes settings for the OpenTelemetryCollector component.

Field	Description	Validation Rule
kubeSpec	tetrateio.api.install.kubernetes.KubernetesComponentSpec Configure Kubernetes specific settings	–

RateLimitServer

Configuration settings for the RateLimit Server

Field	Description	Validation Rule
backend	tetrateio.api.install.controlplane.v1alpha1.RateLimitServer.Backend REQUIRED Configure Database backend settings. This field must be configured by the user.	message = {   required: true }
kubeSpec	tetrateio.api.install.kubernetes.KubernetesComponentSpec Configure Kubernetes specific settings.	–

Backend

External Backend Database types. This points to the backend used by the ratelimit server as a key/value store.

Field	Description	Validation Rule
redis	tetrateio.api.install.controlplane.v1alpha1.RateLimitServer.Backend.RedisSettings Settings for redis database backend.	–

RedisSettings

Configuration for the External Redis Backend Database

Field	Description	Validation Rule
uri	string REQUIRED The Redis Database URI. The value of the URI decides the scope for ratelimiting across multiple clusters.	string = {   min_bytes: 1 }

XCP

Kubernetes settings for the XCP component.

Field	Description	Validation Rule
kubeSpec	tetrateio.api.install.kubernetes.KubernetesComponentSpec Configure Kubernetes specific settings	–

Zipkin

Kubernetes settings for the Zipkin component.

Field	Description	Validation Rule
kubeSpec	tetrateio.api.install.kubernetes.KubernetesComponentSpec Configure Kubernetes specific settings	–