Configuration Model

Service Bridge APIs allow users to onboard clusters and configure the service mesh deployed in these clusters. Service Bridge enables you to create a hierarchy of namespaces across various clusters to simplify management of a multi-tenant mesh. At the root of the hierarchy is a Workspace composed of one or more namespaces across clusters. The namespaces in a Workspace are further subdivided into three types of groups, namely Gateway, Traffic, and Security group. Each group configures a specific type of functionality in the mesh.

Within each group, Service Bridge offers two flavors of APIs to configure the mesh: bridged and direct. Bridged mode APIs are minimalistic macro APIs that allow you to configure all aspects of a component, such as a gateway, with one single API. Under the hood, these APIs are translated into Istio APIs. Direct mode APIs allow you to configure the mesh using Istio's one or more modular APIs.

The rest of this document describes these concepts in greater detail.

note

This TSB version supports direct mode. Bridged mode is supported only for specific scenarios such as tier 1 gateway. The content in the below section will call out support for bridged mode if applicable.

Object Names

Each object in Service Bridge has a name that is unique for that type of resource. There can be only one object of a given kind with a given name at a time. However, if you delete the object, you can make a new one with the same name. All object names must conform to the DNS label standard as defined in RFC 1123, i.e., the following rules apply to object names:

• can contain at most 63 characters
• can contain only lowercase alphanumeric characters or '-'
• must start with an alphanumeric character
• must end with an alphanumeric character

Cluster

A Cluster object should be created for each Kubernetes cluster managed by Service Bridge. The object must be created before installing the Istio control plane on the cluster. The following example creates a cluster named c1 indicating that the cluster is deployed on a network vpc-01 corresponding to the AWS VPC where it resides.

cat <<EOF | tctl apply -f - <<
apiVersion: api.tsb.tetrate.io/v2
kind: Cluster
metadata:
  name: c1
  labels:
    env: uat-demo
spec:
  tokenTtl: "1h"
  network: vpc-01
  # uncomment this if this cluster is running a Tier1 Gateway
  # tier1Cluster: true
EOF

Workspace

Workspaces are at the root of the resource hierarchy in Service Bridge. A workspace is a collection of related namespaces across one or more clusters. For example, you could create a workspace for a team that owns the services in a set of namespaces. Configurations created within the workspace will not impact any other workspace in the system.

The following example creates a workspace w1 containing namespaces ns1 and ns2 namespaces across all clusters (current and future) in Service Bridge.

apiVersion: api.tsb.tetrate.io/v2
kind: Workspace
metadata:
  name: w1
  tenant: mycompany
spec:
  namespaceSelector:
    names:
      - '*/ns1'
      - '*/ns2'

Workspaces can "select" namespaces from a particular cluster as well (or even all namespaces in a cluster). The following example creates workspace w1 with ns1 namespace from the c1 cluster and all namespaces from the c2 cluster.

apiVersion: api.tsb.tetrate.io/v2
kind: Workspace
metadata:
  name: w1
  tenant: mycompany
spec:
  namespaceSelector:
    names:
      - 'c1/ns1'
      - 'c2/*'

note

Two workspaces cannot share a namespace. Ensure that the namespace selectors in each workspace objects do not overlap.

Gateway Group

Gateway Groups allow grouping the gateways in a subset of namespaces owned by its parent workspace. Gateway related configurations can then be applied on the group to control the behavior of these gateways. The group can be in one of two modes: BRIDGED and DIRECT, corresponding to the two API flavors described earlier.

note

There can be multiple gateway groups in a workspace as long as the namespaces in the gateway groups do not overlap.

The following example creates a gateway group for the gateways in ns1 and ns2 namespaces owned by its parent workspace w1 under tenant mycompany

apiVersion: gateway.tsb.tetrate.io/v2
kind: Group
metadata:
  name: g1
  workspace: w1
  tenant: mycompany
spec:
  namespaceSelector:
    names:
      - '*/ns1'
      - '*/ns2'
  configMode: DIRECT

It is possible to create a gateway group for namespaces in a specific cluster as long as the parent workspace owns those namespaces in that cluster. For example,

apiVersion: gateway.tsb.tetrate.io/v2
kind: Group
metadata:
  name: g1
  workspace: w1
  tenant: mycompany
spec:
  namespaceSelector:
    names:
      - 'c1/ns1' # pick ns1 namespace only from c1 cluster
      - '*/ns2'
  configMode: DIRECT

In the DIRECT mode, it is possible to directly attach Istio Networking v1beta1 APIs - VirtualService, and Gateway, and Istio Security v1beta1 APIs - RequestAuthentication, and AuthorizationPolicy to the gateway group. These configurations will be validated for correctness and conflict free operations and then pushed to the appropriate Istio control planes.

The following example declares a Gateway and a VirtualService for a specific workload in the ns1 namespace:

apiVersion: networking.istio.io/v1beta1
kind: Gateway
metadata:
  name: ingress
  namespace: ns1
  annotations:
    tsb.tetrate.io/tenant: mycompany
    tsb.tetrate.io/workspace: w1
    tsb.tetrate.io/gatewayGroup: g1
spec:
  selector:
    app: my-ingress-gateway
  servers:
    - port:
        number: 80
        name: http
        protocol: HTTP
      hosts:
        - uk.bookinfo.com
        - eu.bookinfo.com

and the associated VirtualService

apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  name: ingress-rule
  namespace: ns1
  annotations:
    tsb.tetrate.io/tenant: mycompany
    tsb.tetrate.io/workspace: w1
    tsb.tetrate.io/gatewayGroup: g1
spec:
  hosts:
    - uk.bookinfo.com
    - eu.bookinfo.com
  gateways:
    - ns1/ingress # Has to bind to the same gateway
  http:
    - route:
        - destination:
            port:
              number: 7777
            host: reviews.ns1.svc.cluster.local

The namespace where the Istio APIs are applied will need to be part of the parent gateway group. In addition, each API object will need to have annotations to indicate the tenant, workspace and the gateway group to which it belongs to.

Tier 1 Gateways

Tier1Gateway is a bridged only mode API that allows you to configure a workload to act as a gateway for the following purposes:

• to distribute external traffic (from outside the mesh) to one or more ingress gateways in other clusters.
• to forward mesh internal traffic from a cluster in one network to an ingress gateway in a target cluster in a different network over Istio mutual TLS. The use case for this if two clusters are restricted to have direct communication. In this case a Tier1Gateway can be deployed in a third cluster to act as a trusted intermediary between the other two clusters.

note

Tier 1 gateways cannot be used to route traffic to the same cluster. A cluster with tier 1 gateway cannot have any other gateways or workloads. The cluster object corresponding to the tier 1 gateway must have the tier1Cluster field set to true in order to enable the Tier 1 gateway functionality in that cluster.

The following example declares a Tier1Gateway running on pods with app: gateway labels in the ns1 namespace. The gateway exposes hosts movieinfo.com and bookinfo.com within the mesh. Traffic to movieinfo.com is load balanced across ingress gateways on all clusters in the vpc-02 network only, while traffic to bookinfo.com is load balanced across ingress gateways on any cluster that is reachable from the tier 1 gateway's cluster. Traffic from the source (sidecars) is expected to arrive on the tier 1 gateway over Istio mTLS.

apiVersion: gateway.tsb.tetrate.io/v2
kind: Tier1Gateway
metadata:
  name: tier1
  group: g1
  workspace: w1
  tenant: mycompany
spec:
  workloadSelector:
    namespace: ns1
    labels:
      app: gateway
  internalServers: # forwarding gateway
    - name: movieinfo
      hostname: movieinfo.com
      clusters:
        - network: vpc-02 # the target gateway IPs will be automatically determined
    - name: bookinfo
      hostname: bookinfo.com # route to any ingress gateway exposing bookinfo.com

The configuration above assumes that the gateway group g1 is in BRIDGED mode. In general, it is advisable to create a dedicated gateway group (and preferably a dedicated workspace) for the tier 1 clusters.

note

If two clusters have direct connectivity, a Tier1Gateway is not required.

Multi Cluster Communication

As mentioned above Tier1Gateways provides a solution for where two clusters cannot directly communicate with each other. In order to accomplish this, the rules for which cluster(s) can talk to each other must be set. Take an example where there are three clusters (C1,C2,C3) belonging to different networks (vpc-01, vpc-02 and vpc-03 respectively). In this example, vpc-01 can talk to both vpc-02 and vpc-03 (and vice versa), but vpc-02 and vpc-03 can not talk to each other. Use the following commands to setup the rule.

First retrieve the system settings JSON object:

curl -u admin:<password> -X GET -vk https://${TSBIP}:8443/v1/tenants/${TENANT}/settings/system | jq .  > values.json

Edit the values.json, setting smtpSettings to null and networkReachability to indicate your reachability requirements.

"systemSettings": {
  "smtpSettings": null,
  "networkSettings": {
    "networkReachability": {
      "vpc-01": "vpc-02,vpc-03",
      "vpc-02": "vpc-01",
      "vpc-03": "vpc-01"
    }
  }
}

Now update the modified object in the API:

 curl --request PUT -kv --url https://${TSBIP}:8443/v1/tenants/${TENANT}/settings/system     -u "admin:<password>"      --header 'accept: application/json'      --header 'content-type: application/json'      --data @./values.json

Traffic Groups

Traffic Groups allow grouping the sidecars in a set of namespaces owned by its parent workspace. Networking and routing related configurations can then be applied on the group to control the behavior of these sidecars. The group can be in one of two modes: BRIDGED and DIRECT. The semantics of configuring a Traffic group are same as that of the Gateway group, including the restriction that a namespace cannot belong to two traffic groups simultaneously.

note

A given namespace can be part of Traffic, Security, and Gateway groups as they configure different aspects of the mesh. However, the same namespace cannot be part of two groups of the same type (e.g., two Traffic groups).

The following example creates a traffic group for the sidecars in ns1 and ns2 namespaces owned by its parent workspace w1 under tenant mycompany.

apiVersion: traffic.tsb.tetrate.io/v2
kind: Group
metadata:
  name: t1
  workspace: w1
  tenant: mycompany
spec:
  namespaceSelector:
    names:
      - '*/ns1'
      - '*/ns2'
  configMode: DIRECT

In the DIRECT mode, it is possible to directly attach Istio APIs such as VirtualService, DestinationRule, and Sidecar to the traffic group. These configurations will be validated for correctness and conflict free operations and then pushed to the appropriate Istio control planes.

The following example declares a DestinationRule with two subsets, for the ratings service in the ns1 namespace:

apiVersion: networking.istio.io/v1beta1
kind: DestinationRule
metadata:
  name: ratings-subsets
  namespace: ns1
  annotations:
    tsb.tetrate.io/tenant: mycompany
    tsb.tetrate.io/workspace: w1
    tsb.tetrate.io/trafficGroup: t1
spec:
  host: ratings.ns1.svc.cluster.local
  subsets:
    - name: stableversion
      labels:
        app: ratings
        env: prod
    - name: testversion
      labels:
        app: ratings
        env: uat

The namespace where the Istio APIs are applied will need to be part of the parent traffic group. In addition, each API object will need to have annotations to indicate the tenant, workspace and the traffic group to which it belongs to.

Security Groups

Security Groups allow grouping the sidecars in a set of namespaces owned by its parent workspace. Security related configurations can then be applied on the group to control the behavior of these sidecars. The group can be in one of two modes: BRIDGED and DIRECT. The semantics of configuring a Security group are same as that of the Gateway and Traffic groups, including the restriction that a namespace cannot belong to two security groups simultaneously.

The following example creates a security group for the sidecars in ns1 and ns2 namespaces owned by its parent workspace w1 under tenant mycompany

apiVersion: security.tsb.tetrate.io/v2
kind: Group
metadata:
  name: t1
  workspace: w1
  tenant: mycompany
spec:
  namespaceSelector:
    names:
      - '*/ns1'
      - '*/ns2'
  configMode: DIRECT

In the DIRECT mode, it is possible to directly attach Istio Security v1beta1 APIs - PeerAuthentication, and AuthorizationPolicy to the security group. These configurations will be validated for correctness and conflict free operations and then pushed to the appropriate Istio control planes.

The following example declares a PeerAuthentication policy for a specific workload in the ns1 namespace:

apiVersion: security.istio.io/v1beta1
kind: PeerAuthentication
metadata:
  name: workload-mtls-disable
  namespace: ns1
  annotations:
    tsb.tetrate.io/tenant: mycompany
    tsb.tetrate.io/workspace: w1
    tsb.tetrate.io/securityGroup: t1
spec:
  selector:
    matchLabels:
      app: reviews
  mtls:
    mode: DISABLE

Direct mode resources support

The namespace where the Istio APIs are applied will need to be part of the parent security group. In addition, each API object will need to have annotations to indicate the tenant, workspace and the security group to which it belongs to.

Below there is a table showing you the Istio object kinds that are allowed in each of the TSB groups types.

Group	Allowed Istio API resource
GatewayGroup	VirtualService, Gateway, RequestAuthentication, AuthorizationPolicy
TraffficGroup	Sidecar, VirtualService, DestinationRule
SecurityGroup	PeerAuthentication, AuthorizationPolicy

Any Istio object type not listed in the table above like ServiceEntry and EnvoyFilter need to be applied using kubectl.