Exposing services through UpCloud Load Balancer over HTTPS

Contributed by: Pekka Nurmi

Cloud Controller Manager for UpCloud

UKS has a built-in a Cloud Controller Manager that interfaces with the UpCloud API and manages other cloud resources, such as load balancers.

Note that if your Managed Load Balancer is managed by the UKS cluster, all Load Balancer configuration changes should be made through object annotations. See below for details and examples. All manual modifications done to the Load Balancer through Hub or API are overwritten.

Exposing services through Managed Load Balancer

Create a deployment and expose it to the public Internet over HTTP by running the following commands:

kubectl create deployment --image=ghcr.io/upcloudltd/hello hello-uks
kubectl expose deployment hello-uks --port=80 --target-port=80 --type=LoadBalancer
kubectl get svc -w

This process will take a few minutes as our system will create a new load balancer to handle the traffic.

You can verify that it works by running this simple command:

$ curl http://lb-231912371233.upcloudlb.com

If you want to expose the service over HTTPS instead run the following:

kubectl expose deployment hello-uks --port=443 --target-port=80 --type=LoadBalancer --name=hello-uks-https

We provision a TLS certificate automatically for the autogenerated Load Balancer hostname. Certificate bundle’s name consists of clusters ID and service name and it can be found from the Hub under certificates.

Adding alternative host names to TLS certificate

Alternative names, for example www.example.com, can be added to autogenerated TLS bundle after pointing DNS record of the alternative name to load balancer’s public hostname using CNAME record type. In this case following entry would be added to example.com DNS records:

www.example.com. IN CNAME lb-231912371233.upcloudlb.com.

Once DNS change is propagated, name www.example.com can be added to certificate bundle from the Hub. Additional certificate bundles can be also mapped to load balancer by customising the default configuration.

Customising Load Balancer configuration

Load balancers can be customised by setting the configuration as annotations.

service.beta.kubernetes.io/upcloud-load-balancer-config

This annotation defines the whole LB config in the following format:

{
    "name": "specify here you custom LB name to override service.name",
    "plan": "development", // https://developers.upcloud.com/1.3/17-managed-loadbalancer/#list-plans
     // Please NOTE that there are reserved labels keys: ccm_cluster_id, ccm_cluster_name, ccm_generated_name
     // If labels with those keys are specified below, they will be ignored and the controller will inject it's own value.
    "labels": [ // https://developers.upcloud.com/1.3/17-managed-loadbalancer/#service-labels-usage-examples
        {
          "key": "env",
          "value": "staging"
        },
        {
          "key": "foo",
          "value": "bar"
        }
    ],
    "frontends": [
        {
            "name": "must match port name in spec.ports",
            "mode": "http", // 'http' or 'tcp': when load balancer operating in tcp mode it acts as a layer 4 proxy. In http mode it acts as a layer 7 proxy.
            "port": 443, // Must match to service.spec.ports[x].port
            // Description: https://developers.upcloud.com/1.3/17-managed-loadbalancer/#create-rule
            "rules": [
              {
               "name": "example-rule-1",
               "priority": 100,
               "matchers": [
                {
                 "type": "path",
                 "match_path": {
                  "method": "exact",
                  "value": "/app"
                 }
                }
               ],
               "actions": [ // All actions except 'use_backend' supported
                {
                  "type": "tcp_reject",
                  "action_tcp_reject": {}
                }
               ]
              }
             ],

            // Description: https://developers.upcloud.com/1.3/17-managed-loadbalancer/#create-tls-config
            "tls_configs": [
                {
                    "name": "example-tls-config",
                    "certificate_bundle_uuid": "0aded5c1-c7a3-498a-b9c8-a871611c47a2"
                }
            ],

           // Description: https://developers.upcloud.com/1.3/17-managed-loadbalancer/#frontend-properties
            "properties": {
                "timeout_client": 5,
                "inbound_proxy_protocol": false
            }
        }
    ],
    "backends": [
        {
          "name": "must match port name in spec.ports",
          // you can't define "members" and "resolver" here as
          // backend members are injected automatically (worker nodes, with label selector)

          // Description: https://developers.upcloud.com/1.3/17-managed-loadbalancer/#backend-properties  
          "properties": {
                "timeout_server": 10,
                "timeout_tunnel": 3600,
                "outbound_proxy_protocol": "",
                "health_check_type": "http",
                "health_check_interval": 10,
                "health_check_fall": 5,
                "health_check_rise": 5,
                "health_check_url": "/health",
                "health_check_expected_status": 200,
                "sticky_session_cookie_name": "x-session"
            }
        }
    ]
}

service.beta.kubernetes.io/upcloud-load-balancer-node-selector

This annotation defines node label selector. These nodes will be added automatically as load-balancer backend members for each backend. The format is string representation of labels selector object (yaml or json notation supported):

matchLabels:
  kubernetes.io/os: linux
  topology.kubernetes.io/zone: fi-hel1
matchExpressions:
  - { key: node-role.kubernetes.io/control-plane, operator: DoesNotExist}
  - { key: node-role.kubernetes.io/master, operator: DoesNotExist}

If left empty, then all cluster worker nodes are set as backend members.

Deleting Load Balancer

In order to delete the managed Load Balancer just delete the service object that created it in your Kubernetes cluster.

Please note that:

Managed Load Balancer will not be deleted if you just delete the whole UKS cluster. You either have to delete all service objects before deleting the cluster, or manually delete all managed Load Balancers after deleting the cluster (via the UpCloud Control Panel or upctl.
There is no way to preserve managed Load Balancer IP address when deleting it. In general UpCloud does not recommend to rely on those IP addresses at all - instead you should use Load Balancers hostname (for example for CNAME DNS records).

Auto TLS provisioning

You can force automatic TLS certificate provision for load balancer (1 auto TLS per load balancer). The auto provisioned TLS certificate can be bound to any frontend. This feature can be of help if you want to expose HTTPS endpoint but you don’t have prepared upfront TLS certificates. In order to fulfill it, the following API is used: https://developers.upcloud.com/1.3/17-managed-loadbalancer/#create-dynamic-certificate-bundle (the certificate key type is ecdsa).

The auto TLS provision is supported only for frontends with mode set to http. To enable this feature, you need to provide specific frontend config in service.beta.kubernetes.io/upcloud-loadbalancer-config:

 "frontends": [
        {
            "name": "must match port name in spec.ports",
            "mode": "http", // 'http' 
            "port": 8443, // Any arbitrary port

            // Description: https://developers.upcloud.com/1.3/17-managed-loadbalancer/#create-tls-config
            "tls_configs": [
                {
                    "name": "needs-certificate" // special value that triggers logic for auto TLS provision
                }
            ],
        }
    ]

Special case for 443 port: If the frontend mode set to http and port set to 443 & tls_config is not set, then tls_config for this forntend will be automatically set to:

"tls_configs": [
  {
    "name": "needs-certificate"
  }
]

that triggers the auto TLS.

Examples

See the following:

Create a service with a production-grade load balancer plan with a custom name lb-svc-production-plan.yaml
Create a service with HTTP redirect to HTTPS and configured forwarded-for headers to applications can get and log actual IP addresses of connecting clients: lb-svc-ip-forwarded-for.yaml
Create a service with custom network configuration
Create a service with custom network configuration