Fronting legacy services with Kubernetes

There are many benefits to Kubernetes but what’s not discussed so often is how to migrate your services from their legacy hosting to their new home in Kubernetes. Specifically, I’m looking at the case where you have a single server or a single public IP address and you want to run your services on that server with a mixture of legacy hosting and Kubernetes – either permanently or as part of a migration process.

Let’s suppose you are running an application like ownCloud in a standard way, with Apache httpd bound to ports 80 and 443, with port 80 redirecting to port 443 to force HTTPS/SSL. This is how the simplified config might look:

# /etc/httpd/conf.d/owncloud.conf

<VirtualHost *:80>
  ServerName owncloud.example.com

  DocumentRoot "/var/www/html/owncloud"

  # Redirect non-SSL traffic to SSL site
  RewriteEngine On
  RewriteCond %{HTTPS} off
  RewriteRule (.*) https://%{HTTP_HOST}%{REQUEST_URI}

</VirtualHost>

<VirtualHost *:443>
  ServerName owncloud.example.com

  DocumentRoot "/var/www/html/owncloud"

  ## SSL directives
  SSLEngine on
  SSLCertificateFile      /etc/letsencrypt/live/cert.pem
  SSLCertificateKeyFile   /etc/letsencrypt/live/privkey.pem
  SSLCertificateChainFile /etc/letsencrypt/live/chain.pem
  SSLCACertificatePath    /etc/pki/tls/certs

</VirtualHost>

Now suppose you want to add some new services in a one-node Kubernetes solution like MicroK8s. When you add your Ingress resource to start serving your applications, it will complain because it wants to bind to ports 80 and 443, but they are already reserved by your legacy Apache installation.

The neatest solution is to run your legacy application on a high port, without SSL, thus freeing up 80 and 443. Then set up your Kubernetes Ingress and let it bind to 80 and 443, terminate SSL for your legacy application, and proxy onwards to your application without SSL. You’ll be able to add other Kubernetes Service resources on the same Ingress on the same ports with ease – like Apache’s name-based virtual hosting.

Let’s have a look at the revised Apache config for ownCloud. Notice the Listen directive to bind to an arbitrary high port, and the lack of any SSL directives:

# /etc/httpd/conf.d/owncloud.conf

Listen 5678
<VirtualHost *:5678>
  ServerName owncloud.example.com

  DocumentRoot "/var/www/html/owncloud"

</VirtualHost>

Now we must consider how the Kubernetes infrastructure will look. The typical pattern is to use a Service resource to identify where the application is running, and an Ingress resource to expose the Service to the outside world.

Service resources are usually designed to point an applications running inside a Kubernetes cluster, but by setting the type to ExternalName, we can tell Kubernetes that our legacy service is running on localhost. You could consider an ExternalName type Service to be analogous to a DNS CNAME record.

Here’s how we configure it. Note that we don’t yet specify the port:

kind: Service
apiVersion: v1
metadata:
  name: owncloud
spec:
  type: ExternalName
  externalName: localhost

Now that Kubernetes knows it should look on localhost for your legacy ownCloud application, we need to configure the way it will be presented to the outside world. To begin with, we will set up a dumb proxy without SSL. All the relevant bits are in the spec section, which specifies the domain that the app should be served on, and then specifies the Service resource we created earlier, along with the port number.

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: owncloud
  annotations:
    kubernetes.io/ingress.class: "nginx"
spec:
  rules:
  - host: owncloud.example.com
    http:
      paths:
      - path: /
        backend:
          serviceName: owncloud
          servicePort: 5678

For bonus points, we can use cert-manager and Let’s Encrypt to add SSL, and fully automate the process of issuing SSL certificates. You will need to configure cert-manager in advance – this is beyond the scope of this blog post but there are good docs online. This revised Ingress config is the same as the one above, but with a few extra lines:

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: owncloud
  annotations:
    kubernetes.io/ingress.class: "nginx"
    cert-manager.io/issuer: "letsencrypt-prod"
spec:
  tls:
  - hosts:
    - owncloud.example.com
    secretName: owncloud-tls
  rules:
  - host: owncloud.example.com
    http:
      paths:
      - path: /
        backend:
          serviceName: owncloud
          servicePort: 5678

And that’s it! You can verify the config with the kubectl command:

[jonathan@zeus ~]$ kubectl get service
NAME                   TYPE           CLUSTER-IP      EXTERNAL-IP   PORT(S)   AGE
default-http-backend   ClusterIP      10.152.183.41   <none>        80/TCP    87d
kubernetes             ClusterIP      10.152.183.1    <none>        443/TCP   98d
owncloud               ExternalName   <none>          localhost     <none>    87d

[jonathan@zeus ~]$ kubectl get ingress
NAME       HOSTS                  ADDRESS     PORTS     AGE
owncloud   owncloud.example.com   127.0.0.1   80, 443   87d

Now your legacy ownCloud service is available at owncloud.example.com but fronted by Kubernetes, leaving you free to install as many other services in Kubernetes as you like without having to worry about port clashes.