19. September 2018

4 min

Scale your Spring Boot Application in Kubernetes

The default way to scale applications in Kubernetes (K8s) is by CPU usage. But not all problems are CPU based and scaling on other metrics like requests or latency is sometimes more suitable. In this blogpost I will explain how you can use any kind of metric to scale your application on K8s using Spring Boot and Prometheus.

Objective

At the end of this post we have a running Horizontal Pod Autoscale (HPA) that scales my SampleApp on Actuator metrics. To reach this goal we will deploy Prometheus, which is used to scrape all our Pods. After that we will deploy the k8s-prometheus-adapter, which enables K8s to use the provided metrics for scaling.

Architecture for scaling Kubernetes with custom metrics

Architecture Diagram showing the services needed to scale on custom metrics

Prerequisite

Before starting with deploying everything, I want to list and introduce the required and the recommended tools.

minikube or any other way to run Kubernetes locally
Any way to build a Spring Boot Application, I use maven
go, to get cfssl and other Tools that are really great
hey to generate some load and traffic
Finally jq to make the K8s API readable

After all the preparation we can start with the hand-on part

Sample Application Spring Boot

As we want to scale on Spring Boot Actuator metrics (at least for the sake of this demo), we need an application that provides these metrics to an endpoint. For this you have to copy the following dependencies into your pom.xml.

<!-- Prometheus, allows to publish metrics in Prometheus format -->
<dependency>
  <groupId>io.prometheus</groupId>
  <artifactId>simpleclient</artifactId>
  <version>0.5.0</version>
</dependency>

<!-- Spring Boot Metrics collector and management -->
<dependency>
  <groupId>io.micrometer</groupId>
  <artifactId>micrometer-registry-prometheus</artifactId>
  <version>1.0.6</version>
</dependency>

<!-- Provides basic Metrics like Health state -->
<dependency>
  <groupId>org.springframework.boot</groupId>
  <artifactId>spring-boot-starter-actuator</artifactId>
</dependency>

<groupId>io.prometheus</groupId>

<artifactId>simpleclient</artifactId>

</dependency>

<groupId>io.micrometer</groupId>

<artifactId>micrometer-registry-prometheus</artifactId>

</dependency>

<groupId>org.springframework.boot</groupId>

<artifactId>spring-boot-starter-actuator</artifactId>

</dependency>

Those dependencies add the Endpoint /actuator/prometheus, which publishes the actuator metrics in the Prometheus format.

Deploying everything to Kubernetes

Let’s get to the most interesting part – deploying everything to Kubernetes.

Namespace

First you will have to deploy the namespace custom-metrics. Most of the parts will run in this namespace, for example the Prometheus instance.

Prometheus

Prometheus can either be deployed via the prometheus-operator or (as I did) via some Kubernetes yml files (see here). To keep this easy, you don’t have to configure anything for Prometheus to get your deployed pods, those will be detected later via Kubernetes annotations. The given configuration will add some metrics for Kubernetes objects like cAdvisor.

k8s-prometheus-adapter

This is where the magic happens. To use the provided metrics and Prometheus query tools as a scaling target you have to provide the K8s API Endpoint custom.metrics.k8s.io. This will be done by the adapter. Let’s get this deployed.
I highly recommend reading the documentation for each of the following steps as I can’t go into all the nitty-gritty.

Generate certificates

Yes, you will have to generate some certificates. I used the gencerts.sh script from the Prometheus Operator. You will need to install cfssl and openssl (if not already done) for this.

Deploy Adapter

After creating the certificates you can deploy the k8s-prometheus-adapter with this yaml. This also provides one rule that the adapter will use to fetch metrics.

Configure Adapter

The configuration is provided as YAML within an ConfigMap (already deployed) in Kubernetes.

 rules:
 - seriesQuery: 'http_server_requests_seconds_count{job="kubernetes-pods", app="sample-app"}'
   seriesFilters: []
   resources:
     overrides:
       kubernetes_pod_name:
         resource: pod
       kubernetes_namespace:
         resource: namespace
   name:
     matches: http_server_requests_seconds_count
     as: requests_per_second
   metricsQuery: rate(http_server_requests_seconds_count{job="kubernetes-pods", app="sample-app", uri="/requests", <<.LabelMatchers>>}[5m])

rules:

- seriesQuery: 'http_server_requests_seconds_count{job="kubernetes-pods", app="sample-app"}'

seriesFilters: []

resources:

overrides:

kubernetes_pod_name:

resource: pod

kubernetes_namespace:

resource: namespace

name:

matches: http_server_requests_seconds_count

as: requests_per_second

metricsQuery: rate(http_server_requests_seconds_count{job="kubernetes-pods", app="sample-app", uri="/requests", <<.LabelMatchers>>}[5m])

This provides the metric http_server_requests to use within any HPA. The Adapter provides this metric for each Pod, mapped by the label kubernetes_pod_name. The value of this metric is generated by the following Prometheus query:

rate(http_server_requests_seconds_count{job="kubernetes-pods", app="sample-app", uri="/requests", kubernetes_pod_name="<Inserted Pod name>", kubernetes_namespace="<Inserted Namespace>"}[5m])

1	rate(http_server_requests_seconds_count{job="kubernetes-pods", app="sample-app", uri="/requests", kubernetes_pod_name="<Inserted Pod name>", kubernetes_namespace="<Inserted Namespace>"}[5m])

Sample App

To deploy the Sample App you need a Container Image and a deployment yaml. I will not describe how to create the image needed, as there are mulitple online resources available. For the deployment you can use just copy the following deployment.yaml and enter your image. Note that the important thing are the annotations. You need those so Prometheus scrapes your Actuator metrics from the pods.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: sample-app
  labels:
    app: sample-app
spec:
  replicas: 1
  selector:
    matchLabels:
      app: sample-app
  template:
    metadata:
      labels:
        app: sample-app
      annotations:
        prometheus.io/path: '/actuator/prometheus'
        prometheus.io/port: '8080'
        prometheus.io/scrape: 'true'
    spec:
      containers:
      - name: sample-app
        image: coschapoehler/scale-spring-boot-sample:0.0.1
        ports:
        - containerPort: 8080

apiVersion: apps/v1

kind: Deployment

metadata:

labels:

app: sample-app

spec:

replicas: 1

selector:

matchLabels:

app: sample-app

template:

metadata:

labels:

app: sample-app

annotations:

prometheus.io/path: '/actuator/prometheus'

prometheus.io/port: '8080'

prometheus.io/scrape: 'true'

spec:

containers:

- name: sample-app

image: coschapoehler/scale-spring-boot-sample:0.0.1

ports:

- containerPort: 8080

Test if everything works

Lets check if every deployed object works. First we check our Prometheus instance. It should run and in the web interface show a kubernetes-pods job scaping our deployed sample app.

Prometheus Interface showing scraped kubernetes Pods

If this works we can skip the check our Sample App deployment as Prometheus can only pick it up if everything works correctly. Next we check the adapter. For this, simply query the Kubernetes API:

kubectl get --raw "/apis/custom.metrics.k8s.io/v1beta1/" | jq

1	kubectl get --raw "/apis/custom.metrics.k8s.io/v1beta1/" \| jq

If the output looks like this, everything is ready to go.

k8s output showing the registered API endpoints

Output showing the registered API Endpoints for custom metrics

HPA on custom Metrics

Lets deploy the HPA and see, how Kubernetes autoscales our application.

apiVersion: autoscaling/v2beta1
kind: HorizontalPodAutoscaler
metadata:
  name: sample-app
  namespace: default
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: sample-app
  minReplicas: 1
  maxReplicas: 10
  metrics:
  - type: Pods
    pods:
      metricName: requests_per_second
      targetAverageValue: 500

apiVersion: autoscaling/v2beta1

kind: HorizontalPodAutoscaler

metadata:

namespace: default

spec:

scaleTargetRef:

apiVersion: apps/v1

kind: Deployment

minReplicas: 1

maxReplicas: 10

metrics:

- type: Pods

pods:

metricName: requests_per_second

targetAverageValue: 500

HPA Autoscaling based on custom metrics

Pitfalls that happened to me

Internal Server Error
My metricsQuery within the Config was invalid. You can see this by trying to query a metric using

kubectl get --raw "/apis/custom.metrics.k8s.io/v1beta1/namespaces/<namespace here>/pods/<podname here>/<metric name>" | jq

1

kubectl get --raw "/apis/custom.metrics.k8s.io/v1beta1/namespaces/<namespace here>/pods/<podname here>/<metric name>" | jq

If this returns an Internal Error, than most likely your config is not correct.
Spring Boot App not providing metrics
Most likely this is caused by not exposing the endpoints. In your application-properties (or yaml) dont forget ro expose at least the prometheus endpoint

management.endpoints.web.exposure.include=prometheus (or expose all endpoints, when working locally)

1

management.endpoints.web.exposure.include=prometheus (or expose all endpoints, when working locally)
HPA not picking up metrics
For whatever reason, at some point the deployed HPA did not pick up any metrics. To fix this is redeployed it and it worked, but I don’t think this is the best idea. Check the API Endpoints first to see that the value for metrics is correctly written. After that, check you HPAs events and see what the tell you. Most likely a redeploy will help, but if not it might be a problem with your configurations