Introduction
In recent posts I have shown how to decode AIS messages (what is AIS?), how to create a Spring Boot based AIS message decoder, and how to run this decoder in a Docker container.
Today we will see how to deploy this Docker container into a Kubernetes cluster and to call it once it is running there.
What is Kubernetes?
Explaining Kubernetes can be done both very short and very long.
For in in-depth explanation of Kubernetes, I recommend reading the introduction on kubernetes.io, the documentation on kubernetes.io, or perhaps best: The book “Kubernetes in Action” from Manning Publications.
The short version that I will provide here, is that Kubernetes is a home for Docker containers – both for test and production. Kubernetes is software which allows you to deploy and manage containerized applications in a standardized manner. As “Kubernetes in Action” puts it: “Kubernetes enables you to run your software applications on thousands of computer nodes as if all those nodes were a single, enormous computer”.
I find the standardized container deployment procedure and the management of contains (like scaling and detecting missing ones) especially important.
You can run a Kubernetes cluster on almost any set of machines such as Linux-based PC’s, on Raspberry Pis, virtualized machines (like on AWS or Azure), or even in turn-key hosted environments (read more).
In this post I will focus on setting up a Kubernetes cluster in the Amazon Web Services (AWS) cloud. AWS is a very complex and comprehensive platform. In relation to this post, think of AWS as a place where we will spin up a small number of virtualized standard Linux machines (“EC2 instances”) and a few supporting resources. Nothing else.
To control Kubernetes on AWS we will use kops. Kops is short for “kubernetes operations” and is a fantastic command line tool which we can use to control the operations/sysadmin-stuff of a Kubernetes cluster.
For kops to work, it needs the command line tools for AWS to be installed first. So that’s where we will start.
Installing AWS command line tools – awscli
To get running on AWS you must have an AWS account (create one). Limited use of AWS is free (“free tier”); if you go beyond “limited” you may have to pay for their services.
AWS can be controlled completely through their web interface; but for kops to control AWS we need the alternative AWS user interface installed: The AWS command line tool: awscli. To install awscli on a Mac we will need homebrew. Assuming homebrew is already installed, the installation of awscli is simply:
$ brew install awscliConfiguring an AWS IAM user
For kops to work with AWS we need to create or choose an “IAM user” on your AWS account and grant this user the necessary permissions. Even though awscli is already installed, I find it a lot easier to configure this role and its permissions using the AWS web UI. Whatever approach you choose be sure that you end up with an “IAM user” with following permissions granted:
AmazonEC2FullAccess
AmazonRoute53FullAccess
AmazonS3FullAccess
AmazonVPCFullAccess
Awscli initial configuration
With awscli installed and an IAM user prepared we need a one-time configuration to connect awscli to your account:
$ aws configure
AWS Access Key ID [None]: XXXXXXXXXXXXXXXXX
AWS Secret Access Key [None]: XXXXXXXXXXXXXXXXXXXXXXXXXX
Default region name [None]: eu-central-1
Default output format [None]:To know your values of the X’es – follow the AWS user guide.
After this – awscli is now configured and ready to use.
Installing Kubernetes Operations – kops
Next we need kops. To install kops using homebrew:
$ brew install kopsThis usually runs straight with no problems.
Preparing AWS for kops
Before we can use kops, we need to prepare the AWS surroundings.
First, we need to create an “AWS S3 bucket” where kops can store its on administrative information. AWS S3 is a key-value store on AWS, which you can think of as a database. We need to find a name for this bucket – unfortunately the S3 bucket namespace is shared by all users of the system, so you may need to be somewhat creative to find a name which is both descriptive and globally unique. Here I will try with “tbsalling-kops-state-store” and use awscli to create the bucket like this:
$ aws s3api create-bucket --bucket tbsalling-kops-state-store --region eu-central-1 --create-bucket-configuration LocationConstraint=eu-central-1
{
"Location": "http://tbsalling-kops-state-store.s3.amazonaws.com/"
}
That went well. Now kops requires a simple single configuration of this bucket:
$ aws s3api put-bucket-versioning --bucket tbsalling-kops-state-store --versioning-configuration Status=EnabledAWS S3 is now ready for kops.
Configuring kops
So far we have installed awscli, prepared an AWS IAM user, configured awscli, installed kops, and prepared AWS for kops. Before we reach the real fun, we still need to configure kops. First we need to create an SSH keypair which kops can use:
$ ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (~/.ssh/id_rsa): ~/.ssh/id_rsa_kops
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in ~/.ssh/id_rsa_kops.
Your public key has been saved in ~/.ssh/id_rsa_kops.pub.
The key fingerprint is:
SHA256:KJRCn5Ejjm5IB9DNc0DgsuzHNK6gOnwNPxUj8XHnJY0 tbsalling@Thomass-MBP
The key's randomart image is:
+---[RSA 2048]----+
|o.o=+o . |
| +o.*=.. . o o E |
|.o+.=+o o o E . |
|o+.+ . = . |
|=..o. o S |
|oo+... . |
|+. ++ . |
|+.o. + |
|+o. . |
+----[SHA256]-----+
Before using the kops cli we set two environment varibles:
$ export KOPS_CLUSTER_NAME=tbsalling-kops.k8s.local
$ export KOPS_STATE_STORE=s3://tbsalling-kops-state-store
Note that the KOPS_STATE_STORE must match the name you chose for the AWS S3 bucket above.
Creating a Kubernetes cluster on AWS using kops
Phew 🙂 Now our preparations are completed and we are ready to create our first Kubernetes cluster on AWS!
Let’s start out by creating a Kubernetes cluster with 3 nodes. We don’t need big machines for this – so I will choose will use AWS instance type “t2.micro”. I prefer the machines to run in Frankfurt, Germany – so I choose AWS availability zone “eu-central-1a”. With these few choices, I perform the following kops command:
$ kops create cluster --node-count=3 --master-size=t2.micro --node-size=t2.micro --zones=eu-central-1a --ssh-public-key=~/.ssh/id_rsa_kops.pub --name=${KOPS_CLUSTER_NAME} --yes
I1012 10:03:51.711946 33011 create_cluster.go:1351] Using SSH public key: /Users/tbsalling/.ssh/id_rsa_kops.pub
I1012 10:03:52.843113 33011 create_cluster.go:480] Inferred --cloud=aws from zone "eu-central-1a"
I1012 10:03:53.105492 33011 subnets.go:184] Assigned CIDR 172.20.32.0/19 to subnet eu-central-1a
I1012 10:03:55.371638 33011 apply_cluster.go:505] Gossip DNS: skipping DNS validation
I1012 10:03:55.833340 33011 executor.go:103] Tasks: 0 done / 77 total; 30 can run
I1012 10:03:56.757590 33011 vfs_castore.go:735] Issuing new certificate: "apiserver-aggregator-ca"
I1012 10:03:56.853766 33011 vfs_castore.go:735] Issuing new certificate: "ca"
I1012 10:03:57.417478 33011 executor.go:103] Tasks: 30 done / 77 total; 24 can run
I1012 10:03:58.212893 33011 vfs_castore.go:735] Issuing new certificate: "kubecfg"
I1012 10:03:58.267395 33011 vfs_castore.go:735] Issuing new certificate: "kube-scheduler"
I1012 10:03:58.297758 33011 vfs_castore.go:735] Issuing new certificate: "kops"
I1012 10:03:58.330616 33011 vfs_castore.go:735] Issuing new certificate: "kube-controller-manager"
I1012 10:03:58.374286 33011 vfs_castore.go:735] Issuing new certificate: "kube-proxy"
I1012 10:03:58.402156 33011 vfs_castore.go:735] Issuing new certificate: "kubelet-api"
I1012 10:03:58.402657 33011 vfs_castore.go:735] Issuing new certificate: "kubelet"
I1012 10:03:58.413877 33011 vfs_castore.go:735] Issuing new certificate: "apiserver-proxy-client"
I1012 10:03:58.429905 33011 vfs_castore.go:735] Issuing new certificate: "apiserver-aggregator"
I1012 10:03:59.393739 33011 executor.go:103] Tasks: 54 done / 77 total; 19 can run
I1012 10:04:00.068211 33011 launchconfiguration.go:380] waiting for IAM instance profile "masters.tbsalling-kops.k8s.local" to be ready
I1012 10:04:00.188953 33011 launchconfiguration.go:380] waiting for IAM instance profile "nodes.tbsalling-kops.k8s.local" to be ready
I1012 10:04:10.725027 33011 executor.go:103] Tasks: 73 done / 77 total; 3 can run
I1012 10:04:11.623059 33011 vfs_castore.go:735] Issuing new certificate: "master"
I1012 10:04:12.107182 33011 executor.go:103] Tasks: 76 done / 77 total; 1 can run
I1012 10:04:12.382870 33011 executor.go:103] Tasks: 77 done / 77 total; 0 can run
I1012 10:04:12.508591 33011 update_cluster.go:290] Exporting kubecfg for cluster
kops has set your kubectl context to tbsalling-kops.k8s.local
Cluster is starting. It should be ready in a few minutes.
Suggestions:
* validate cluster: kops validate cluster
* list nodes: kubectl get nodes --show-labels
* ssh to the master: ssh -i ~/.ssh/id_rsa admin@api.tbsalling-kops.k8s.local
* the admin user is specific to Debian. If not using Debian please use the appropriate user based on your OS.
* read about installing addons at: https://github.com/kubernetes/kops/blob/master/docs/addons.md.
It may take a while for the cluster to get completely up to speed, which you can see with kops validate cluster like this:
$ kops validate cluster
Validating cluster tbsalling-kops.k8s.local
INSTANCE GROUPS
NAME ROLE MACHINETYPE MIN MAX SUBNETS
master-eu-central-1a Master t2.micro 1 1 eu-central-1a
nodes Node t2.micro 3 3 eu-central-1a
NODE STATUS
NAME ROLE READY
VALIDATION ERRORS
KIND NAME MESSAGE
Machine i-0018ee6b015bcb65d machine "i-0018ee6b015bcb65d" has not yet joined cluster
Machine i-00d3dcb9425a2ae37 machine "i-00d3dcb9425a2ae37" has not yet joined cluster
Machine i-07a70e1ecd5b650ff machine "i-07a70e1ecd5b650ff" has not yet joined cluster
Machine i-0f0e79bcef34422cc machine "i-0f0e79bcef34422cc" has not yet joined cluster
Validation Failed
But after a while you will see something like this:
$ kops validate cluster
Validating cluster tbsalling-kops.k8s.local
INSTANCE GROUPS
NAME ROLE MACHINETYPE MIN MAX SUBNETS
master-eu-central-1a Master t2.micro 1 1 eu-central-1a
nodes Node t2.micro 3 3 eu-central-1a
NODE STATUS
NAME ROLE READY
ip-172-20-42-171.eu-central-1.compute.internal node True
ip-172-20-50-187.eu-central-1.compute.internal master True
ip-172-20-56-73.eu-central-1.compute.internal node True
ip-172-20-56-98.eu-central-1.compute.internal node True
Your cluster tbsalling-kops.k8s.local is ready
And this makes us happy: “Your cluster tbsalling-kops.k8s.local is ready” 🙂
So now we have a Kubernetes cluster with 1 master and 3 slave nodes running in Frankfurt!
Deploying AISdecoder to the AWS Kubernetes cluster
With the Kubernetes cluster up and running we will now deploy our AISdecoder Docker image to it.
The main tool to control a kubernetes cluster is kubectl. You can read a lot about this in the official Kubernetes tutorial. Can you also simply type kubectl at the command line to see its usage. For now, we can see that kubectl is working and correctly configured by kops like this:
$ kubectl get nodes
NAME STATUS ROLES AGE VERSION
ip-172-20-42-171.eu-central-1.compute.internal Ready node 1m v1.10.6
ip-172-20-50-187.eu-central-1.compute.internal Ready master 2m v1.10.6
ip-172-20-56-73.eu-central-1.compute.internal Ready node 1m v1.10.6
ip-172-20-56-98.eu-central-1.compute.internal Ready node 1m v1.10.6
To deploy our Docker image – which is already on Docker hub – we can use the kubectl run command and point it to our Docker image on Docker hub:
$ kubectl run aisdecoder --image=tbsalling/aisdecoder --port=8080
kubectl run --generator=deployment/apps.v1beta1 is DEPRECATED and will be removed in a future version. Use kubectl create instead.
deployment.apps/aisdecoder created
We can verify, that Kubernetes now indeed has created a deployment for us:
$ kubectl get deployments
NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE
aisdecoder 1 1 1 1 2m
This shows, that we desire 1 instance of aisdecoder to be running at any time, and that 1 instance of aisdecoder is actually running.
To be able to make a REST call to the aisdecoder application running inside this Kubernetes cluster we need to go through a “proxy”. This is because applications running in Kubernetes are on a private network which can only be reached through a proxy. To start such a proxy we type:
$ kubectl proxy
Starting to serve on 127.0.0.1:8001
Now it is time to make a call to our application. So in a second terminal window we will now call the aisdecoder application through the Kubernetes proxy:
$ curl -X POST http://localhost:8001/api/v1/namespaces/default/pods/aisdecoder-c4c46474-rzd2x/proxy/decode -H 'Content-Type: application/json' -d '[ "!AIVDM,1,1,,A,18UG;P0012G?Uq4EdHa=c;7@051@,0*53" ]'
Note the URL: “http://localhost:8001/api/v1/namespaces/default/pods/aisdecoder-c4c46474-rzd2x/proxy/decode”.
- The hostname and port number (localhost:8001) point to the Kubernetes proxy. So this is what came out of
kubectl proxy. - For now “/api/v1/namespaces/default/pods/” is just a static string which is appended.
- “aisdecoder-c4c46474-rzd2x” is the name of the pod containing our application – this is obtained from
kubectl get pods. - “/proxy” is a static string.
- And finally “/decode” is the URI defined by our own application.
Hence, we see the following output of the curl command:
[{"nmeaMessages":[{"rawMessage":"!AIVDM,1,1,,A,18UG;P0012G?Uq4EdHa=c;7@051@,0*53","valid":true,"sequenceNumber":null,"encodedPayload":"18UG;P0012G?Uq4EdHa=c;7@051@","fillBits":0,"numberOfFragments":1,"fragmentNumber":1,"radioChannelCode":"A","checksum":83,"messageType":"AIVDM"}],"metadata":{"source":"SRC1","received":"2018-10-17T08:26:43.756037Z","decoderVersion":"2.2.2","category":"AIS"},"repeatIndicator":0,"sourceMmsi":{"mmsi":576048000},"navigationStatus":"UnderwayUsingEngine","rateOfTurn":0,"speedOverGround":6.6,"positionAccuracy":false,"latitude":37.912167,"longitude":-122.42299,"courseOverGround":350.0,"trueHeading":355,"second":40,"specialManeuverIndicator":"NotAvailable","raimFlag":false,"communicationState":{"syncState":"UTCDirect","slotTimeout":1,"numberOfReceivedStations":null,"slotNumber":null,"utcHour":8,"utcMinute":20,"slotOffset":null},"messageType":"PositionReportClassAScheduled","transponderClass":"A","valid":true}]
Which is absolutely fantastic! Because now we have successfully called our own application running inside the Kubernetes cluster!
After all this work please take time to celebrate a bit before reading on :-).
In a later post I will show how to play more with our application in Kubernetes now that it is running there. But to keep this post at a reasonable length, it is now time for:
Cleaning up the cluster
Once we are finished playing we can stop and delete the application from our Kubernetes cluster:
$ $ kubectl delete deployment/aisdecoder
deployment.extensions "aisdecoder" deleted
Cleaning up on AWS
Also, since we are just playing around here we want to avoid unnecessary AWS hosting costs by freeing up the machines, that we just spun up.
One way to stop these machines would be to use the awscli. But since kops knows exactly what it has done and which AWS resources it has allocated (thanks to its S3 bucket), kops can conveniently clean up after itself and free all the AWS resources related to the Kubernetes cluster.
It can be done like this:
$ kops delete cluster --yes --name=${KOPS_CLUSTER_NAME}
... <a few minutes pass> ...
Not all resources deleted; waiting before reattempting deletion
vpc:vpc-01d6bcee76e847162
dhcp-options:dopt-05f07d75a03571a2b
subnet:subnet-0de74da43276b07e8
security-group:sg-0b173ee5b5afc79a4
route-table:rtb-06e4a6f7ef889bde2
subnet:subnet-0de74da43276b07e8 ok
security-group:sg-0b173ee5b5afc79a4 ok
route-table:rtb-06e4a6f7ef889bde2 ok
vpc:vpc-01d6bcee76e847162 ok
dhcp-options:dopt-05f07d75a03571a2b ok
Deleted kubectl config for tbsalling-kops.k8s.local
Deleted cluster: "tbsalling-kops.k8s.local"
It may take a few minutes for kops to get everything spun down; but when the command completes with “Deleted cluster: …” then all the AWS resources (except for the adminsitrative AWS S3 bucket) are freed and we are back where we started.
Conclusion
In this post, I showed how to install the command line interface for Amazon web services and also “kops” – kubernetes operations.
Then I showed how create a 3-node Kubernetes cluster on AWS, how to deploy and run a Docker image there, and finally how to clean it all up.
Important resources
- “Kubernetes in Action” by Marko LukÅ¡a, ISBN 9781617293726, Manning Publications, https://www.manning.com/books/kubernetes-in-action
- “Installing Kubernetes on AWS with kops”, https://kubernetes.io/docs/setup/custom-cloud/kops/.
- Official Kubernetes tutorials – https://kubernetes.io/docs/tutorials/; especially https://kubernetes.io/docs/tutorials/kubernetes-basics/deploy-app/deploy-intro/