KubeCon 2017 Europe

KubeCon was held in Berlin this spring. As a developer focused conference the tee-shirts where everywhere. Intel had a small booth where we had continuous demos of Secure Clear Containers and Kubernetes Federation. The big announcement was the release of Kubernetes 1.6 with its added features.

• Rolling updates with DaemonSets
• Beta release of kubernetes federation
• improved networking functionality and tools
• Improved scheduling
• Storage Improvements
• New adm tool for enterprise customers.

The biggest buzz around the show was default networking, storage and security. Typically Kubernetes chooses configurability over convention, which leads to longer setup time and variability in deployments specify around networking and storage. Security is a hot topic/issue with all container technologies, not just kubernetes.

One of Kubernetes biggest complaints is it is hard to get up and running, especially around network configurations. With 1.6 some network aspects come configured out of the box. For example etcd comes installed and configured (Service Discovery), CNI is now integrated with CRI by default and a stand bridge plugin has been validated with the combination. This decreases the amount of time and variability in previous releases. These are welcomed changes in the distro.

Another big issue with Kubernetes and Containers in general is lack of support of storage. Kubernetes is taking a clue from OpenStack here
and are supporting more Software Defined Storage options. Kubernetes gives the ability to plugin to Ceph, Swift, Lustre and other basic Storage sub-systems. But they are not planning on supporting a storage solution themselves. The announcement at KubeCon was an increased focus on Persistent Volumes. It will be interesting to see how a focus in this area will change the community from compute focused to complete solution focused. Time will tell if it takes.

As I worked the booth for two days and attended sessions which were standing room only,  it was good to interact with developers and hear their problems and concerns about working in the data-center. There was interest in the Kubernetes Federation demo which was somewhat problematic, but gave plenty of talking points. The Secure Clear Containers got lots of traffic and buzz. Many of the conversations were around secure as it is still a major problem with containers in general. Everyone was looking for what was available in the security area.

On a personal note I got the opportunity to meet a long lost cousin from the Pulsipher/Pulsifer side of my family. He was excited to see another Pulsipher and thought he was the last of his family out there. It was fun to share family stories and he got to hear about our common Ancestor which came into the Americans in the 1640s. It was also a great technical contact as he works for Spotify and works as the Director of Security in their data center.

DWP

OSCON 2015 Day 1 - Kubernetes Training

The Kubernetes Training (k8s) was built on top of the understanding of Docker and running a simple webapp (NodeJS) in a container and accessing it through security.

This is the second blog in a series of blogs about Docker and Kubernetes training. The first blog covers Docker Containers and a tutorial on how to set things up in Google Compute Engine. Check it out here (Docker Training)

Key Concepts

One of the things I quickly learned was that managing a single Container in Docker is very easy, but I can see how managing several instances of the container can be difficult and managing hundreds of heterogeneous containers would be impossible.

This is where k8s comes in. It basically lets me manage clusters of containers running on different machines in my cloud. To start k8s has a whole new nomenclature

• Master - maintains the state of k8s server runtime, control entry point for nodes, pods, services
• Node - represents a resource (machine) that pods are provisioned on. It runs docker, etcd, and kubelet daemons.
• Pod - a collections of containers that run on a machine in a node. Way to group containers together. Good way to share storage volumes across multiple containers in the same pod.
• Service and Labels - Defines a logical set of pods that make up a service or application. Policies are set up to the containers in the pods can communicate with each other. Sets up IP/Port configurations for inter-service communication. Creates a kube-proxy to front end the Service for external access.
• Volume - Storage element served up from etcd. Can connect to AWS EBS, GCE Persistent disk, iSCSI volumem etc...
• Container - Docker Container.

Hands On Tutorial

After the session went over the concepts we dove right into getting everything up and running. This is when things began to get hard. Mostly because I was on a windows box and "kubectl" (Command to control the k8s master) did not run on my windows box. So I created another VM in the Google Cloud to run through my tutorial. That ran into problems with ssh keys and security so it took much longer than it should have. I was not the only one having problems at this point. So we threw together a quick diagram to see how the k8s installation worked with the docker installation I had set up from the previous session (Docker Training)  This diagram on the right shows how this all fits together.

Kubernetes introduces another command line tool "kubectl" This is accessible after loading components via the glcoud components command. The problem mentioned before is it is not supported on Windows without jumping through some hoops. So I had to create a linux VM instance to run the commands. But I had a hard time getting the right ssh keys on the VM to talk to my Google Cloud Engine Instance. The presenter of and his helpers were having a hard time getting everyone up and running on this step as well. After over an hour of trying to get this set up. about 70% of the people were able to create a k8s pod. "kubeclt create"

One thing that k8s gives developers is the ability to define higher level services with a simple script. They have chosen YAML for the description language to do this. Since YAML is quickly becoming a de facto standard for service description this was the logical choice. My first task was to create a multi-tierd web application. The tutorial walk me through a wordpress installation that had a mysql database, and the wordpress web application.

There are 4 YAML files that need to be created.

1. mysql pod - Describes the mysql node. Which volume to mount for the containers in the node.
2. mysql-service file - Describes the mysql-service. It contains the docker container description, ports to expose, volumes to access, user names and passwords.
3. wordpress pod file - Describes the wordpress node.
4. wordpress-service file - Describes the wordpress service. It contains the docker container description, ports, volumes and configuration information.

Some of the things to watch out for using k8s to define services:

• There is no way to reference secure passwords/usernames for authenticated services. These are stored as clear text in the YAML file descriptions.
• Don't even try this with Windows boxes or images. It just is not there yet.
• There is not a federated k8s Master. So if your master goes down, you have lost controller of your containers and volumes. They will still be running and consuming resources, with no way to control and monitor them.
• k8s really requires a higher level scheduler to set up kubelet nodes. Mesos is a good option for this.
• SSH key management is not as easy as it was with just a simple docker set up.

Darren

Tool box of Container Technology

This last weekend I started cleaning out my garage. I said I started, because my garage will never completely be cleaned out. But I did manage to clean up my tool box. I found lots of interesting things in my tool box. Some broken tools, some tools that I have no idea how they got there. I am sure my neighbors and friends probably are missing them.  I also found half started projects that just got thrown in their that don't matter anymore. Probably in a hurry to clean up the mess in the garage. Anyway. I started thinking about all of these tools in my tool box, and of course comparing them to the tools I use at work. They are not things I can touch like the hardware in my tool box, but they are invaluable tools that I can use to get jobs done. I quickly broke down the tools into categories and found a group of tools that I have downloaded, played with and am using everyday that fit in the "Container Technology" category. So I gathered those tools together and decided to write this blog to help me keep track of what container tool to use for what job. This is what I have so far. Please note this is not a complete list, but a good start I think.

Container Tools

• Docker - Container Definition and nice CLI to control Containers
• Docker Compose - Define applications that contain more than one container.
• Mesos - Meta-scheduler, Distribute Kernel
• Docker Swarm - Container Scheduler that can talk to MesosS
• Kubernetes - Container Scheduler that can schedule and control Containers.
• Marathon - Service based scheduler for containers. Plugin to Mesos
• Chronos - Temporal Based Scheduler for containers. Plugin to Mesos

This is how I see everything fitting together.

Docker (Production - 1.7.0)

This helps developers define how to configure a container. Starts with a base image and then allows you to run commands, expose ports, and copy data into the container through a simple configuration file. Docker also gives CLI and REST API to allow users to control the individual containers. It requires a Docker controller running on a physical or virtual machine.

Docker Compose (Beta - 1.7.0)

Gives developers the ability to define an application as a set of micro-services. It shows the dependencies between the micro-services (containers). Which ports they expose to each other, their startup order, and any shared resources(data) between the containers that make up the application.

Mesos (Production - 0.23.0)

"A distributed systems kernel" from the mesos web site.

Mesos describes itself as a distributed system kernel. It is responsible for picking physical compute nodes to run containers, jobs, micro-services, etc... I gets telemetry from the physical hardware and can determine which machines can best handle incoming jobs. It then provisions machines and executes job on those machines. Schedulers like Kubernetes, Marathon, Chronos, and Docker Swarm sit ontop of Mesos and act as the orchestrator for the containers running in the cloud.

Schedulers

Docker Swarm (Still in Alpha - 0.3.0 )

If you want to use a familiar Docker API you can use Docker Swarm with Mesos to control multiple containers on mutilple hosts. There is rumor it may also allow you to talk to Kubernetes in the future. Check out http://www.techrepublic.com/article/docker-and-mesos-like-peanut-butter-and-jelly/ for more information.

Kubernetes ( pre-production Beta - 0.19.0 )

• If you want to launch groups of containers (K8 Pods) co-scheduled and co-located together, sharing resources on the same machine.
• If you want to launch a service alongside one or more sidekick containers (e.g. log archiver, metrics monitor) that live next to the parent container.
• if you want to use the K8s label-based service-discovery, load-balancing, and replication control. Very cool stuff for managing large number of containers.
• If you want to manage groups of containers(K8 Pods).
• If you want to load balance in isolation (K8 Pods).

Marathon (Production - 0.8.2)

• If you want to launch applications that contain long running heterogeneous apps/services (Docker and Non-Docker).
• If you want to use Mesos attributes for constraint-based scheduling.
• If you want to use application groups and dependencies to launch, scale, or upgrade related services.
• If you want to use event driven health checks to automatically restart unhealthy services.
• If you want to integrate HAProxy or Consul for service discovery.
• If you want a nice web UI or REST API to launch and monitor apps.

Chronos (Production - 2.3.4)

• If you want to launch applications that contain short running heterogeneous apps/services (Docker and Non-Docker).
• If you want to schedule a remporal task to run at a specific time/schedule, just like cron.
• If you want to schedule a DAG workflow of dependent tasks.
• If you want a nice web UI or REST API to launch and monitor apps.
• If  you want to use a scheduler that was built from the start with Mesos in mind.

Ok. This is the start. I am sure many of you have more to add to the list. Please let me know what I am missing.

DWP.