Kubernetes
https://github.com/kubernetes/minikube
minikube start
I. Introduction
1. What is Kubernetes
Kubernetes
Kubernetes is a container orchestration tool. Which means that is helps orchestrate the deployment of multiple containers and help manage its state and make sure the container is running smoothly. There are many nice feature built into kubernetes which we will go into detail in later sections. But let's just say if you want to launch a containerised solution for your web app you almost certainly want to use an orchestration tool such as Kubernetes.
Kubernetes setups
-Kubernetes should really be able to run anywhere
-but, there are more integration for certain Cloud Providers, like AWS & GCE
*things like Volumes and External Load Balancer work only with support Cloud Providers
-minikube to quickly spin up a local single machine with a Kubernetes cluster
-spin up cluster on AWS using kops
*this tool can be used to spin up a highly available production cluster
Minikube setup
-minikube is a tool that makes it easy to run Kuberenetes locally
-Minikube runs a single-node Kubernetes cluster inside a Linux VM
-it's aimed on users who want to just test it out or use it out or use if for development
-it cannot spin up a production cluster, it's a one node machine with no high availability
-It works on Windows, Linux, and MacOS
-You will need Virtualization Software installed to run minikube:
*example : VirtualBox
-you can downlad minikube from: https://github.com/kubernetes/minikube
-to lunch your cluster you just need to enter (in a shell/terminal/powershell): minikube start
minikube start
ls ~/.kube/
cat ~/.kube/config
kubectl run hello-minikube --image=k8s.gcr.io/echoserver:1.4 --port=8080
kubectl expose deployment hello-minikube --type=NodePort
minikube service hello-minikube --url
kubectl cluster-info
minikube service list
minikube stop
Docker client setup
Kubernetes cluster setup -tools
minikube/docker client/kops/kubeadm
-there are multiple tools to install a kubernetes cluster
-I showed you how to use minikube /docker client, to do local installs
-if you want a production cluster, you'll need different tooling
*minikube and docker client are greate for local setups, but not for real clusters
*kops and kubeadm are tools to spin up a production cluster
-On AWS, the best tool is kops
* at some point AWS EKS (hosted Kubernetes) will be avaible,at that point this will probably be the preferred option
-For other installs, or if you can't get kops to work, you can use kubeadm
*kubeadm is an alternative approach, kops is still recommended (on AWS) - you also have AWS inntegrations with kops automatically
Kubernetes Setup
Setting up Kubernetes on AWS using kops
-To setup Kubernetes on AWS, you can use a tools called kops
*kops stands for Kubenrnetes Operations
-the tool allows you to do production grade Kubernetes installations, upgrades and managment
*i will use this tool to start a Kubernetes cluster on AWS
-there is also a legacy tool called kube-up.sh
*this was a simple tool to bring up a cluster, but is now deprecated, it doesn't create a production ready environment
Kops
-Kops only works on Mac/Linux
-If you're on windows, you'll need to boot a virtual machine first
-you can use Vagrant to quickly boot up a linux box
-you can download Virtualbox from virtualbox.org Vagrant from vagrantup.com (you need both)
-once downloaded, to boot up a new linux VM, just type in cmd/powershell:
mkdir ubuntu
cd ubuntu
vagarnt init ubuntu/xenial64
vagrant up
vagsrant ssh-config
vagrant ssh
install kops
create domain and hosted zone on route53
create s3 bucket
kops create cluster --name=kubernetes.domain.com --state=s3://kops-state-rds --zones=eu-west-1a --node-count=2 --node-size=t2.micro --master-size=t2.micro --dns-zone=kubernetes.domain.com
kops update cluster kubernetes.domain.com --state=s3://kops-state-rds --yes
kubectl run hello-minikube --image=k8s.gcr.io/echoserver:1.4 --port=8080
kubectl expose deployment hello-minikube --type=NodePort
minikube service hello-minikube --url
kubectl cluster-info
kubectl get services (forwording to uniq port for every node except master)
kops delete cluster kubernetes.domain.com --state=s3://kops-state-rds --yes
2. Cloud / On - premise setup
3. Cluster Setup
4. Building Containers
Dockerfile
FROM node:4.6
WORKDIR /app
ADD . /app
RUN npm install
EXPOSE 3000
CMD npm start
index.js
var express = require('express');
var app = express();
app.get('/', function (req, res) {
res.send('Hello World!')
});
var server = app.listen(3000, function () {
var host = server.address(.address;
var port = server.address().port;
console.log('Example app listeining at http://%s:%s', host, port);
});
package.json
{
"name": "myapp",
"version": "0.0.1",
"private": true,
"scripts": {
"start": "node index.js"
},
"engines": {
"node": "^4.6.1"
},
"dependencies": {
"express": "^4.14.0"
}
}
5. Running your first app
-Let's run our newly built application on the new Kubernetes cluster
-before we can launch a container based on the image, we need to create a pod definitation
*a pod describes an application running on Kubernetes
*a pod can conatiain one or more tightly coupled containers, that make up the app
**those apps can easily communicate which each other their local port numbers
Create a pod
-create a file pod-helloworld.yml with the pod definition:
apiVersion: v1
kind: Pod
metadata:
name: nodehelloworld.example.com
labels:
app: helloworld
spec:
containers:
- name: k8s-demo
image: 212131316/nodeapp
ports:
- containerPort: 3000
-Use kubectl to create the pod on the kubernetes cluster:
kubectl create -f pod-helloworld.yml
Useful commands
kubectl get pod // get information about all running pods
kubectl describe pod <pod> // describe one pod
kubectl expose pod <pod> --port=444 --name=forntend // expose the port of a pod (creates a new service)
kubectl port-forward <pod> 8080 // port forward the exposed pod port to your local machine
kubectl attach <podname> -i // attach to the pod
kubectl exec <pod> -- command // execute a command on the pod
kubectl label pods <pod> mylabel=awesome // add a new label to a pod
kubectl run -i --tty busybox --image=busybox --restart=Never -- sh // run a shell in a pod - very useful for debugging
git clone git@github.com:wardviaene/kubernetes-course.git
kubectl get node
cd kubernetes-course/
cat first-app/helloworld.yml
apiVersion: v1
kind: Pod
metadata:
name: nodehelloworld.example.com
labels:
app: helloworld
spec:
containers:
- name: k8s-demo
image: wardviaene/k8s-demo
ports:
- name: nodejs-port
containerPort: 3000
kubectl create -f first-app/helloworld.yml
kubectl get pod
kubectl describe pod nodehelloworld.example.com
Name: nodehelloworld.example.com
Namespace: default
Node: minikube/10.0.2.15
Start Time: Sat, 24 Nov 2018 15:44:53 +0100
Labels: app=helloworld
Annotations: <none>
Status: Running
IP: 172.17.0.7
Containers:
k8s-demo:
Container ID: docker://4cc525f9aa7049f0c7752bbfd166d6b704cd68aecda413558ffbfb8cf53780f2
Image: wardviaene/k8s-demo
Image ID: docker-pullable://wardviaene/k8s-demo@sha256:2c050f462f5d0b3a6430e7869bcdfe6ac48a447a89da79a56d0ef61460c7ab9e
Port: 3000/TCP
Host Port: 0/TCP
State: Running
Started: Sat, 24 Nov 2018 15:45:50 +0100
Ready: True
Restart Count: 0
Environment: <none>
Mounts:
/var/run/secrets/kubernetes.io/serviceaccount from default-token-zhz87 (ro)
Conditions:
Type Status
Initialized True
Ready True
PodScheduled True
Volumes:
default-token-zhz87:
Type: Secret (a volume populated by a Secret)
SecretName: default-token-zhz87
Optional: false
QoS Class: BestEffort
Node-Selectors: <none>
Tolerations: node.kubernetes.io/not-ready:NoExecute for 300s
node.kubernetes.io/unreachable:NoExecute for 300s
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 2m50s default-scheduler Successfully assigned nodehelloworld.example.com to minikube
Normal SuccessfulMountVolume 2m50s kubelet, minikube MountVolume.SetUp succeeded for volume "default-token-zhz87"
Normal Pulling 2m49s kubelet, minikube pulling image "wardviaene/k8s-demo"
Normal Pulled 114s kubelet, minikube Successfully pulled image "wardviaene/k8s-demo"
Normal Created 113s kubelet, minikube Created container
Normal Started 113s kubelet, minikube Started container
kubectl port-forward nodehelloworld.example.com 8081:3000
curl localhost:8081
kubectl expose pod nodehelloworld.example.com --type=NodePort --name nodehelloworld-service
minikube service nodehelloworld-service --url
kubectl get service
Commands to debug
kubectl attach nodehelloworld.example.com
kubectl exec nodehelloworld.example.com -- ls /app
kubectl exec nodehelloworld.example.com -- touch /app/test.txt
kubectl exec nodehelloworld.example.com -- ls /app
kubectl get service
kubectl describe nodehelloworld-service
kubectl describe service nodehelloworld-service
Name: nodehelloworld-service
Namespace: default
Labels: app=helloworld
Annotations: <none>
Selector: app=helloworld
Type: NodePort
IP: 10.105.187.86
Port: <unset> 3000/TCP
TargetPort: 3000/TCP
NodePort: <unset> 31034/TCP
Endpoints: 172.17.0.7:3000
Session Affinity: None
External Traffic Policy: Cluster
Events: <none>
kubectl run -i --tty busybox --image=busybox --restart=Never -- sh
telnet 172.17.0.7 3000
GET /
Service with Load Balancer
-in a real world scenario, you need to be able to access the app from outside cluster
-on AWS, you can easily add an external Load Balancer
-this AWS Load Balancer will route the traffic to the correctt pod in Kubernetes
-there are other solutions for other cloud providers that don't have a Load Balancer
*your own haproxy/nginx load balancer in front of your cluster
*expose ports directly
cat first-app/helloworld.yml
apiVersion: v1
kind: Pod
metadata:
name: nodehelloworld.example.com
labels:
app: helloworld
spec:
containers:
- name: k8s-demo
image: wardviaene/k8s-demo
ports:
- name: nodejs-port
containerPort: 3000
cat first-app/helloworld-service.yml
apiVersion: v1
kind: Service
metadata:
name: helloworld-service
spec:
ports:
- port: 80
targetPort: nodejs-port
protocol: TCP
selector:
app: helloworld
type: LoadBalancer
kubectl create -f first-app/helloworld.yml
kubectl create -f first-app/helloworld-service.ym
kubectl describe service helloworld-service
6. Building Container Images
1. How does the AWS LoadBalancer routes traffic to the correct pod -The LoadBalancer uses a NodePort that is exposed on all non-master nodes
The LoadBalancer Service will also create a port that is open on every node, that can be used by the AWS LoadBalancer to connect. That port is owned by the Service, so that the traffic can be routed to the correct pod
2. The command "kubectl expose" is used for what purpose? - To create a Service object for an existing object
Kubectl expose creates a new service for a pod (or other kubernetes objects). A service allows you to connect to a pod from outside the cluster, using Service type NodePort or LoadBalancer
3. How do containers within a pod communicate? - Using 127.0.0.1/loalhost and the port number
Containers within a pod can communicate using the local port numbers
4. How many containers can you run in a pod - You can run multiple containers in one pod
II. Kubernetes Basics
1. Node Architecture
2. Scaling pods
-If your application is stateless you can horizontally scale it
*Stateless = your application does't have a state, it doesn't write any local files / keeps local sessions
*All traditional databases (MySQL, Postgres) are stateful, they have database files that can't be split over multiple instaces
-most web applications can be mase stateless
*session managment needs to be done outside the container
*any files that need to be saved can't be saved locally on the container
-Our example app is stateless, if the same app would run multiple times, it doesn't change state.
-For more information about best practices, have a look at 12factor.net
*or see my course: Learn DevOps: Continuously delivering better software /scaling apps on-premise and in the cloud
-later in this course I'll explain how to use volumes to still run stateful apps
*those stateful apps can't horizontally scale, but you can run them in a single container and vertically scale (allocate more CPU / Memory / Disk)
-Scaling in Kubernetes can be done using the Replication Controller
-The replicartion controller will ensure a specified number of pod replicas will run at all time
- a pods created with the repica controller will automatically be replaced if they fail, get deleted, or are terminated
-using the replication controller is also recommended if you just want to make sure 1 pod is always running, even after reboots
*you can then run a replication controller with just 1 replica
*this makes sure that the pod is always running
-to replicate our example app 2 times
apiVersion: v1
kind: ReplicationController
metadata:
name: helloworld-controller
spec:
replicas: 2
selector:
app: helloworld
template:
metadata:
labels:
app: helloworld
spec:
containers:
- name: k8s-demo
image: wardviaene/k8s-demo
ports:
- name: nodejs-port
containerPort: 3000
kubectl get node
kubectl create -f replication-controller/helloworld-repl-controller.yml
kubectl get pods
kubectl edit -f replication-controller/helloworld-repl-controller.yml
kubectl get pods
kubectl describe pod helloworld-controller-9dnbz
Name: helloworld-controller-9dnbz
Namespace: default
Node: minikube/10.0.2.15
Start Time: Sat, 24 Nov 2018 19:05:11 +0100
Labels: app=helloworld
Annotations: <none>
Status: Running
IP: 172.17.0.8
Controlled By: ReplicationController/helloworld-controller
Containers:
k8s-demo:
Container ID: docker://d150846754af213d2534d5376915b86cdb67c60f03e2092889eb98cadaf6e3ba
Image: wardviaene/k8s-demo
Image ID: docker-pullable://wardviaene/k8s-demo@sha256:2c050f462f5d0b3a6430e7869bcdfe6ac48a447a89da79a56d0ef61460c7ab9e
Port: 3000/TCP
Host Port: 0/TCP
State: Running
Started: Sat, 24 Nov 2018 19:05:15 +0100
Ready: True
Restart Count: 0
Environment: <none>
Mounts:
/var/run/secrets/kubernetes.io/serviceaccount from default-token-zhz87 (ro)
Conditions:
Type Status
Initialized True
Ready True
PodScheduled True
Volumes:
default-token-zhz87:
Type: Secret (a volume populated by a Secret)
SecretName: default-token-zhz87
Optional: false
QoS Class: BestEffort
Node-Selectors: <none>
Tolerations: node.kubernetes.io/not-ready:NoExecute for 300s
node.kubernetes.io/unreachable:NoExecute for 300s
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 23m default-scheduler Successfully assigned helloworld-controller-9dnbz to minikube
Normal SuccessfulMountVolume 23m kubelet, minikube MountVolume.SetUp succeeded for volume "default-token-zhz87"
Normal Pulling 23m kubelet, minikube pulling image "wardviaene/k8s-demo"
Normal Pulled 23m kubelet, minikube Successfully pulled image "wardviaene/k8s-demo"
Normal Created 23m kubelet, minikube Created container
Normal Started 23m kubelet, minikube Started container
kubectl delete pod helloworld-controller-9dnbz
kubectl get pods
kubectl scale --replicas=4 -f replication-controller/helloworld-repl-controller.yml
kubectl get rc
kubectl delete rc/helloworld-controller
3. Deployments
Replication Set
-Replica Set is the next-generation Replication Controller
-It supports a new selector that can do selection based on filtering according a set of values
*e.g. "environment" either "dev" or "qa"
*not only based on equality, like the Replication Controller
**e.g. "environment" == "dev"
-this Replica Set, rather than the Replication Controller, is used by the Deploymnet object
Deployments
-A deployment declaration in Kubernetes allows you to do app deployments and updates
-when using the deployment object, you define the state of your application
*Kubernetes will then make sure the clusters matches your desired state
-Just using the replication controller or replication set might be cumbersome to deploy apps
*The Deployment Object is easier to use and gives you more possibilities
-with a deployment object you . can:
*create a deployment (e.g. deploying an app)
*update a deployment (e.g. deploying a new version)
*do rolling updates (zero downtime deployments)
*roll back to a previous version
*pause /resume a deployment (e.g. to roll-out to only certain percentage)
-this is an example of a deployment:
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: helloworld-deployment
spec:
replicas: 3
template:
metadata:
labels:
app: helloworld
spec:
containers:
- name: k8s-demo
image: wardviaene/k8s-demo
ports:
- name: nodejs-port
containerPort: 3000
Commands:
kubectl get depoyments // get information on current deployments
kubectl get rs // get information about the replica sets
kubectl get pods --show-labels // get pods, and also show labels attached to those pods
kubectl rollout status deployment/helloworld-deployment //get deployment status
kubectl set image deployment/helloworld-deployment k8s-demo=k8s-demo:2 // run k8s-demo with the image label version 2
kubectl edit deployment/helloworld-deployment // edit the deployment object
kubectl rollout status deployment/helloworld-deployment // get the status of the rollout
kubectl rollout history deployment/helloworld-deployment // get the rollout history
kubectl rollout undo deployment/helloworld-deployment // rollback to previous version
kubectl rollout undo deployment/helloworld-deployment --to-version=n // rollback to any version version
kubectl get deployments
kubectl get rs
kubectl get pods
kubectl get pods --show-labels
kubectl rollout status deployment/helloworld-deployment
kubectl expose deployment helloworld-deployment --type=NodePort
kubectl get service
kubectl describe service helloworld-deployment
minikube service helloworld-deployment --url
kubectl set image deployment/helloworld-deployment k8s-demo=wardviaene/k8s-demo:2
kubectl rollout status deployment/helloworld-deployment
kubectl rollout history deployment/helloworld-deployment
kubectl create -f deployment/helloworld.yml --record
kubectl rollout undo deployment/helloworld-deployment
kubectl rollout history deployment/helloworld-deployment
kubectl edit deployment/helloworld-deployment
...
spec:
progressDeadlineSeconds: 600
replicas: 3
revisionHistoryLimit: 10
selector:
matchLabels:
app: helloworld
...
4. Services
Services
-Pods are very dynamic, they come and go on the Kubernetes cluster
*When using a Replication Controller, pods are terminated and created during scaling operations
*When using Deployments, when updating the image version, pods are terminated and new pods take the place of older pods
-That's why Pods should never be accessed directly, but always through a Service
-A service is the logical bridge between the "mortal" pods and other services or end-users
- when using the "kubectl expose" command earlier, you created a new Service for your pod, so it could be accessed externally
-creating a service will create an endpoint for your pod(s);
*a ClusterIP: a virtual IP address only reachable from within the cluster (this is the default)
*a NodePort: a port that is the same on each node that is also reachable externally
*A LoadBalancer: a LoadBalancer created by the cloud provider that will route external traffic to every node on the NodePort (ELB on AWS)
-the options just shown only allow you to create virtual IPs or ports
-there is also a possiblility to use DNS names
*ExternalName can provide a DNS name for the service
*e.g. for service discovery using DNS
*this only works when the DNS add-on is enabled
-I will discuss this later in a separate lecture
-this is an example of a Service definition (also created using kubectl expose):
apiVersion: v1
kind: Service
metadata:
name: helloworld-service
spec:
ports:
- port: 31001
nodePort: 31001
targetPort: nodejs-port
protocol: TCP
selector:
app: helloworld
type: NodePort
5. Labels
Labels:
-labels are key/value pairs that can be attached to objects
*labels are like tags in AWS or other cloud providers, used to tag resources
-you can label your objects, for instance your pod, following an organizational structure
*key: environment - value: dev / staging /qa / prod
*key: department - value: engineering / finace /marketing
-in our previous examples I already have been using labels to tag pods:
metadata:
name: nodehelloworld.example.com
labels:
app: helloworld
-labels are not unique and multiple labels can be added to one object
-once labels are attached to an object, you can use filters to narrow down results
* this is called Label Selectors
-using Label Selectors, you can use matching expressions to match labels
*for instance, a particular pod can only run on a node labeled with "environment" equals "development"
*more cmplex matching: "environment" in "development" or "qa"
Node Labels
-you can also use labels to tag nodes
-once nodes are tagged, you can use label selectors to let pods only run on specific nodes
-there are 2 steps required to run . a pod on a specific set of nodes:
*first you tag the node
*then you add a nodeSelector to your pod configuration
-First step, add a label or multiple labels to your nodes:
kubectl label nodes node1 hardware=high-spec
kubectl label nodes node2 hardware=low-spec
-Secondly, add a pod that uses those labels:
apiVersion: v1
kind: Pod
metadata:
name: nodehelloworld.example.com
labels:
app: helloworld
spec:
containers:
- name: k8s-demo
image: wardviaene/k8s-demo
ports:
- name: nodejs-port
containerPort: 3000
nodeSelector:
hardware: high-spec
NodeSelector using labels:
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: helloworld-deployment
spec:
replicas: 3
template:
metadata:
labels:
app: helloworld
spec:
containers:
- name: k8s-demo
image: wardviaene/k8s-demo
ports:
- name: nodejs-port
containerPort: 3000
nodeSelector:
hardware: high-spec
kubectl get nodes --show-labels
kubectl create -f deployment/helloworld-nodeselector.yml
kubectl get deployment
kubectl get pod
kubectl describe pod helloworld-deployment-6576b7749d-8n624
kubectl label nodes minikube hardware=high-spec
kubectl get nodes --show-labels
kubectl describe pod helloworld-deployment-6576b7749d-8n624
6. Healthchecks
Healthchecks
-it your application malfunctions, the pod and container can still be running, but the application might not work anymore
-to detect and resolve problems with your application, you can run health checks
-you can run 2 different type of health checks
*running a command in the container periodically
*periodic checks on a URL (HTTP)
-the typical production application behind a load balancer should always have health checks implemented in some way to ensure availability and resiliency of the app
7. ReadinessProbe
8. Pod State & LifeCycle
9. Secrets
10. WebUI
III. Advanced topics
1. Service auto-discovery
2. ConfigMap
3. Ingress
4. External DNS
5. Volumes
6. Pod Presets
7. StatefulSets
8. Deamon Sets
9. Monitoring
10. Autoscaling
11. Node Affinity
12. InterPod (Anti-)Affinity
13. Taints and Tolerations
14. Operators
IV. Administration
1. Master Services
2.Quotas and Limits
3. Namespaces
4. User Managmeent
5. RBAC
6. Networking
7. Node Maintenance
8. High Availability
9. TLS on ELB
V. Packing
1. Introduction to Helm
2. Creating Helm Charts
3. Helm Repository
4. Building & Deploying
VI
1. kubeadm
2. TLS Certificcates with cert -manager
VII. Logging
Elasticsearch (as SeatefullSet), fluentd, Kibana, LogTrail
Logging with fluentd
Logging:
-logging is important to show errors, information and debuging ata about application
-when you only run one app, it is pretty obvious how to look for the logs
* you'd just open the "app.log" file to see what's going on
*or if deployed as a pod with "kubectl logs"
-with kubernetes 1 application will be running as one or many pods
*Finding a error will be much more difficult: what pod do you have to look at?
- Up until now you might have been using "kubectl logs" to get the log details of a pod
-to get the correct information, you might have to look up:
*the pod name
*the containers names
-and run "kubectl logs" for every container in every pod that is running for an application
-note: you can use both "kubectl log" and "kubectl logs"
-the solution for this problem is to do Log Aggregation
-It's not Kuibernetes specific
*It's already applied for years, even with syslog (a standard for message logging since 1980s)
*Log Aggregation is nowadays often implemented with more modern tools
** the ELK Stack (ElasticSearch + Logstash + Kibana)
** several hosted services like loggly.com, papertrailapp.com
-Demo
*Fluentd (for log forwarding)
*ElasticSearch (for log indexing)
*Kibana (for visualisation)
*LogTrail (an easy to use UI to show logs)
*this solution can be easily customaized, you can create custom dashboard to show what is important for you
VIII. Authentication
OpenID Connect (OIDC), Auth0
IX. Authorization
Kubernetes RBAC
X. Packing
Helm
XI. The Job Resource
Job
XII. Scheduling
CronJob
XIII. Deploying on Kubernetes
Spinnaker
XIV. Microservices on Kuberenetes
Linkerd
XV. Federation
kubefed
XVI. Monitioring
Prometheus
Kubernetes
Kubernetes is a container orchestration tool. Which means that is helps orchestrate the deployment of multiple containers and help manage its state and make sure the container is running smoothly. There are many nice feature built into kubernetes which we will go into detail in later sections. But let's just say if you want to launch a containerised solution for your web app you almost certainly want to use an orchestration tool such as Kubernetes.
Master and nodes in Kubernetes
A kubernetes deployment in generally referred to as a kubernetes cluster.
cluster = master + nodes
master
-manages all cluster activities and your requests.
node
- is where the work happens, a worker machine or a worker node.
- A node is typically a Virtual Machine or VM.
Master components
a) Kube-Apiserver
-It is the component on the master the exposes the Kubernetes API. It is the front-end for the Kubernetes control plane. Basically when you issue commands to your cluster to perform an action for example deploy something, this is the component that receives your commands. You will see the use of commands in the practicals.
b) Kube-Scheduler
-it is the component on the master that watches newly created pods that have no node assigned , and selects a node for them to run on. It basically manages the resources available on on the kubernetes cluster and helps manages the launching of new pods (we will come back to pods later) to where there are available resources.
c) Kube-Controller Manager
-Component on the master that runs controllers. The controllers have many functions and we will touch on them in practicals but two major controllers you should know is the node controller (responsible for noticing when a node dies) and replication controller which is responsible for maintaining the correct number of pods.
Node Compoenents
a) Kubelet
-It is the primary node agent. The kubelet does the following:
*watches for pods that have been assigned to the node.
*it runs the pods containers via docker
*it interacts with Kube-Api in the master, to update status of each pod back to the cluster master.
b) Kube Proxy
-Kube-proxy enables the Kubernetes service abstraction by maintaining network rules on the host and performing connection forwarding.
c) Docker
-Docker needs to be installed on the node to actually run containers.
The nodes has to have a kubelet and docker installed for it to be considered a node.
Is it true that master and node are separate VMs? So you have one VM for the master and VMs for the worker nodes. Your answer from the diagram should be yes. It seems that way. However, this is actually been changed. Currently, many default settings will allow master and worker to be on the same VM.
This is actually preferred in mosrt cases because you can use the available resources that's not used on you master VM to launch pods as well. So in short a master can also be a worker node.
What defines a master is actually just the components mentioned earlier and same for the worker so on a single VM both the master and worker co-exists. for our practicals, this is what will be done. We will only launch a single VM with both master and worker node.
Practically speaking many of the components mentioned above for the master and Nodes are background processes and in most cases you will not be interacting with it directly. We will be mainly controlling the cluster in a high level manner by interacting with the Kubernetes API (served through kub-apiserver on the master) using the kubernetes command line interface kubectl. In the practical we will do everything using kubectl.
Pods
-Pods are the unit of measure in kubernetes. What do I mean by that? In kubernetes we do not talk about containers we talk about pods. Everything in Kubernetes revolve around the concept of Pods. A Pod encapsulates a container or multiple containers. Within the pod resources are shared such as a uniques network IP, storage resources and configurations.
-The most common way to setup a pod is the "one-container-per-pod" model, which is shown on the left. You only have one container in your pod and kubernetes will interact with this pod.
-The second way to setup a pod is with multiple containers. You want to have multiple containers if you have a tightly coupled application that needs to work together . Then the volume which is the storage unit will be shared among the two containers as well as the configs. Notice that the IP address is for the Pod and not for the individual containers.
Pod networking
-Very important concept when working with pods is that the Pods have a private network within. So containers within the same Pod can communicate freely with one another however, since it is a private network to access the outside network a proxy must be setup. We will go deeper into this topic when we reach the proxy section.
Services
-Pods which have a privatew network can commucate to the external internet only via a proxy. And the concept of services adds to that logic. A Kubernetes service is an abstraction on top of the selected Pods which allows the pods within that service to interact with external traffic. Although each Pod has a unique IP address, those IPs are not exposed outside the cluster without a Service.
Services are related to the selected Pods using the concept of labels.
-There are 4 VMs//nodes in the diagram. One for the master, the other three for the worker nodes. The pods have unique IP addresses linked to them. The top 3 pods which are distributed across two VMs have the same label, app = B. Now the service qhich is defined by the dotted line has s:app=B which means that this service relates to the pods with the label app=B. The service also has a unique IP address. This IP address is the proxy address which is where the external network will interact with and not the Pod IP addresses.
Notice also that that within the Mastewr Node there is two prcocesses running ine with s:app=B and one for s:app=A. This is called a label selector (which selects the labels). Service A is much the same with service B the difference is the number of pods and VMs.
-There are three major configuration options when using services and they are:
*ClusterIP
*NodePort
*Load Balancer
In the practical we will test each of these options and see the difference.
Services
a) ClusterIP
-Expose the Service on an internal IP in the cluster. This type makes the Service only reachable from within the cluster. So a service with this configuration will not be exposed to external traffic.
b) NodePort
-Expose the Service on the same port of each selected Node in the cluster. Makes a Service accessible from outside the cluster using <NodeIP>:<NodePort>.
-Using our previous example. If the NodePort Option was selected. We will be exposing two endpoints for service B instead of just one. One for each node. The NodeIP is the VM's external IP address and the port can be specified and is the same for both end points. So if we only have one VM there would be only one end point.
c) LoadBalancer
-This option is the most comonly used option. It exposes only one end point for our service and the user can only access this point and the subsequent load is distributed among all available pods in the service.
-Using our previous example, this showcases the option for LoadBalancer. The workload is distributed among all pods. We will see this in action in the pracrtical. But in short the load is balanced so every pod should have a similar % of workload and not leaving one overburdened.
Deployments
-imagine that you have finished creating docker images for a web app, an image for the frontend interface and an image for the backend data storage. Now as you know kubernetes manages and ochestrates containers so how do you launch your web app using kubernetes so that you have a working web app with both a frontend and a backend? This is where kubernetes deployments comein.
-the deployment instructs Kubernetes how to create and update instances of your application. Once you've created a Deployment, the Kubernetes master schedulkes mentioned application instances individual Nodes in the cluster.
-You have seen services before using the same diagram. Now I want to bring to you the idea that deployments and services in fact go hand in hand. You create a deployment with a label app=B and you use the same label selector for your services to deployments.
-In fact this is the preferred method of telling kubernetes how to deploy your application. First you create a deployment and you link your deployment to a service to allow external traffic to your deployment. Deployments therefore can have multiple pods as seen in the diagram. app=B and app=A are both deployements with its own attached services. We will dive deeper and give example of this in the practicals.
Replication set
-Inherent in the deployment concept is the idea of replication and scaling. For example, in the diagram appB has 3 pods while appA only has q. How does kubernetes know how many pods to deploy for each deployment? You define this using a replication set. You can kubernetes I want 3 replicas of my application and kubernetes will deploy three instances of you application and since the pod encapsulates your application, 3 pods will be deployed.
-Hence, Kubernetes is known its ability to scale as you can easily request kubernetes to deploy more or less instances of your application and it will happily do so changing the number of pods to your deployment and service for that particular app.
-For exapmple if you change the number of replicas for appB to 5, kubernetes will simply add two more pods to your app=B deployment and since serviceB is attached to your app=B deployment, the newly launched pods will autmatically be configured to interact with serviceB and adopt the same configurations as the rest of the pods. In this case the service in configured as a LoadBalancer. So the new pods will start to share the workload autmatically. This is the beauty of kubernetes, everything is smooth and automatic. Without kubernetes every time you deploy new instances of your application you may need to reset your load balancer to work with the new instaces and configure your instances to talk to other instances.
Kubernetes objects
-Whow do you actually define a deployment and service and tell kubernetes please deploy my applicartion? You do this by defining objects in a yaml file.
-there are many diffrerent type of objects you can define in kubernetes however, for the purpose of this course I will only explain the two most commonly used objects, a deployment and a service.
a) deployments
////////////////////////////////////////////////
apiVersion: apps/v1 # for versions before 1.9.0 use apps//v1beta2
kind: Deployment
metadata:
name :frontend
spec:
selector:
matchLabels:
app: guestbook
tier: frontend
replicas: 3
template:
metadata:
labels:
app: guestbook
tier: frontend
spec:
containers:
- name: php-redis
image: gcr.io/google-sample//gb-frtend:v4
resources:
requests:
cpu: 100m
memory: 100Mi
env:
- name: GET_HOSTS_FROM
value: dns
ports:
- containerPort: 80
////////////////////////////////////////////////
-you can define a deployment by simply creating a deployment object using a .yaml file. The apiVersion is the kubernetes API version. Kind: defines what kind of object this yaml file is defining, in this case we are defining a deployment object.
-under spec > selector > matchLabels, this is important because the labels defined here will be used to match tith the service object to connect the service to this deployment object as we will see later.
-the replicas value define how many pods or instances will be created as part of this deployment. This is where you want to edit for scaling purposes.
-under containers this defines the docker image you will be using to create the containers for your application and when scaling or creating replicas the new containers will be created using this image. You can request the amount of resources is required by each of your pods under resources. But this is not mandatory
-under env this is where you define the default configuration for networking for your pod. The newer kubernetes versions use DNS (Domain name service). Finally the ports define which port your powd will be using and since it is a front end application it will be taking the common port 80 for external traffic.
b) Services
/////////////////////
apiVersion: v1
kind: Service
metadata:
name : frontend
labels:
app: guestbook
tier: frontend
spec:
#try diffrent proxy types, ClusterIP, NodePort
type: LoadBalancer
ports:
- port: 80
#Add in ehwn using NodePort as an option
#nodePort: 30080
#name: http
selector:
app: guestbook
tier: frontend
/////////////////////
-notice the kind has switched from a deployment to a service object. The metadata is the same as for deployment and type defines the type of service it can be. Remember how we discussed about LoadBalancer, NodePort and ClusterIP? Again ports open up the service to interact with external traffic (port 80 is used for web pages). Now the crucial point is the selector, notice how the selector is exactly the same as the match label attribute for the deployment. They must be the same fpr the service to correctly identify the deployment to attach to. Usually Deployments and Services are launched together as we will see in the practicals.
Desired state
-States in kubernetes are very important. When you define a deployment or service object using the example before you are actually defining a desired state for the kubernetes cluster.
-What do I mean by desired state? A desired state is a state in which the kubernetes cluster pushes to achieve, I will give you a practical example.
-Practical example: Let's say you have defined a deployment object with replica:3 so there are 3 pods in your deployment. What happens when one of the pods are deleted by accident or it dies somehow. The answer is that the cluster will regenerate the lost pod because the desired state specifies 3 pods. The cluster will always try to match the desired state.
Again, what happens when you manually edit the deployment object to replica:5 and apply the changes? Since the desired state has changed the cluster will want meet the desired state and thus will create two more pods to meet replica:5.
Rolling update
-For example you are developer and you need to update a website which you have deployed. Say the title of the webpage needs updating. How would you do it? Your system is live so you cannot bring your system down to update the web page and relaunch again.
-This is where you would use a rolling update. Rolling updates are the standard nowadays because a rolling update if done properly should have zero downtime for your end users. A rolling update basically updates a percentage of your instaces (or it could be just 1) at a time leaving enough active resources to serve customers at any time.
-Rolling update are a feature in kubernetes, it is implemented automatically if you require updates to your desired state. For example if you need to change the image (which is your application) of your deployment you can simply change the deployment object in the yaml file and apply the new changes. Then a rolling update will automatically be triggered to update all your pods with the new image. Again this is the beauty of kubernetes; rolling updates are generally not easy to implement if you need to write this from scratch. We will see an example of rolling update in our practicals.
Stateless & Stateful theory
-when deploying applications on Kubernetes there are two major architectual concepts to choose from. Application are designed with these two concepts in mind. These two concepts are called stateless and stateful.
-Stateless & stateful relate to data persistence. A stateless application does not have data persistence whereas a stateful application prersists the data.
Data persistence
- what do I mean by data persistence? In Kubernetes terms data persistence directly relate to Pods. Data is said to be persistent if after all pods in the kubernetes cluster was terminated, the data that was there before still remains. On the other hand if the data is removed after the pods in the deployment have been terminated then the data is not persisted.
-If data is not persisted your application would be considered a stateless application while if the data was persisted your application would be a stateful application.
Stateless theory
-This is a working stateless application with two deployments a frontend deployment and a backend deployment. The backend deployment process all the data.
Now if the redis pod which is a data store for our application gets terminated will the data that is stored on the redis store be kept? The answer to that is no. All the data stored in the pod gets removed.
Stateful theory
-Here we have a MySQL server Pod which stores its data on the virtual machine hard drive called a persistent volume, we will touch on this more later on. So since the data is saved on the VM and not just in the pods, even if all the MySQL pods were terminated the data will still be there.
This is an example of data persistence using a Persistent Volume on the Virtual machine which hosts these Pods.
Persistent volumes
-Persistent volumes can be defined in the .yaml file using a kubernetes object called a Persistent Volume Claim. This object will claim the allocated space from the VM and turn it into a persistent volume which then can be used for the MySQL server Pod (in this case) to store its data persistently. We will see this in action as it is the setup for the stateful applications practical.
Stateful problems
-Data persistence sounds better right? Well, in theory yes but practically no, and let's make it clear that, in a real production environment where your application is serving a wide audiance, be it a eCommerce website, social websites, etc. You will almost use stateless design.
There are certain use cases for a stateful kubernetes design however it's just not a production ready design.
- There a re major issues with a stateful design for production such as:
*It does not scale. -> In the example shown above data persistence was achived by saving the data into a persistent volume on the VM. The problem is what if we have more data then the volume on the VM. The problem is what if we have more data then the volume can store what do we do? Can you increase the volume, yes you can but not easily and also what if we have so much data that the VM does not have enough hard drive space to store them. Can we easily add another VM for storage? You see this is the problem, you cannot easily add more VMs and use it for storage.
*Data Synchronisation -> Also, lets just say you managed to add more VMs for data storage then there is a problem of synchronisation. All your data needs to be synced otherwise you will start having dara quality problems. But there is no easy way to sync the data across different VMs.
*Possible Loss of Data -> Remmeber all of this is stored on a VM. What if the VM hard drive fails, what happens to the data stored on it? You guessed it, it is lost. Which means this solution cannot possibly fit a production solution
Stateless & Stateful theory
-So what can you use stateful kubernetes application for? The stateful design can be used for quick mock ups of ideas and you do not want to think about scaling yet. Or if you know the users for your app will not grow and you are happy with just one VM's hard drive space. An example of this app could be an in house employee application. The amount of employees in your company probably won't reach millions and the workload might be okay for just a single VM to handle.
-So the question comes back to then how is the data persisted for my production application using stateless design, since the data is not persisted in stateless applications.
-Well, that is true, that is why the data is persistes outside of the kubernetes cluster, basically it is saved in an external storage solution. We cannect the Pods within kubernetes with the external storage solution to persist the data do even if all the pods were terminated we can still retrive the data from our external store. Howecver, setting this connection up may be quite time consuming and hence why you will want to use a stateful application for a quick mock up of your ideas first.
Creating Kubernetes cluster from UI:
-Click create cluster (create options)
-Click "connect to the cluster"
-somethimes you need to authenticate using "gcloud auth login mymial@gmail.com"
Kubectl commands
kubectl get pods
Pratical 1 - Guestbook
-This practical involves deploying a stateless application called Guestbook. The application conatins 3 deployments one for fronted deployments one for frontend deployment and one for redis master and another for redis slaves. All deployments have attached services both only the frontend service open to external traffic. Deployment 2 and 3 are both considered backend dseployments and handles data that comes through from the frontend. we will have a look at the yaml file which defines these objects later.
-In this practical you will learn:
*the use of kubectl commands to interact with Kube-apiserver in the master to make changes to and monitor the cluster
-desired state in action - of you delete a pod it will auto regenerate.
-viewing different resources on the cluster such as pods deployments, services and logs using kubectl.
-different services options such as LoadBalncer, NodePort and clusterIP. You will see LoadBalancer in action to further your understanding.
-scaling your frontend pods up and down simply by changing the replica number.
-Perform a rolling update on your front end application by changing the image used to generate your pods and see the rolling update in action.
kubectl apply -f frontend.yaml
kubectl apply -f redismaster.yaml
kubectl apply -f redisslave.yaml
kubectl get deployments
kubectl get svc
kubectl get pods
kubectl get pods -l tier=frontend
kubectl describe deployment -l tier=frontend
/////////////////////
macOs
curl -O https://bootstrap.pypa.io/get-pip.py
python3 get-pip.py --user
pip3 install awscli --upgrade --user
aws --version
How to configure awscli
⋊> ~/.aws vim credentials
⋊> ~/.aws vim config
⋊> ~/.aws aws iam get-user
///////////////////////
http://wbarczynski.pl/
https://github.com/skarab7/kubernetes-memcached
https://github.com/kubernetes/examples
https://blog.tekspace.io/kubernetes-dashboard-remote-access/
https://pascalnaber.wordpress.com/2018/06/17/access-dashboard-on-aks-with-rbac-enabled/
https://docs.microsoft.com/en-gb/azure/aks/kubernetes-dashboard
Azure Kubernetes Service
https://github.com/Azure/AKS
https://docs.microsoft.com/en-us/cli/azure/aks?view=azure-cli-latest#az-aks-get-upgrades
https://docs.microsoft.com/en-us/azure/aks/tutorial-kubernetes-deploy-cluster
https://pascalnaber.wordpress.com/2018/01/26/persistent-storage-and-volumes-using-kubernetes-on-azure-with-aks-or-azure-container-service/
Ingress nginx
https://github.com/kubernetes/ingress-nginx
https://hackernoon.com/setting-up-nginx-ingress-on-kubernetes-2b733d8d2f45
static IP
https://docs.microsoft.com/pl-pl/azure/aks/static-ip
brew cask install minikube brew install kubernetes-climinikube status
minikube start
kubectl run hello-minikube --image=k8s.gcr.io/echoserver:1.4 --port=8080
kubectl expose deployment hello-minikube --type=NodePort
kubectl expose deployment hello-minikube --type=NodePort
minikube service hello-minikube --url
<open a browser and go to that url>
https://github.com/kubernetes/kops
brew update && brew install kops
1. What is Kubernetes
Kubernetes
Kubernetes is a container orchestration tool. Which means that is helps orchestrate the deployment of multiple containers and help manage its state and make sure the container is running smoothly. There are many nice feature built into kubernetes which we will go into detail in later sections. But let's just say if you want to launch a containerised solution for your web app you almost certainly want to use an orchestration tool such as Kubernetes.
-it lets you shedule containers on a cluster of machines
-you can run multiple containers on one machine
-you can run long long running service (like web applications)
-Kubernetes will manage the state of these containers
*can start container on specific nodes
*will restart a container when it gets killed
*can move containers from one node to another node
-instead of just running a few docker containers on one host manually, Kubernetes is a platfrorm that will manage the containers for you
-Kubernetes clusters can start with one node until thousands of nodes
-Some other popular docker orchestrators are:
*Docker Swarm
*Mesos
-You can run Kuberntes anywhere:
*on-premise (own datacenter)
*Public (google cloud, aWS)
*Hybrid: public & private
-highly modular
-open source
-backed by google
Containers
Docker
-Docker is the most popular container software
*an alternative to Docker is rkt - which also works with Kubernetes
-Docker Engine
*the docker runtime
*software to make run docker images
-Docker Hub
*online service to store and fetch docker images
* also allows you to build images online
-Docker benefits
*isolation: you ship a binary with all the dependencies
**no more it works on my machine, but not in production
*closer parity between dev, QA, and production enviorments
*docker makes development teams able to ship faster
*you can run the same docker image, unchanged, on laptops, data center VMs, and Cloud providers.
*docker uses Linuc Containers (a kernel feature) for operating system-level isolation
-highly modular
-open source
-backed by google
Containers
Docker
-Docker is the most popular container software
*an alternative to Docker is rkt - which also works with Kubernetes
-Docker Engine
*the docker runtime
*software to make run docker images
-Docker Hub
*online service to store and fetch docker images
* also allows you to build images online
-Docker benefits
*isolation: you ship a binary with all the dependencies
**no more it works on my machine, but not in production
*closer parity between dev, QA, and production enviorments
*docker makes development teams able to ship faster
*you can run the same docker image, unchanged, on laptops, data center VMs, and Cloud providers.
*docker uses Linuc Containers (a kernel feature) for operating system-level isolation
Kubernetes setups
-Kubernetes should really be able to run anywhere
-but, there are more integration for certain Cloud Providers, like AWS & GCE
*things like Volumes and External Load Balancer work only with support Cloud Providers
-minikube to quickly spin up a local single machine with a Kubernetes cluster
-spin up cluster on AWS using kops
*this tool can be used to spin up a highly available production cluster
Minikube setup
-minikube is a tool that makes it easy to run Kuberenetes locally
-Minikube runs a single-node Kubernetes cluster inside a Linux VM
-it's aimed on users who want to just test it out or use it out or use if for development
-it cannot spin up a production cluster, it's a one node machine with no high availability
-It works on Windows, Linux, and MacOS
-You will need Virtualization Software installed to run minikube:
*example : VirtualBox
-you can downlad minikube from: https://github.com/kubernetes/minikube
-to lunch your cluster you just need to enter (in a shell/terminal/powershell): minikube start
minikube start
ls ~/.kube/
cat ~/.kube/config
kubectl run hello-minikube --image=k8s.gcr.io/echoserver:1.4 --port=8080
kubectl expose deployment hello-minikube --type=NodePort
minikube service hello-minikube --url
kubectl cluster-info
minikube service list
minikube stop
Docker client setup
Kubernetes cluster setup -tools
minikube/docker client/kops/kubeadm
-there are multiple tools to install a kubernetes cluster
-I showed you how to use minikube /docker client, to do local installs
-if you want a production cluster, you'll need different tooling
*minikube and docker client are greate for local setups, but not for real clusters
*kops and kubeadm are tools to spin up a production cluster
-On AWS, the best tool is kops
* at some point AWS EKS (hosted Kubernetes) will be avaible,at that point this will probably be the preferred option
-For other installs, or if you can't get kops to work, you can use kubeadm
*kubeadm is an alternative approach, kops is still recommended (on AWS) - you also have AWS inntegrations with kops automatically
Kubernetes Setup
Setting up Kubernetes on AWS using kops
-To setup Kubernetes on AWS, you can use a tools called kops
*kops stands for Kubenrnetes Operations
-the tool allows you to do production grade Kubernetes installations, upgrades and managment
*i will use this tool to start a Kubernetes cluster on AWS
-there is also a legacy tool called kube-up.sh
*this was a simple tool to bring up a cluster, but is now deprecated, it doesn't create a production ready environment
Kops
-Kops only works on Mac/Linux
-If you're on windows, you'll need to boot a virtual machine first
-you can use Vagrant to quickly boot up a linux box
-you can download Virtualbox from virtualbox.org Vagrant from vagrantup.com (you need both)
-once downloaded, to boot up a new linux VM, just type in cmd/powershell:
mkdir ubuntu
cd ubuntu
vagarnt init ubuntu/xenial64
vagrant up
vagsrant ssh-config
vagrant ssh
install kops
create domain and hosted zone on route53
create s3 bucket
kops create cluster --name=kubernetes.domain.com --state=s3://kops-state-rds --zones=eu-west-1a --node-count=2 --node-size=t2.micro --master-size=t2.micro --dns-zone=kubernetes.domain.com
kops update cluster kubernetes.domain.com --state=s3://kops-state-rds --yes
kubectl run hello-minikube --image=k8s.gcr.io/echoserver:1.4 --port=8080
kubectl expose deployment hello-minikube --type=NodePort
minikube service hello-minikube --url
kubectl cluster-info
kubectl get services (forwording to uniq port for every node except master)
kops delete cluster kubernetes.domain.com --state=s3://kops-state-rds --yes
2. Cloud / On - premise setup
3. Cluster Setup
4. Building Containers
Dockerfile
FROM node:4.6
WORKDIR /app
ADD . /app
RUN npm install
EXPOSE 3000
CMD npm start
index.js
var express = require('express');
var app = express();
app.get('/', function (req, res) {
res.send('Hello World!')
});
var server = app.listen(3000, function () {
var host = server.address(.address;
var port = server.address().port;
console.log('Example app listeining at http://%s:%s', host, port);
});
package.json
{
"name": "myapp",
"version": "0.0.1",
"private": true,
"scripts": {
"start": "node index.js"
},
"engines": {
"node": "^4.6.1"
},
"dependencies": {
"express": "^4.14.0"
}
}
docker build .
docker run -p 3000:3000 -t a871ae01e52a
to push an image to Docker Hub
docker login
docker tag imageid login/docker-demo
docker push login/docker-demo
5. Running your first app
-Let's run our newly built application on the new Kubernetes cluster
-before we can launch a container based on the image, we need to create a pod definitation
*a pod describes an application running on Kubernetes
*a pod can conatiain one or more tightly coupled containers, that make up the app
**those apps can easily communicate which each other their local port numbers
Create a pod
-create a file pod-helloworld.yml with the pod definition:
apiVersion: v1
kind: Pod
metadata:
name: nodehelloworld.example.com
labels:
app: helloworld
spec:
containers:
- name: k8s-demo
image: 212131316/nodeapp
ports:
- containerPort: 3000
-Use kubectl to create the pod on the kubernetes cluster:
kubectl create -f pod-helloworld.yml
Useful commands
kubectl get pod // get information about all running pods
kubectl describe pod <pod> // describe one pod
kubectl expose pod <pod> --port=444 --name=forntend // expose the port of a pod (creates a new service)
kubectl port-forward <pod> 8080 // port forward the exposed pod port to your local machine
kubectl attach <podname> -i // attach to the pod
kubectl exec <pod> -- command // execute a command on the pod
kubectl label pods <pod> mylabel=awesome // add a new label to a pod
kubectl run -i --tty busybox --image=busybox --restart=Never -- sh // run a shell in a pod - very useful for debugging
git clone git@github.com:wardviaene/kubernetes-course.git
kubectl get node
cd kubernetes-course/
cat first-app/helloworld.yml
apiVersion: v1
kind: Pod
metadata:
name: nodehelloworld.example.com
labels:
app: helloworld
spec:
containers:
- name: k8s-demo
image: wardviaene/k8s-demo
ports:
- name: nodejs-port
containerPort: 3000
kubectl create -f first-app/helloworld.yml
kubectl get pod
kubectl describe pod nodehelloworld.example.com
Name: nodehelloworld.example.com
Namespace: default
Node: minikube/10.0.2.15
Start Time: Sat, 24 Nov 2018 15:44:53 +0100
Labels: app=helloworld
Annotations: <none>
Status: Running
IP: 172.17.0.7
Containers:
k8s-demo:
Container ID: docker://4cc525f9aa7049f0c7752bbfd166d6b704cd68aecda413558ffbfb8cf53780f2
Image: wardviaene/k8s-demo
Image ID: docker-pullable://wardviaene/k8s-demo@sha256:2c050f462f5d0b3a6430e7869bcdfe6ac48a447a89da79a56d0ef61460c7ab9e
Port: 3000/TCP
Host Port: 0/TCP
State: Running
Started: Sat, 24 Nov 2018 15:45:50 +0100
Ready: True
Restart Count: 0
Environment: <none>
Mounts:
/var/run/secrets/kubernetes.io/serviceaccount from default-token-zhz87 (ro)
Conditions:
Type Status
Initialized True
Ready True
PodScheduled True
Volumes:
default-token-zhz87:
Type: Secret (a volume populated by a Secret)
SecretName: default-token-zhz87
Optional: false
QoS Class: BestEffort
Node-Selectors: <none>
Tolerations: node.kubernetes.io/not-ready:NoExecute for 300s
node.kubernetes.io/unreachable:NoExecute for 300s
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 2m50s default-scheduler Successfully assigned nodehelloworld.example.com to minikube
Normal SuccessfulMountVolume 2m50s kubelet, minikube MountVolume.SetUp succeeded for volume "default-token-zhz87"
Normal Pulling 2m49s kubelet, minikube pulling image "wardviaene/k8s-demo"
Normal Pulled 114s kubelet, minikube Successfully pulled image "wardviaene/k8s-demo"
Normal Created 113s kubelet, minikube Created container
Normal Started 113s kubelet, minikube Started container
kubectl port-forward nodehelloworld.example.com 8081:3000
curl localhost:8081
kubectl expose pod nodehelloworld.example.com --type=NodePort --name nodehelloworld-service
minikube service nodehelloworld-service --url
kubectl get service
Commands to debug
kubectl attach nodehelloworld.example.com
kubectl exec nodehelloworld.example.com -- ls /app
kubectl exec nodehelloworld.example.com -- touch /app/test.txt
kubectl exec nodehelloworld.example.com -- ls /app
kubectl get service
kubectl describe nodehelloworld-service
kubectl describe service nodehelloworld-service
Name: nodehelloworld-service
Namespace: default
Labels: app=helloworld
Annotations: <none>
Selector: app=helloworld
Type: NodePort
IP: 10.105.187.86
Port: <unset> 3000/TCP
TargetPort: 3000/TCP
NodePort: <unset> 31034/TCP
Endpoints: 172.17.0.7:3000
Session Affinity: None
External Traffic Policy: Cluster
Events: <none>
kubectl run -i --tty busybox --image=busybox --restart=Never -- sh
telnet 172.17.0.7 3000
GET /
Service with Load Balancer
-in a real world scenario, you need to be able to access the app from outside cluster
-on AWS, you can easily add an external Load Balancer
-this AWS Load Balancer will route the traffic to the correctt pod in Kubernetes
-there are other solutions for other cloud providers that don't have a Load Balancer
*your own haproxy/nginx load balancer in front of your cluster
*expose ports directly
cat first-app/helloworld.yml
apiVersion: v1
kind: Pod
metadata:
name: nodehelloworld.example.com
labels:
app: helloworld
spec:
containers:
- name: k8s-demo
image: wardviaene/k8s-demo
ports:
- name: nodejs-port
containerPort: 3000
cat first-app/helloworld-service.yml
apiVersion: v1
kind: Service
metadata:
name: helloworld-service
spec:
ports:
- port: 80
targetPort: nodejs-port
protocol: TCP
selector:
app: helloworld
type: LoadBalancer
kubectl create -f first-app/helloworld.yml
kubectl create -f first-app/helloworld-service.ym
kubectl describe service helloworld-service
6. Building Container Images
1. How does the AWS LoadBalancer routes traffic to the correct pod -The LoadBalancer uses a NodePort that is exposed on all non-master nodes
The LoadBalancer Service will also create a port that is open on every node, that can be used by the AWS LoadBalancer to connect. That port is owned by the Service, so that the traffic can be routed to the correct pod
2. The command "kubectl expose" is used for what purpose? - To create a Service object for an existing object
Kubectl expose creates a new service for a pod (or other kubernetes objects). A service allows you to connect to a pod from outside the cluster, using Service type NodePort or LoadBalancer
3. How do containers within a pod communicate? - Using 127.0.0.1/loalhost and the port number
Containers within a pod can communicate using the local port numbers
4. How many containers can you run in a pod - You can run multiple containers in one pod
II. Kubernetes Basics
1. Node Architecture
2. Scaling pods
-If your application is stateless you can horizontally scale it
*Stateless = your application does't have a state, it doesn't write any local files / keeps local sessions
*All traditional databases (MySQL, Postgres) are stateful, they have database files that can't be split over multiple instaces
-most web applications can be mase stateless
*session managment needs to be done outside the container
*any files that need to be saved can't be saved locally on the container
-Our example app is stateless, if the same app would run multiple times, it doesn't change state.
-For more information about best practices, have a look at 12factor.net
*or see my course: Learn DevOps: Continuously delivering better software /scaling apps on-premise and in the cloud
-later in this course I'll explain how to use volumes to still run stateful apps
*those stateful apps can't horizontally scale, but you can run them in a single container and vertically scale (allocate more CPU / Memory / Disk)
-Scaling in Kubernetes can be done using the Replication Controller
-The replicartion controller will ensure a specified number of pod replicas will run at all time
- a pods created with the repica controller will automatically be replaced if they fail, get deleted, or are terminated
-using the replication controller is also recommended if you just want to make sure 1 pod is always running, even after reboots
*you can then run a replication controller with just 1 replica
*this makes sure that the pod is always running
-to replicate our example app 2 times
apiVersion: v1
kind: ReplicationController
metadata:
name: helloworld-controller
spec:
replicas: 2
selector:
app: helloworld
template:
metadata:
labels:
app: helloworld
spec:
containers:
- name: k8s-demo
image: wardviaene/k8s-demo
ports:
- name: nodejs-port
containerPort: 3000
kubectl get node
kubectl create -f replication-controller/helloworld-repl-controller.yml
kubectl get pods
kubectl edit -f replication-controller/helloworld-repl-controller.yml
kubectl get pods
kubectl describe pod helloworld-controller-9dnbz
Name: helloworld-controller-9dnbz
Namespace: default
Node: minikube/10.0.2.15
Start Time: Sat, 24 Nov 2018 19:05:11 +0100
Labels: app=helloworld
Annotations: <none>
Status: Running
IP: 172.17.0.8
Controlled By: ReplicationController/helloworld-controller
Containers:
k8s-demo:
Container ID: docker://d150846754af213d2534d5376915b86cdb67c60f03e2092889eb98cadaf6e3ba
Image: wardviaene/k8s-demo
Image ID: docker-pullable://wardviaene/k8s-demo@sha256:2c050f462f5d0b3a6430e7869bcdfe6ac48a447a89da79a56d0ef61460c7ab9e
Port: 3000/TCP
Host Port: 0/TCP
State: Running
Started: Sat, 24 Nov 2018 19:05:15 +0100
Ready: True
Restart Count: 0
Environment: <none>
Mounts:
/var/run/secrets/kubernetes.io/serviceaccount from default-token-zhz87 (ro)
Conditions:
Type Status
Initialized True
Ready True
PodScheduled True
Volumes:
default-token-zhz87:
Type: Secret (a volume populated by a Secret)
SecretName: default-token-zhz87
Optional: false
QoS Class: BestEffort
Node-Selectors: <none>
Tolerations: node.kubernetes.io/not-ready:NoExecute for 300s
node.kubernetes.io/unreachable:NoExecute for 300s
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 23m default-scheduler Successfully assigned helloworld-controller-9dnbz to minikube
Normal SuccessfulMountVolume 23m kubelet, minikube MountVolume.SetUp succeeded for volume "default-token-zhz87"
Normal Pulling 23m kubelet, minikube pulling image "wardviaene/k8s-demo"
Normal Pulled 23m kubelet, minikube Successfully pulled image "wardviaene/k8s-demo"
Normal Created 23m kubelet, minikube Created container
Normal Started 23m kubelet, minikube Started container
kubectl delete pod helloworld-controller-9dnbz
kubectl get pods
kubectl scale --replicas=4 -f replication-controller/helloworld-repl-controller.yml
kubectl get rc
kubectl delete rc/helloworld-controller
3. Deployments
Replication Set
-Replica Set is the next-generation Replication Controller
-It supports a new selector that can do selection based on filtering according a set of values
*e.g. "environment" either "dev" or "qa"
*not only based on equality, like the Replication Controller
**e.g. "environment" == "dev"
-this Replica Set, rather than the Replication Controller, is used by the Deploymnet object
Deployments
-A deployment declaration in Kubernetes allows you to do app deployments and updates
-when using the deployment object, you define the state of your application
*Kubernetes will then make sure the clusters matches your desired state
-Just using the replication controller or replication set might be cumbersome to deploy apps
*The Deployment Object is easier to use and gives you more possibilities
-with a deployment object you . can:
*create a deployment (e.g. deploying an app)
*update a deployment (e.g. deploying a new version)
*do rolling updates (zero downtime deployments)
*roll back to a previous version
*pause /resume a deployment (e.g. to roll-out to only certain percentage)
-this is an example of a deployment:
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: helloworld-deployment
spec:
replicas: 3
template:
metadata:
labels:
app: helloworld
spec:
containers:
- name: k8s-demo
image: wardviaene/k8s-demo
ports:
- name: nodejs-port
containerPort: 3000
Commands:
kubectl get depoyments // get information on current deployments
kubectl get rs // get information about the replica sets
kubectl get pods --show-labels // get pods, and also show labels attached to those pods
kubectl rollout status deployment/helloworld-deployment //get deployment status
kubectl set image deployment/helloworld-deployment k8s-demo=k8s-demo:2 // run k8s-demo with the image label version 2
kubectl edit deployment/helloworld-deployment // edit the deployment object
kubectl rollout status deployment/helloworld-deployment // get the status of the rollout
kubectl rollout history deployment/helloworld-deployment // get the rollout history
kubectl rollout undo deployment/helloworld-deployment // rollback to previous version
kubectl rollout undo deployment/helloworld-deployment --to-version=n // rollback to any version version
kubectl get deployments
kubectl get rs
kubectl get pods
kubectl get pods --show-labels
kubectl rollout status deployment/helloworld-deployment
kubectl expose deployment helloworld-deployment --type=NodePort
kubectl get service
kubectl describe service helloworld-deployment
minikube service helloworld-deployment --url
kubectl set image deployment/helloworld-deployment k8s-demo=wardviaene/k8s-demo:2
kubectl rollout status deployment/helloworld-deployment
kubectl rollout history deployment/helloworld-deployment
kubectl create -f deployment/helloworld.yml --record
kubectl rollout undo deployment/helloworld-deployment
kubectl rollout history deployment/helloworld-deployment
kubectl edit deployment/helloworld-deployment
...
spec:
progressDeadlineSeconds: 600
replicas: 3
revisionHistoryLimit: 10
selector:
matchLabels:
app: helloworld
...
4. Services
Services
-Pods are very dynamic, they come and go on the Kubernetes cluster
*When using a Replication Controller, pods are terminated and created during scaling operations
*When using Deployments, when updating the image version, pods are terminated and new pods take the place of older pods
-That's why Pods should never be accessed directly, but always through a Service
-A service is the logical bridge between the "mortal" pods and other services or end-users
- when using the "kubectl expose" command earlier, you created a new Service for your pod, so it could be accessed externally
-creating a service will create an endpoint for your pod(s);
*a ClusterIP: a virtual IP address only reachable from within the cluster (this is the default)
*a NodePort: a port that is the same on each node that is also reachable externally
*A LoadBalancer: a LoadBalancer created by the cloud provider that will route external traffic to every node on the NodePort (ELB on AWS)
-the options just shown only allow you to create virtual IPs or ports
-there is also a possiblility to use DNS names
*ExternalName can provide a DNS name for the service
*e.g. for service discovery using DNS
*this only works when the DNS add-on is enabled
-I will discuss this later in a separate lecture
-this is an example of a Service definition (also created using kubectl expose):
apiVersion: v1
kind: Service
metadata:
name: helloworld-service
spec:
ports:
- port: 31001
nodePort: 31001
targetPort: nodejs-port
protocol: TCP
selector:
app: helloworld
type: NodePort
-Note: by default service can only run between ports 30000-32767, but you could change this behavior by adding the --service-node-port-range= argument to the kube-apiserver (in the init scripts)
kubectl create -f first-app/helloworld.yml
kubectl get pods
kubectl describe pod nodehelloworld.example.com
kubectl create -f first-app/helloworld-nodeport-service.yml
minikube service helloworld-service --url
kubectl get svc
kubectl describe svc helloworld-service
5. Labels
Labels:
-labels are key/value pairs that can be attached to objects
*labels are like tags in AWS or other cloud providers, used to tag resources
-you can label your objects, for instance your pod, following an organizational structure
*key: environment - value: dev / staging /qa / prod
*key: department - value: engineering / finace /marketing
-in our previous examples I already have been using labels to tag pods:
metadata:
name: nodehelloworld.example.com
labels:
app: helloworld
-labels are not unique and multiple labels can be added to one object
-once labels are attached to an object, you can use filters to narrow down results
* this is called Label Selectors
-using Label Selectors, you can use matching expressions to match labels
*for instance, a particular pod can only run on a node labeled with "environment" equals "development"
*more cmplex matching: "environment" in "development" or "qa"
Node Labels
-you can also use labels to tag nodes
-once nodes are tagged, you can use label selectors to let pods only run on specific nodes
-there are 2 steps required to run . a pod on a specific set of nodes:
*first you tag the node
*then you add a nodeSelector to your pod configuration
-First step, add a label or multiple labels to your nodes:
kubectl label nodes node1 hardware=high-spec
kubectl label nodes node2 hardware=low-spec
-Secondly, add a pod that uses those labels:
apiVersion: v1
kind: Pod
metadata:
name: nodehelloworld.example.com
labels:
app: helloworld
spec:
containers:
- name: k8s-demo
image: wardviaene/k8s-demo
ports:
- name: nodejs-port
containerPort: 3000
nodeSelector:
hardware: high-spec
NodeSelector using labels:
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: helloworld-deployment
spec:
replicas: 3
template:
metadata:
labels:
app: helloworld
spec:
containers:
- name: k8s-demo
image: wardviaene/k8s-demo
ports:
- name: nodejs-port
containerPort: 3000
nodeSelector:
hardware: high-spec
kubectl get nodes --show-labels
kubectl create -f deployment/helloworld-nodeselector.yml
kubectl get deployment
kubectl get pod
kubectl describe pod helloworld-deployment-6576b7749d-8n624
kubectl label nodes minikube hardware=high-spec
kubectl get nodes --show-labels
kubectl describe pod helloworld-deployment-6576b7749d-8n624
6. Healthchecks
Healthchecks
-it your application malfunctions, the pod and container can still be running, but the application might not work anymore
-to detect and resolve problems with your application, you can run health checks
-you can run 2 different type of health checks
*running a command in the container periodically
*periodic checks on a URL (HTTP)
-the typical production application behind a load balancer should always have health checks implemented in some way to ensure availability and resiliency of the app
7. ReadinessProbe
8. Pod State & LifeCycle
9. Secrets
10. WebUI
III. Advanced topics
1. Service auto-discovery
2. ConfigMap
3. Ingress
4. External DNS
5. Volumes
6. Pod Presets
7. StatefulSets
8. Deamon Sets
9. Monitoring
10. Autoscaling
11. Node Affinity
12. InterPod (Anti-)Affinity
13. Taints and Tolerations
14. Operators
IV. Administration
1. Master Services
2.Quotas and Limits
3. Namespaces
4. User Managmeent
5. RBAC
6. Networking
7. Node Maintenance
8. High Availability
9. TLS on ELB
V. Packing
1. Introduction to Helm
2. Creating Helm Charts
3. Helm Repository
4. Building & Deploying
VI
1. kubeadm
2. TLS Certificcates with cert -manager
Kubernetes Procedure Document
Github repository [Read this first]
- Download all the course material from: https://github.com/wardviaene/advanced-kubernetes-course
Questions?
- Send me a message
- Use Q&A
- Join our facebook group: https://www.facebook.com/groups/840062592768074/
Download Kubectl
- Download from: https://kubernetes.io/docs/tasks/tools/install-kubectl/
Minikube (doesn’t work with all demos)
- Project URL: https://github.com/kubernetes/minikube
- Latest Release and download instructions: https://github.com/kubernetes/minikube/releases
- VirtualBox: http://www.virtualbox.org
Minikube on windows:
- Download the latest minikube-version.exe
- Rename the file to minikube.exe and put it in C:\minikube
- Open a cmd (search for the app cmd or powershell)
- Run: cd C:\minikube and enter minikube start
Kops (recommended)
Project URL
Free DNS Service
- Sign up at http://freedns.afraid.org/
- Choose for subdomain hosting
- Enter the AWS nameservers given to you in route53 as nameservers for the subdomain
- http://www.dot.tk/ provides a free .tk domain name you can use and you can point it to the amazon AWS nameservers
Cluster Commands
- kops create cluster --name=kubernetes.newtech.academy --state=s3://kops-state-b429b --zones=eu-west-1a --node-count=2 --node-size=t2.micro --master-size=t2.micro --dns-zone=kubernetes.newtech.academy
- kops update cluster kubernetes.newtech.academy --yes --state=s3://kops-state-b429b
- kops delete cluster --name kubernetes.newtech.academy --state=s3://kops-state-b429b
- kops delete cluster --name kubernetes.newtech.academy --state=s3://kops-state-b429b --yes
Kubeadm
Kubernetes from scratch
- You can setup your cluster manually from scratch
- If you’re planning to deploy on AWS / Google / Azure, use the tools that are fit for these platforms
- If you have an unsupported cloud platform, and you still want Kubernetes, you can install it manually
- CoreOS + Kubernetes: https://coreos.com/kubernetes/docs/latest/getting-started.html
VII. Logging
Elasticsearch (as SeatefullSet), fluentd, Kibana, LogTrail
Logging with fluentd
Logging:
-logging is important to show errors, information and debuging ata about application
-when you only run one app, it is pretty obvious how to look for the logs
* you'd just open the "app.log" file to see what's going on
*or if deployed as a pod with "kubectl logs"
-with kubernetes 1 application will be running as one or many pods
*Finding a error will be much more difficult: what pod do you have to look at?
- Up until now you might have been using "kubectl logs" to get the log details of a pod
-to get the correct information, you might have to look up:
*the pod name
*the containers names
-and run "kubectl logs" for every container in every pod that is running for an application
-note: you can use both "kubectl log" and "kubectl logs"
-the solution for this problem is to do Log Aggregation
-It's not Kuibernetes specific
*It's already applied for years, even with syslog (a standard for message logging since 1980s)
*Log Aggregation is nowadays often implemented with more modern tools
** the ELK Stack (ElasticSearch + Logstash + Kibana)
** several hosted services like loggly.com, papertrailapp.com
-Demo
*Fluentd (for log forwarding)
*ElasticSearch (for log indexing)
*Kibana (for visualisation)
*LogTrail (an easy to use UI to show logs)
*this solution can be easily customaized, you can create custom dashboard to show what is important for you
VIII. Authentication
OpenID Connect (OIDC), Auth0
IX. Authorization
Kubernetes RBAC
X. Packing
Helm
XI. The Job Resource
Job
XII. Scheduling
CronJob
XIII. Deploying on Kubernetes
Spinnaker
XIV. Microservices on Kuberenetes
Linkerd
XV. Federation
kubefed
XVI. Monitioring
Prometheus
Kubernetes
Kubernetes is a container orchestration tool. Which means that is helps orchestrate the deployment of multiple containers and help manage its state and make sure the container is running smoothly. There are many nice feature built into kubernetes which we will go into detail in later sections. But let's just say if you want to launch a containerised solution for your web app you almost certainly want to use an orchestration tool such as Kubernetes.
Master and nodes in Kubernetes
A kubernetes deployment in generally referred to as a kubernetes cluster.
cluster = master + nodes
master
-manages all cluster activities and your requests.
node
- is where the work happens, a worker machine or a worker node.
- A node is typically a Virtual Machine or VM.
Master components
a) Kube-Apiserver
-It is the component on the master the exposes the Kubernetes API. It is the front-end for the Kubernetes control plane. Basically when you issue commands to your cluster to perform an action for example deploy something, this is the component that receives your commands. You will see the use of commands in the practicals.
b) Kube-Scheduler
-it is the component on the master that watches newly created pods that have no node assigned , and selects a node for them to run on. It basically manages the resources available on on the kubernetes cluster and helps manages the launching of new pods (we will come back to pods later) to where there are available resources.
c) Kube-Controller Manager
-Component on the master that runs controllers. The controllers have many functions and we will touch on them in practicals but two major controllers you should know is the node controller (responsible for noticing when a node dies) and replication controller which is responsible for maintaining the correct number of pods.
Node Compoenents
a) Kubelet
-It is the primary node agent. The kubelet does the following:
*watches for pods that have been assigned to the node.
*it runs the pods containers via docker
*it interacts with Kube-Api in the master, to update status of each pod back to the cluster master.
b) Kube Proxy
-Kube-proxy enables the Kubernetes service abstraction by maintaining network rules on the host and performing connection forwarding.
c) Docker
-Docker needs to be installed on the node to actually run containers.
The nodes has to have a kubelet and docker installed for it to be considered a node.
Is it true that master and node are separate VMs? So you have one VM for the master and VMs for the worker nodes. Your answer from the diagram should be yes. It seems that way. However, this is actually been changed. Currently, many default settings will allow master and worker to be on the same VM.
This is actually preferred in mosrt cases because you can use the available resources that's not used on you master VM to launch pods as well. So in short a master can also be a worker node.
What defines a master is actually just the components mentioned earlier and same for the worker so on a single VM both the master and worker co-exists. for our practicals, this is what will be done. We will only launch a single VM with both master and worker node.
Practically speaking many of the components mentioned above for the master and Nodes are background processes and in most cases you will not be interacting with it directly. We will be mainly controlling the cluster in a high level manner by interacting with the Kubernetes API (served through kub-apiserver on the master) using the kubernetes command line interface kubectl. In the practical we will do everything using kubectl.
Pods
-Pods are the unit of measure in kubernetes. What do I mean by that? In kubernetes we do not talk about containers we talk about pods. Everything in Kubernetes revolve around the concept of Pods. A Pod encapsulates a container or multiple containers. Within the pod resources are shared such as a uniques network IP, storage resources and configurations.
-The most common way to setup a pod is the "one-container-per-pod" model, which is shown on the left. You only have one container in your pod and kubernetes will interact with this pod.
-The second way to setup a pod is with multiple containers. You want to have multiple containers if you have a tightly coupled application that needs to work together . Then the volume which is the storage unit will be shared among the two containers as well as the configs. Notice that the IP address is for the Pod and not for the individual containers.
Pod networking
-Very important concept when working with pods is that the Pods have a private network within. So containers within the same Pod can communicate freely with one another however, since it is a private network to access the outside network a proxy must be setup. We will go deeper into this topic when we reach the proxy section.
Services
-Pods which have a privatew network can commucate to the external internet only via a proxy. And the concept of services adds to that logic. A Kubernetes service is an abstraction on top of the selected Pods which allows the pods within that service to interact with external traffic. Although each Pod has a unique IP address, those IPs are not exposed outside the cluster without a Service.
Services are related to the selected Pods using the concept of labels.
-There are 4 VMs//nodes in the diagram. One for the master, the other three for the worker nodes. The pods have unique IP addresses linked to them. The top 3 pods which are distributed across two VMs have the same label, app = B. Now the service qhich is defined by the dotted line has s:app=B which means that this service relates to the pods with the label app=B. The service also has a unique IP address. This IP address is the proxy address which is where the external network will interact with and not the Pod IP addresses.
Notice also that that within the Mastewr Node there is two prcocesses running ine with s:app=B and one for s:app=A. This is called a label selector (which selects the labels). Service A is much the same with service B the difference is the number of pods and VMs.
-There are three major configuration options when using services and they are:
*ClusterIP
*NodePort
*Load Balancer
In the practical we will test each of these options and see the difference.
Services
a) ClusterIP
-Expose the Service on an internal IP in the cluster. This type makes the Service only reachable from within the cluster. So a service with this configuration will not be exposed to external traffic.
b) NodePort
-Expose the Service on the same port of each selected Node in the cluster. Makes a Service accessible from outside the cluster using <NodeIP>:<NodePort>.
-Using our previous example. If the NodePort Option was selected. We will be exposing two endpoints for service B instead of just one. One for each node. The NodeIP is the VM's external IP address and the port can be specified and is the same for both end points. So if we only have one VM there would be only one end point.
c) LoadBalancer
-This option is the most comonly used option. It exposes only one end point for our service and the user can only access this point and the subsequent load is distributed among all available pods in the service.
-Using our previous example, this showcases the option for LoadBalancer. The workload is distributed among all pods. We will see this in action in the pracrtical. But in short the load is balanced so every pod should have a similar % of workload and not leaving one overburdened.
Deployments
-imagine that you have finished creating docker images for a web app, an image for the frontend interface and an image for the backend data storage. Now as you know kubernetes manages and ochestrates containers so how do you launch your web app using kubernetes so that you have a working web app with both a frontend and a backend? This is where kubernetes deployments comein.
-the deployment instructs Kubernetes how to create and update instances of your application. Once you've created a Deployment, the Kubernetes master schedulkes mentioned application instances individual Nodes in the cluster.
-You have seen services before using the same diagram. Now I want to bring to you the idea that deployments and services in fact go hand in hand. You create a deployment with a label app=B and you use the same label selector for your services to deployments.
-In fact this is the preferred method of telling kubernetes how to deploy your application. First you create a deployment and you link your deployment to a service to allow external traffic to your deployment. Deployments therefore can have multiple pods as seen in the diagram. app=B and app=A are both deployements with its own attached services. We will dive deeper and give example of this in the practicals.
Replication set
-Inherent in the deployment concept is the idea of replication and scaling. For example, in the diagram appB has 3 pods while appA only has q. How does kubernetes know how many pods to deploy for each deployment? You define this using a replication set. You can kubernetes I want 3 replicas of my application and kubernetes will deploy three instances of you application and since the pod encapsulates your application, 3 pods will be deployed.
-Hence, Kubernetes is known its ability to scale as you can easily request kubernetes to deploy more or less instances of your application and it will happily do so changing the number of pods to your deployment and service for that particular app.
-For exapmple if you change the number of replicas for appB to 5, kubernetes will simply add two more pods to your app=B deployment and since serviceB is attached to your app=B deployment, the newly launched pods will autmatically be configured to interact with serviceB and adopt the same configurations as the rest of the pods. In this case the service in configured as a LoadBalancer. So the new pods will start to share the workload autmatically. This is the beauty of kubernetes, everything is smooth and automatic. Without kubernetes every time you deploy new instances of your application you may need to reset your load balancer to work with the new instaces and configure your instances to talk to other instances.
Kubernetes objects
-Whow do you actually define a deployment and service and tell kubernetes please deploy my applicartion? You do this by defining objects in a yaml file.
-there are many diffrerent type of objects you can define in kubernetes however, for the purpose of this course I will only explain the two most commonly used objects, a deployment and a service.
a) deployments
////////////////////////////////////////////////
apiVersion: apps/v1 # for versions before 1.9.0 use apps//v1beta2
kind: Deployment
metadata:
name :frontend
spec:
selector:
matchLabels:
app: guestbook
tier: frontend
replicas: 3
template:
metadata:
labels:
app: guestbook
tier: frontend
spec:
containers:
- name: php-redis
image: gcr.io/google-sample//gb-frtend:v4
resources:
requests:
cpu: 100m
memory: 100Mi
env:
- name: GET_HOSTS_FROM
value: dns
ports:
- containerPort: 80
////////////////////////////////////////////////
-you can define a deployment by simply creating a deployment object using a .yaml file. The apiVersion is the kubernetes API version. Kind: defines what kind of object this yaml file is defining, in this case we are defining a deployment object.
-under spec > selector > matchLabels, this is important because the labels defined here will be used to match tith the service object to connect the service to this deployment object as we will see later.
-the replicas value define how many pods or instances will be created as part of this deployment. This is where you want to edit for scaling purposes.
-under containers this defines the docker image you will be using to create the containers for your application and when scaling or creating replicas the new containers will be created using this image. You can request the amount of resources is required by each of your pods under resources. But this is not mandatory
-under env this is where you define the default configuration for networking for your pod. The newer kubernetes versions use DNS (Domain name service). Finally the ports define which port your powd will be using and since it is a front end application it will be taking the common port 80 for external traffic.
b) Services
/////////////////////
apiVersion: v1
kind: Service
metadata:
name : frontend
labels:
app: guestbook
tier: frontend
spec:
#try diffrent proxy types, ClusterIP, NodePort
type: LoadBalancer
ports:
- port: 80
#Add in ehwn using NodePort as an option
#nodePort: 30080
#name: http
selector:
app: guestbook
tier: frontend
/////////////////////
-notice the kind has switched from a deployment to a service object. The metadata is the same as for deployment and type defines the type of service it can be. Remember how we discussed about LoadBalancer, NodePort and ClusterIP? Again ports open up the service to interact with external traffic (port 80 is used for web pages). Now the crucial point is the selector, notice how the selector is exactly the same as the match label attribute for the deployment. They must be the same fpr the service to correctly identify the deployment to attach to. Usually Deployments and Services are launched together as we will see in the practicals.
Desired state
-States in kubernetes are very important. When you define a deployment or service object using the example before you are actually defining a desired state for the kubernetes cluster.
-What do I mean by desired state? A desired state is a state in which the kubernetes cluster pushes to achieve, I will give you a practical example.
-Practical example: Let's say you have defined a deployment object with replica:3 so there are 3 pods in your deployment. What happens when one of the pods are deleted by accident or it dies somehow. The answer is that the cluster will regenerate the lost pod because the desired state specifies 3 pods. The cluster will always try to match the desired state.
Again, what happens when you manually edit the deployment object to replica:5 and apply the changes? Since the desired state has changed the cluster will want meet the desired state and thus will create two more pods to meet replica:5.
Rolling update
-For example you are developer and you need to update a website which you have deployed. Say the title of the webpage needs updating. How would you do it? Your system is live so you cannot bring your system down to update the web page and relaunch again.
-This is where you would use a rolling update. Rolling updates are the standard nowadays because a rolling update if done properly should have zero downtime for your end users. A rolling update basically updates a percentage of your instaces (or it could be just 1) at a time leaving enough active resources to serve customers at any time.
-Rolling update are a feature in kubernetes, it is implemented automatically if you require updates to your desired state. For example if you need to change the image (which is your application) of your deployment you can simply change the deployment object in the yaml file and apply the new changes. Then a rolling update will automatically be triggered to update all your pods with the new image. Again this is the beauty of kubernetes; rolling updates are generally not easy to implement if you need to write this from scratch. We will see an example of rolling update in our practicals.
Stateless & Stateful theory
-when deploying applications on Kubernetes there are two major architectual concepts to choose from. Application are designed with these two concepts in mind. These two concepts are called stateless and stateful.
-Stateless & stateful relate to data persistence. A stateless application does not have data persistence whereas a stateful application prersists the data.
Data persistence
- what do I mean by data persistence? In Kubernetes terms data persistence directly relate to Pods. Data is said to be persistent if after all pods in the kubernetes cluster was terminated, the data that was there before still remains. On the other hand if the data is removed after the pods in the deployment have been terminated then the data is not persisted.
-If data is not persisted your application would be considered a stateless application while if the data was persisted your application would be a stateful application.
Stateless theory
-This is a working stateless application with two deployments a frontend deployment and a backend deployment. The backend deployment process all the data.
Now if the redis pod which is a data store for our application gets terminated will the data that is stored on the redis store be kept? The answer to that is no. All the data stored in the pod gets removed.
Stateful theory
-Here we have a MySQL server Pod which stores its data on the virtual machine hard drive called a persistent volume, we will touch on this more later on. So since the data is saved on the VM and not just in the pods, even if all the MySQL pods were terminated the data will still be there.
This is an example of data persistence using a Persistent Volume on the Virtual machine which hosts these Pods.
Persistent volumes
-Persistent volumes can be defined in the .yaml file using a kubernetes object called a Persistent Volume Claim. This object will claim the allocated space from the VM and turn it into a persistent volume which then can be used for the MySQL server Pod (in this case) to store its data persistently. We will see this in action as it is the setup for the stateful applications practical.
Stateful problems
-Data persistence sounds better right? Well, in theory yes but practically no, and let's make it clear that, in a real production environment where your application is serving a wide audiance, be it a eCommerce website, social websites, etc. You will almost use stateless design.
There are certain use cases for a stateful kubernetes design however it's just not a production ready design.
- There a re major issues with a stateful design for production such as:
*It does not scale. -> In the example shown above data persistence was achived by saving the data into a persistent volume on the VM. The problem is what if we have more data then the volume on the VM. The problem is what if we have more data then the volume can store what do we do? Can you increase the volume, yes you can but not easily and also what if we have so much data that the VM does not have enough hard drive space to store them. Can we easily add another VM for storage? You see this is the problem, you cannot easily add more VMs and use it for storage.
*Data Synchronisation -> Also, lets just say you managed to add more VMs for data storage then there is a problem of synchronisation. All your data needs to be synced otherwise you will start having dara quality problems. But there is no easy way to sync the data across different VMs.
*Possible Loss of Data -> Remmeber all of this is stored on a VM. What if the VM hard drive fails, what happens to the data stored on it? You guessed it, it is lost. Which means this solution cannot possibly fit a production solution
Stateless & Stateful theory
-So what can you use stateful kubernetes application for? The stateful design can be used for quick mock ups of ideas and you do not want to think about scaling yet. Or if you know the users for your app will not grow and you are happy with just one VM's hard drive space. An example of this app could be an in house employee application. The amount of employees in your company probably won't reach millions and the workload might be okay for just a single VM to handle.
-So the question comes back to then how is the data persisted for my production application using stateless design, since the data is not persisted in stateless applications.
-Well, that is true, that is why the data is persistes outside of the kubernetes cluster, basically it is saved in an external storage solution. We cannect the Pods within kubernetes with the external storage solution to persist the data do even if all the pods were terminated we can still retrive the data from our external store. Howecver, setting this connection up may be quite time consuming and hence why you will want to use a stateful application for a quick mock up of your ideas first.
Creating Kubernetes cluster from UI:
-Click create cluster (create options)
-Click "connect to the cluster"
-somethimes you need to authenticate using "gcloud auth login mymial@gmail.com"
Kubectl commands
kubectl get pods
Pratical 1 - Guestbook
-This practical involves deploying a stateless application called Guestbook. The application conatins 3 deployments one for fronted deployments one for frontend deployment and one for redis master and another for redis slaves. All deployments have attached services both only the frontend service open to external traffic. Deployment 2 and 3 are both considered backend dseployments and handles data that comes through from the frontend. we will have a look at the yaml file which defines these objects later.
-In this practical you will learn:
*the use of kubectl commands to interact with Kube-apiserver in the master to make changes to and monitor the cluster
-desired state in action - of you delete a pod it will auto regenerate.
-viewing different resources on the cluster such as pods deployments, services and logs using kubectl.
-different services options such as LoadBalncer, NodePort and clusterIP. You will see LoadBalancer in action to further your understanding.
-scaling your frontend pods up and down simply by changing the replica number.
-Perform a rolling update on your front end application by changing the image used to generate your pods and see the rolling update in action.
kubectl apply -f frontend.yaml
kubectl apply -f redismaster.yaml
kubectl apply -f redisslave.yaml
kubectl get deployments
kubectl get svc
kubectl get pods
kubectl get pods -l tier=frontend
kubectl describe deployment -l tier=frontend
kubectl get svc
kubectl get pods --watch
kubectl logs podname
kubectl delete deployments --all
kubectl logs podname
kubectl delete deployments --all
kubectl delete svc -l tiers=frontend
kubectl delete svc -l tiers=backend
MySQL Database
-In practical we deploy a simple stateful MySQL databases application by defining a MySQL Server deployment with a single pod and defining a persistent volume claim object within the same yaml file as we can see later. The MySQL Client Pod is launched manually to connect to the MySQL server. A service is attached to the server deployment however this service has been configured to not allow external traffic.
Data is persistent volume on the virtual machine which runs the kubernetes cluster.
-Learning targets
*learn hot to setup a local persistent volume on the VM by defining a persistent volume claim object
*see in action if MySQL pods are terminated and regenerated the data in the MySQL server remains as data is persisted on the VM
kubectl delete svc -l tiers=backend
MySQL Database
-In practical we deploy a simple stateful MySQL databases application by defining a MySQL Server deployment with a single pod and defining a persistent volume claim object within the same yaml file as we can see later. The MySQL Client Pod is launched manually to connect to the MySQL server. A service is attached to the server deployment however this service has been configured to not allow external traffic.
Data is persistent volume on the virtual machine which runs the kubernetes cluster.
-Learning targets
*learn hot to setup a local persistent volume on the VM by defining a persistent volume claim object
*see in action if MySQL pods are terminated and regenerated the data in the MySQL server remains as data is persisted on the VM
kubectl apply -f frontend.yaml
kubectl apply -f frontend.yaml
/////////////////////
macOs
curl -O https://bootstrap.pypa.io/get-pip.py
python3 get-pip.py --user
pip3 install awscli --upgrade --user
aws --version
How to configure awscli
⋊> ~/.aws vim credentials
⋊> ~/.aws vim config
⋊> ~/.aws aws iam get-user
///////////////////////
http://wbarczynski.pl/
https://github.com/skarab7/kubernetes-memcached
https://github.com/kubernetes/examples
https://blog.tekspace.io/kubernetes-dashboard-remote-access/
https://pascalnaber.wordpress.com/2018/06/17/access-dashboard-on-aks-with-rbac-enabled/
https://docs.microsoft.com/en-gb/azure/aks/kubernetes-dashboard
Azure Kubernetes Service
https://github.com/Azure/AKS
https://docs.microsoft.com/en-us/cli/azure/aks?view=azure-cli-latest#az-aks-get-upgrades
https://docs.microsoft.com/en-us/azure/aks/tutorial-kubernetes-deploy-cluster
https://pascalnaber.wordpress.com/2018/01/26/persistent-storage-and-volumes-using-kubernetes-on-azure-with-aks-or-azure-container-service/
Ingress nginx
https://github.com/kubernetes/ingress-nginx
https://hackernoon.com/setting-up-nginx-ingress-on-kubernetes-2b733d8d2f45
static IP
https://docs.microsoft.com/pl-pl/azure/aks/static-ip
Komentarze
Prześlij komentarz