aljaz/kata-containers
1
0
Fork 0
mirror of https://github.com/aljazceru/kata-containers.git synced 2026-01-14 03:44:25 +01:00
Code Issues Packages Projects Releases Wiki Activity
Files
fc012a2baba4debe456b8609625c720ab2ccdf05
kata-containers/tools/packaging/kata-deploy
History
Snir Sheriber a2e524f356 release: Kata Containers 2.3.1 
- stable-2.3 | kata-deploy: fix tar command in dockerfile
- stable-2.3 | versions: Upgrade to Cloud Hypervisor v20.2
- stable-2.3 Missing backports
- stable-2.3 | docs: Fix kernel configs README spelling errors
- docs: Fix outdated links
- stable-2.3 | versions: Upgrade to Cloud Hypervisor v20.1
- Backport osbuilder: Revert to using apk.static for Alpine
- stable-2.3 | runtime: only call stopVirtiofsd when shared_fs is virtio-fs
- Backport versions: Use Ubuntu initrd for non-musl archs
- stable-2.3 | Upgrade to Cloud Hypervisor v20.0 and Openapi-generator v5.3.0
- stable-2.3 | packaging: Fix missing commit message in building kata-runtime
- stable-2.3 | runtime: enable vhost-net for rootless hypervisor
- [backport] agent: create directories for watchable-bind mounts
- runtime: enable FUSE_DAX kernel config for DAX

dfbe74c4 kata-deploy: fix tar command in dockerfile
9e7eed7c versions: Upgrade to Cloud Hypervisor v20.2
53cf1dd0 tools/packaging: add copyright to kata-monitor's Dockerfile
a4dee6a5 packaging: delint tests dockerfiles
fd87b60c packaging: delint kata-deploy dockerfiles
2cb4f7ba ci/openshift-ci: delint dockerfiles
993dcc94 osbuilder: delint dockerfiles
bbd7cc2f packaging: delint kata-monitor dockerfiles
9837ec72 packaging: delint static-build dockerfiles
8785106f packaging/qemu: Use QEMU script to update submodules
a915f082 packaging/qemu: Use partial git clone
ec3faab8 security: Update rust crate versions
1f61be84 osbuilder: Add protoc to the alpine container
d2d8f9ac osbuilder: avoid to copy versions.txt which already deprecated
ca30eee3 kata-manager: Retrieve static tarball
0217abce kata-deploy: Deal with empty containerd conf file
572b25dd osbuilder: be runtime consistent also with podman build
84e69ecb agent: user container ID as watchable storage key for hashmap
77b6cfbd docs: Fix kernel configs README spelling errors
24085c95 docs: Fix outdated k8s link
514bf74f docs: Replicate branch rename on runtime-spec
77a2502a cri-o: Update links for the CRI-O github page
6413ecf4 docs: Backport source reorganization links
a0bed72d versions: Upgrade to Cloud Hypervisor v20.1
d03e05e8 versions: Use fixed, minor version for Alpine
0f7db91c osbuilder: Revert to using apk.static for Alpine
271d67a8 runtime: only call stopVirtiofsd when shared_fs is virtio-fs
7c15335d versions: Use Ubuntu initrd for non-musl archs
15080f20 virtcontainers: clh: Upgrade to openapi-generator v5.3.0
c2b8eb3c virtcontainers: clh: Re-generate the client code
fe0fbab5 versions: Upgrade to Cloud Hypervisor v20.0
be5468fd packaging: Fix missing commit message in building kata-runtime
18bb9a5d runtime: enable vhost-net for rootless hypervisor
3458073d agent: create directories for watchable-bind mounts
0e91503c runtime: enable FUSE_DAX kernel config for DAX

Signed-off-by: Snir Sheriber <ssheribe@redhat.com>
2022-01-06 20:51:21 +02:00
..
action
kata-deploy: fix tar command in dockerfile
2022-01-06 08:26:36 +02:00
examples
kata-deploy: QEMU, for 2.x, already includes virtiofs
2021-02-23 14:24:48 +01:00
kata-cleanup
release: Kata Containers 2.3.1
2022-01-06 20:51:21 +02:00
kata-deploy
release: Kata Containers 2.3.1
2022-01-06 20:51:21 +02:00
kata-rbac/base
packaging: merge packaging repository
2020-06-23 22:58:18 -07:00
local-build
packaging: delint kata-deploy dockerfiles
2022-01-03 10:52:53 +02:00
runtimeclasses
kata-deploy: Remove qemu-virtiofs runtime class
2021-09-21 22:48:04 +02:00
scripts
kata-deploy: Deal with empty containerd conf file
2021-12-29 14:39:14 +02:00
.dockerignore
kata-deploy: add .dockerignore file
2021-10-15 12:00:14 +02:00
Dockerfile
packaging: delint kata-deploy dockerfiles
2022-01-03 10:52:53 +02:00
README.md
kata-deploy: Add more info about the stable tag
2021-09-21 23:13:45 +02:00

README.md

kata-deploy

kata-deploy provides a Dockerfile, which contains all of the binaries and artifacts required to run Kata Containers, as well as reference DaemonSets, which can be utilized to install Kata Containers on a running Kubernetes cluster.

Note, installation through DaemonSets successfully installs katacontainers.io/kata-runtime on a node only if it uses either containerd or CRI-O CRI-shims.

Kubernetes quick start

Install Kata on a running Kubernetes cluster

Installing the latest image

The latest image refers to pre-release and release candidate content. For stable releases, please, use the "stable" instructions.

$ kubectl apply -f https://raw.githubusercontent.com/kata-containers/kata-containers/main/tools/packaging/kata-deploy/kata-rbac/base/kata-rbac.yaml
$ kubectl apply -f https://raw.githubusercontent.com/kata-containers/kata-containers/main/tools/packaging/kata-deploy/kata-deploy/base/kata-deploy.yaml

Installing the stable image

The stable image refers to the last stable releases content.

Note that if you use a tagged version of the repo, the stable image does match that version. For instance, if you use the 2.2.1 tagged version of the kata-deploy.yaml file, then the version 2.2.1 of the kata runtime will be deployed.

$ kubectl apply -f https://raw.githubusercontent.com/kata-containers/kata-containers/main/tools/packaging/kata-deploy/kata-rbac/base/kata-rbac.yaml
$ kubectl apply -f https://raw.githubusercontent.com/kata-containers/kata-containers/main/tools/packaging/kata-deploy/kata-deploy/base/kata-deploy-stable.yaml

For your k3s cluster, do:

$ GO111MODULE=auto go get github.com/kata-containers/kata-containers
$ cd $GOPATH/src/github.com/kata-containers/kata-containers/tools/packaging/kata-deploy
$ kubectl apply -k kata-deploy/overlays/k3s

Ensure kata-deploy is ready

kubectl -n kube-system wait --timeout=10m --for=condition=Ready -l name=kata-deploy pod

Run a sample workload

Workloads specify the runtime they'd like to utilize by setting the appropriate runtimeClass object within the Pod specification. The runtimeClass examples provided define a node selector to match node label katacontainers.io/kata-runtime:"true", which will ensure the workload is only scheduled on a node that has Kata Containers installed

runtimeClass is a built-in type in Kubernetes. To apply each Kata Containers runtimeClass:

  $ kubectl apply -f https://raw.githubusercontent.com/kata-containers/kata-containers/main/tools/packaging/kata-deploy/runtimeclasses/kata-runtimeClasses.yaml

The following YAML snippet shows how to specify a workload should use Kata with Cloud Hypervisor:

spec:
  template:
    spec:
      runtimeClassName: kata-clh

The following YAML snippet shows how to specify a workload should use Kata with Firecracker:

spec:
  template:
    spec:
      runtimeClassName: kata-fc

The following YAML snippet shows how to specify a workload should use Kata with QEMU:

spec:
  template:
    spec:
      runtimeClassName: kata-qemu

To run an example with kata-clh:

$ kubectl apply -f https://raw.githubusercontent.com/kata-containers/kata-containers/main/tools/packaging/kata-deploy/examples/test-deploy-kata-clh.yaml

To run an example with kata-fc:

$ kubectl apply -f https://raw.githubusercontent.com/kata-containers/kata-containers/main/tools/packaging/kata-deploy/examples/test-deploy-kata-fc.yaml

To run an example with kata-qemu:

$ kubectl apply -f https://raw.githubusercontent.com/kata-containers/kata-containers/main/tools/packaging/kata-deploy/examples/test-deploy-kata-qemu.yaml

The following removes the test pods:

$ kubectl delete -f https://raw.githubusercontent.com/kata-containers/kata-containers/main/tools/packaging/kata-deploy/examples/test-deploy-kata-clh.yaml
$ kubectl delete -f https://raw.githubusercontent.com/kata-containers/kata-containers/main/tools/packaging/kata-deploy/examples/test-deploy-kata-fc.yaml
$ kubectl delete -f https://raw.githubusercontent.com/kata-containers/kata-containers/main/tools/packaging/kata-deploy/examples/test-deploy-kata-qemu.yaml

Remove Kata from the Kubernetes cluster

Removing the latest image

$ kubectl delete -f https://raw.githubusercontent.com/kata-containers/kata-containers/main/tools/packaging/kata-deploy/kata-deploy/base/kata-deploy.yaml
$ kubectl -n kube-system wait --timeout=10m --for=delete -l name=kata-deploy pod

After ensuring kata-deploy has been deleted, cleanup the cluster:

$ kubectl apply -f https://raw.githubusercontent.com/kata-containers/kata-containers/main/tools/packaging/kata-deploy/kata-cleanup/base/kata-cleanup.yaml

The cleanup daemon-set will run a single time, cleaning up the node-label, which makes it difficult to check in an automated fashion. This process should take, at most, 5 minutes.

After that, let's delete the cleanup daemon-set, the added RBAC and runtime classes:

$ kubectl delete -f https://raw.githubusercontent.com/kata-containers/kata-containers/main/tools/packaging/kata-deploy/kata-cleanup/base/kata-cleanup.yaml
$ kubectl delete -f https://raw.githubusercontent.com/kata-containers/kata-containers/main/tools/packaging/kata-deploy/kata-rbac/base/kata-rbac.yaml
$ kubectl delete -f https://raw.githubusercontent.com/kata-containers/kata-containers/main/tools/packaging/kata-deploy/runtimeclasses/kata-runtimeClasses.yaml

Removing the stable image

$ kubectl delete -f https://raw.githubusercontent.com/kata-containers/kata-containers/main/tools/packaging/kata-deploy/kata-deploy/base/kata-deploy-stable.yaml
$ kubectl -n kube-system wait --timeout=10m --for=delete -l name=kata-deploy pod

After ensuring kata-deploy has been deleted, cleanup the cluster:

$ kubectl apply -f https://raw.githubusercontent.com/kata-containers/kata-containers/main/tools/packaging/kata-deploy/kata-cleanup/base/kata-cleanup-stabe.yaml

The cleanup daemon-set will run a single time, cleaning up the node-label, which makes it difficult to check in an automated fashion. This process should take, at most, 5 minutes.

After that, let's delete the cleanup daemon-set, the added RBAC and runtime classes:

$ kubectl delete -f https://raw.githubusercontent.com/kata-containers/kata-containers/main/tools/packaging/kata-deploy/kata-cleanup/base/kata-cleanup-stable.yaml
$ kubectl delete -f https://raw.githubusercontent.com/kata-containers/kata-containers/main/tools/packaging/kata-deploy/kata-rbac/base/kata-rbac.yaml
$ kubectl delete -f https://raw.githubusercontent.com/kata-containers/kata-containers/main/tools/packaging/kata-deploy/runtimeclasses/kata-runtimeClasses.yaml

kata-deploy details

Dockerfile

The Dockerfile used to create the container image deployed in the DaemonSet is provided here. This image contains all the necessary artifacts for running Kata Containers, all of which are pulled from the Kata Containers release page.

Host artifacts:

  • cloud-hypervisor, firecracker, qemu-system-x86_64, and supporting binaries
  • containerd-shim-kata-v2
  • kata-collect-data.sh
  • kata-runtime

Virtual Machine artifacts:

  • kata-containers.img and kata-containers-initrd.img: pulled from Kata GitHub releases page
  • vmlinuz.container and vmlinuz-virtiofs.container: pulled from Kata GitHub releases page

DaemonSets and RBAC

Two DaemonSets are introduced for kata-deploy, as well as an RBAC to facilitate applying labels to the nodes.

Kata deploy

This DaemonSet installs the necessary Kata binaries, configuration files, and virtual machine artifacts on the node. Once installed, the DaemonSet adds a node label katacontainers.io/kata-runtime=true and reconfigures either CRI-O or containerd to register three runtimeClasses: kata-clh (for Cloud Hypervisor isolation), kata-qemu (for QEMU isolation), and kata-fc (for Firecracker isolation). As a final step the DaemonSet restarts either CRI-O or containerd. Upon deletion, the DaemonSet removes the Kata binaries and VM artifacts and updates the node label to katacontainers.io/kata-runtime=cleanup.

Kata cleanup

This DaemonSet runs of the node has the label katacontainers.io/kata-runtime=cleanup. These DaemonSets removes the katacontainers.io/kata-runtime label as well as restarts either CRI-O or containerd systemctl daemon. You cannot execute these resets during the preStopHook of the Kata installer DaemonSet, which necessitated this final cleanup DaemonSet.