aljaz/kata-containers
1
0
Fork 0
mirror of https://github.com/aljazceru/kata-containers.git synced 2026-01-21 23:34:22 +01:00
Code Issues Packages Projects Releases Wiki Activity
Files
c4e8e86acfaf4c387802f5c7cb1fc0cdf5e7d081
kata-containers/tools/packaging/kata-deploy
History
Fabiano Fidêncio 688cc8e2bd release: Kata Containers 2.2.1 
- stable-2.2 | watcher: ensure we create target mount point for storage
- stable-2.2 | virtiofs: Create shared directory with 0700 mode, not 0750
- [backport]sandbox: Allow the device to be accessed,such as /dev/null and /dev/u…
- stable-2.2 | kata-deploy: Also provide "stable" & "latest" tags
- stable-2.2 | runtime: tracing: Fix logger passed in newContainer
- stable-2.2 | runtime: tracing: Use root context to stop tracing
- packaging: Backport QEMU's GitLab switch to 5.1.x
- stable-2.2 | workflows,release: Upload the vendored cargo code
- backport: Call agent shutdown test only in the correspondent CI_JOB
- packaging: Backport QEMU's switch to GitLab repos
- stable-2.2 | virtcontainers: fc: parse vcpuID correctly
- shimv2: Backport fixes for #2527
- backport-2.2: remove default config for arm64.
- stable-2.2 | versions: Upgrade to Cloud Hypervisor v18.0
- [backport]sandbox: Add device permissions such as /dev/null to cgroup
- [backport] runtime: Fix README link
- [backport] snap: Test variable instead of executing "branch"

d9b41fc5 watcher: ensure we create target mount point for storage
2b6327ac kata-deploy: Add more info about the stable tag
5256e085 kata-deploy: Improve README
02b46268 kata-deploy: Remove qemu-virtiofs runtime class
1b3058dd release: update the kata-deploy yaml files accordingly
98e2e935 kata-deploy: Add "stable" info to the README
8f25c7da kata-deploy: Update the README
84da2f8d workflows: Add "stable" & "latest" tags to kata-deploy
5c76f1c6 packaging: Backport QEMU's GitLab switch to 5.1.x
ba6fc328 packaging: Backport QEMU's switch to GitLab repos
d5f5da43 workflows,release: Upload the vendored cargo code
017cd3c5 ci: Call agent shutdown test only in the correspondent CI_JOB
2ca867da runtime: Add container field to logs
f4da502c shimv2: add information to method comment
16164241 shimv2: add logging to shimv2 api calls
25c7e118 virtiofs: Create shared directory with 0700 mode, not 0750
4c5bf057 virtcontainers: fc: parse vcpuID correctly
b3e620db runtime: tracing: Fix logger passed in newContainer
98c2ca13 runtime: tracing: Use root context to stop tracing
0481c507 backport-2.2: remove default config for arm64.
56920bc9 sandbox: Allow the device to be accessed,such as /dev/null and /dev/urandom
a1874ccd virtcontainers: clh: Revert the workaround incorrect default values
c2c65050 virtcontainers: clh: Re-generate the client code
7ee43f94 versions: Upgrade to Cloud Hypervisor v18.0
1792a9fe runtime: Fix README link
807cc8a3 sandbox: Add device permissions such as /dev/null to cgroup
5987f3b5 snap: Test variable instead of executing "branch"

Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
2021-09-24 12:34:35 +02:00
..
action
tools: Update the image repository to quay.io
2021-08-05 22:53:20 +02:00
examples
kata-deploy: QEMU, for 2.x, already includes virtiofs
2021-02-23 14:24:48 +01:00
kata-cleanup
release: Kata Containers 2.2.1
2021-09-24 12:34:35 +02:00
kata-deploy
release: Kata Containers 2.2.1
2021-09-24 12:34:35 +02:00
kata-rbac/base
packaging: merge packaging repository
2020-06-23 22:58:18 -07:00
local-build
kata-deploy: shorten directory path
2021-08-02 22:37:39 +02:00
runtimeclasses
kata-deploy: Remove qemu-virtiofs runtime class
2021-09-23 14:04:50 +02:00
scripts
kata-deploy: Adapt CRI-O config to use drop-in files
2021-04-30 23:14:19 +02:00
Dockerfile
kata-deploy: Rely directly on a centos:7 image
2021-08-06 13:22:45 +02:00
README.md
kata-deploy: Add more info about the stable tag
2021-09-23 14:05:17 +02:00

README.md

kata-deploy

kata-deploy provides a Dockerfile, which contains all of the binaries and artifacts required to run Kata Containers, as well as reference DaemonSets, which can be utilized to install Kata Containers on a running Kubernetes cluster.

Note, installation through DaemonSets successfully installs katacontainers.io/kata-runtime on a node only if it uses either containerd or CRI-O CRI-shims.

Kubernetes quick start

Install Kata on a running Kubernetes cluster

Installing the latest image

The latest image refers to pre-release and release candidate content. For stable releases, please, use the "stable" instructions.

$ kubectl apply -f https://raw.githubusercontent.com/kata-containers/kata-containers/main/tools/packaging/kata-deploy/kata-rbac/base/kata-rbac.yaml
$ kubectl apply -f https://raw.githubusercontent.com/kata-containers/kata-containers/main/tools/packaging/kata-deploy/kata-deploy/base/kata-deploy.yaml

Installing the stable image

The stable image refers to the last stable releases content.

Note that if you use a tagged version of the repo, the stable image does match that version. For instance, if you use the 2.2.1 tagged version of the kata-deploy.yaml file, then the version 2.2.1 of the kata runtime will be deployed.

$ kubectl apply -f https://raw.githubusercontent.com/kata-containers/kata-containers/main/tools/packaging/kata-deploy/kata-rbac/base/kata-rbac.yaml
$ kubectl apply -f https://raw.githubusercontent.com/kata-containers/kata-containers/main/tools/packaging/kata-deploy/kata-deploy/base/kata-deploy-stable.yaml

For your k3s cluster, do:

$ GO111MODULE=auto go get github.com/kata-containers/kata-containers
$ cd $GOPATH/src/github.com/kata-containers/kata-containers/tools/packaging/kata-deploy
$ kubectl apply -k kata-deploy/overlays/k3s

Ensure kata-deploy is ready

kubectl -n kube-system wait --timeout=10m --for=condition=Ready -l name=kata-deploy pod

Run a sample workload

Workloads specify the runtime they'd like to utilize by setting the appropriate runtimeClass object within the Pod specification. The runtimeClass examples provided define a node selector to match node label katacontainers.io/kata-runtime:"true", which will ensure the workload is only scheduled on a node that has Kata Containers installed

runtimeClass is a built-in type in Kubernetes. To apply each Kata Containers runtimeClass:

  $ kubectl apply -f https://raw.githubusercontent.com/kata-containers/kata-containers/main/tools/packaging/kata-deploy/runtimeclasses/kata-runtimeClasses.yaml

The following YAML snippet shows how to specify a workload should use Kata with Cloud Hypervisor:

spec:
  template:
    spec:
      runtimeClassName: kata-clh

The following YAML snippet shows how to specify a workload should use Kata with Firecracker:

spec:
  template:
    spec:
      runtimeClassName: kata-fc

The following YAML snippet shows how to specify a workload should use Kata with QEMU:

spec:
  template:
    spec:
      runtimeClassName: kata-qemu

To run an example with kata-clh:

$ kubectl apply -f https://raw.githubusercontent.com/kata-containers/kata-containers/main/tools/packaging/kata-deploy/examples/test-deploy-kata-clh.yaml

To run an example with kata-fc:

$ kubectl apply -f https://raw.githubusercontent.com/kata-containers/kata-containers/main/tools/packaging/kata-deploy/examples/test-deploy-kata-fc.yaml

To run an example with kata-qemu:

$ kubectl apply -f https://raw.githubusercontent.com/kata-containers/kata-containers/main/tools/packaging/kata-deploy/examples/test-deploy-kata-qemu.yaml

The following removes the test pods:

$ kubectl delete -f https://raw.githubusercontent.com/kata-containers/kata-containers/main/tools/packaging/kata-deploy/examples/test-deploy-kata-clh.yaml
$ kubectl delete -f https://raw.githubusercontent.com/kata-containers/kata-containers/main/tools/packaging/kata-deploy/examples/test-deploy-kata-fc.yaml
$ kubectl delete -f https://raw.githubusercontent.com/kata-containers/kata-containers/main/tools/packaging/kata-deploy/examples/test-deploy-kata-qemu.yaml

Remove Kata from the Kubernetes cluster

Removing the latest image

$ kubectl delete -f https://raw.githubusercontent.com/kata-containers/kata-containers/main/tools/packaging/kata-deploy/kata-deploy/base/kata-deploy.yaml
$ kubectl -n kube-system wait --timeout=10m --for=delete -l name=kata-deploy pod

After ensuring kata-deploy has been deleted, cleanup the cluster:

$ kubectl apply -f https://raw.githubusercontent.com/kata-containers/kata-containers/main/tools/packaging/kata-deploy/kata-cleanup/base/kata-cleanup.yaml

The cleanup daemon-set will run a single time, cleaning up the node-label, which makes it difficult to check in an automated fashion. This process should take, at most, 5 minutes.

After that, let's delete the cleanup daemon-set, the added RBAC and runtime classes:

$ kubectl delete -f https://raw.githubusercontent.com/kata-containers/kata-containers/main/tools/packaging/kata-deploy/kata-cleanup/base/kata-cleanup.yaml
$ kubectl delete -f https://raw.githubusercontent.com/kata-containers/kata-containers/main/tools/packaging/kata-deploy/kata-rbac/base/kata-rbac.yaml
$ kubectl delete -f https://raw.githubusercontent.com/kata-containers/kata-containers/main/tools/packaging/kata-deploy/runtimeclasses/kata-runtimeClasses.yaml

Removing the stable image

$ kubectl delete -f https://raw.githubusercontent.com/kata-containers/kata-containers/main/tools/packaging/kata-deploy/kata-deploy/base/kata-deploy-stable.yaml
$ kubectl -n kube-system wait --timeout=10m --for=delete -l name=kata-deploy pod

After ensuring kata-deploy has been deleted, cleanup the cluster:

$ kubectl apply -f https://raw.githubusercontent.com/kata-containers/kata-containers/main/tools/packaging/kata-deploy/kata-cleanup/base/kata-cleanup-stabe.yaml

The cleanup daemon-set will run a single time, cleaning up the node-label, which makes it difficult to check in an automated fashion. This process should take, at most, 5 minutes.

After that, let's delete the cleanup daemon-set, the added RBAC and runtime classes:

$ kubectl delete -f https://raw.githubusercontent.com/kata-containers/kata-containers/main/tools/packaging/kata-deploy/kata-cleanup/base/kata-cleanup-stable.yaml
$ kubectl delete -f https://raw.githubusercontent.com/kata-containers/kata-containers/main/tools/packaging/kata-deploy/kata-rbac/base/kata-rbac.yaml
$ kubectl delete -f https://raw.githubusercontent.com/kata-containers/kata-containers/main/tools/packaging/kata-deploy/runtimeclasses/kata-runtimeClasses.yaml

kata-deploy details

Dockerfile

The Dockerfile used to create the container image deployed in the DaemonSet is provided here. This image contains all the necessary artifacts for running Kata Containers, all of which are pulled from the Kata Containers release page.

Host artifacts:

  • cloud-hypervisor, firecracker, qemu-system-x86_64, and supporting binaries
  • containerd-shim-kata-v2
  • kata-collect-data.sh
  • kata-runtime

Virtual Machine artifacts:

  • kata-containers.img and kata-containers-initrd.img: pulled from Kata GitHub releases page
  • vmlinuz.container and vmlinuz-virtiofs.container: pulled from Kata GitHub releases page

DaemonSets and RBAC

Two DaemonSets are introduced for kata-deploy, as well as an RBAC to facilitate applying labels to the nodes.

Kata deploy

This DaemonSet installs the necessary Kata binaries, configuration files, and virtual machine artifacts on the node. Once installed, the DaemonSet adds a node label katacontainers.io/kata-runtime=true and reconfigures either CRI-O or containerd to register three runtimeClasses: kata-clh (for Cloud Hypervisor isolation), kata-qemu (for QEMU isolation), and kata-fc (for Firecracker isolation). As a final step the DaemonSet restarts either CRI-O or containerd. Upon deletion, the DaemonSet removes the Kata binaries and VM artifacts and updates the node label to katacontainers.io/kata-runtime=cleanup.

Kata cleanup

This DaemonSet runs of the node has the label katacontainers.io/kata-runtime=cleanup. These DaemonSets removes the katacontainers.io/kata-runtime label as well as restarts either CRI-O or containerd systemctl daemon. You cannot execute these resets during the preStopHook of the Kata installer DaemonSet, which necessitated this final cleanup DaemonSet.