Commit Graph

450 Commits

Author SHA1 Message Date
Jeremi Piotrowski
4417641803 local-build: Remove GID before creating group
docker install now creates a group with gid 999 which happens to match what we
need to get docker-in-docker to work. Remove the group first as we don't need
it.

Fixes: #7726
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
(cherry picked from commit 3b881fbc0e)
2023-08-24 14:17:58 +02:00
Fabiano Fidêncio
a7f01b4456 kata-deploy: Don't try to remove /opt/kata
The directory is a host path mount and cannot be removed from within the
container.  What we actually want to remove is whatever is inside that
directory.

This may raise errors like:
```
rm: cannot remove '/opt/kata/': Device or resource busy
```

Fixes: #7746

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-08-24 14:01:45 +02:00
Fabiano Fidêncio
b64891c5f5 kata-deploy: Avoid failing on content removal
We can simply use `rm -f` all over the place and avoid the container
returning any error.

Fixes: #7733

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
(cherry picked from commit 5cba38c175)
2023-08-23 20:05:01 +02:00
Steve Horsman
0e9a8f22ca Merge pull request #7601 from ChengyuZhu6/install_dmsetup
CC | tools: Install dependencies with dm-verity in rootfs
2023-08-23 17:24:43 +01:00
Aurélien Bombo
723c44a7c4 kata-deploy: Properly create default runtime class
The default `kata` runtime class would get created with the `kata`
handler instead of `kata-$KATA_HYPERVISOR`. This made Kata use the wrong
hypervisor and broke CI.

Fixes: #7681

Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
2023-08-17 10:28:58 -07:00
ChengyuZhu6
d053f848b4 tools: Install the dependencies with dm-verity
Fixes #7636

Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
2023-08-16 21:47:52 +08:00
Fabiano Fidêncio
516468815e cc: Merge from main to CCv0 - Aug 14th
Conflicts:
	src/agent/src/rpc.rs

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-08-14 09:22:03 +02:00
Steve Fan
72cbcf040b kata-deploy: Add k0s support
Add k0s support to kata-deploy, in the very same way kata-containers
already supports k3s, and rke2.

k0s support requires v1.27.1, which is noted as part of the kata-deploy
documentation, as it's the way to use dynamic configuration on
containerd CRI runtimes.

This support will only be part of the `main` branch, as it's not a bug
fix that can be backported to the `stable-3.2` branch, and this is also
noted as part of the documentation.

Fixes: #7548
Signed-off-by: Steve Fan <29133953+stevefan1999-personal@users.noreply.github.com>
2023-08-11 21:17:23 +02:00
Fabiano Fidêncio
5f5e05a77f CC: Merge from main to CCv0 - Aug 7th, 2023
Conflicts:
	src/runtime/pkg/containerd-shim-v2/create.go
	tools/packaging/kata-deploy/local-build/kata-deploy-binaries.sh
	tools/packaging/scripts/lib.sh

Fixes: #7563
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-08-07 11:12:04 +02:00
Fabiano Fidêncio
2df6cb7609 kata-deploy: Add missing kata-remote runtimeclass
It's CCv0 specific for now, and it's needed as the Operator is now
delegating the runtimeclass creation to the kata-deploy daemonset.

Fixes: #7550

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-08-07 10:24:21 +02:00
Fabiano Fidêncio
83e866a37d kata-deploy: Add missing kata-qemu-se runtimeclass
It's CCv0 specific for now, and it's needed as the Operator is now
delegating the runtimeclass creation to the kata-deploy daemonset.

Fixes: #7550

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-08-05 20:23:54 +02:00
Fabiano Fidêncio
bde0e72da5 kata-deploy: Add missing kata-clh-tdx runtimeclass
It's CCv0 specific for now, and it's needed as the Operator is now
delegating the runtimeclass creation to the kata-deploy daemonset.

Fixes: #7550

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-08-05 20:23:42 +02:00
Fabiano Fidêncio
7164ced4dc CCv0: Merge from main -- August 1st
Conflicts:
	src/runtime/pkg/katautils/config.go
	src/runtime/virtcontainers/container.go
	src/runtime/virtcontainers/hypervisor.go
	src/runtime/virtcontainers/qemu_arch_base.go
	src/runtime/virtcontainers/sandbox.go
	tests/integration/kubernetes/gha-run.sh
	tests/integration/kubernetes/setup.sh
	tools/packaging/kata-deploy/kata-deploy/base/kata-deploy.yaml
	tools/packaging/kata-deploy/local-build/kata-deploy-binaries.sh
	tools/packaging/kata-deploy/scripts/kata-deploy.sh
	tools/packaging/kernel/kata_config_version
	versions.yaml

Fixes: #7433

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-08-04 22:15:09 +02:00
Jeremi Piotrowski
3230dec950 kata-deploy: Use host's systemctl
when interacting with systemd. We have occasionally faced issues with
compatibility between the systemctl version used inside the kata-deploy
container and the systemd version on the host. Instead of using a containerized
systemctl with bind mounted sockets, nsenter the host and run systemctl from
there. This provides less coupling between the kata-deploy container and the
host.

Fixes: #7511
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
2023-08-02 15:32:01 +02:00
Jianyong Wu
35d6d86ab5 static-build: enable cross-build for image build
It's too long a time to cross build agent based on docker buildx, thus
we cross build rootfs based on a container with cross compile toolchain
of gcc and rust with musl libc. Then we get fast build just like native
build.

rootfs initrd cross build is disabled as no cross compile tolchain for
rust with musl lib if found for alpine and based on docker buildx takes
too long a time.

Fixes: #6557
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
2023-08-01 23:28:52 +02:00
Jianyong Wu
e2c31fce23 kata-deploy: enable cross build for kata deploy script
kata-deploy-binaries-in-docker.sh is the entry to build kata components.
set some environment to facilitate the following cross build work.

Fixes: #6557
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
2023-08-01 22:10:46 +02:00
Fabiano Fidêncio
288296dacd kata-deploy: We need shim to an array, not a string
In order to do so, we need the `()` around the `shim_{arch}`.

Fixes: #7422

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-28 15:51:45 +02:00
Fabiano Fidêncio
40e678164a kata-deploy: Fix if-elif-else statement
We were doing "if - else  if - else", while bash expects "if - elif -
else", and that should never have happened in the first place, but it
happend as part of b8b73939ea

Fixes: #7422

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-28 10:36:57 +02:00
Fabiano Fidêncio
39192c6084 kata-deploy: Print variables passed to the script
This will help folks to debug / understand what's been passed to the
kata-deploy.sh script.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-28 10:04:33 +02:00
Fabiano Fidêncio
0e157be6f2 kata-deploy: Allow runtimeclasses to be created by the daemonset
Let's allow the daemonset to create the runtimeclasses, which will
decrease one manual step a user of kata-deploy should take, and also
help us in the Confidential Containers land as the Operator can just
delegate it to this script.

Fixes: #7409

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-28 10:04:33 +02:00
Fabiano Fidêncio
a274333248 kata-deploy: Change default values of DEBUG
This can be easily done as there was no official release with the
previous values.

The reason we're doing so is because when using `yq` to replace the
value, even when forcing `--tag '!!str' "yes"`, the content is placed
without quotes, causing errors in our CI.

While here, we're also removing the fallback value for DEBUG, as it is
**always** set in the kata-deploy.yaml file.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-28 09:50:39 +02:00
Fabiano Fidêncio
69535b8089 kata-deploy: runtimeclass: Split out entries
This will make things simpler to only create the handlers defined by the
kata-deploy user.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-28 09:43:45 +02:00
Fabiano Fidêncio
9e1710674a kata-runtimeClasses: Alphabetically sort the enrties
This will become handy in the near future, as we want to have separate
enrties for each file, while still keeping this one.

Having the entries sorted will make our lives easier to test those are
always in sync.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-28 09:43:45 +02:00
Fabiano Fidêncio
b8b73939ea kata-deploy: Use different shim arrays for different arches
On main we will not have this problem as we can easily configure which
shims will be installed according to an environment variable passed to
the kata-deploy.yaml file.

However, on CCV0, at least for now, we better keep the list of shims
separated by architecture, as we've found out that s390x CoCo Operator
CI is breaking because we try to install a shim that's not even built
for that architecture (dragonball).

Fixes: #7422

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-27 22:57:42 +02:00
Fabiano Fidêncio
03478ad064 kata-deploy: Add the runtime-classes that are not yet on main
This is another piece that got dropped as part of
6f552b010c and is causing regressions on
the operator tests.

Fixes: #7422

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-27 17:02:18 +02:00
Fabiano Fidêncio
61cbae6c39 Merge pull request #7463 from fidencio/topic/CCv0-converge-build-and-payload-scripts-follow-up-5
cc: kata-deploy: Configure cri-handler="cc"
2023-07-26 14:00:28 +02:00
Fabiano Fidêncio
eff98f5795 cc: kata-deploy: Configure cri-handler="cc"
This has been mistakenly dropped as part of
441399df1f

Fixes: #7422

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-26 12:57:03 +02:00
Jeremi Piotrowski
5ae7a74846 kata-deploy: Add executable bit to script
We need the executable bit set because it is preserved into the
runtime-payload-ci image.

Fixes: #7460
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
2023-07-26 12:07:37 +02:00
Fabiano Fidêncio
f6197f60b4 cc: cache: shim-v2: Re-enable cached artefact
Now that the shim-v2 for CCv0 has been rebuilt with the correct path,
let's re-enable the cache.

Fixes: #7422

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-25 21:48:20 +02:00
Fabiano Fidêncio
ad8c96b6c0 cc: cache: Enable more cached components after rebuild was done
Let's re-enabled caching for the following components, as those were
rebuilt with the new prefix:
* cc-rootfs-image
* cc-rootfs-initrd
* cc-tdx-rootfs-image
* cc-tdx-td-shim
* cc-sev-rootfs-initrd

"cc-se-image" was part of the list, but we never had a target for it.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-25 18:33:27 +02:00
Fabiano Fidêncio
2213660bf3 cc: cache: shim-v2: Allow root_hashes to be downloaded
We should not return, in case cache is not used, before actually
downloading the root_hash_*.txt provided by the other components,
otherwise the job used to do the caching will always fail.

Fixes: #7422

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-25 18:33:27 +02:00
Fabiano Fidêncio
471e23cb12 cc: kernel-tdx: Ensure we try the cache for the specific CC version
Otherwise we'd have to build the component every single time as the main
version is different from the CC one.

Fixes: #7422

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-25 13:06:43 +02:00
Fabiano Fidêncio
3f309fad01 cc: qemu-tdx: Ensure we try the cache for the specific CC version
Otherwise we'd have to build the component every single time as the main
version is different from the CC one.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-25 13:01:15 +02:00
Fabiano Fidêncio
d03685004e cc: tdvf: Ensure we try the cache for the specific CC version
Otherwise we'd have to build the component every single time as the main
version is different from the CC one.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-25 13:01:15 +02:00
Fabiano Fidêncio
0749022f8c cc: clh: Ensure we try the cache for the specific CC version
Otherwise we'd have to build the component every single time as the main
version is different from the CC one.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-25 12:41:17 +02:00
Fabiano Fidêncio
a56f96bb2b kata-deploy: Allow shim creation based on what's passed to the daemonset
Instead of hardcoding shims as part of the script, let's ensure we can
allow them to be created based on environment variables passed to the
daemonset.

This change brings no functionality change as the default values in the
daemonset are exactly what has been used as part of the scripts.

Fixes: #7407

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-25 08:30:00 +02:00
Fabiano Fidêncio
7204b991e7 kata-deploy-binaries: kernel_cache: Take module_dir into account
`module_dir` has been passed to the function but was never assigned to a
var, leading to errors when trying to use it.

Fixes: #7416

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
(cherry picked from commit d4eba36980)
2023-07-25 00:19:26 +02:00
Fabiano Fidêncio
b8abd6bfee kata-deploy-binaries: Adjust TDVF edk2 tarball name
We must use "edk2-staging-tdx" instead of "edk2-tdx".  The reason for
that is versions diverging between main and CCv0.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-25 00:18:35 +02:00
Fabiano Fidêncio
344921849c kata-deploy-binaries: Temporarily disable using cached components
We need to rebuild those with the appropriate path.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-25 00:12:21 +02:00
Fabiano Fidêncio
ef6c0be984 kata-depkoy-binarues: Add tarballs from main to the cc target
Same as the others, it'll help us in the merges.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-25 00:12:20 +02:00
Fabiano Fidêncio
20a523f81b kata-deloy-binaries: Get rid of cc_prefix
We'll be using prefix (/opt/kata) from now on, as it simplifies things
on our side.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-25 00:12:20 +02:00
Fabiano Fidêncio
4d0b319a8b kata-deploy-binaries: Remove CC OVMF / TDVF
Let's just rely on whatever we have on main.  The big execption here is
TDVF, but we have a big note saying to not update the version n this
branch.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-25 00:12:20 +02:00
Fabiano Fidêncio
8d1e1d4b0a kata-deploy-binaries: Remove CC kernel builds
We can simply rely on those coming from main.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-25 00:12:20 +02:00
Fabiano Fidêncio
3fa936e492 kata-deploy-binaires: Remove CC virtiofsd build
We can simply ship the one from main.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-25 00:12:20 +02:00
Fabiano Fidêncio
f62a88f179 kata-deploy-binaries: Remove CC hypervisor builds
We can just rely on the hypervisors builds from `main`, with the TDX one
being the only discrepancy here.

However, we have a big note in the versions.yaml to **not** update the
TDX hypervisor versions on this branch, so we should be good.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-25 00:12:20 +02:00
Fabiano Fidêncio
6f552b010c kata-deploy: Make sure kata-deploy handles kata-deploy-cc content
This will also help us immensely on main -> CCv0 merges

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-25 00:12:20 +02:00
Fabiano Fidêncio
d4eba36980 kata-deploy-binaries: kernel_cache: Take module_dir into account
`module_dir` has been passed to the function but was never assigned to a
var, leading to errors when trying to use it.

Fixes: #7416

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-24 18:19:13 +02:00
Fabiano Fidêncio
e149a3c783 Merge pull request #7404 from fidencio/topic/cache-consider-changes-in-the-scripts-used-to-build-the-kernel
cache: kernel: Consider changes in tools/packaging/kernel
2023-07-21 15:05:01 +02:00
Fabiano Fidêncio
ae6e8d2b38 kata-deploy: Properly get the path of the versions.yaml file
We need to correctly get the full path of the versions.yaml file as part
of the merge-builds.sh script, as we do a `pushd` there and that leads
to a fail merging the artefacts as the `versions.yaml` file does not
exists in that path.

Fixes: #7405

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-21 12:02:11 +02:00
Fabiano Fidêncio
309e232553 cache: kernel: Consider changes in tools/packaging/kernel
Any change in the script used to build the kernel should invalidate the
cache.

Fixes: #7403

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-21 11:48:29 +02:00