1100 Commits

Author SHA1 Message Date
Hyounggyu Choi
981f0a1f0f cc|rootfs: Define SEALED_SECRET for cc-rootfs-initrd-tarball
This is to define `SEALED_SECRET` as yes for a make target `cc-rootfs-initrd-tarball`,
which makes a service `confidential-data-hub` available with an initrd-based VM creation.

Fixes: #8210

Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
2023-10-12 14:28:46 +02:00
Fabiano Fidêncio
5e6cecf01a kata-deploy: Remove cri_handler = cc
This won't be used anymore as we won't be depending on the forked
version of containerd.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-09-25 22:58:59 +02:00
Fabiano Fidêncio
9fb0eb4076 CC: kata-deploy: Set the snapshotter in the containerd runtime config
This is a patch that should **NOT** be forward ported to main, as there
we want to take a cleaner approach on configuring specific snapshotters
for specific runtime handlers.

However, for CC, for the v0.8.0 release of CC, this is good enough as it
is, and it'll allow us to set one snapshotter for all the deployments
done with the CoCo Operator.

This is the Kata Containers counterpart of the work, and there's still
work to be done on the Confidential Containers in order to make it work
as expected, as:
* Confidential Containers Operator has to expose to the users which
  snapshotter will be configured
* Confidential Containers Opereator, specifically the pre-install hook,
  will have to take care of actually installing and configuring the
  snapshotter, so it can be used.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-09-25 22:57:18 +02:00
Fabiano Fidêncio
b588c1128c Merge pull request #7676 from ChengyuZhu6/pull_image_in_guest
CC | image pulling in the guest without forked containerd
2023-09-20 21:11:24 +02:00
Fabiano Fidêncio
3dbbbc88ac cache: Fix rootfs-image-tdx and rootfs-initrd-sev cached artefacts
The name of the tarballs changed on main, but we didn't follow up
changing this on the CCv0 branch. :-/

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-09-20 18:49:27 +02:00
ChengyuZhu6
e18a425fbb static-build: Fix arch error on nydus build
Fix the arch error when downloading the nydus tarball.

Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
Signed-off-by: Steven Horsman <steven@uk.ibm.com>
(cherry picked from commit f6df3d6efb)
2023-09-20 16:09:38 +08:00
Fabiano Fidêncio
d5d0befc61 build: Pass RUSTUP_UPDATE_ROOT & RUSTUP_DIST_SERVER down
This may help us to decrease the amount of issues we're having with the
TDX CI.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-09-18 21:05:11 +02:00
Hyounggyu Choi
5e5e78cd6f cc|osbuilder: Correct a typo in the initrd-image
This PR is to prevent rootfs.sh from running twice by fixing the typo `initrd-image`.

Fixes: #7980

Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
2023-09-15 19:18:31 +02:00
Fabiano Fidêncio
bbe0db55ac kernel: Add erofs patches needed for CC related work
All the patches have already been merged upstream and they've just been
cherry-picked to this branch.

Fixes: #7885

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
(cherry picked from commit fde34610cd)

 Conflicts:
	tools/packaging/kernel/kata_config_version
2023-09-11 13:21:21 +02:00
Fabiano Fidêncio
08b87a4eab Revert "CC | versions: Bump to 6.4.14 stable kernel" 2023-09-11 09:01:53 +02:00
Archana Shinde
911ab9c306 Merge pull request #7847 from fidencio/topic/CC-versions-update-kernel-to-6.4.14-stable
CC | versions: Bump to 6.4.14 stable kernel
2023-09-08 16:53:30 -07:00
Fabiano Fidêncio
d70ed93173 Merge pull request #7570 from LindaYu17/CCv0
CC | add sealed secret support in Kata
2023-09-07 09:46:33 +02:00
Linda Yu
212229df83 runtime: add sealed secret configuration
Fixes: #7555

Signed-off-by: Linda Yu <linda.yu@intel.com>
2023-09-07 09:55:46 +08:00
Hyounggyu Choi
c3a8ce53e6 CCv0: packaging: do not install docker-compose-plugin for s390x|ppc64le
This PR is to skip installing docker-compose-plugin while buiding a `build-kata-deploy` image for s390x|ppc64le.
It is a temporary solution to fix current CI failures for s390x regarding `hash sum mismatch`.

Fixes: #7848
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
(cherry picked from commit 2efda20c77)
2023-09-06 13:16:16 +02:00
Fabiano Fidêncio
f64041e686 kernel: Add more configs to the whitelist
This is a partial backport of 8115a0522d,
which added those configs to the whitelist.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-09-06 11:55:37 +02:00
Fabiano Fidêncio
57001431b4 versions: Bump to 6.4.14 stable kernel
This kernel update is needed in order to get the latest and greatest
commits related to EROFS, which will be used for allowing sharing the
container images between the guest and host for Confidential Containers
using the tarfs mode of EROFS.

We're removing a few options here, because:
* SECURITY_SELINUX_CHECKREQPROT_VALUE was deprecated as part of
  a7e4676e8e2c.
* CONFIG_IP_NF_TARGET_CLUSTERIP was removed as part of 9db5d918e2c0.
* CONFIG_NET_SCH_CBQ was removed as part of 051d44209842.

Fixes: #7845
Backports: #7846

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-09-06 11:03:31 +02:00
Biao Lu
acd0a75efd agent: rootfs: add sealed-secret in Makefile
When set SEALED_SECRET to "yes", the kata-agent is built with
sealed-secret capability, default value is "no".

Fixes: #7544

Signed-off-by: Biao Lu <biao.lu@intel.com>
2023-09-06 10:57:34 +08:00
Jeremi Piotrowski
4417641803 local-build: Remove GID before creating group
docker install now creates a group with gid 999 which happens to match what we
need to get docker-in-docker to work. Remove the group first as we don't need
it.

Fixes: #7726
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
(cherry picked from commit 3b881fbc0e)
2023-08-24 14:17:58 +02:00
Fabiano Fidêncio
a7f01b4456 kata-deploy: Don't try to remove /opt/kata
The directory is a host path mount and cannot be removed from within the
container.  What we actually want to remove is whatever is inside that
directory.

This may raise errors like:
```
rm: cannot remove '/opt/kata/': Device or resource busy
```

Fixes: #7746

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-08-24 14:01:45 +02:00
Fabiano Fidêncio
b64891c5f5 kata-deploy: Avoid failing on content removal
We can simply use `rm -f` all over the place and avoid the container
returning any error.

Fixes: #7733

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
(cherry picked from commit 5cba38c175)
2023-08-23 20:05:01 +02:00
Steve Horsman
0e9a8f22ca Merge pull request #7601 from ChengyuZhu6/install_dmsetup
CC | tools: Install dependencies with dm-verity in rootfs
2023-08-23 17:24:43 +01:00
Aurélien Bombo
723c44a7c4 kata-deploy: Properly create default runtime class
The default `kata` runtime class would get created with the `kata`
handler instead of `kata-$KATA_HYPERVISOR`. This made Kata use the wrong
hypervisor and broke CI.

Fixes: #7681

Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
2023-08-17 10:28:58 -07:00
ChengyuZhu6
d053f848b4 tools: Install the dependencies with dm-verity
Fixes #7636

Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
2023-08-16 21:47:52 +08:00
Fabiano Fidêncio
516468815e cc: Merge from main to CCv0 - Aug 14th
Conflicts:
	src/agent/src/rpc.rs

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-08-14 09:22:03 +02:00
Steve Fan
72cbcf040b kata-deploy: Add k0s support
Add k0s support to kata-deploy, in the very same way kata-containers
already supports k3s, and rke2.

k0s support requires v1.27.1, which is noted as part of the kata-deploy
documentation, as it's the way to use dynamic configuration on
containerd CRI runtimes.

This support will only be part of the `main` branch, as it's not a bug
fix that can be backported to the `stable-3.2` branch, and this is also
noted as part of the documentation.

Fixes: #7548
Signed-off-by: Steve Fan <29133953+stevefan1999-personal@users.noreply.github.com>
2023-08-11 21:17:23 +02:00
Fabiano Fidêncio
5f5e05a77f CC: Merge from main to CCv0 - Aug 7th, 2023
Conflicts:
	src/runtime/pkg/containerd-shim-v2/create.go
	tools/packaging/kata-deploy/local-build/kata-deploy-binaries.sh
	tools/packaging/scripts/lib.sh

Fixes: #7563
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-08-07 11:12:04 +02:00
Fabiano Fidêncio
2df6cb7609 kata-deploy: Add missing kata-remote runtimeclass
It's CCv0 specific for now, and it's needed as the Operator is now
delegating the runtimeclass creation to the kata-deploy daemonset.

Fixes: #7550

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-08-07 10:24:21 +02:00
Fabiano Fidêncio
83e866a37d kata-deploy: Add missing kata-qemu-se runtimeclass
It's CCv0 specific for now, and it's needed as the Operator is now
delegating the runtimeclass creation to the kata-deploy daemonset.

Fixes: #7550

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-08-05 20:23:54 +02:00
Fabiano Fidêncio
bde0e72da5 kata-deploy: Add missing kata-clh-tdx runtimeclass
It's CCv0 specific for now, and it's needed as the Operator is now
delegating the runtimeclass creation to the kata-deploy daemonset.

Fixes: #7550

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-08-05 20:23:42 +02:00
Fabiano Fidêncio
7164ced4dc CCv0: Merge from main -- August 1st
Conflicts:
	src/runtime/pkg/katautils/config.go
	src/runtime/virtcontainers/container.go
	src/runtime/virtcontainers/hypervisor.go
	src/runtime/virtcontainers/qemu_arch_base.go
	src/runtime/virtcontainers/sandbox.go
	tests/integration/kubernetes/gha-run.sh
	tests/integration/kubernetes/setup.sh
	tools/packaging/kata-deploy/kata-deploy/base/kata-deploy.yaml
	tools/packaging/kata-deploy/local-build/kata-deploy-binaries.sh
	tools/packaging/kata-deploy/scripts/kata-deploy.sh
	tools/packaging/kernel/kata_config_version
	versions.yaml

Fixes: #7433

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-08-04 22:15:09 +02:00
Jeremi Piotrowski
3230dec950 kata-deploy: Use host's systemctl
when interacting with systemd. We have occasionally faced issues with
compatibility between the systemctl version used inside the kata-deploy
container and the systemd version on the host. Instead of using a containerized
systemctl with bind mounted sockets, nsenter the host and run systemctl from
there. This provides less coupling between the kata-deploy container and the
host.

Fixes: #7511
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
2023-08-02 15:32:01 +02:00
Jianyong Wu
7ffc0c1225 static-build: enable cross build for qemu
Depends on mutiarch feature of ubuntu, we can set up cross build
environment easily and achive as good build performance as native
build.

Fixes: #6557
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
2023-08-01 23:28:52 +02:00
Jianyong Wu
35d6d86ab5 static-build: enable cross-build for image build
It's too long a time to cross build agent based on docker buildx, thus
we cross build rootfs based on a container with cross compile toolchain
of gcc and rust with musl libc. Then we get fast build just like native
build.

rootfs initrd cross build is disabled as no cross compile tolchain for
rust with musl lib if found for alpine and based on docker buildx takes
too long a time.

Fixes: #6557
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
2023-08-01 23:28:52 +02:00
Jianyong Wu
2205fb9d05 static-build: enable cross build for virtiofsd
Based on messense/rust-musl-cross which offer cross build musl lib
environment to cross compile virtiofsd.

Fixes: #6557
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
2023-08-01 22:10:46 +02:00
Jianyong Wu
11631c681a static-build: enable cross build for shim-v2
shim-v2 has go and rust code. For rust code, we use messense/rust-musl-cross
to build for speed up as it doesn't depends on qemu emulation. Build go
code based on docker buildx as it doesn't support cross build now.

Fixes: #6557
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
2023-08-01 22:10:46 +02:00
Jianyong Wu
7923de8999 static-build: cross build kernel
Prepare cross build environment based on current Dockerfile.

Fixes: #6557
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
2023-08-01 22:10:46 +02:00
Jianyong Wu
e2c31fce23 kata-deploy: enable cross build for kata deploy script
kata-deploy-binaries-in-docker.sh is the entry to build kata components.
set some environment to facilitate the following cross build work.

Fixes: #6557
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
2023-08-01 22:10:46 +02:00
Jianyong Wu
2fc5f0e2e0 kata-depoly: prepare env for cross build in lib.sh
We leverage three env, TARGET_ARCH means the buid target tuple;
ARCH nearly the same meaning with TARGET_ARCH but has been widely
used in kata; CROSS_BUILD means if you want to do cross compile.

Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
2023-08-01 22:10:46 +02:00
Fabiano Fidêncio
f5e9985afe release: release-note.sh: Fix typos and reference to images
diferent -> different

And also let's make sure we escape the backticks around the kata-deploy
environment variables, otherwise bash will try to interpret those.

Fixes: #7497

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-08-01 12:42:03 +02:00
Fabiano Fidêncio
288296dacd kata-deploy: We need shim to an array, not a string
In order to do so, we need the `()` around the `shim_{arch}`.

Fixes: #7422

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-28 15:51:45 +02:00
Fabiano Fidêncio
40e678164a kata-deploy: Fix if-elif-else statement
We were doing "if - else  if - else", while bash expects "if - elif -
else", and that should never have happened in the first place, but it
happend as part of b8b73939ea

Fixes: #7422

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-28 10:36:57 +02:00
Fabiano Fidêncio
39192c6084 kata-deploy: Print variables passed to the script
This will help folks to debug / understand what's been passed to the
kata-deploy.sh script.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-28 10:04:33 +02:00
Fabiano Fidêncio
0e157be6f2 kata-deploy: Allow runtimeclasses to be created by the daemonset
Let's allow the daemonset to create the runtimeclasses, which will
decrease one manual step a user of kata-deploy should take, and also
help us in the Confidential Containers land as the Operator can just
delegate it to this script.

Fixes: #7409

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-28 10:04:33 +02:00
Fabiano Fidêncio
a274333248 kata-deploy: Change default values of DEBUG
This can be easily done as there was no official release with the
previous values.

The reason we're doing so is because when using `yq` to replace the
value, even when forcing `--tag '!!str' "yes"`, the content is placed
without quotes, causing errors in our CI.

While here, we're also removing the fallback value for DEBUG, as it is
**always** set in the kata-deploy.yaml file.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-28 09:50:39 +02:00
Fabiano Fidêncio
69535b8089 kata-deploy: runtimeclass: Split out entries
This will make things simpler to only create the handlers defined by the
kata-deploy user.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-28 09:43:45 +02:00
Fabiano Fidêncio
9e1710674a kata-runtimeClasses: Alphabetically sort the enrties
This will become handy in the near future, as we want to have separate
enrties for each file, while still keeping this one.

Having the entries sorted will make our lives easier to test those are
always in sync.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-28 09:43:45 +02:00
Fabiano Fidêncio
b8b73939ea kata-deploy: Use different shim arrays for different arches
On main we will not have this problem as we can easily configure which
shims will be installed according to an environment variable passed to
the kata-deploy.yaml file.

However, on CCV0, at least for now, we better keep the list of shims
separated by architecture, as we've found out that s390x CoCo Operator
CI is breaking because we try to install a shim that's not even built
for that architecture (dragonball).

Fixes: #7422

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-27 22:57:42 +02:00
Fabiano Fidêncio
03478ad064 kata-deploy: Add the runtime-classes that are not yet on main
This is another piece that got dropped as part of
6f552b010c and is causing regressions on
the operator tests.

Fixes: #7422

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-27 17:02:18 +02:00
Fabiano Fidêncio
61cbae6c39 Merge pull request #7463 from fidencio/topic/CCv0-converge-build-and-payload-scripts-follow-up-5
cc: kata-deploy: Configure cri-handler="cc"
2023-07-26 14:00:28 +02:00
Fabiano Fidêncio
eff98f5795 cc: kata-deploy: Configure cri-handler="cc"
This has been mistakenly dropped as part of
441399df1f

Fixes: #7422

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-26 12:57:03 +02:00