Commit Graph

1382 Commits

Author SHA1 Message Date
Fabiano Fidêncio
7164ced4dc CCv0: Merge from main -- August 1st
Conflicts:
	src/runtime/pkg/katautils/config.go
	src/runtime/virtcontainers/container.go
	src/runtime/virtcontainers/hypervisor.go
	src/runtime/virtcontainers/qemu_arch_base.go
	src/runtime/virtcontainers/sandbox.go
	tests/integration/kubernetes/gha-run.sh
	tests/integration/kubernetes/setup.sh
	tools/packaging/kata-deploy/kata-deploy/base/kata-deploy.yaml
	tools/packaging/kata-deploy/local-build/kata-deploy-binaries.sh
	tools/packaging/kata-deploy/scripts/kata-deploy.sh
	tools/packaging/kernel/kata_config_version
	versions.yaml

Fixes: #7433

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-08-04 22:15:09 +02:00
Fabiano Fidêncio
288296dacd kata-deploy: We need shim to an array, not a string
In order to do so, we need the `()` around the `shim_{arch}`.

Fixes: #7422

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-28 15:51:45 +02:00
Fabiano Fidêncio
40e678164a kata-deploy: Fix if-elif-else statement
We were doing "if - else  if - else", while bash expects "if - elif -
else", and that should never have happened in the first place, but it
happend as part of b8b73939ea

Fixes: #7422

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-28 10:36:57 +02:00
Fabiano Fidêncio
39192c6084 kata-deploy: Print variables passed to the script
This will help folks to debug / understand what's been passed to the
kata-deploy.sh script.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-28 10:04:33 +02:00
Fabiano Fidêncio
0e157be6f2 kata-deploy: Allow runtimeclasses to be created by the daemonset
Let's allow the daemonset to create the runtimeclasses, which will
decrease one manual step a user of kata-deploy should take, and also
help us in the Confidential Containers land as the Operator can just
delegate it to this script.

Fixes: #7409

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-28 10:04:33 +02:00
Fabiano Fidêncio
a274333248 kata-deploy: Change default values of DEBUG
This can be easily done as there was no official release with the
previous values.

The reason we're doing so is because when using `yq` to replace the
value, even when forcing `--tag '!!str' "yes"`, the content is placed
without quotes, causing errors in our CI.

While here, we're also removing the fallback value for DEBUG, as it is
**always** set in the kata-deploy.yaml file.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-28 09:50:39 +02:00
Fabiano Fidêncio
69535b8089 kata-deploy: runtimeclass: Split out entries
This will make things simpler to only create the handlers defined by the
kata-deploy user.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-28 09:43:45 +02:00
Fabiano Fidêncio
9e1710674a kata-runtimeClasses: Alphabetically sort the enrties
This will become handy in the near future, as we want to have separate
enrties for each file, while still keeping this one.

Having the entries sorted will make our lives easier to test those are
always in sync.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-28 09:43:45 +02:00
Fabiano Fidêncio
b8b73939ea kata-deploy: Use different shim arrays for different arches
On main we will not have this problem as we can easily configure which
shims will be installed according to an environment variable passed to
the kata-deploy.yaml file.

However, on CCV0, at least for now, we better keep the list of shims
separated by architecture, as we've found out that s390x CoCo Operator
CI is breaking because we try to install a shim that's not even built
for that architecture (dragonball).

Fixes: #7422

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-27 22:57:42 +02:00
Fabiano Fidêncio
03478ad064 kata-deploy: Add the runtime-classes that are not yet on main
This is another piece that got dropped as part of
6f552b010c and is causing regressions on
the operator tests.

Fixes: #7422

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-27 17:02:18 +02:00
Fabiano Fidêncio
61cbae6c39 Merge pull request #7463 from fidencio/topic/CCv0-converge-build-and-payload-scripts-follow-up-5
cc: kata-deploy: Configure cri-handler="cc"
2023-07-26 14:00:28 +02:00
Fabiano Fidêncio
eff98f5795 cc: kata-deploy: Configure cri-handler="cc"
This has been mistakenly dropped as part of
441399df1f

Fixes: #7422

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-26 12:57:03 +02:00
Jeremi Piotrowski
5ae7a74846 kata-deploy: Add executable bit to script
We need the executable bit set because it is preserved into the
runtime-payload-ci image.

Fixes: #7460
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
2023-07-26 12:07:37 +02:00
Hyounggyu Choi
46f04d762f Merge pull request #7448 from fidencio/topic/CCv0-converge-build-and-payload-scripts-follow-up-4
cc: cache: shim-v2: Re-enable cached artefact
2023-07-26 08:18:53 +02:00
Hyounggyu Choi
a578266b26 CCv0: fix prefix to kata for IBM SE image build
This is to change a prefix from `confidential-containers` to `kata` for IBM SE image build.

Fixes: #7444

Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
2023-07-25 22:54:18 +02:00
Hyounggyu Choi
78262695d2 Merge pull request #7445 from BbolroC/remove-cc-from-kernel-ibm-se
CCv0: Remove `cc` from kernel for IBM SE image build
2023-07-25 21:52:01 +02:00
Fabiano Fidêncio
f6197f60b4 cc: cache: shim-v2: Re-enable cached artefact
Now that the shim-v2 for CCv0 has been rebuilt with the correct path,
let's re-enable the cache.

Fixes: #7422

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-25 21:48:20 +02:00
Hyounggyu Choi
1093e71cc5 CCv0: Remove cc from kernel for IBM SE image build
This is a quick fix for the error on IBM SE image build.

Fixes: #7444

Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
2023-07-25 19:46:31 +02:00
Fabiano Fidêncio
ad8c96b6c0 cc: cache: Enable more cached components after rebuild was done
Let's re-enabled caching for the following components, as those were
rebuilt with the new prefix:
* cc-rootfs-image
* cc-rootfs-initrd
* cc-tdx-rootfs-image
* cc-tdx-td-shim
* cc-sev-rootfs-initrd

"cc-se-image" was part of the list, but we never had a target for it.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-25 18:33:27 +02:00
Fabiano Fidêncio
2213660bf3 cc: cache: shim-v2: Allow root_hashes to be downloaded
We should not return, in case cache is not used, before actually
downloading the root_hash_*.txt provided by the other components,
otherwise the job used to do the caching will always fail.

Fixes: #7422

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-25 18:33:27 +02:00
Fabiano Fidêncio
cdf04e5018 Merge pull request #7437 from jepio/fix-sev-kernel-cache
cache: kernel: Fix kernel caching
2023-07-25 18:10:03 +02:00
Fabiano Fidêncio
471e23cb12 cc: kernel-tdx: Ensure we try the cache for the specific CC version
Otherwise we'd have to build the component every single time as the main
version is different from the CC one.

Fixes: #7422

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-25 13:06:43 +02:00
Fabiano Fidêncio
3f309fad01 cc: qemu-tdx: Ensure we try the cache for the specific CC version
Otherwise we'd have to build the component every single time as the main
version is different from the CC one.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-25 13:01:15 +02:00
Fabiano Fidêncio
d03685004e cc: tdvf: Ensure we try the cache for the specific CC version
Otherwise we'd have to build the component every single time as the main
version is different from the CC one.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-25 13:01:15 +02:00
Fabiano Fidêncio
0749022f8c cc: clh: Ensure we try the cache for the specific CC version
Otherwise we'd have to build the component every single time as the main
version is different from the CC one.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-25 12:41:17 +02:00
Jeremi Piotrowski
c8ac56569a cache: kernel: Harmonize commit with fetching side
kata-deploy-binaries.sh uses the last commit in
tools/packaging/static-build/kernel for its version check, while the cache
generation uses tools/packaging/kernel. Use tools/packaging/static-build/kernel
as $kata_config_version is already part of the version string and covers any
changes to tools/packaging/kernel.

Fixes: #7403
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
2023-07-25 12:23:05 +02:00
Jeremi Piotrowski
81775ab1b3 cache: kernel: Fix SEV kernel caching
The SEV kernel cache calls create_cache_asset() twice, once for the kernel and
once for modules. Both calls need to use the same version string, otherwise the
second call overwrites the "latest" file of the first one and the cache is not
used.

Fixes: #7403
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
2023-07-25 11:58:19 +02:00
Fabiano Fidêncio
a56f96bb2b kata-deploy: Allow shim creation based on what's passed to the daemonset
Instead of hardcoding shims as part of the script, let's ensure we can
allow them to be created based on environment variables passed to the
daemonset.

This change brings no functionality change as the default values in the
daemonset are exactly what has been used as part of the scripts.

Fixes: #7407

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-25 08:30:00 +02:00
Fabiano Fidêncio
7204b991e7 kata-deploy-binaries: kernel_cache: Take module_dir into account
`module_dir` has been passed to the function but was never assigned to a
var, leading to errors when trying to use it.

Fixes: #7416

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
(cherry picked from commit d4eba36980)
2023-07-25 00:19:26 +02:00
Fabiano Fidêncio
b8abd6bfee kata-deploy-binaries: Adjust TDVF edk2 tarball name
We must use "edk2-staging-tdx" instead of "edk2-tdx".  The reason for
that is versions diverging between main and CCv0.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-25 00:18:35 +02:00
Fabiano Fidêncio
0f022d5771 guest-image: Update kernel_module_dir to main sev kernel
As we're building SEV kernel from the main branch, we can stop relying
on the path produced by the one from the CCv0 branch (which is now
removed).

Fixes: #7422

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-25 00:18:35 +02:00
Fabiano Fidêncio
344921849c kata-deploy-binaries: Temporarily disable using cached components
We need to rebuild those with the appropriate path.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-25 00:12:21 +02:00
Fabiano Fidêncio
ef6c0be984 kata-depkoy-binarues: Add tarballs from main to the cc target
Same as the others, it'll help us in the merges.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-25 00:12:20 +02:00
Fabiano Fidêncio
20a523f81b kata-deloy-binaries: Get rid of cc_prefix
We'll be using prefix (/opt/kata) from now on, as it simplifies things
on our side.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-25 00:12:20 +02:00
Fabiano Fidêncio
4d0b319a8b kata-deploy-binaries: Remove CC OVMF / TDVF
Let's just rely on whatever we have on main.  The big execption here is
TDVF, but we have a big note saying to not update the version n this
branch.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-25 00:12:20 +02:00
Fabiano Fidêncio
8d1e1d4b0a kata-deploy-binaries: Remove CC kernel builds
We can simply rely on those coming from main.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-25 00:12:20 +02:00
Fabiano Fidêncio
3fa936e492 kata-deploy-binaires: Remove CC virtiofsd build
We can simply ship the one from main.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-25 00:12:20 +02:00
Fabiano Fidêncio
f62a88f179 kata-deploy-binaries: Remove CC hypervisor builds
We can just rely on the hypervisors builds from `main`, with the TDX one
being the only discrepancy here.

However, we have a big note in the versions.yaml to **not** update the
TDX hypervisor versions on this branch, so we should be good.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-25 00:12:20 +02:00
Fabiano Fidêncio
6f552b010c kata-deploy: Make sure kata-deploy handles kata-deploy-cc content
This will also help us immensely on main -> CCv0 merges

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-25 00:12:20 +02:00
David Esparza
b11d618a3f Merge pull request #7413 from fidencio/topic/release-publish-builder-images
release: Mention the container images used to build the project
2023-07-24 15:46:31 -06:00
Fabiano Fidêncio
d4eba36980 kata-deploy-binaries: kernel_cache: Take module_dir into account
`module_dir` has been passed to the function but was never assigned to a
var, leading to errors when trying to use it.

Fixes: #7416

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-24 18:19:13 +02:00
Fabiano Fidêncio
b7c9867d60 release: Mention the container images used to build the project
This is a small step towards build reproducibility.

Fixes: #7412

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-24 18:01:57 +02:00
Fabiano Fidêncio
6a59e227b6 Merge pull request #7399 from fidencio/topic/add-kata-debug
packaging/tools: Add kata-debug and use it as part of our CI
2023-07-21 17:05:27 +02:00
Fabiano Fidêncio
e149a3c783 Merge pull request #7404 from fidencio/topic/cache-consider-changes-in-the-scripts-used-to-build-the-kernel
cache: kernel: Consider changes in tools/packaging/kernel
2023-07-21 15:05:01 +02:00
Fabiano Fidêncio
38a7b5325f packaging/tools: Add kata-debug
kata-debug is a tool that is used as part of the Kata Containers CI to gather
information from the node, in order to help debugging issues with Kata
Containers.

As one can imagine, this can be expanded and used outside of the CI context,
and any contribution back to the script is very much welcome.

The resulting container is stored at the [Kata Containers quay.io
space](https://quay.io/repository/kata-containers/kata-debug) and can
be used as shown below:
```sh
kubectl debug $NODE_NAME -it --image=quay.io/kata-containers/kata-debug:latest
```

Fixes: #7397

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-21 14:05:30 +02:00
Fabiano Fidêncio
ae6e8d2b38 kata-deploy: Properly get the path of the versions.yaml file
We need to correctly get the full path of the versions.yaml file as part
of the merge-builds.sh script, as we do a `pushd` there and that leads
to a fail merging the artefacts as the `versions.yaml` file does not
exists in that path.

Fixes: #7405

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-21 12:02:11 +02:00
Fabiano Fidêncio
309e232553 cache: kernel: Consider changes in tools/packaging/kernel
Any change in the script used to build the kernel should invalidate the
cache.

Fixes: #7403

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-21 11:48:29 +02:00
Fabiano Fidêncio
59fdd69b85 kata-deploy: Add VERSION and versions.yaml to the final tarball
Let's make things simpler to figure out which version of Kata
Containers has been deployed, and also which artefacts come with it.

This will help us immensely in the future, for the TEEs use case, so we
can easily know whether we can deploy a specific guest kernel for a
specific host kernel.

Fixes: #7394

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-20 18:33:14 +02:00
Fabiano Fidêncio
87d99a71ec versions: Remove "kernel-experimental"
We've not been using nor shipping this kernel for a very long time.

Regardless, we're leaving behind the logic in the kernel scripts to
build it, in case it becomes necessary in the future.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-20 17:14:22 +02:00
Wainer dos Santos Moschetta
94b3ab8339 versions: migrate out of k8s.gcr.io
The k8s.gcr.io is deprecated for a while now and has been redirected to
registry.k8s.io. However on some bare-metal machines in our testing
pools that redirection is not working, so let's just replace the
registries.

Fixes #6461
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
2023-07-18 15:17:59 +02:00