Commit Graph

1423 Commits

Author SHA1 Message Date
ChengyuZhu6
e18a425fbb static-build: Fix arch error on nydus build
Fix the arch error when downloading the nydus tarball.

Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
Signed-off-by: Steven Horsman <steven@uk.ibm.com>
(cherry picked from commit f6df3d6efb)
2023-09-20 16:09:38 +08:00
Fabiano Fidêncio
d5d0befc61 build: Pass RUSTUP_UPDATE_ROOT & RUSTUP_DIST_SERVER down
This may help us to decrease the amount of issues we're having with the
TDX CI.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-09-18 21:05:11 +02:00
Hyounggyu Choi
5e5e78cd6f cc|osbuilder: Correct a typo in the initrd-image
This PR is to prevent rootfs.sh from running twice by fixing the typo `initrd-image`.

Fixes: #7980

Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
2023-09-15 19:18:31 +02:00
Fabiano Fidêncio
b71443a7ae Merge pull request #7878 from ChengyuZhu6/check_rust
CC | osbuilder: check rust enviornment before building agent and AA
2023-09-13 17:15:59 +02:00
ChengyuZhu6
8f38dcb850 osbuilder: check rust enviornment before building agent and AA
We should configure the Rust environment when AGENT_SOURCE_BIN is empty or AA_KBC is not empty.

Fixes #7877

Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
2023-09-13 15:22:49 +08:00
ChengyuZhu6
fd33309475 image-builder: fix udev error when using docker to build image
Incorporate the `DM_VERITY` parameter when building the image with docker.

Fixes #7879

Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
2023-09-13 09:34:22 +08:00
Fabiano Fidêncio
bbe0db55ac kernel: Add erofs patches needed for CC related work
All the patches have already been merged upstream and they've just been
cherry-picked to this branch.

Fixes: #7885

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
(cherry picked from commit fde34610cd)

 Conflicts:
	tools/packaging/kernel/kata_config_version
2023-09-11 13:21:21 +02:00
Fabiano Fidêncio
08b87a4eab Revert "CC | versions: Bump to 6.4.14 stable kernel" 2023-09-11 09:01:53 +02:00
Archana Shinde
911ab9c306 Merge pull request #7847 from fidencio/topic/CC-versions-update-kernel-to-6.4.14-stable
CC | versions: Bump to 6.4.14 stable kernel
2023-09-08 16:53:30 -07:00
Fabiano Fidêncio
d70ed93173 Merge pull request #7570 from LindaYu17/CCv0
CC | add sealed secret support in Kata
2023-09-07 09:46:33 +02:00
Biao Lu
e865359f4e osbuilder: add api-server-rest in rootfs
Integrate api-server-rest into rootfs image.

Fixes: #7555

Signed-off-by: Biao Lu <biao.lu@intel.com>
2023-09-07 09:55:46 +08:00
Linda Yu
212229df83 runtime: add sealed secret configuration
Fixes: #7555

Signed-off-by: Linda Yu <linda.yu@intel.com>
2023-09-07 09:55:46 +08:00
Hyounggyu Choi
c3a8ce53e6 CCv0: packaging: do not install docker-compose-plugin for s390x|ppc64le
This PR is to skip installing docker-compose-plugin while buiding a `build-kata-deploy` image for s390x|ppc64le.
It is a temporary solution to fix current CI failures for s390x regarding `hash sum mismatch`.

Fixes: #7848
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
(cherry picked from commit 2efda20c77)
2023-09-06 13:16:16 +02:00
Fabiano Fidêncio
f64041e686 kernel: Add more configs to the whitelist
This is a partial backport of 8115a0522d,
which added those configs to the whitelist.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-09-06 11:55:37 +02:00
Fabiano Fidêncio
57001431b4 versions: Bump to 6.4.14 stable kernel
This kernel update is needed in order to get the latest and greatest
commits related to EROFS, which will be used for allowing sharing the
container images between the guest and host for Confidential Containers
using the tarfs mode of EROFS.

We're removing a few options here, because:
* SECURITY_SELINUX_CHECKREQPROT_VALUE was deprecated as part of
  a7e4676e8e2c.
* CONFIG_IP_NF_TARGET_CLUSTERIP was removed as part of 9db5d918e2c0.
* CONFIG_NET_SCH_CBQ was removed as part of 051d44209842.

Fixes: #7845
Backports: #7846

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-09-06 11:03:31 +02:00
Biao Lu
b4092023bf osbuilder: add confidential-data-hub in rootfs
Fixes: #7544

Signed-off-by: Biao Lu <biao.lu@intel.com>
2023-09-06 10:57:34 +08:00
Biao Lu
acd0a75efd agent: rootfs: add sealed-secret in Makefile
When set SEALED_SECRET to "yes", the kata-agent is built with
sealed-secret capability, default value is "no".

Fixes: #7544

Signed-off-by: Biao Lu <biao.lu@intel.com>
2023-09-06 10:57:34 +08:00
Fabiano Fidêncio
badba8058c rootfs: Fail in case attestation-agent fails to build
Today I learned, I must say.

When running a basic script, such as:
```bash
 #/usr/bin/env bash

 set -o errexit
 set -o pipefail
 set -o errtrace

 cat junk && echo "hello"
 echo "didn't fail"

 cat junk
 echo "hello"
 echo "didn't fail"
```

One will get as a result:
```bash
cat: junk: No such file or directory
didn't fail
cat: junk: No such file or directory
```

Meaning that although there was an error on `cat junk && echo "hello"`,
and the `echo "hello"` part was not executed, an error was not reported
for that failure.

On the second part, though, it just breaks and returns an error as
expected.

Small scripts aside, this is exactly what was happening with the
attestation-agent, where a `make ... && make install ...` was being
called, make was failing but not actually breaking the script.

Let's change the logic and avoid such situations in the future, as it
caused our CI to be broken for quite some time without a simple way to
detect that line in the huge amount of logs left behind.

Here goes a reference to the documentation:
```
-e      Exit immediately if a pipeline (which may consist
        of a single simple command), a list, or a compound
        command (see SHELL GRAMMAR above), exits with a
        non-zero status.  The shell does not exit if the
        command that fails is part of the command list
        immediately following a while or until keyword,
        part of the test following the if or elif reserved
        words, part of any command executed in a && or ||
        list except the command following the final && or
        ||, any command in a pipeline but the last, or if
        the command's return value is being inverted with
        !.  If a compound command other than a subshell
        returns a non-zero status because a command failed
        while -e was being ignored, the shell does not
        exit.  A trap on ERR, if set, is executed before
        the shell exits.  This option applies to the shell
        environment and each subshell environment
        separately (see COMMAND EXECUTION ENVIRONMENT
        above), and may cause subshells to exit before
        executing all the commands in the subshell.

        If a compound command or shell function executes
        in a context where -e is being ignored, none of
        the commands executed within the compound command
        or function body will be affected by the -e
        setting, even if -e is set and a command returns a
        failure status.  If a compound command or shell
        function sets -e while executing in a context
        where -e is ignored, that setting will not have
        any effect until the compound command or the
        command containing the function call completes.
```

This comes from https://www.man7.org/linux/man-pages/man1/bash.1.html

Fixes: #7793

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-08-30 17:27:27 +02:00
Jeremi Piotrowski
4417641803 local-build: Remove GID before creating group
docker install now creates a group with gid 999 which happens to match what we
need to get docker-in-docker to work. Remove the group first as we don't need
it.

Fixes: #7726
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
(cherry picked from commit 3b881fbc0e)
2023-08-24 14:17:58 +02:00
Fabiano Fidêncio
a7f01b4456 kata-deploy: Don't try to remove /opt/kata
The directory is a host path mount and cannot be removed from within the
container.  What we actually want to remove is whatever is inside that
directory.

This may raise errors like:
```
rm: cannot remove '/opt/kata/': Device or resource busy
```

Fixes: #7746

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-08-24 14:01:45 +02:00
Fabiano Fidêncio
aef93c7aaf Merge pull request #7576 from surajssd/update-aa-imagers-tdshim
Update AA, image-rs td-shim
2023-08-24 08:38:29 +02:00
Fabiano Fidêncio
b64891c5f5 kata-deploy: Avoid failing on content removal
We can simply use `rm -f` all over the place and avoid the container
returning any error.

Fixes: #7733

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
(cherry picked from commit 5cba38c175)
2023-08-23 20:05:01 +02:00
Steve Horsman
0e9a8f22ca Merge pull request #7601 from ChengyuZhu6/install_dmsetup
CC | tools: Install dependencies with dm-verity in rootfs
2023-08-23 17:24:43 +01:00
Aurélien Bombo
723c44a7c4 kata-deploy: Properly create default runtime class
The default `kata` runtime class would get created with the `kata`
handler instead of `kata-$KATA_HYPERVISOR`. This made Kata use the wrong
hypervisor and broke CI.

Fixes: #7681

Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
2023-08-17 10:28:58 -07:00
ChengyuZhu6
d053f848b4 tools: Install the dependencies with dm-verity
Fixes #7636

Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
2023-08-16 21:47:52 +08:00
Suraj Deshmukh
32d347aa25 tools/static-checks: Install clang
Without this library the builds are failing with the following error:

```
...
error: failed to run custom build command for `devicemapper-sys v0.1.5`

Caused by: process didn't exit successfully:
    `/kata-containers/src/agent/target/release/build/devicemapper-sys-d8eae524a127e049/build-script-build`
    (exit status: 101) --- stderr thread 'main' panicked at 'Unable to
    find libclang: "couldn't find any valid shared libraries matching:
    ['libclang.so', 'libclang-*.so', 'libclang.so.*', 'libclang-*.so.*'],
    set the `LIBCLANG_PATH` environment variable to a path where one of
    these files can be found (invalid: [])"',
    /root/.cargo/registry/src/github.com-1ecc6299db9ec823/bindgen-0.63.0/./lib.rs:2338:31
```

Fixes: #7580

Signed-off-by: Suraj Deshmukh <suraj.deshmukh@microsoft.com>
2023-08-16 13:12:49 +00:00
Fabiano Fidêncio
516468815e cc: Merge from main to CCv0 - Aug 14th
Conflicts:
	src/agent/src/rpc.rs

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-08-14 09:22:03 +02:00
Steve Fan
72cbcf040b kata-deploy: Add k0s support
Add k0s support to kata-deploy, in the very same way kata-containers
already supports k3s, and rke2.

k0s support requires v1.27.1, which is noted as part of the kata-deploy
documentation, as it's the way to use dynamic configuration on
containerd CRI runtimes.

This support will only be part of the `main` branch, as it's not a bug
fix that can be backported to the `stable-3.2` branch, and this is also
noted as part of the documentation.

Fixes: #7548
Signed-off-by: Steve Fan <29133953+stevefan1999-personal@users.noreply.github.com>
2023-08-11 21:17:23 +02:00
Fabiano Fidêncio
5f5e05a77f CC: Merge from main to CCv0 - Aug 7th, 2023
Conflicts:
	src/runtime/pkg/containerd-shim-v2/create.go
	tools/packaging/kata-deploy/local-build/kata-deploy-binaries.sh
	tools/packaging/scripts/lib.sh

Fixes: #7563
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-08-07 11:12:04 +02:00
Fabiano Fidêncio
2df6cb7609 kata-deploy: Add missing kata-remote runtimeclass
It's CCv0 specific for now, and it's needed as the Operator is now
delegating the runtimeclass creation to the kata-deploy daemonset.

Fixes: #7550

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-08-07 10:24:21 +02:00
Fabiano Fidêncio
83e866a37d kata-deploy: Add missing kata-qemu-se runtimeclass
It's CCv0 specific for now, and it's needed as the Operator is now
delegating the runtimeclass creation to the kata-deploy daemonset.

Fixes: #7550

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-08-05 20:23:54 +02:00
Fabiano Fidêncio
bde0e72da5 kata-deploy: Add missing kata-clh-tdx runtimeclass
It's CCv0 specific for now, and it's needed as the Operator is now
delegating the runtimeclass creation to the kata-deploy daemonset.

Fixes: #7550

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-08-05 20:23:42 +02:00
Fabiano Fidêncio
7164ced4dc CCv0: Merge from main -- August 1st
Conflicts:
	src/runtime/pkg/katautils/config.go
	src/runtime/virtcontainers/container.go
	src/runtime/virtcontainers/hypervisor.go
	src/runtime/virtcontainers/qemu_arch_base.go
	src/runtime/virtcontainers/sandbox.go
	tests/integration/kubernetes/gha-run.sh
	tests/integration/kubernetes/setup.sh
	tools/packaging/kata-deploy/kata-deploy/base/kata-deploy.yaml
	tools/packaging/kata-deploy/local-build/kata-deploy-binaries.sh
	tools/packaging/kata-deploy/scripts/kata-deploy.sh
	tools/packaging/kernel/kata_config_version
	versions.yaml

Fixes: #7433

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-08-04 22:15:09 +02:00
Jeremi Piotrowski
3230dec950 kata-deploy: Use host's systemctl
when interacting with systemd. We have occasionally faced issues with
compatibility between the systemctl version used inside the kata-deploy
container and the systemd version on the host. Instead of using a containerized
systemctl with bind mounted sockets, nsenter the host and run systemctl from
there. This provides less coupling between the kata-deploy container and the
host.

Fixes: #7511
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
2023-08-02 15:32:01 +02:00
Jianyong Wu
7ffc0c1225 static-build: enable cross build for qemu
Depends on mutiarch feature of ubuntu, we can set up cross build
environment easily and achive as good build performance as native
build.

Fixes: #6557
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
2023-08-01 23:28:52 +02:00
Jianyong Wu
35d6d86ab5 static-build: enable cross-build for image build
It's too long a time to cross build agent based on docker buildx, thus
we cross build rootfs based on a container with cross compile toolchain
of gcc and rust with musl libc. Then we get fast build just like native
build.

rootfs initrd cross build is disabled as no cross compile tolchain for
rust with musl lib if found for alpine and based on docker buildx takes
too long a time.

Fixes: #6557
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
2023-08-01 23:28:52 +02:00
Jianyong Wu
2205fb9d05 static-build: enable cross build for virtiofsd
Based on messense/rust-musl-cross which offer cross build musl lib
environment to cross compile virtiofsd.

Fixes: #6557
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
2023-08-01 22:10:46 +02:00
Jianyong Wu
11631c681a static-build: enable cross build for shim-v2
shim-v2 has go and rust code. For rust code, we use messense/rust-musl-cross
to build for speed up as it doesn't depends on qemu emulation. Build go
code based on docker buildx as it doesn't support cross build now.

Fixes: #6557
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
2023-08-01 22:10:46 +02:00
Jianyong Wu
7923de8999 static-build: cross build kernel
Prepare cross build environment based on current Dockerfile.

Fixes: #6557
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
2023-08-01 22:10:46 +02:00
Jianyong Wu
e2c31fce23 kata-deploy: enable cross build for kata deploy script
kata-deploy-binaries-in-docker.sh is the entry to build kata components.
set some environment to facilitate the following cross build work.

Fixes: #6557
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
2023-08-01 22:10:46 +02:00
Jianyong Wu
2fc5f0e2e0 kata-depoly: prepare env for cross build in lib.sh
We leverage three env, TARGET_ARCH means the buid target tuple;
ARCH nearly the same meaning with TARGET_ARCH but has been widely
used in kata; CROSS_BUILD means if you want to do cross compile.

Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
2023-08-01 22:10:46 +02:00
Fabiano Fidêncio
f5e9985afe release: release-note.sh: Fix typos and reference to images
diferent -> different

And also let's make sure we escape the backticks around the kata-deploy
environment variables, otherwise bash will try to interpret those.

Fixes: #7497

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-08-01 12:42:03 +02:00
Fabiano Fidêncio
288296dacd kata-deploy: We need shim to an array, not a string
In order to do so, we need the `()` around the `shim_{arch}`.

Fixes: #7422

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-28 15:51:45 +02:00
Fabiano Fidêncio
40e678164a kata-deploy: Fix if-elif-else statement
We were doing "if - else  if - else", while bash expects "if - elif -
else", and that should never have happened in the first place, but it
happend as part of b8b73939ea

Fixes: #7422

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-28 10:36:57 +02:00
Fabiano Fidêncio
39192c6084 kata-deploy: Print variables passed to the script
This will help folks to debug / understand what's been passed to the
kata-deploy.sh script.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-28 10:04:33 +02:00
Fabiano Fidêncio
0e157be6f2 kata-deploy: Allow runtimeclasses to be created by the daemonset
Let's allow the daemonset to create the runtimeclasses, which will
decrease one manual step a user of kata-deploy should take, and also
help us in the Confidential Containers land as the Operator can just
delegate it to this script.

Fixes: #7409

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-28 10:04:33 +02:00
Fabiano Fidêncio
a274333248 kata-deploy: Change default values of DEBUG
This can be easily done as there was no official release with the
previous values.

The reason we're doing so is because when using `yq` to replace the
value, even when forcing `--tag '!!str' "yes"`, the content is placed
without quotes, causing errors in our CI.

While here, we're also removing the fallback value for DEBUG, as it is
**always** set in the kata-deploy.yaml file.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-28 09:50:39 +02:00
Fabiano Fidêncio
69535b8089 kata-deploy: runtimeclass: Split out entries
This will make things simpler to only create the handlers defined by the
kata-deploy user.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-28 09:43:45 +02:00
Fabiano Fidêncio
9e1710674a kata-runtimeClasses: Alphabetically sort the enrties
This will become handy in the near future, as we want to have separate
enrties for each file, while still keeping this one.

Having the entries sorted will make our lives easier to test those are
always in sync.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-28 09:43:45 +02:00
Fabiano Fidêncio
b8b73939ea kata-deploy: Use different shim arrays for different arches
On main we will not have this problem as we can easily configure which
shims will be installed according to an environment variable passed to
the kata-deploy.yaml file.

However, on CCV0, at least for now, we better keep the list of shims
separated by architecture, as we've found out that s390x CoCo Operator
CI is breaking because we try to install a shim that's not even built
for that architecture (dragonball).

Fixes: #7422

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-27 22:57:42 +02:00