Commit Graph

143 Commits

Author SHA1 Message Date
stevenhorsman
9f3b2aaf6a CCv0: Merge main into CCv0 branch
Merge remote-tracking branch 'upstream/main' into CCv0

Fixes: #3573
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2022-02-14 16:25:09 +00:00
Bin Liu
cf53ec2c71 Merge pull request #2977 from luodw/support_nydus
feature(nydusd): add nydusd support to introduce lazyload ability
2022-02-14 13:08:50 +08:00
luodaowen.backend
2d9f89aec7 feature(nydusd): add nydusd support to introduse lazyload ability
Pulling image is the most time-consuming step in the container lifecycle. This PR
introduse nydus to kata container, it can lazily pull image when container start. So it
can speed up kata container create and start.

Fixes #2724

Signed-off-by: luodaowen.backend <luodaowen.backend@bytedance.com>
2022-02-11 21:41:17 +08:00
Daniel Höxtermann
b19b6938a8 docs: Fix relative links in Markdown
Relative links within this repository allow for easier navigation to
the corresponding file / directory in the current commit / for the
selected version.

Link text was slightly changed / fixed in
- docs/Unit-Test-Advice.md
- docs/how-to/how-to-run-docker-with-kata.md

Fixes #3045

Signed-off-by: Daniel Höxtermann <daniel@hxtm.dev>
2022-02-11 13:49:42 +01:00
Manabu Sugimoto
df6ae1e789 osbuilder: Remove libseccomp from Dockerfile
Remove the libseccomp package from Dockerfile of `alpine` and `clearlinux`
because the libseccomp library is installed by the `ci/install_libseccomp.sh`
script when building the kata-agent.

Fixes: #3508

Signed-off-by: Manabu Sugimoto <Manabu.Sugimoto@sony.com>
2022-01-21 00:32:57 +09:00
Carlos Venegas
5f41e199dd Merge pull request #3438 from haslersn/usr-bin-env-bash
scripts: Use shebang /usr/bin/env bash
2022-01-17 15:39:42 -06:00
Sebastian Hasler
adffd3f8b6 scripts: Use shebang /usr/bin/env bash
Not all distros have `/bin/bash`, e.g. NixOS.

Fixes: #3450

Signed-off-by: Sebastian Hasler <sebastian.hasler@stuvus.uni-stuttgart.de>
2022-01-13 22:53:28 +01:00
Fabiano Fidêncio
e4b7a12bf3 qat: Add Debian to the distro examples
Debian is a supported rootfs that uses systemd as init, thus, it should
be mentioned in the QAT README document.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-01-13 21:54:05 +01:00
Fabiano Fidêncio
6979d5be69 osbuilder: Remove gentoo rootfs-builder
As the gentoo rootfs is not tested in our CI, we can't guarantee it
actually works as expected.

Whenever we have someone willing to maintain this rootfs we can have it
added back, and also add a CI job to test it altogether, avoiding then
any possible regression.

Fixes: #2144

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-01-13 21:54:05 +01:00
Fabiano Fidêncio
22c1a093d7 osbuilder: Remove suse rootfs-builder
As the suse rootfs is not tested in our CI, we can't guarantee it
actually works as expected.

Whenver we have someone willing to maintain this rootfs we can have it
added back, and also add a CI job to test it altogether, avoiding then
any possible regression.

Fixes: #2145

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-01-13 21:54:05 +01:00
Fabiano Fidêncio
85dd587382 osbuilder: Remove fedora rootfs-builder
As the fedora rootfs is not tested in our CI, we can't guarantee it
actually works as expected.

Whenever we have someone willing to maintain the rootfs we can have it
added back, and also add a CI job to test it altogether, avoiding then
any possible regression.

Fixes: #2143

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-01-13 21:54:05 +01:00
Fabiano Fidêncio
06fae29f49 osbuilder: Remove centos rootfs-builder
As the centos rootfs is not tested in our CI, we can't guarantee it
actually works as expected.

Whenever we have someone willing to maintain the rootfs we can have it
added back, and also add a CI job to test it altogether, avoiding then
any possible regression.

Fixes: #2140

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-01-13 21:54:05 +01:00
Bin Liu
b2166560fa Merge pull request #3375 from zhaojizhuang/debianrootfs
osbuilder: Restore Debian as a rootfs
2022-01-05 10:27:47 +08:00
stevenhorsman
5f7115eca8 CCv0: Merge main into CCv0 branch
Merge remote-tracking branch 'upstream/main' into CCv0

Fixes: #3376
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2022-01-04 10:29:03 +00:00
zhaojizhuang
3093f93a6f osbuilder: Restore Debian as a rootfs
Restore Debian as a rootfs.
1. revert of #3154, but some change
2. update debian version to 10.11
3. update  `libstdc++-6-dev` to `libstdc++-8-dev`
4.  changes discarded in QAT are not restored

Fixes: #3372
Signed-off-by: zhaojizhuang <571130360@qq.com>
2022-01-04 11:54:34 +08:00
Wainer Moschetta
820dc930db Merge pull request #3109 from wainersm/delint_dockerfiles
Delint dockerfiles
2021-12-28 10:11:51 -03:00
Fabiano Fidêncio
67f0ab4092 Merge pull request #3294 from Kvasscn/kata_dev_osbuilder_makefile
osbuilder: avoid to copy versions.txt which already deprecated
2021-12-21 16:07:01 +01:00
Wainer dos Santos Moschetta
aeb2b673b3 osbuilder: delint dockerfiles
Removed all errors/warnings pointed out by hadolint version 2.7.0, except for the following
ignored rules:
  - "DL3008 warning: Pin versions in apt get install"
  - "DL3041 warning: Specify version with `dnf install -y <package>-<version>`"
  - "DL3033 warning: Specify version with `yum install -y <package>-<version>`"
  - "DL3048 style: Invalid label key"
  - "DL3003 warning: Use WORKDIR to switch to a directory"
  - "DL3018 warning: Pin versions in apk add. Instead of apk add <package> use apk add <package>=<version>"
  - "DL3037 warning: Specify version with zypper install -y <package>[=]<version>"

Fixes #3107
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
2021-12-21 09:54:44 -05:00
Fabiano Fidêncio
99ef52a35d osbuilder: Add protoc to the alpine container
It seems the lack of protoc in the alpine containers is causing issues
with some of our CIs, such as the VFIO one.

Fixes: #3323

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2021-12-21 13:57:18 +01:00
stevenhorsman
85bb1e5e45 CCv0: Merge main into CCv0 branch
Merge remote-tracking branch 'upstream/main' into CCv0

Fixes: #3295

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2021-12-17 10:33:30 +00:00
zhanghj
d1bc409d57 osbuilder: avoid to copy versions.txt which already deprecated
Currently the versions.txt in rootfs-builder dir is already removed,
so avoid to copy it in list of helper files.

Fixes: #3267

Signed-off-by: zhanghj <zhanghj.lc@inspur.com>
2021-12-17 17:23:05 +08:00
stevenhorsman
e1ba87408c CCv0: Merge main into CCv0 branch
Weekly merge of new commits in main into CCv0

Fixes: #3259

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2021-12-13 14:15:32 +00:00
bin
3f7cf7ae67 osbuilder: show usage if no options/arguments specified
Now if no options/arguments specified, the shell scripts will return an error:

ERROR: Invalid rootfs directory: ''

This commit will show usage if no options/arguments specified.

Fixes: #3256

Signed-off-by: bin <bin@hyper.sh>
2021-12-13 16:10:55 +08:00
Bin Liu
978b13c9e8 Merge pull request #3235 from Kvasscn/kata_dev_image_builer_help
image_build: add help info for '-f' option and 'BLOCK_SIZE' env.
2021-12-09 22:55:24 +08:00
Julio Montes
70062e1563 Merge pull request #3238 from snir911/wip/build_with_runtime
osbuilder: be runtime consistent with podman build
2021-12-09 08:06:00 -06:00
Snir Sheriber
2ebaaac73d osbuilder: be runtime consistent also with podman build
Use the same runtime used for podman run also for the podman build cmd
Additionally remove "docker" from the docker_run_args variable

Fixes: #3239
Signed-off-by: Snir Sheriber <ssheribe@redhat.com>
2021-12-09 11:28:16 +02:00
Jakob Naucke
2204ecac39 versions: Upgrade Alpine, using minor version
- Upgrade Alpine guest rootfs to 3.15
- Specify a minor version rather than patch level as the Alpine
  repositories use that.

Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
2021-12-08 15:18:44 +01:00
Jakob Naucke
dfd0732ff9 osbuilder: Revert to using apk.static for Alpine
#2399 partially reverted #418, missing on returning to bootstrapping a
rootfs with `apk.static` instead of copying the entire root, which can
result in drastically larger (more than 10x) images. Revert this as well
(requires some updates to URL building).

Fixes: #3216
Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
2021-12-08 15:18:43 +01:00
zhanghj
6b3e4c212c image_build: add help info for '-f' option and 'BLOCK_SIZE' env.
The help information of '-f' option is missing, and same issue
with 'BLOCK_SIZE' env variables, fix it in usage() function.

Fixes: #3231

Signed-off-by: zhanghj <zhanghj.lc@inspur.com>
2021-12-08 17:33:07 +08:00
stevenhorsman
15494d7e06 CCv0: Merge main into CCv0 branch
Resolve merge conflicts during merge

Fixes: #3209
Depends-on: github.com/kata-containers/tests#4256

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2021-12-06 11:12:47 +00:00
stevenhorsman
bb66dbdccc osbuilder: Stop skopeo being installed by default
With the new rust image pull service skopeo we can parameterise whether to build
and install skopeo and turn it off by default if we don't need
signature verification support

Fixes: #3170

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2021-12-03 14:53:39 +00:00
Jakob Naucke
af91084887 Merge pull request #3151 from liangzhou121/osbuilder-dev-1
CCv0|osbuilder: integrate AA's EAA KBC into ubuntu rootfs image
2021-12-03 09:37:29 +01:00
zhouliang121
365acda643 CCv0|osbuilder: add EAA KBC into ubuntu rootfs
Integrate EAA KBC into ubuntu rootfs image.
Fix build failure if build with AA_KBC=eaa_kbc option.

Fixes: #3167
Signed-off-by: zhouliang121 <liang.a.zhou@linux.alibaba.com>
2021-12-02 12:07:45 +08:00
Arron Wang
263bbe937f osbuilder: Add build dependency for openssl and protobuf
The openssl-sys crate will automatically detect:
https://docs.rs/openssl/0.10.16/openssl/#automatic

Signed-off-by: Arron Wang <arron.wang@intel.com>
2021-12-01 23:42:50 +01:00
Gabriela Cervantes
923e098db6 osbuilder: Remove debian as a rootfs
Currently we do not have debian as part of the kata CI as we
do not have a mantainer, this PR removes debian as a supported
rootfs in order to have only the distros that we are supporting
and mantainining.

Fixes #3153

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2021-11-30 19:31:33 +00:00
Binbin Zhang
8ee67aae4f osbuilder: fix missing cpio package when building rootfs-initrd image
1. install cpio package before building rootfs-initrd image
2. add `pipefaili;errexit` check to the scripts

Fixes: #3144

Signed-off-by: Binbin Zhang <binbin36520@gmail.com>
2021-11-29 23:42:44 +08:00
Peng Tao
f59d3ff600 osbuilder: add coreutils to guest rootfs
So that the debug console is more useful. In the meantime, remove
iptables as it is not used by kata-agent any more.

Fixes: #3138
Signed-off-by: Peng Tao <bergwolf@hyper.sh>
2021-11-29 11:22:07 +08:00
Julio Montes
857501d8dd tools/osbuilder: build QAT kernel in fedora 34
kernel compiled in fedora 35 (latest) is not working, following error
is reported:

```
qemu-system-x86_64: Error loading uncompressed kernel without PVH ELF
Note
```

Build QAT kernel in fedora 34 container to fix it

fixes #3135

Signed-off-by: Julio Montes <julio.montes@intel.com>
2021-11-26 13:56:43 -06:00
stevenhorsman
6f1bdd7079 doc: Add PoC container signature validation
- Document how to test the signature validation with
a number of different scenarios and test images
- Update ccv0.sh to add policy_path to kernel_params

Fixes: #2682

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2021-11-22 11:51:05 +01:00
stevenhorsman
7670792f97 osbuilder: Add PoC container signature validation
- Add hardcoded gpg, signature and polict files
- Modify rootfs.sh to put these in the correct place in the kata image
if skopeo and umoci are being used

Fixes: #2682

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2021-11-22 11:51:05 +01:00
Jakob Naucke
13f6418c46 osbuilder: Build Skopeo, umoci, attestation-agent
When the environment variable $SKOPEO_UMOCI is set to "yes", Skopeo and
umoci are built inside the guest build container and installed to the
guest rootfs. The respective build- and runtime dependencies are added.
This respects the (existing) $LIBC variable (gnu/musl) and avoids issues
with glibc mismatches.
This is currently only supported for Ubuntu guests, as the system Golang
packages included in the versions of other distros that we use are too
old to build these packages, and re-enabling installing Golang from
golang.org is cumbersome, given especially that it is unclear how long
we will keep using Skopeo and umoci.

Additionally, when the environment variable $AA_KBC is set,
attestation-agent (with that KBC) is included.

This replaces some logic in ccv0.sh that is removed.

Fixes: #2907
Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
2021-11-09 18:04:21 +01:00
stevenhorsman
85d455791c osbuilder: update base rootfs to be ubuntu
- Revert fedora OS changes made in #2556 as we aren't using it anymore.
- They should be done in main under #2116

Fixes #2849

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2021-11-05 14:49:45 +00:00
Georgina Kinge
af44b7a591 agent: Add PullImage endpoint
This commit adds the PullImge endpoint to the agent
and the agent-ctl command to test it.

Fixes: #2509

Signed-off-by: Georgina Kinge <georgina.kinge@ibm.com>
Co-authored-by: stevenhorsman <steven@uk.ibm.com>
2021-11-05 14:49:20 +00:00
Wainer dos Santos Moschetta
a239a38f45 osbuilder: build image-builder image from Fedora 34
Currently the image-builder image is built from `fedora:latest` and
this is error-prone as any update of the base image can lead to
breakage. Instead let's create the image from Fedora 34, which is the
last known version to build fine.

Fixes #2960
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
2021-11-03 13:07:42 -04:00
Manabu Sugimoto
a3647e3486 osbuilder: Set up libseccomp library
The osbuilder needs to set up libseccomp library to build the kata-agent
because the kata-agent supports seccomp currently.
The library is built from the sources to create a static library for musl libc.
In addition, environment variables for the libseccomp crate are set to
link the library statically.

Fixes: #1476

Signed-off-by: Manabu Sugimoto <Manabu.Sugimoto@sony.com>
2021-10-27 19:06:13 +09:00
Eric Adams
37fa453dd2 osbuilder: Update QAT driver in Dockerfile
This is to bump the OOT QAT 1.7 driver version to the
latest version. I dida test on my QAT enabled system and
everything functioned as expected.

Fixes: #2877

Signed-off-by: Eric Adams <eric.adams@intel.com>
2021-10-22 00:08:24 +00:00
Wainer dos Santos Moschetta
50da26d3e6 osbuilder: Call detect_rust_version() right before install_rust.sh
When building with dracut method the build_rootfs_distro() is not called, in turn
detect_rust_version() isn't either, so the install_rust.sh script is gave a null
rust version. This changed the script to call detect_rust_version() right before
install_rust.sh.

Related to commit: f34f67d610
Fixes #2862
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
2021-10-18 16:54:53 -04:00
Jakob Naucke
f34f67d610 osbuilder: Specify version when installing Rust
and update the script in `ci/` accordingly.
When only parts of the Kata Containers repositories are checked out
(e.g. when building with Snap) and no Rust version is provided in
calling `install_rust.sh`, the scripts will attempt to clone the
appropriate repos to read the version, which will fail because the
directories already exist. Since we have read the version already, we
can just specify it.

Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
2021-10-15 16:27:40 +02:00
Jakob Naucke
135a0802c5 osbuilder: Pass CI env to container agent build
The agent build inside a Docker or Podman container has been re-enabled,
but we have since introduced the `$CI` environment variable. Pass it to
avoid checking out the tests repo to main when there is a dependency.

Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
2021-10-15 16:27:39 +02:00
Jakob Naucke
eb5dd76e9d osbuilder: Re-enable building the agent in Docker
or Podman. This is a partial revert of
76c18aa345. The rationale behind that
commit was the fact that the agent could not be built on Alpine, and
then this capability was removed altogether. The issue in Alpine has
since been resolved (see
https://github.com/kata-containers/osbuilder/issues/386). At the same
time, this ensures being able to run a glibc agent on hosts with distros
more recent than the osbuilder distro used (i.e. as of now, when you
build the agent on the host, and its glibc is newer than the one used in
the guest, the agent may encounter unresolved symbols).

Fixes #2398
Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
2021-10-15 16:27:37 +02:00