Commit Graph

79 Commits

Author SHA1 Message Date
stevenhorsman
85bb1e5e45 CCv0: Merge main into CCv0 branch
Merge remote-tracking branch 'upstream/main' into CCv0

Fixes: #3295

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2021-12-17 10:33:30 +00:00
stevenhorsman
e1ba87408c CCv0: Merge main into CCv0 branch
Weekly merge of new commits in main into CCv0

Fixes: #3259

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2021-12-13 14:15:32 +00:00
bin
3f7cf7ae67 osbuilder: show usage if no options/arguments specified
Now if no options/arguments specified, the shell scripts will return an error:

ERROR: Invalid rootfs directory: ''

This commit will show usage if no options/arguments specified.

Fixes: #3256

Signed-off-by: bin <bin@hyper.sh>
2021-12-13 16:10:55 +08:00
Julio Montes
70062e1563 Merge pull request #3238 from snir911/wip/build_with_runtime
osbuilder: be runtime consistent with podman build
2021-12-09 08:06:00 -06:00
Snir Sheriber
2ebaaac73d osbuilder: be runtime consistent also with podman build
Use the same runtime used for podman run also for the podman build cmd
Additionally remove "docker" from the docker_run_args variable

Fixes: #3239
Signed-off-by: Snir Sheriber <ssheribe@redhat.com>
2021-12-09 11:28:16 +02:00
Jakob Naucke
2204ecac39 versions: Upgrade Alpine, using minor version
- Upgrade Alpine guest rootfs to 3.15
- Specify a minor version rather than patch level as the Alpine
  repositories use that.

Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
2021-12-08 15:18:44 +01:00
Jakob Naucke
dfd0732ff9 osbuilder: Revert to using apk.static for Alpine
#2399 partially reverted #418, missing on returning to bootstrapping a
rootfs with `apk.static` instead of copying the entire root, which can
result in drastically larger (more than 10x) images. Revert this as well
(requires some updates to URL building).

Fixes: #3216
Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
2021-12-08 15:18:43 +01:00
stevenhorsman
15494d7e06 CCv0: Merge main into CCv0 branch
Resolve merge conflicts during merge

Fixes: #3209
Depends-on: github.com/kata-containers/tests#4256

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2021-12-06 11:12:47 +00:00
stevenhorsman
bb66dbdccc osbuilder: Stop skopeo being installed by default
With the new rust image pull service skopeo we can parameterise whether to build
and install skopeo and turn it off by default if we don't need
signature verification support

Fixes: #3170

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2021-12-03 14:53:39 +00:00
Jakob Naucke
af91084887 Merge pull request #3151 from liangzhou121/osbuilder-dev-1
CCv0|osbuilder: integrate AA's EAA KBC into ubuntu rootfs image
2021-12-03 09:37:29 +01:00
zhouliang121
365acda643 CCv0|osbuilder: add EAA KBC into ubuntu rootfs
Integrate EAA KBC into ubuntu rootfs image.
Fix build failure if build with AA_KBC=eaa_kbc option.

Fixes: #3167
Signed-off-by: zhouliang121 <liang.a.zhou@linux.alibaba.com>
2021-12-02 12:07:45 +08:00
Arron Wang
263bbe937f osbuilder: Add build dependency for openssl and protobuf
The openssl-sys crate will automatically detect:
https://docs.rs/openssl/0.10.16/openssl/#automatic

Signed-off-by: Arron Wang <arron.wang@intel.com>
2021-12-01 23:42:50 +01:00
Gabriela Cervantes
923e098db6 osbuilder: Remove debian as a rootfs
Currently we do not have debian as part of the kata CI as we
do not have a mantainer, this PR removes debian as a supported
rootfs in order to have only the distros that we are supporting
and mantainining.

Fixes #3153

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2021-11-30 19:31:33 +00:00
Peng Tao
f59d3ff600 osbuilder: add coreutils to guest rootfs
So that the debug console is more useful. In the meantime, remove
iptables as it is not used by kata-agent any more.

Fixes: #3138
Signed-off-by: Peng Tao <bergwolf@hyper.sh>
2021-11-29 11:22:07 +08:00
stevenhorsman
6f1bdd7079 doc: Add PoC container signature validation
- Document how to test the signature validation with
a number of different scenarios and test images
- Update ccv0.sh to add policy_path to kernel_params

Fixes: #2682

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2021-11-22 11:51:05 +01:00
stevenhorsman
7670792f97 osbuilder: Add PoC container signature validation
- Add hardcoded gpg, signature and polict files
- Modify rootfs.sh to put these in the correct place in the kata image
if skopeo and umoci are being used

Fixes: #2682

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2021-11-22 11:51:05 +01:00
Jakob Naucke
13f6418c46 osbuilder: Build Skopeo, umoci, attestation-agent
When the environment variable $SKOPEO_UMOCI is set to "yes", Skopeo and
umoci are built inside the guest build container and installed to the
guest rootfs. The respective build- and runtime dependencies are added.
This respects the (existing) $LIBC variable (gnu/musl) and avoids issues
with glibc mismatches.
This is currently only supported for Ubuntu guests, as the system Golang
packages included in the versions of other distros that we use are too
old to build these packages, and re-enabling installing Golang from
golang.org is cumbersome, given especially that it is unclear how long
we will keep using Skopeo and umoci.

Additionally, when the environment variable $AA_KBC is set,
attestation-agent (with that KBC) is included.

This replaces some logic in ccv0.sh that is removed.

Fixes: #2907
Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
2021-11-09 18:04:21 +01:00
stevenhorsman
85d455791c osbuilder: update base rootfs to be ubuntu
- Revert fedora OS changes made in #2556 as we aren't using it anymore.
- They should be done in main under #2116

Fixes #2849

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2021-11-05 14:49:45 +00:00
Georgina Kinge
af44b7a591 agent: Add PullImage endpoint
This commit adds the PullImge endpoint to the agent
and the agent-ctl command to test it.

Fixes: #2509

Signed-off-by: Georgina Kinge <georgina.kinge@ibm.com>
Co-authored-by: stevenhorsman <steven@uk.ibm.com>
2021-11-05 14:49:20 +00:00
Manabu Sugimoto
a3647e3486 osbuilder: Set up libseccomp library
The osbuilder needs to set up libseccomp library to build the kata-agent
because the kata-agent supports seccomp currently.
The library is built from the sources to create a static library for musl libc.
In addition, environment variables for the libseccomp crate are set to
link the library statically.

Fixes: #1476

Signed-off-by: Manabu Sugimoto <Manabu.Sugimoto@sony.com>
2021-10-27 19:06:13 +09:00
Wainer dos Santos Moschetta
50da26d3e6 osbuilder: Call detect_rust_version() right before install_rust.sh
When building with dracut method the build_rootfs_distro() is not called, in turn
detect_rust_version() isn't either, so the install_rust.sh script is gave a null
rust version. This changed the script to call detect_rust_version() right before
install_rust.sh.

Related to commit: f34f67d610
Fixes #2862
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
2021-10-18 16:54:53 -04:00
Jakob Naucke
f34f67d610 osbuilder: Specify version when installing Rust
and update the script in `ci/` accordingly.
When only parts of the Kata Containers repositories are checked out
(e.g. when building with Snap) and no Rust version is provided in
calling `install_rust.sh`, the scripts will attempt to clone the
appropriate repos to read the version, which will fail because the
directories already exist. Since we have read the version already, we
can just specify it.

Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
2021-10-15 16:27:40 +02:00
Jakob Naucke
135a0802c5 osbuilder: Pass CI env to container agent build
The agent build inside a Docker or Podman container has been re-enabled,
but we have since introduced the `$CI` environment variable. Pass it to
avoid checking out the tests repo to main when there is a dependency.

Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
2021-10-15 16:27:39 +02:00
Jakob Naucke
eb5dd76e9d osbuilder: Re-enable building the agent in Docker
or Podman. This is a partial revert of
76c18aa345. The rationale behind that
commit was the fact that the agent could not be built on Alpine, and
then this capability was removed altogether. The issue in Alpine has
since been resolved (see
https://github.com/kata-containers/osbuilder/issues/386). At the same
time, this ensures being able to run a glibc agent on hosts with distros
more recent than the osbuilder distro used (i.e. as of now, when you
build the agent on the host, and its glibc is newer than the one used in
the guest, the agent may encounter unresolved symbols).

Fixes #2398
Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
2021-10-15 16:27:37 +02:00
Aaron Simmons
80f6b97710 osbuilder: fixing centos gpg key url for ppc64le
The centos ppc64le gpg key at mirror.centos.org doesn't exist (link rot?).
Replacing it with url from CentOS/sig-core-AltArch on github.

Fixes: #2676

Signed-off-by: Aaron Simmons <paleozogt@gmail.com>
2021-09-29 09:20:51 -06:00
Binbin Zhang
924a68d08d osbuilder: Change to "=" operator to make script more portable
zsh doesn't support "==" as equal comparison operator, so
replace "==" with "=" to make the script more portable

Fixes: #2584

Signed-off-by: Binbin Zhang <binbin36520@gmail.com>
2021-09-08 16:57:34 +08:00
Marcel Apfelbaum
9de1129bf7 osbuilder: Fix rootfs-builder when running in VMs
The script runs apt sync at some point which scans all possible fds
in order to close them. The operation is incredibly slow on VMs
and may lead to build timeouts.

Fix it by limiting the container runtime fds to a sane limit.

Fixes: #2510

Signed-off-by: Marcel Apfelbaum <marcel@redhat.com>
2021-08-25 19:58:21 +00:00
Marcel Apfelbaum
65a1e13195 osbuilder: Allow running the tool several times
Once the ${ROOTFS_DIR} is created, the tool can't run the second
time since the directory is populated and the debootstrap tool
will fail.

Fix by deleting the contents of ${ROOTFS_DIR} if the directory exists.
Note that running make clean will also allow the re-run, it
is only an optimization for some cases the build fails in the middle.

Signed-off-by: Marcel Apfelbaum <marcel@redhat.com>
2021-08-25 19:49:14 +00:00
Jakob Naucke
7effbdebcb osbuilder: Upgrade Ubuntu guest to 20.04
- no need to create `/usr/lib/systemd/systemd` link any more
- install `chrony` as extra package and install extra packages in chroot
  rather than `debootstrap`, because `chrony` provides `time-daemon`,
  which under 20.04 is provided by `systemd-timesyncd`, which is
  required by `systemd`, and `debootstrap`'s conflict resolvement can't
  handle this, but `apt`'s can.

Fixes: #2147
Depends-on: github.com/kata-containers/tests#3636
Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
2021-08-10 16:31:21 +02:00
Jakob Naucke
a72b08117f osbuilder: pass env OS_VERSION
With lines like
0a2e2c6038/tools/osbuilder/rootfs-builder/fedora/config.sh (L8)
we imply that one can set another OS_VERSION and it will get picked up.
This is not the case when building inside Docker/Podman because the
variable is not passed to the container, which can lead to confusion.
Forward this env.

Fixes: #2378
Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
2021-08-03 18:28:17 +02:00
James O. D. Hunt
4f0726bc49 docs: Remove table of contents
Removed all TOCs now that GitHub auto-generates them.

Also updated the documentation requirements doc removing the requirement
to add a TOC.

Fixes: #2022.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2021-07-30 10:58:22 +01:00
Fabiano Fidêncio
2d142bc92d Merge pull request #2155 from jcvenegas/kata-deploy-2021-06-29
kata-deploy: Allow build  kata-deploy  tarball from HEAD
2021-07-29 22:50:03 +02:00
Carlos Venegas
4895747f35 Rootfs: Add curl to alpine rootfs builder.
If alpine image is created inside a container,
it does not get any golang version data. It will try
to get it by installing yq. To install yq curl is used.

Signed-off-by: Carlos Venegas <jos.c.venegas.munoz@intel.com>
2021-07-28 19:45:35 +00:00
Fabiano Fidêncio
d75c01bd67 Merge pull request #2186 from YchauWang/yc-osbuilder-arm
osbuilder: update centos arm rootfs image config 'GPG_KEY_ARCH_URL'
2021-07-23 11:17:08 +02:00
wangyongchao.bj
3882619471 osbuilder: update centos arm rootfs image config 'GPG_KEY_ARCH_URL'
fix GPG_KEY_ARCH_URL config of centos's config_aarch64,
update to "http://mirror.centos.org/altarch/7/os/aarch64/RPM-GPG-KEY-CentOS-7".

Fixes: #2181

Signed-off-by: wangyongchao.bj <wangyongchao.bj@inspur.com>
2021-07-20 13:49:17 +08:00
bin
b12b21f337 osbuilder: Skip installing golang for building rootfs
Building rootfs does not depend on golang, delete intalling
golang may save build time.

And there is only rust agent now, the code for golang agent should
be deleted too.

Fixes: #2170

Signed-off-by: bin <bin@hyper.sh>
2021-07-15 23:59:15 +08:00
Manabu Sugimoto
88e7075929 osbuilder: Fix the order of checking the distro config directory
Check if the distro config directory exists before using the directory

Fixes: #2148

Signed-off-by: Manabu Sugimoto <Manabu.Sugimoto@sony.com>
2021-07-11 15:24:25 +09:00
Tim Zhang
c881899903 Merge pull request #1990 from liubin/1989/fix-log-message
osbuilder: fix log message that is not error but seems like an error
2021-06-15 15:13:21 +08:00
bin
45fd58d11c osbuilder: fix log message that is not error but seems like an error
Only show checkout failed message if AGENT_VERSION is set
and the checkout is failed.

Fixes: #1989

Signed-off-by: bin <bin@hyper.sh>
2021-06-15 10:26:52 +08:00
Fabiano Fidêncio
dc4307d3cc Merge pull request #1974 from Jakob-Naucke/fix-cc-fedora-s390x
Update CC=gcc setting for Fedora s390x
2021-06-11 00:31:51 +02:00
Jakob Naucke
a484d6db87 osbuilder: Streamline s390x CMake & musl handling
- Merge codepath in lib.sh with ppc64le -- do not install CMake
- Like ppc64le, do not install musl rather than just not using it

Fixes: #1975

Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
2021-06-07 17:39:31 +02:00
Jakob Naucke
da2d9ab813 osbuilder: Remove CC=gcc for Fedora s390x
since that was required specifically for the Go agent

Fixes: #1973

Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
2021-06-07 17:32:05 +02:00
Chelsea Mafrica
e8038718aa osbuilder: Upgrade alpine version to 3.13.5
We are using an older version of alpine, so upgrade to latest 3.13.5.

Fixes #1817

Signed-off-by: Chelsea Mafrica <chelsea.e.mafrica@intel.com>
2021-05-07 16:34:45 -07:00
Eric Ernst
935460e549 osbuilder: update dockerfiles to utilize IMAGE_REGISTRY
While we introduced IMAGE_REGISTRY, we didn't actually update the
corresponding Dockerfiles to utilize it. Let's add

Fixes: #1622

Signed-off-by: Eric Ernst <eric.g.ernst@gmail.com>
2021-04-02 09:46:16 -07:00
Chelsea Mafrica
8c1abc3b57 Merge pull request #1418 from Amulyam24/fix-docker-ppc64le
osbuiler: fixing USE_DOCKER for ppc64le
2021-03-19 17:22:12 -07:00
Amulya Meka
5096103e7e osbuiler: fixing USE_DOCKER for ppc64le
For building rootfs with docker, glibc based rust target should be installed on ppc64le.
Additionally, protobuf-compiler would be required on ppc64le as it is not present by default.

Fixes: #1417

Signed-off-by: Amulya Meka <amulmek1@in.ibm.com>
2021-02-18 05:33:51 +00:00
Eric Ernst
49bdbac606 osbuilder: Allow image registry to be customizable
Give the user chance to specify their own registry in event the default
provided are not accessible, desirable.

Fixes: #1393

Signed-off-by: Eric Ernst <eric.g.ernst@gmail.com>
2021-02-17 16:49:58 -08:00
Eric Ernst
cb6d2f3c40 osbuilder: alphabetize fields
Let's go ahead and list the usage info / fields in alphabetical order!

Signed-off-by: Eric Ernst <eric.g.ernst@gmail.com>
2021-02-10 12:39:10 -08:00
Jakob-Naucke
11fe6a3552 osbuilder: Fix USE_DOCKER on s390x
- Install the required protobuf-compiler on Ubuntu
- Install correct libc Rust target (glibc on s390x)
- Do not skip Rust installation on s390x

Fixes: #1266

Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
2021-02-09 12:37:24 +01:00
Jakob Naucke
afb4197813 osbuilder: Build for glibc on s390x
Since there is no Rust target for musl on s390x, builds on s390x should use
glibc. This commit removes the "Cannot build Rust agent on s390x" restriction
in rootfs.sh and only installs musl in the build environment when it is
required.

Fixes: #1262

Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
2021-02-04 09:52:28 +01:00