Commit Graph

51 Commits

Author SHA1 Message Date
Nitesh Konkar
862b077598 image_builder: create /etc/resolv.conf
Since the image rootfs is readonly, we
create an empty /etc/resolv.conf which
the agent would later bind-remount as
read-write.

Fixes: #345

Signed-off-by: Nitesh Konkar <niteshkonkar@in.ibm.com>
2019-08-12 19:15:32 +05:30
Salvador Fuentes
269d31a9ac Merge pull request #337 from devimc/topic/image-builder/fixMissingAgent
image-builder: check /sbin/init when AGENT_INIT=yes
2019-08-02 09:46:29 -05:00
Julio Montes
b1b8ce427a image-builder: check /sbin/init when AGENT_INIT=yes
kata-agent is moved to /sbin/init when AGENT_INIT is yes. Check
/sbin/init exists and it's not systemd

fixes #322

Signed-off-by: Julio Montes <julio.montes@intel.com>
2019-07-30 13:23:11 +00:00
ClarkLee
6d5aef531a osbuilder: Get mount directory size with Mb format
While calculating directory size in imagebuild script, we shoul use
`df -BM` instead of `df -h` to keep Mb format.

Fixes #336

image-builder: get mount directory size with Mb format

Signed-off-by: ClarkLee <clarklee1992@hotmail.com>
2019-07-29 23:03:17 +08:00
Archana Shinde
6f294f43d4 tmp: Do not remove tmp.mount unit file from the rootfs
We should start this unit so that systemd can mount /tmp as
tmpfs.

Fixes #300

Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
2019-05-30 15:12:05 -07:00
Julio Montes
a438d086b2 image-builder: create /etc/machine-id
systemd complains if `/etc/machine-id` does not exist. Create the `machine-id`
file to make systemd happy, it'll bind-mount that file to write the machine id.

fixes #296
fixes github.com/kata-containers/runtime#1537

Signed-off-by: Julio Montes <julio.montes@intel.com>
2019-05-23 07:38:12 -05:00
Julio Montes
02b3b3b977 image-builder: remove unneeded systemd units and files
Remove systemd units and files that are not needed in Kata Containers.
Removing this files we can improve the boot time.

fixes #289

Signed-off-by: Julio Montes <julio.montes@intel.com>
2019-05-16 13:19:55 -05:00
Marco Vedovati
7cfe5b4dd2 image-builder: set default value of AGENT_INIT
Set as default AGENT_INIT=no when not explicitly set, to make it
consistent with the other scripts in osbuilder

Signed-off-by: Marco Vedovati <mvedovati@suse.com>
2019-04-26 15:22:50 +02:00
Julio Montes
88b85231ca image-builder: share the mkfs configuration file
Share with the container the mkfs configuration file to use only the
filesystem features supported in the host.

fixes #270

Signed-off-by: Julio Montes <julio.montes@intel.com>
2019-04-01 09:18:28 -06:00
Julio Montes
77fb8085f4 image-builder: check format_loop return code
check format_loop return code and die if it fails

Signed-off-by: Julio Montes <julio.montes@intel.com>
2019-03-29 07:47:22 -06:00
Julio Montes
8debe95b61 image-builder: add fs type
Add filesystem type to format the loop device.

fixes #270

Signed-off-by: Julio Montes <julio.montes@intel.com>
2019-03-28 15:10:16 -06:00
Julio Montes
d8cdd88ace image-builder: re-implement image builder script
Re-implement image builder script to generate an image with a double MBR +
a DAX metadata. The DAX metadata is read by the NVDIMM driver to know the
beginning of the data in the pmem device.
This new image format is required to enable DAX in the kernels and hypervisors
that support NVDIMM, without breaking the compatibility with the kernels and
hypervisors that don't support it.

Following diagram shows how the resulting image will look like

```
	.-----------.----------.---------------.-----------.
	| 0 - 512 B | 4 - 8 Kb |  2M - 2M+512B |    3M     |
	|-----------+----------+---------------+-----------+
	|   MBR #1  |   DAX    |    MBR #2     |  Rootfs   |
	'-----------'----------'---------------'-----------+
	      |          |      ^      |        ^
	      |          '-data-'      '--------'
	      |                                 |
	      '--------rootfs-partition---------'
```

MBR: Master boot record.
DAX: Metadata required by the NVDIMM driver to enable DAX in the guest [1][2]
(struct nd_pfn_sb).
Rootfs: partition that contains the root filesystem (/usr, /bin, etc).

Kernels and hypervisors that support DAX/NVDIMM read the MBR #2,
otherwise MBR #1 is read.

[1] - https://github.com/kata-containers/osbuilder/blob/master/image-builder/\
nsdax.gpl.c
[2] - https://github.com/torvalds/linux/blob/master/drivers/nvdimm/pfn.h

fixes #263

Signed-off-by: Julio Montes <julio.montes@intel.com>
2019-03-26 09:02:45 -06:00
Julio Montes
6c7f3077c9 image-builder: add xfsprogs to Dockerfile
xfsprogs is a package required to generate xfs images.

Signed-off-by: Julio Montes <julio.montes@intel.com>
2019-03-25 08:20:49 -06:00
Julio Montes
e888ef5a60 Merge pull request #260 from darfux/pass_debug_to_docker
builder: Pass the DEBUG flag when using docker
2019-03-25 07:25:12 -06:00
James O. D. Hunt
0b7e456d47 Merge pull request #250 from matthewfischer/max_attempts
Print attempt number after max check
2019-03-25 10:32:49 +00:00
Li Yuxuan
0d2ba4766e builder: Pass the DEBUG flag when using docker
When using docker, pass the `DEBUG` flag to trace the commands as well.

Fixes: #261

Signed-off-by: Li Yuxuan <liyuxuan04@baidu.com>
2019-03-25 17:36:38 +08:00
Matt Fischer
ad5d879f8c rootfs: Print attempt number after max check
With the old code it was possible to see odd messages like:
"INFO: Create root disk image. Attempt 6 out of 5."

Move the attempt number print to after we check against the max

Fixes #251

Signed-off-by: Matt Fischer <matt@mattfischer.com>
2019-03-19 21:04:40 -06:00
Julio Montes
02fa22bbdd image-builder: make DAX support optional
Not all hypervisor support NVDIMM hence DAX support MUST BE enabled explicitly
setting the DAX environment variable to 'yes'

fixes #246

Signed-off-by: Julio Montes <julio.montes@intel.com>
2019-03-15 13:14:52 -06:00
Julio Montes
726f798ff7 image-builder: fill out device namespace information into kata image
The new NVDIMM driver implementation (kernel >= 4.16) needs to know the device
namespace information to map pages, this metadata is read from the nvdimm
namespace at 4k offset.

fixes #235

Signed-off-by: Julio Montes <julio.montes@intel.com>
2019-03-13 13:05:06 -06:00
Julio Montes
cbe5642b9d image-builder: add gcc as dependecy to generate the image
gcc is required to build the binary in charge to fill out the device
namespace information (matadata) into the kata containers image.

Signed-off-by: Julio Montes <julio.montes@intel.com>
2019-03-13 13:05:06 -06:00
Julio Montes
7620066c8a image-builder: sync rootfs data after copying it into the image
Rootfs data must be sync'd after copying it into the image to avoid
data corruption

Signed-off-by: Julio Montes <julio.montes@intel.com>
2019-03-13 13:05:06 -06:00
Julio Montes
dc5bc07825 image-builder: fix mem boundary recalculation
$/${} is unnecessary on arithmetic variables. [SC2004]

Signed-off-by: Julio Montes <julio.montes@intel.com>
2019-03-13 13:05:06 -06:00
Julio Montes
71ccc0a6ea mage-builder: remove -s option and IMG_SIZE envar
guest kernel needs 64 bytes of DRAM per 4K page of emulated PMEM, hence the
image size should be as small as possible to reduce the container's memory
footprint. The image size is recalculated automatically if it's too small
to contain the rootfs.

Signed-off-by: Julio Montes <julio.montes@intel.com>
2019-03-13 13:05:06 -06:00
James O. D. Hunt
f38c67da0c arch: Remove calls to arch command
The `arch(1)` command is not available on some systems so use the
`uname(1)` command for the equivalent functionality.

Fixes #150.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2019-02-18 08:36:22 +00:00
Penny Zheng
e4d5c1b751 memory_hotplug: MEM_BOUNDARY_MB should be arch-dependent
As runtime/#624(https://github.com/kata-containers/runtime/pull/624#discussion_r212534586)
discussed before, the size of memory section is arch-dependent.
For arm64, it should be 1G, not 128MB.

Fixes: #224

Signed-off-by: Penny Zheng <penny.zheng@arm.com>
2019-01-16 11:16:26 +08:00
Yonatan Gefen
06437bb51e docs: Fix link in README
Correct rootfs builder link in the image builder README.

Fixes #185.

Signed-off-by: Yonatan Gefen <yoni.gefen@gmail.com>
2018-10-24 13:52:00 -04:00
Marco Vedovati
4a2fdee972 osbuilder: fix loop devices manipulation in image-builder.sh
Improve image-builder.sh to avoid the sporadic failures observed during
the CI builds of images. Also, some cosmetic changes to indentation.

Fixes: #172

Signed-off-by: Marco Vedovati <mvedovati@suse.com>
2018-09-20 17:52:06 +02:00
Stefan Hajnoczi
7b1bbac600 image-builder: require root earlier for better error messages
The image_builder.sh script must be run as root.  The following check is
performed before the script checks for root:

  [ "${AGENT_INIT}" == "yes" ] || [ -x "${ROOTFS}/usr/bin/${AGENT_BIN}" ] || \
      die "/usr/bin/${AGENT_BIN} is not installed in ${ROOTFS}
      use AGENT_BIN env variable to change the expected agent binary name"

The -x test is "True if the file is executable by you".  It may evaluate
to true as root and false as non-root, depending on the file
permissions.

The permissions for kata-agent given in the Developer Guide are 0550
(https://github.com/kata-containers/documentation/blob/master/Developer-Guide.md#add-a-custom-agent-to-the-image---optional).

Therefore image_builder.sh fails with "/usr/bin/${AGENT_BIN} is not
installed" when run as non-root.  This is confusing since the agent
binary is really installed!

Move the root check to the beginning of the script.  This solves the
confusing error and prevents similar problems where the script doesn't
take into account that the user may be non-root.

Fixes: #127
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2018-06-28 14:16:26 +01:00
Julio Montes
52d015e283 builder: support proxy in distros based on yum or dnf
proxy server must be specified in configuration file
of the package manager, /etc/yum.conf for yum or
/etc/dnf/dnf.conf for dnf

fixes #123

Signed-off-by: Julio Montes <julio.montes@intel.com>
2018-06-26 10:17:07 -05:00
Penny Zheng
e86380aab7 rootfs: add '-rm' to delete intermediate container
If we set env USE_DOCKER true, we will use container as development
environment. After docker run command, this temporary container
would be no use. we could add -rm flag to automatically delete
intermediate container.

Fixes: #115

Signed-off-by: Penny Zheng <penny.zheng@arm.com>
2018-06-12 08:16:56 +00:00
James O. D. Hunt
aca45c5820 image: Require systemd
Building an image requires systemd to be installed in the rootfs as the
init daemon, so assert that systemd is available.

Updated tests so that alpine is only tested as an initrd (it cannot be
an image as it doesn't use systemd).

Added warning note about alpine to the docs.

Fixes #98.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2018-05-18 13:01:08 +01:00
James O. D. Hunt
171eceb426 image: Use variable for referring to init
Use a variable rather than hard-coding the expected init daemon path.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2018-05-18 11:04:14 +01:00
Stefan Hajnoczi
32aee00673 image-builder: fix "paratition" typo
Fixes: #89
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2018-04-26 10:26:06 +01:00
James O. D. Hunt
f90f65247e rootfs: Create a summary file inside the image
Create a YAML metadata file inside the rootfs image
containing information about the environment:

```
/var/lib/osbuilder/osbuilder.yaml
```

Example contents:

```
---
osbuilder:
  url: "https://github.com/kata-containers/osbuilder"
  version: "unknown"
rootfs-creation-time: "2018-04-19T16:19:30.254610305+0000Z"
description: "osbuilder rootfs"
file-format-version: "0.0.1"
architecture: "x86_64"
base-distro:
  name: "Centos"
  version: "7"
  packages:
    - "iptables"
    - "systemd"
agent:
  url: "https://github.com/kata-containers/agent"
  name: "kata-agent"
  version: "0.0.1-2ec0b9593845b9a5e0eab5a85b20d74c35a2ca52-dirty"
  agent-is-init-daemon: "no"
```

This change adds a new `-o` option to `rootfs.sh` for
specifying the version of osbuilder to the rootfs builder.

Fixes #35.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2018-04-20 11:46:55 +01:00
James O. D. Hunt
b14d117a89 image-builder: Fix incorrect error message
Fixed an error message which was referring to an incorrect rootfs
variable name.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2018-04-20 11:17:15 +01:00
James O. D. Hunt
019a80f304 refactor: Move more functions to script library
Moved the display functions to the script library to avoid duplication.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2018-04-20 08:29:23 +01:00
James O. D. Hunt
a18753b2ff refactor: Remove duplicate variable
Some of the scripts had two variables to refer to the program name.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2018-04-19 16:00:37 +01:00
James O. D. Hunt
f3e89d38a9 refactor: Simplify enabling debug
Simplify the logic to enable debug and also enable debug as early as
possible.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2018-04-19 15:55:34 +01:00
Erick Cardona
ea4063095d image-builder: Allow to specify root partition free space
There is no way to specify the remaining free space of the root partition.
It can vary depending on the upper bound size of the image aligned to 128MB
and the size of the root filesystem.
The following patch allow the user to specify that at least a certain amount
of space (defined in MB) will be kept in the root partition.

Fixes: #45

Signed-off-by: Erick Cardona <erick.cardona.ruiz@intel.com>
2018-02-14 14:52:25 -06:00
Harshal Patil
132e812fe4 scripts : handle 0 and negative root disk sizes
When user manually specifies root disk image
size the value should be greater than 0.

Fixes : #57

Signed-off-by: Harshal Patil <harshal.patil@in.ibm.com>
2018-02-05 15:43:26 +05:30
Peng Tao
de6e4dc93f builder: fix agent binary path
kata agent is installed in `${ROOTFS}/usr/bin/` rather than `${ROOTFS}/bin/`.
It just happended to work because `bin` is a symlink to `usr/bin`, which
might not be true in some distros. So do not rely on it.

Signed-off-by: Peng Tao <bergwolf@gmail.com>
2018-02-05 17:18:52 +08:00
James O. D. Hunt
17e01fd32f Merge pull request #49 from bergwolf/agent_init
Allow to use agent as init process and create initrd image based on rootfs
2018-02-05 09:11:17 +00:00
Harshal Patil
5fe3f4ae0b scripts: Add an "auto-size" feature
Changed image-builder/image_builder.sh to
automatically calculate the size of the root disk

Fixes #25.

Signed-off-by: Harshal Patil <harshal.patil@in.ibm.com>
2018-02-02 14:32:09 +05:30
Peng Tao
82759dac14 initrd: add script to build kata initrd image
Build a kata initrd image based on rootfs created by rootfs.sh.

Fixes: #5

Signed-off-by: Peng Tao <bergwolf@gmail.com>
2018-02-01 23:27:36 +08:00
Peng Tao
94e7b1da4f rootfs: allow to use agent as init process
Add AGENT_INIT env to make it configurable.

Signed-off-by: Peng Tao <bergwolf@gmail.com>
2018-01-31 00:59:29 +08:00
James O. D. Hunt
1ca1b71f9c docs: Fix image builder instructions.
The `image_builder.sh` script must be run as `root`.

Fixes #36.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2018-01-23 14:29:54 +00:00
Jose Carlos Venegas Munoz
052b8af497 Merge pull request #29 from jcvenegas/travis
ci: Add travis basic testing
2018-01-19 13:00:07 -06:00
Jose Carlos Venegas Munoz
a9fef80feb image_builder: Allow build in a docker container
In case host not has requried programs allow build image
in a container when USE_DOCKER is set.

Signed-off-by: Jose Carlos Venegas Munoz <jose.carlos.venegas.munoz@intel.com>
2018-01-19 12:51:07 -06:00
James O. D. Hunt
57617ea4af docs: General cleanup
Rework the docs to make them simpler and more consistent. Also added of
contents and corrected a few mistakes.

Fixes #26.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2018-01-18 09:36:47 +00:00
Jose Carlos Venegas Munoz
d9fe322a5f image: Add README
Add readme do document how to use ./image-builder/image_builder.sh

Signed-off-by: Jose Carlos Venegas Munoz <jose.carlos.venegas.munoz@intel.com>
2017-12-06 13:32:03 -06:00