Fixes: #7573 To enable this feature, build your rootfs using AGENT_POLICY=yes. The default is AGENT_POLICY=no. Building rootfs using AGENT_POLICY=yes has the following effects: 1. The kata-opa service gets included in the Guest image. 2. The agent gets built using AGENT_POLICY=yes. After this patch, the shim calls SetPolicy if and only if a Policy annotation is attached to the sandbox/pod. When creating a sandbox/pod that doesn't have an attached Policy annotation: 1. If the agent was built using AGENT_POLICY=yes, the new sandbox uses the default agent settings, that might include a default Policy too. 2. If the agent was built using AGENT_POLICY=no, the new sandbox is executed the same way as before this patch. Any SetPolicy calls from the shim to the agent fail if the agent was built using AGENT_POLICY=no. If the agent was built using AGENT_POLICY=yes: 1. The agent reads the contents of a default policy file during sandbox start-up. 2. The agent then connects to the OPA service on localhost and sends the default policy to OPA. 3. If the shim calls SetPolicy: a. The agent checks if SetPolicy is allowed by the current policy (the current policy is typically the default policy mentioned above). b. If SetPolicy is allowed, the agent deletes the current policy from OPA and replaces it with the new policy it received from the shim. A typical new policy from the shim doesn't allow any future SetPolicy calls. 4. For every agent rpc API call, the agent asks OPA if that call should be allowed. OPA allows or not a call based on the current policy, the name of the agent API, and the API call's inputs. The agent rejects any calls that are rejected by OPA. When building using AGENT_POLICY_DEBUG=yes, additional Policy logging gets enabled in the agent. In particular, information about the inputs for agent rpc API calls is logged in /tmp/policy.txt, on the Guest VM. These inputs can be useful for investigating API calls that might have been rejected by the Policy. Examples: 1. Load a failing policy file test1.rego on a different machine: opa run --server --addr 127.0.0.1:8181 test1.rego 2. Collect the API inputs from Guest's /tmp/policy.txt and test on the machine where the failing policy has been loaded: curl -X POST http://localhost:8181/v1/data/agent_policy/CreateContainerRequest \ --data-binary @test1-inputs.json Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Building a Guest OS rootfs for Kata Containers
The Kata Containers rootfs is created using the rootfs.sh script.
Supported base OSs
The rootfs.sh script builds a rootfs based on a particular Linux*
distribution. The script supports multiple distributions and can be extended
to add further ones.
Extra features
Supported distributions list
List the supported distributions by running the following:
$ ./rootfs.sh -l
Generate Kata specific files
The rootfs.sh script can be used to populate a directory with only Kata specific files and
components, without creating a full usable rootfs.
This feature is used to create a rootfs based on a distribution not officially
supported by osbuilder, and when building an image using the dracut build method.
To achieve this, simply invoke rootfs.sh without specifying a target rootfs, e.g.:
$ mkdir kata-overlay
$ ./rootfs.sh -r "$PWD/kata-overlay"
Rootfs requirements
The rootfs must provide at least the following components:
-
Path:
/bin/kata-agent- Kata Containers guest. -
An
initsystem (e.g.systemd) to start the Kata agent when the guest OS boots.Path:
/sbin/init- init binary called by the kernel.
When the AGENT_INIT environment variable is set to yes, use Kata agent as /sbin/init.
Note
:
AGENT_INIT=yesmust be used for the Alpine distribution since it does not usesystemdas its init daemon.
Creating a rootfs
To build a rootfs for your chosen distribution, run:
$ sudo ./rootfs.sh <distro>
Creating a rootfs with kernel modules
To build a rootfs with additional kernel modules, run:
$ sudo KERNEL_MODULES_DIR=${kernel_mod_dir} ./rootfs.sh <distro>
Where kernel_mod_dir points to the kernel modules directory to be put under the
/lib/modules/ directory of the created rootfs.
Build a rootfs using Docker
Depending on the base OS to build the rootfs guest OS, it is required some
specific programs that probably are not available or installed in the system
that will build the guest image. For this case rootfs.sh can use
a Docker* container to build the rootfs. The following requirements
must be met:
-
Docker 1.12+ installed.
-
runcis configured as the default runtime.To check if
runcis the default runtime:$ docker info | grep 'Default Runtime: runc' -
Export
USE_DOCKERvariable.$ export USE_DOCKER=true -
Use
rootfs.sh:Example:
$ export USE_DOCKER=true $ # build guest O/S rootfs based on debian $ ./rootfs-builder/rootfs.sh -r "${PWD}/debian_rootfs" debian $ # build image based rootfs created above $ ./image-builder/image_builder.sh "${PWD}/debian_rootfs"
Adding support for a new guest OS
The rootfs.sh script will check for immediate sub-directories
containing the following expected files:
-
A
bash(1)script calledconfig.shThis represents the specific configuration for
<distro>. It must provide configuration specific variables for the user to modify as needed. Theconfig.shfile will be loaded before executingbuild_rootfs()to provide all the needed configuration to the function.Path:
rootfs-builder/<distro>/config.sh. -
(OPTIONAL) A
bash(1)script calledrootfs_lib.shThis file must contain a function called
build_rootfs(), which must receive the path to where the rootfs is created, as its first argument. Normally, this file is needed if a new distro with a special requirement is needed. This function will override thebuild_rootfs()function inscripts/lib.sh.Path:
rootfs-builder/<distro>/rootfs_lib.sh.
Create template files
To create a directory with the expected file structure run:
$ make -f template/Makefile ROOTFS_BASE_NAME=my_new_awesome_rootfs
After running the previous command, a new directory is created in
rootfs-builder/my_new_awesome_rootfs/.
To verify the directory can be used to build a rootfs, run ./rootfs.sh -h.
Running this script shows my_new_awesome_rootfs as one of the options for
use. To use the new guest OS, follow the instructions in Creating a rootfs.
Modify template files
After the new directory structure is created:
-
If needed, add configuration variables to
rootfs-builder/my_new_awesome_rootfs/config.sh. -
Implement the stub
build_rootfs()function fromrootfs-builder/my_new_awesome_rootfs/rootfs_lib.sh.
Expected rootfs directory content
After the function build_rootfs is called, the script expects the
rootfs directory to contain /sbin/init and /sbin/kata-agent binaries.
Optional - Customize the rootfs
For particular use cases developers might want to modify the guest OS.
Adding extra packages
To add additional packages, use one of the following methods:
-
Use the environment variable
EXTRA_PKGSto provide a list of space-separated packages to install.Note:
The package names might vary among Linux distributions, the extra package names must exist in the base OS flavor you use to build the rootfs from.
Example:
$ EXTRA_PKGS="vim emacs" ./rootfs-builder/rootfs.sh -r ${PWD}/myrootfs debian -
Modify the variable
PACKAGESinrootfs-builder/<distro>/config.sh.This variable specifies the minimal set of packages needed. The configuration file must use the package names from the distro for which they were created.
Arbitrary rootfs changes
Once the rootfs directory is created, you can add and remove files as needed. Changes affect the files included in the final guest image.