Files
Dan Mihai ef1614edb2 agent: runtime: add the Agent Policy feature
Fixes: #7573

To enable this feature, build your rootfs using AGENT_POLICY=yes. The
default is AGENT_POLICY=no.

Building rootfs using AGENT_POLICY=yes has the following effects:

1. The kata-opa service gets included in the Guest image.

2. The agent gets built using AGENT_POLICY=yes.

After this patch, the shim calls SetPolicy if and only if a Policy
annotation is attached to the sandbox/pod. When creating a sandbox/pod
that doesn't have an attached Policy annotation:

1. If the agent was built using AGENT_POLICY=yes, the new sandbox uses
   the default agent settings, that might include a default Policy too.

2. If the agent was built using AGENT_POLICY=no, the new sandbox is
   executed the same way as before this patch.

Any SetPolicy calls from the shim to the agent fail if the agent was
built using AGENT_POLICY=no.

If the agent was built using AGENT_POLICY=yes:

1. The agent reads the contents of a default policy file during sandbox
   start-up.

2. The agent then connects to the OPA service on localhost and sends
   the default policy to OPA.

3. If the shim calls SetPolicy:

   a. The agent checks if SetPolicy is allowed by the current
      policy (the current policy is typically the default policy
      mentioned above).

   b. If SetPolicy is allowed, the agent deletes the current policy
      from OPA and replaces it with the new policy it received from
      the shim.

   A typical new policy from the shim doesn't allow any future SetPolicy
   calls.

4. For every agent rpc API call, the agent asks OPA if that call
   should be allowed. OPA allows or not a call based on the current
   policy, the name of the agent API, and the API call's inputs. The
   agent rejects any calls that are rejected by OPA.

When building using AGENT_POLICY_DEBUG=yes, additional Policy logging
gets enabled in the agent. In particular, information about the inputs
for agent rpc API calls is logged in /tmp/policy.txt, on the Guest VM.
These inputs can be useful for investigating API calls that might have
been rejected by the Policy. Examples:

1. Load a failing policy file test1.rego on a different machine:

opa run --server --addr 127.0.0.1:8181 test1.rego

2. Collect the API inputs from Guest's /tmp/policy.txt and test on the
   machine where the failing policy has been loaded:

curl -X POST http://localhost:8181/v1/data/agent_policy/CreateContainerRequest \
--data-binary @test1-inputs.json

Signed-off-by: Dan Mihai <dmihai@microsoft.com>
(cherry picked from commit ab829d1038)

Note: this is cherrypicked to help with the following:
- Provide a building block to continue experimenting with policy and identify issues at the earliest.
This is especially helpful for remote hypervisor (peer-pods) as currently we have no way to test
this feature and identify areas of improvements as part of merge to main.

- Provide a building building block to prototype and understand any potential gaps or integration
issues with the initdata specification discussed in the following issue - https://github.com/confidential-containers/confidential-containers/issues/171

There are no tests for this feature in CCv0 branch and you should use it at your own risk.

Signed-off-by: Pradipta Banerjee <pradipta.banerjee@gmail.com>
2023-11-30 18:07:40 +05:30
..
2022-06-08 15:51:18 +01:00
2022-05-31 09:21:02 -07:00

Building a Guest OS rootfs for Kata Containers

The Kata Containers rootfs is created using the rootfs.sh script.

Supported base OSs

The rootfs.sh script builds a rootfs based on a particular Linux* distribution. The script supports multiple distributions and can be extended to add further ones.

Extra features

Supported distributions list

List the supported distributions by running the following:

$ ./rootfs.sh -l

Generate Kata specific files

The rootfs.sh script can be used to populate a directory with only Kata specific files and components, without creating a full usable rootfs. This feature is used to create a rootfs based on a distribution not officially supported by osbuilder, and when building an image using the dracut build method.

To achieve this, simply invoke rootfs.sh without specifying a target rootfs, e.g.:

$ mkdir kata-overlay
$ ./rootfs.sh -r "$PWD/kata-overlay"

Rootfs requirements

The rootfs must provide at least the following components:

  • Kata agent

    Path: /bin/kata-agent - Kata Containers guest.

  • An init system (e.g. systemd) to start the Kata agent when the guest OS boots.

    Path: /sbin/init - init binary called by the kernel.

When the AGENT_INIT environment variable is set to yes, use Kata agent as /sbin/init.

Note

: AGENT_INIT=yes must be used for the Alpine distribution since it does not use systemd as its init daemon.

Creating a rootfs

To build a rootfs for your chosen distribution, run:

$ sudo ./rootfs.sh <distro>

Creating a rootfs with kernel modules

To build a rootfs with additional kernel modules, run:

$ sudo KERNEL_MODULES_DIR=${kernel_mod_dir} ./rootfs.sh <distro>

Where kernel_mod_dir points to the kernel modules directory to be put under the /lib/modules/ directory of the created rootfs.

Build a rootfs using Docker

Depending on the base OS to build the rootfs guest OS, it is required some specific programs that probably are not available or installed in the system that will build the guest image. For this case rootfs.sh can use a Docker* container to build the rootfs. The following requirements must be met:

  1. Docker 1.12+ installed.

  2. runc is configured as the default runtime.

    To check if runc is the default runtime:

    $ docker info | grep 'Default Runtime: runc'
    
  3. Export USE_DOCKER variable.

    $ export USE_DOCKER=true
    
  4. Use rootfs.sh:

    Example:

    $ export USE_DOCKER=true
    $ # build guest O/S rootfs based on debian
    $ ./rootfs-builder/rootfs.sh -r "${PWD}/debian_rootfs" debian
    $ # build image based rootfs created above
    $ ./image-builder/image_builder.sh "${PWD}/debian_rootfs"
    

Adding support for a new guest OS

The rootfs.sh script will check for immediate sub-directories containing the following expected files:

  • A bash(1) script called config.sh

    This represents the specific configuration for <distro>. It must provide configuration specific variables for the user to modify as needed. The config.sh file will be loaded before executing build_rootfs() to provide all the needed configuration to the function.

    Path: rootfs-builder/<distro>/config.sh.

  • (OPTIONAL) A bash(1) script called rootfs_lib.sh

    This file must contain a function called build_rootfs(), which must receive the path to where the rootfs is created, as its first argument. Normally, this file is needed if a new distro with a special requirement is needed. This function will override the build_rootfs() function in scripts/lib.sh.

    Path: rootfs-builder/<distro>/rootfs_lib.sh.

Create template files

To create a directory with the expected file structure run:

$ make -f template/Makefile  ROOTFS_BASE_NAME=my_new_awesome_rootfs

After running the previous command, a new directory is created in rootfs-builder/my_new_awesome_rootfs/.

To verify the directory can be used to build a rootfs, run ./rootfs.sh -h. Running this script shows my_new_awesome_rootfs as one of the options for use. To use the new guest OS, follow the instructions in Creating a rootfs.

Modify template files

After the new directory structure is created:

  • If needed, add configuration variables to rootfs-builder/my_new_awesome_rootfs/config.sh.

  • Implement the stub build_rootfs() function from rootfs-builder/my_new_awesome_rootfs/rootfs_lib.sh.

Expected rootfs directory content

After the function build_rootfs is called, the script expects the rootfs directory to contain /sbin/init and /sbin/kata-agent binaries.

Optional - Customize the rootfs

For particular use cases developers might want to modify the guest OS.

Adding extra packages

To add additional packages, use one of the following methods:

  • Use the environment variable EXTRA_PKGS to provide a list of space-separated packages to install.

    Note:

    The package names might vary among Linux distributions, the extra package names must exist in the base OS flavor you use to build the rootfs from.

    Example:

    $ EXTRA_PKGS="vim emacs" ./rootfs-builder/rootfs.sh -r ${PWD}/myrootfs debian
    
  • Modify the variable PACKAGES in rootfs-builder/<distro>/config.sh.

    This variable specifies the minimal set of packages needed. The configuration file must use the package names from the distro for which they were created.

Arbitrary rootfs changes

Once the rootfs directory is created, you can add and remove files as needed. Changes affect the files included in the final guest image.