3879 Commits

Author SHA1 Message Date
Liu Jiang
1d5c898d7f libs/sys-util: add utilities to parse NUMA information
Add utilities to parse NUMA information.

Signed-off-by: Liu Jiang <gerry@linux.alibaba.com>
Signed-off-by: Qingyuan Hou <qingyuan.hou@linux.alibaba.com>
Signed-off-by: Simon Guo <wei.guo.simon@linux.alibaba.com>
2022-06-10 19:15:12 +08:00
Liu Jiang
87887026f6 libs/sys-util: add utilities to manipulate cgroup
Add utilities to manipulate cgroup, currently only v1 is supported.

Signed-off-by: Liu Jiang <gerry@linux.alibaba.com>
Signed-off-by: He Rongguang <herongguang@linux.alibaba.com>
Signed-off-by: Jiahuan Chao <jhchao@linux.alibaba.com>
Signed-off-by: Qingyuan Hou <qingyuan.hou@linux.alibaba.com>
Signed-off-by: Quanwei Zhou <quanweiZhou@linux.alibaba.com>
Signed-off-by: Tim Zhang <tim@hyper.sh>
2022-06-10 19:14:59 +08:00
Liu Jiang
ccd03e2cae libs/sys-util: add wrappers for mount and fs
Add some wrappers for mount and fs syscall.

Signed-off-by: Liu Jiang <gerry@linux.alibaba.com>
Signed-off-by: Bin Liu <bin@hyper.sh>
Signed-off-by: Fupan Li <lifupan@gmail.com>
Signed-off-by: Huamin Tang <huamin.thm@alibaba-inc.com>
Signed-off-by: Lei Wang <wllenyj@linux.alibaba.com>
Signed-off-by: Quanwei Zhou <quanweiZhou@linux.alibaba.com>
2022-06-10 19:14:06 +08:00
Liu Jiang
45a00b4f02 libs/sys-util: add kata-sys-util crate under src/libs
The kata-sys-util crate is a collection of modules that provides helpers
and utilities used by multiple Kata Containers components.

Fixes: #3305

Signed-off-by: Liu Jiang <gerry@linux.alibaba.com>
2022-06-10 19:10:40 +08:00
Zhongtao Hu
48c201a1ac libs/types: make the variable name easier to understand
1. modify default values for hypervisor
2. change the variable name
3. check the min memory limit

Signed-off-by: Zhongtao Hu <zhongtaohu.tim@linux.alibaba.com>
2022-06-10 19:01:31 +08:00
Zhongtao Hu
b9b6d70aae libs/types: modify implementation details
1. fix nit problems
2. use generic type when parsing different type

Signed-off-by: Zhongtao Hu <zhongtaohu.tim@linux.alibaba.com>
2022-06-10 19:01:24 +08:00
Zhongtao Hu
05ad026fc0 libs/types: fix implementation details
use ok_or_else to handle get_mut(hypervisor) to substitue unwrap

Signed-off-by: Zhongtao Hu <zhongtaohu.tim@linux.alibaba.com>
2022-06-10 19:01:17 +08:00
Zhongtao Hu
d96716b4d2 libs/types:fix styles and implementation details
1. Some Nit problems are fixed
2. Make the code more readable
3. Modify some implementation details

Signed-off-by: Zhongtao Hu <zhongtaohu.tim@linux.alibaba.com>
2022-06-10 19:01:09 +08:00
Zhongtao Hu
6cffd943be libs/types:return Result to handle parse error
If there is a parse error when we are trying to get the annotations, we
will return Result<Option<type>> to handle that.

Signed-off-by: Zhongtao Hu <zhongtaohu.tim@linux.alibaba.com>
2022-06-10 19:00:58 +08:00
Zhongtao Hu
6ae87d9d66 libs/types: use contains to make code more readable
use contains to when validate hypervisor block_device_driver

Signed-off-by: Zhongtao Hu <zhongtaohu.tim@linux.alibaba.com>
2022-06-10 19:00:50 +08:00
Zhongtao Hu
45e5780e7c libs/types: fixed spelling and grammer error
fixed spelling and grammer error in some files

Signed-off-by: Zhongtao Hu <zhongtaohu.tim@linux.alibaba.com>
2022-06-10 19:00:43 +08:00
Zhongtao Hu
2599a06a56 libs/types:use include_str! in test file
use include_str! to load toml file to string fmt

Signed-off-by: Zhongtao Hu <zhongtaohu.tim@linux.alibaba.com>
2022-06-10 18:28:14 +08:00
Zhongtao Hu
8ffff40af4 libs/types:Option type to handle empty tomlconfig
loading from empty string is only used to identity that the config is
not initialized yet, so Option<TomlConfig> is a better option

Signed-off-by: Zhongtao Hu <zhongtaohu.tim@linux.alibaba.com>
2022-06-10 18:28:05 +08:00
Zhongtao Hu
626828696d libs/types: add license for test-config.rs
add SPDX license identifier: Apache-2.0

Signed-off-by: Zhongtao Hu <zhongtaohu.tim@linux.alibaba.com>
2022-06-10 18:27:57 +08:00
Liu Jiang
8cdd70f6c2 libs/types: change method to update config by annotation
Some annotations are used to override hypervisor configurations, and you
know it's dangerous. We must be careful when overriding hypervisor configuration
by annotations, to avoid security flaws.
There are two existing mechanisms to prevent attacks by annotations:
1) config.hypervisor.enable_annotations defines the allowed annotation
keys for config.hypervisor.
2) config.hyperisor.xxxx_paths defines allowd values for specific keys.

The access methods for config.hypervisor.xxx enforces the permisstion
checks for above rules.

To update conifg, traverse the annotation hashmap,check if the key is enabled in hypervisor or not.
If it is enabled. For path related annotation, check whether it is valid or not
before updating conifg. For cpu and memory related annotation, check whether it
is more than or less than the limitation for DB and qemu beforing updating config.

If it is not enabled, there will be three possibilities, agent related
annotation, runtime related annotation and hypervisor related annotation
but not enabled. The function will handle agent and runtime annotation
first, then the option left will be the invlaid hypervisor, err message
will be returned.

add more edge cases tests for updating config

clean up unused functions, delete unused files and fix warnings

Fixes: #3523

Signed-off-by: Zhongtao Hu <zhongtaohu.tim@linux.alibaba.com>
Signed-off-by: Liu Jiang <gerry@linux.alibaba.com>
2022-06-10 18:27:36 +08:00
Liu Jiang
e19d04719f libs/types: implement KataConfig to wrap TomlConfig
The TomlConfig structure is a parsed form of Kata configuration file,
but it's a little inconveneient to access those configuration
information directly. So introduce a wrapper KataConfig to easily
access those configuration information.

Two singletons of KataConfig is provided:
- KATA_DEFAULT_CONFIG: the original version directly loaded from Kata
configuration file.
- KATA_ACTIVE_CONFIG: the active version is the KATA_DEFAULT_CONFIG
patched by annotations.

So the recommended to way to use these two singletons:
- Load TomlConfig from configuration file and set it as the default one.
- Clone the default one and patch it with values from annotations.
- Use the default one for permission checks, such as to check for
  allowed annotation keys/values.
- The patched version may be set as the active one or passed to clients.
- The clients directly accesses information from the active/passed one,
  and do not need to check annotation for override.

Signed-off-by: Liu Jiang <gerry@linux.alibaba.com>
2022-06-10 18:26:48 +08:00
Liu Jiang
387ffa914e libs/types: support load Kata agent configuration from file
Add structures to load Kata agent configuration from configuration files.
Also define a mechanism for vendor to extend the Kata configuration
structure.

Signed-off-by: Liu Jiang <gerry@linux.alibaba.com>
2022-06-10 18:26:37 +08:00
Liu Jiang
69f10afb71 libs/types: support load Kata hypervisor configuration from file
Add structures to load Kata hypevisor configuration from configuration
files. Also define a mechanisms to:
1) for hypervisors to handle the configuration info.
2) for vendor to extend the Kata configuration structure.

Signed-off-by: Liu Jiang <gerry@linux.alibaba.com>
Signed-off-by: Zhongtao Hu <zhongtaohu.tim@linux.alibaba.com>
2022-06-10 18:25:24 +08:00
Liu Jiang
21cc02d724 libs/types: support load Kata runtime configuration from file
Add structures to load Kata runtime configuration from configuration
files. Also define a mechanism for vendor to extend the Kata
configuration structure.

Signed-off-by: Liu Jiang <gerry@linux.alibaba.com>
Signed-off-by: Zhongtao Hu <zhongtaohu.tim@linux.alibaba.com>
2022-06-10 18:25:24 +08:00
Liu Jiang
5b89c1df2f libs/types: add kata-types crate under src/libs
Add kata-types crate to host constants and data types shared by multiple
Kata Containers components.

Fixes: #3305

Signed-off-by: Liu Jiang <gerry@linux.alibaba.com>
Signed-off-by: Fupan Li <lifupan@gmail.com>
Signed-off-by: Huamin Tang <huamin.thm@alibaba-inc.com>
Signed-off-by: Lei Wang <wllenyj@linux.alibaba.com>
Signed-off-by: yanlei <yl.on.the.way@gmail.com>
2022-06-10 18:25:24 +08:00
Liu Jiang
4f62a7618c libs/logging: fix clippy warnings
Fix clippy warnings of libs/logging.

Signed-off-by: Liu Jiang <gerry@linux.alibaba.com>
2022-06-10 18:25:24 +08:00
Liu Jiang
6f8acb94c2 libs: refine Makefile rules
Refine Makefile rules to better support the KATA ci env.

Fixes: #3536

Signed-off-by: Liu Jiang <gerry@linux.alibaba.com>
2022-06-10 18:25:24 +08:00
Liu Jiang
7cdee4980c libs/logging: introduce a wrapper writer for logging
Introduce a wrapper writer `LogWriter` which converts every line written
to it into a log record.

Signed-off-by: Liu Jiang <gerry@linux.alibaba.com>
Signed-off-by: Wei Yang <wei.yang1@linux.alibaba.com>
Signed-off-by: yanlei <yl.on.the.way@gmail.com>
2022-06-10 18:25:24 +08:00
Liu Jiang
426f38de94 libs/logging: implement rotator for log files
Add FileRotator to rotate log files.

The FileRotator structure may be used as writer for create_logger()
and limits the storage space occupied by log files.

Fixes: #3304

Signed-off-by: Liu Jiang <gerry@linux.alibaba.com>
Signed-off-by: Wei Yang <wei.yang1@linux.alibaba.com>
Signed-off-by: yanlei <yl.on.the.way@gmail.com>
2022-06-10 18:25:24 +08:00
Liu Jiang
392f1ecdf5 libs: convert to a cargo workspace
Convert libs into a Cargo workspace, so all libraries could share the
build infrastructure.

Fixes #3282

Signed-off-by: Liu Jiang <gerry@linux.alibaba.com>
2022-06-10 18:25:24 +08:00
James O. D. Hunt
412441308b docs: Add more kata monitor details
Add more detail to the `kata-monitor` doc to allow an admin to make a
more informed decision about where and how to run the daemon.

Fixes: #4416.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2022-06-09 09:20:11 +01:00
Bin Liu
ae911d0cd3 Merge pull request #4378 from cmaf/update-containerd-docs-critools
docs: Update source for cri-tools
2022-06-09 15:12:37 +08:00
Bin Liu
05022975c8 Merge pull request #4413 from jodh-intel/tools-full-err-output
tools: Enable extra detail on error
2022-06-09 13:52:08 +08:00
Chelsea Mafrica
aaa74e8a2b Merge pull request #4415 from jodh-intel/agent-ctl-doc-examples
docs: Add agent-ctl examples section
2022-06-08 09:51:30 -07:00
Eric Ernst
4ebf9d38b9 Merge pull request #4310 from egernst/core-sched
shim: add support for core scheduling
2022-06-08 17:42:45 +02:00
Bin Liu
eff4e1017d shim: change the log level for GetOOMEvent call failures
GetOOMEvent is a blocking call that will fail if
the container exit, in this case, it's not an error or warning.

Changing the log level for logs in case of GetOOMEvent call fails
will reduce log noise in a large cluster that has pods
creating/deleting frequently.

Fixes: #4376

Signed-off-by: Bin Liu <bin@hyper.sh>
2022-06-08 22:17:24 +08:00
dependabot[bot]
5d7fb7b7b0 build(deps): bump github.com/containerd/containerd in /src/runtime
Bumps [github.com/containerd/containerd](https://github.com/containerd/containerd) from 1.6.1 to 1.6.6.
- [Release notes](https://github.com/containerd/containerd/releases)
- [Changelog](https://github.com/containerd/containerd/blob/main/RELEASES.md)
- [Commits](https://github.com/containerd/containerd/compare/v1.6.1...v1.6.6)

---
updated-dependencies:
- dependency-name: github.com/containerd/containerd
  dependency-type: direct:production
...

Fixes: #4421
Signed-off-by: dependabot[bot] <support@github.com>
2022-06-08 10:54:46 +03:00
dependabot[bot]
d0ca2fcbbc build(deps): bump crossbeam-utils in /src/tools/trace-forwarder
Bumps [crossbeam-utils](https://github.com/crossbeam-rs/crossbeam) from 0.8.5 to 0.8.8.
- [Release notes](https://github.com/crossbeam-rs/crossbeam/releases)
- [Changelog](https://github.com/crossbeam-rs/crossbeam/blob/master/CHANGELOG.md)
- [Commits](https://github.com/crossbeam-rs/crossbeam/compare/crossbeam-utils-0.8.5...crossbeam-utils-0.8.8)

---
updated-dependencies:
- dependency-name: crossbeam-utils
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-06-08 10:47:58 +03:00
dependabot[bot]
a60dcff4d8 build(deps): bump regex from 1.5.4 to 1.5.6 in /src/tools/agent-ctl
Bumps [regex](https://github.com/rust-lang/regex) from 1.5.4 to 1.5.6.
- [Release notes](https://github.com/rust-lang/regex/releases)
- [Changelog](https://github.com/rust-lang/regex/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/regex/compare/1.5.4...1.5.6)

---
updated-dependencies:
- dependency-name: regex
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-06-08 10:47:58 +03:00
dependabot[bot]
dbf50672e1 build(deps): bump crossbeam-utils in /src/tools/agent-ctl
Bumps [crossbeam-utils](https://github.com/crossbeam-rs/crossbeam) from 0.8.5 to 0.8.8.
- [Release notes](https://github.com/crossbeam-rs/crossbeam/releases)
- [Changelog](https://github.com/crossbeam-rs/crossbeam/blob/master/CHANGELOG.md)
- [Commits](https://github.com/crossbeam-rs/crossbeam/compare/crossbeam-utils-0.8.5...crossbeam-utils-0.8.8)

---
updated-dependencies:
- dependency-name: crossbeam-utils
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-06-08 10:47:58 +03:00
dependabot[bot]
8e2847bd52 build(deps): bump crossbeam-utils from 0.8.6 to 0.8.8 in /src/libs
Bumps [crossbeam-utils](https://github.com/crossbeam-rs/crossbeam) from 0.8.6 to 0.8.8.
- [Release notes](https://github.com/crossbeam-rs/crossbeam/releases)
- [Changelog](https://github.com/crossbeam-rs/crossbeam/blob/master/CHANGELOG.md)
- [Commits](https://github.com/crossbeam-rs/crossbeam/compare/crossbeam-utils-0.8.6...crossbeam-utils-0.8.8)

---
updated-dependencies:
- dependency-name: crossbeam-utils
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-06-08 10:47:58 +03:00
dependabot[bot]
e9ada165ff build(deps): bump regex from 1.5.4 to 1.5.5 in /src/agent
Bumps [regex](https://github.com/rust-lang/regex) from 1.5.4 to 1.5.5.
- [Release notes](https://github.com/rust-lang/regex/releases)
- [Changelog](https://github.com/rust-lang/regex/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/regex/compare/1.5.4...1.5.5)

---
updated-dependencies:
- dependency-name: regex
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-06-08 10:47:58 +03:00
dependabot[bot]
adad9cef18 build(deps): bump crossbeam-utils from 0.8.5 to 0.8.8 in /src/agent
Bumps [crossbeam-utils](https://github.com/crossbeam-rs/crossbeam) from 0.8.5 to 0.8.8.
- [Release notes](https://github.com/crossbeam-rs/crossbeam/releases)
- [Changelog](https://github.com/crossbeam-rs/crossbeam/blob/master/CHANGELOG.md)
- [Commits](https://github.com/crossbeam-rs/crossbeam/compare/crossbeam-utils-0.8.5...crossbeam-utils-0.8.8)

---
updated-dependencies:
- dependency-name: crossbeam-utils
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-06-08 10:47:58 +03:00
James O. D. Hunt
34bcef8846 docs: Add agent-ctl examples section
Add a new `Examples` section to the `agent-ctl` docs giving some
examples of how to use the tool with QEMU and stand-alone.

Fixes: #4414.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2022-06-08 08:39:38 +01:00
James O. D. Hunt
815157bf02 docs: Remove erroneous whitespace
Deleted an extra blank line.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2022-06-08 08:39:38 +01:00
James O. D. Hunt
f5099620f1 tools: Enable extra detail on error
The `agent-ctl` and `trace-forwarder` tools make use of
`anyhow::Context` to provide additional call site information on error.

However, previously neither tool was using the "alternate debug" format
to display the error, meaning full error output was not displayed.

Fixes: #4411.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2022-06-07 14:00:29 +01:00
Bin Liu
a238d8c6bd Merge pull request #4300 from justxuewei/fix/rustjail/home-env
rustjail: get home dir using nix crate
2022-06-06 11:03:46 +08:00
Bin Liu
f981190621 Merge pull request #4383 from cyyzero/runk-list
runk: Support `list` sub-command
2022-06-06 10:25:33 +08:00
David Gibson
8f10e13e07 config: Allow enable_iommu pod annotation by default
Since #902 the `io.katacontainers.config.hypervisor` pod annotations
have only been permitted if explicitly allowed in the global
configuration.  The default global configuration allows no such
annotations.  That's important because several of those annotations
would cause Kata to execute arbitrary binaries, and so were wildly
unsafe.

However, this is inconvenient for the
`io.katacontainers.config.hypervisor.enable_iommu` annotation
specifically, which controls whether the sandbox VM includes a vIOMMU.
A guest side vIOMMU is necessary to implement VFIO passthrough devices
with `vfio_mode = vfio`, so enabling that mode of operation currently
requires a global configuration change, and can't just be enabled
per-pod.

Unlike some of the other hypervisor annotations, the `enable_iommu`
annotation is quite safe.  By default the vIOMMU is not present, so
allowing a user to override it for a pod only improves their
facilities for isolation.  Even if the global default were changed to
enable the vIOMMU, that doesn't compel the guest kernel to use it, so
allowing a user to disable the vIOMMU doesn't materially affect
isolation either.

Therefore, allow the io.katacontainers.config.hypervisor.enable_iommu
annotation to work in the default configurations.

fixes #4330

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2022-06-04 13:02:05 +10:00
Eric Ernst
430da47215 Merge pull request #4360 from fengwang666/shim-leak
runtime: ignore ESRCH error from stop container
2022-06-02 12:42:19 -07:00
Feng Wang
9d27c1fced agent: ignore ESRCH error when destroying containers
destroy() method should ignore the ESRCH error from signal::kill
and continue the operation as ESRCH is often considered harmless.

Fixes: #4359

Signed-off-by: Feng Wang <feng.wang@databricks.com>
2022-06-02 08:19:48 -07:00
Feng Wang
9726f56fdc runtime: force stop container after the container process exits
Set thestop container force flag to true so that the container state is always set to
“StateStopped” after the container wait goroutine is finished. This is necessary for
the following delete container step to succeed.

Fixes: #4359

Signed-off-by: Feng Wang <feng.wang@databricks.com>
2022-06-02 08:17:08 -07:00
Chen Yiyang
38a3188206 runk: Support list sub-command
Support list sub-command. It will traverse the root directory, parse
status file and print basic information of containers. Behavior and
print format consistent with runc. To handle race with runk delete
or system user modify, the loop will continue to traverse when errors
are encountered.

Fixes: #4362

Signed-off-by: Chen Yiyang <cyyzero@qq.com>
2022-06-02 18:24:51 +08:00
Peng Tao
295a01f9b1 Merge pull request #4159 from egernst/topic/iptables
feature: add ability to interact with IPTables within the guest
2022-06-02 11:19:41 +08:00
Tim Zhang
b8e98b175c Merge pull request #4355 from liubin/fix/add-debug-info-for-parse-mount-error
agent: return mount file content if parse mountinfo failed
2022-06-02 10:31:46 +08:00