3879 Commits

Author SHA1 Message Date
Eric Ernst
e5be5cb086 runtime: device: cleanup outdated comments
Prior device config move didn't update the comments. Let's address this,
and make sure comments match the new path...

Signed-off-by: Eric Ernst <eric_ernst@apple.com>
2022-06-28 18:22:28 -07:00
Eric Ernst
5f936f268f virtcontainers: config validation is host specific
Ideally this config validation would be in a seperate package
(katautils?), but that would introduce circular dependency since we'd
call it from vc, and it depends on vc types (which, shouldn't be vc, but
probably a hypervisor package instead).

Signed-off-by: Eric Ernst <eric_ernst@apple.com>
2022-06-28 18:22:28 -07:00
Fabiano Fidêncio
323271403e virtcontainers: Remove unused function
While working on the previous commits, some of the functions become
non-used.  Let's simply remove them.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-06-28 21:19:24 +02:00
Fabiano Fidêncio
0939f5181b config: Expose default_maxmemory
Expose the newly added `default_maxmemory` to the project's Makefile and
to the configuration files.

Fixes: #4516

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-06-28 21:19:24 +02:00
Fabiano Fidêncio
58ff2bd5c9 clh,qemu: Adapt to using default_maxmemory
Let's adapt Cloud Hypervisor's and QEMU's code to properly behave to the
newly added `default_maxmemory` config.

While implementing this, a change of behaviour (or a bug fix, depending
on how you see it) has been introduced as if a pod requests more memory
than the amount avaiable in the host, instead of failing to start the
pod, we simply hotplug the maximum amount of memory available, mimicing
better the runc behaviour.

Fixes: #4516

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-06-28 21:19:24 +02:00
Zhongtao Hu
f3907aa127 runtime-rs:Merge remote-tracking branch 'origin/main' into runtime-rs-newv
Fixes:#4536
Signed-off-by: Zhongtao Hu <zhongtaohu.tim@linux.alibaba.com>
2022-06-28 20:58:40 +08:00
Bin Liu
badbbcd8be Merge pull request #4400 from openanolis/anolis/dragonball-2
runtime-rs: built-in Dragonball sandbox part II - vCPU manager
2022-06-28 20:41:36 +08:00
Tim Zhang
916ffb75d7 Merge pull request #4432 from liubin/fix/4420-binary-log
shim: support shim v2 logging plugin
2022-06-28 16:29:07 +08:00
Fabiano Fidêncio
afdc960424 hypervisor: Add default_maxmemory configuration
Let's add a `default_maxmemory` configuration, which allows the admins
to set the maximum amount of memory to be used by a VM, considering the
initial amount + whatever ends up being hotplugged via the pod limits.

By default this value is 0 (zero), and it means that the whole physical
RAM is the limit.

Fixes: #4516

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-06-28 08:32:15 +02:00
Bin Liu
4e30e11b31 shim: support shim v2 logging plugin
Now kata shim only supports stdout/stderr of fifo from
containerd/CRI-O, but shim v2 supports logging plugins,
and nerdctl default will use the binary schema for logs.

This commit will add the others type of log plugins:

- file
- binary

In case of binary, kata shim will receive a stdout/stderr like:

binary:///nerdctl?_NERDCTL_INTERNAL_LOGGING=/var/lib/nerdctl/1935db59

That means the nerdctl process will handle the logs(stdout/stderr)

Fixes: #4420

Signed-off-by: Bin Liu <bin@hyper.sh>
2022-06-28 13:54:22 +08:00
Eric Ernst
bdf5e5229b virtcontainers: validate hypervisor config outside of hypervisor itself
Depending on the user of it, the hypervisor from hypervisor interface
could have differing view on what is valid or not. To help decouple,
let's instead check the hypervisor config validity as part of the
sandbox creation, rather than as part of the CreateVM call within the
hypervisor interface implementation.

Fixes: #4251

Signed-off-by: Eric Ernst <eric_ernst@apple.com>
2022-06-27 11:53:41 -07:00
Eric Ernst
469e098543 katautils: don't do validation when loading hypervisor config
Policy for whats valid/invalid within the config varies by VMM, host,
and by silicon architecture. Let's keep katautils simple for just
translating a toml to the hypervisor config structure, and leave
validation to virtcontainers.

Without this change, we're doing duplicate validation.

Signed-off-by: Eric Ernst <eric_ernst@apple.com>
2022-06-27 10:13:26 -07:00
Chao Wu
71db2dd5b8 hotplug: add room for future acpi hotplug mechanism
In order to support ACPI hotplug in the future with the cooperative work
from the Kata community, we add ACPI feature and dbs-upcall feature to
add room for ACPI hotplug.

Signed-off-by: Chao Wu <chaowu@linux.alibaba.com>
2022-06-27 21:52:36 +08:00
Zizheng Bian
8bb00a3dc8 dragonball: fix a bug when generating kernel boot args
We should refuse to generate boot args when hotplugging, not cold starting.

Signed-off-by: Zizheng Bian <zizheng.bian@linux.alibaba.com>
2022-06-27 18:12:50 +08:00
Chao Wu
2aedd4d12a doc: add document for vCPU, api and device
Create the document for vCPU and api.

Add some detail in the device document.

Fixes: #4257

Signed-off-by: Chao Wu <chaowu@linux.alibaba.com>
2022-06-27 18:12:50 +08:00
wllenyj
bec22ad01f dragonball: add api module
It is used to define the vmm communication interface.

Signed-off-by: Chao Wu <chaowu@linux.alibaba.com>
Signed-off-by: wllenyj <wllenyj@linux.alibaba.com>
2022-06-27 18:12:50 +08:00
wllenyj
07f44c3e0a dragonball: add vcpu manager
Manage vcpu related operations.

Signed-off-by: Liu Jiang <gerry@linux.alibaba.com>
Signed-off-by: jingshan <jingshan@linux.alibaba.com>
Signed-off-by: Chao Wu <chaowu@linux.alibaba.com>
Signed-off-by: wllenyj <wllenyj@linux.alibaba.com>
2022-06-27 18:12:48 +08:00
wllenyj
78c9718752 dragonball: add upcall support
Upcall is a direct communication tool between VMM and guest developed
upon vsock. It is used to implement device hotplug.

Signed-off-by: Liu Jiang <gerry@linux.alibaba.com>
Signed-off-by: jingshan <jingshan@linux.alibaba.com>
Signed-off-by: Chao Wu <chaowu@linux.alibaba.com>
Signed-off-by: wllenyj <wllenyj@linux.alibaba.com>
Signed-off-by: Zizheng Bian <zizheng.bian@linux.alibaba.com>
2022-06-27 17:04:47 +08:00
wllenyj
7d1953b52e dragonball: add vcpu
Virtual CPU manager for virtual machines.

Signed-off-by: Liu Jiang <gerry@linux.alibaba.com>
Signed-off-by: jingshan <jingshan@linux.alibaba.com>
Signed-off-by: Chao Wu <chaowu@linux.alibaba.com>
Signed-off-by: wllenyj <wllenyj@linux.alibaba.com>
2022-06-27 17:04:42 +08:00
wllenyj
468c73b3cb dragonball: add kvm context
KVM operation context for virtual machines.

Signed-off-by: Liu Jiang <gerry@linux.alibaba.com>
Signed-off-by: jingshan <jingshan@linux.alibaba.com>
Signed-off-by: Chao Wu <chaowu@linux.alibaba.com>
Signed-off-by: wllenyj <wllenyj@linux.alibaba.com>
2022-06-27 16:02:06 +08:00
Bin Liu
27b1bb5ed9 Merge pull request #4467 from egernst/device-pkg
device package cleanup/refactor
2022-06-27 14:40:53 +08:00
Eric Ernst
e32bf53318 device: deduplicate state structures
Before, we maintained almost identical structures between our persist
API and what we keep for our devices, with the persist API being a
slight subset of device structures.

Let's deduplicate this, now that persist is importing device package.
Json unmarshal of prior persist structure will work fine, since it was
an exact subset of fields.

Fixes: #4468

Signed-off-by: Eric Ernst <eric_ernst@apple.com>
2022-06-26 21:31:29 -07:00
Eric Ernst
f97d9b45c8 runtime: device/persist: drop persist dependency from device pkgs
Rather than have device package depend on persist, let's define the
(almost duplicate) structures within device itself, and have the Kata
Container's persist pkg import these.

This'll help avoid unecessary dependencies within our core packages.

Signed-off-by: Eric Ernst <eric_ernst@apple.com>
2022-06-26 21:31:29 -07:00
Eric Ernst
f9e96c6506 runtime: device: move to top level package
Let's move device package to runtime/pkg instead of being buried under
virtcontainers.

Signed-off-by: Eric Ernst <eric_ernst@apple.com>
2022-06-26 21:31:29 -07:00
Bin Liu
3880e0c077 agent: refactor reading file timing for debugging
In the original code, reads mountstats file and return
the content in the error, but at this time the file maybe
changed, we should return the file content that parsed
line by line to check why there is not a fstype option.

Fixes: #4246

Signed-off-by: Bin Liu <bin@hyper.sh>
2022-06-26 21:27:43 -07:00
Fabiano Fidêncio
083ca5f217 Merge pull request #4505 from yoheiueda/agent-debug-build
agent: Allow BUILD_TYPE=debug
2022-06-24 14:04:23 +02:00
Fabiano Fidêncio
c70d3a2c35 agent: Update the dependencies
Let's run a `cargo update` and ensure the deps are up-to-date before we
cut the "-rc0" release.

Fixes: #4525

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-06-24 11:37:25 +02:00
Fabiano Fidêncio
612fd79bae random: Fix "nonminimal-bool" clippy warning
The error shown below was caught during a dependency bump in the CCv0
branch, but we better fix it here first.
```
error: this boolean expression can be simplified
  --> src/random.rs:85:21
   |
85 |             assert!(!ret.is_ok());
   |                     ^^^^^^^^^^^^ help: try: `ret.is_err()`
   |
   = note: `-D clippy::nonminimal-bool` implied by `-D warnings`
   = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#nonminimal_bool

error: this boolean expression can be simplified
  --> src/random.rs:93:17
   |
93 |         assert!(!ret.is_ok());
   |                 ^^^^^^^^^^^^ help: try: `ret.is_err()`
   |
   = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#nonminimal_bool
```

Fixes: #4523

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-06-24 11:37:05 +02:00
Fabiano Fidêncio
d4417f210e netlink: Fix "or-fun-call" clippy warnings
The error shown below was caught during a dependency bump in the CCv0
branch, but we better fix it here first.
```
error: use of `ok_or` followed by a function call
   --> src/netlink.rs:526:14
    |
526 |             .ok_or(anyhow!(nix::Error::EINVAL))?;
    |              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ help: try this: `ok_or_else(|| anyhow!(nix::Error::EINVAL))`
    |
    = note: `-D clippy::or-fun-call` implied by `-D warnings`
    = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#or_fun_call
error: use of `ok_or` followed by a function call
   --> src/netlink.rs:615:49
    |
615 |         let v = u8::from_str_radix(split.next().ok_or(anyhow!(nix::Error::EINVAL))?, 16)?;
    |                                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ help: try this: `ok_or_else(|| anyhow!(nix::Error::EINVAL))`
    |
    = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#or_fun_call
```

Fixes: #4523

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-06-24 11:37:01 +02:00
Fabiano Fidêncio
133528dd14 Merge pull request #4503 from amshinde/multi-queue-block
block: Leverage multiqueue for virtio-block
2022-06-23 12:17:11 +02:00
Yohei Ueda
1b7d36fdb0 agent: Allow BUILD_TYPE=debug
The cargo command creates debug build binaries, when the --release
option is not specified. Specifying --debug option causes an error.
This patch specifies --release option when BUILD_TYPE=release,
and does not specify any build type option when BUILD_TYPE=debug.

Fixes #4504

Signed-off-by: Yohei Ueda <yohei@jp.ibm.com>
2022-06-23 13:54:32 +09:00
Fabiano Fidêncio
78e27de6c3 Merge pull request #4358 from zvonkok/memreserve
runtime: Add heuristic to get the right value(s) for mem-reserve
2022-06-22 13:41:23 +02:00
Archana Shinde
e227b4c404 block: Leverage multiqueue for virtio-block
Similar to network, we can use multiple queues for virtio-block
devices. This can help improve storage performance.
This commit changes the number of queues for block devices to
the number of cpus for cloud-hypervisor and qemu.

Today the default number of cpus a VM starts with is 1.
Hence the queues used will be 1. This change will help
improve performance when the default cold-plugged cpus is greater
than one by changing this in the config file. This may also help
when we use the sandboxing feature with k8s that passes down
the sum of the resources required down to Kata.

Fixes #4502

Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
2022-06-21 12:38:53 -07:00
Eric Ernst
72049350ae Merge pull request #4288 from fengwang666/enable-qemu-sandbox
runtime: enable sandbox feature on qemu
2022-06-21 09:22:26 -07:00
Zvonko Kaiser
e7e7dc9dfe runtime: Add heuristic to get the right value(s) for mem-reserve
Fixes: #2938

Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
2022-06-21 03:44:28 -07:00
Chao Wu
86123f49f2 Merge branch 'main' into runtime-rs
In order to keep update with the main, we will update runtime-rs every
week.

Fixes: #4485
Signed-off-by: Chao Wu <chaowu@linux.alibaba.com>
2022-06-20 10:01:58 +08:00
Liang Zhou
ef925d40ce runtime: enable sandbox feature on qemu
Enable "-sandbox on" in qemu can introduce another protect layer
on the host, to make the secure container more secure.

The default option is disable because this feature may introduce some
performance cost, even though user can enable
/proc/sys/net/core/bpf_jit_enable to reduce the impact.

Fixes: #2266

Signed-off-by: Feng Wang <feng.wang@databricks.com>
2022-06-17 15:30:46 -07:00
Chelsea Mafrica
28995301b3 tracing: Remove whitespace from root span
Remove space from root span name to follow camel casing of other tracing
span names in the runtime and to make parsing easier in testing.

Fixes #4483

Signed-off-by: Chelsea Mafrica <chelsea.e.mafrica@intel.com>
2022-06-17 12:07:37 -07:00
wllenyj
e89e6507a4 dragonball: add signal handler
Used to register dragonball's signal handler.

Signed-off-by: Liu Jiang <gerry@linux.alibaba.com>
Signed-off-by: jingshan <jingshan@linux.alibaba.com>
Signed-off-by: Chao Wu <chaowu@linux.alibaba.com>
Signed-off-by: wllenyj <wllenyj@linux.alibaba.com>
2022-06-16 17:31:58 +08:00
Fabiano Fidêncio
f30fe86dc1 Merge pull request #4456 from Bevisy/fixIssue4454
docs: Update outdated URLs and keep them available
2022-06-16 10:26:24 +02:00
Bin Liu
553ec46115 Merge pull request #4436 from alex-matei/fix/sandbox-mem-overflow
runtime: fix error when trying to parse sandbox sizing annotations
2022-06-16 11:18:24 +08:00
James O. D. Hunt
9766a285a4 Merge pull request #4422 from snir911/dependabot_bumps
deps: Resolve dependabot bumps of containerd, crossbeam-utils, regex
2022-06-15 15:57:53 +01:00
James O. D. Hunt
d06dd8fcdc Merge pull request #4312 from fidencio/topic/pass-the-tuntap-fd-to-clh
Allow Cloud Hypervisor to run under the `container_kvm_t`
2022-06-15 09:37:49 +01:00
Binbin Zhang
a305bafeef docs: Update outdated URLs and keep them available
By comparing the content of the old url and the new url,
ensure that their content is consistent and does not contain ambiguities

Fixes: #4454

Signed-off-by: Binbin Zhang <binbin36520@gmail.com>
2022-06-15 16:34:28 +08:00
Chelsea Mafrica
db2a4d6cdf Merge pull request #4441 from liubin/fix/refactor-reading-mountstat-log
agent: refactor reading file timing for debugging
2022-06-14 14:18:14 -07:00
Fabiano Fidêncio
ac5dbd8598 clh: Improve logging related to the net dev addition
Let's improve the log so we make it clear that we're only *actually*
adding the net device to the Cloud Hypervisor configuration when calling
our own version of VmAddNetPut().

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-06-14 10:53:09 +00:00
Fabiano Fidêncio
0b75522e1f network: Set queues to 1 to ensure we get the network fds
We want to have the file descriptors of the opened tuntap device to pass
them down to the VMMs, so the VMMs don't have to explicitly open a new
tuntap device themselves, as the `container_kvm_t` label does not allow
such a thing.

With this change we ensure that what's currently done when using QEMU as
the hypervisor, can be easily replicated with other VMMs, even if they
don't support multiqueue.

As a side effect of this, we need to close the received file descriptors
in the code of the VMMs which are not going to use them.

Fixes: #3533

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-06-14 10:53:09 +00:00
Fabiano Fidêncio
93b61e0f07 network: Add FFI_NO_PI to the netlink flags
Adding FFI_NO_PI to the netlink flags causes no harm to the supported
and tested hypervisors as when opening the device by its name Cloud
Hypervisor[0], Firecracker[1], and QEMU[2] do set the flag already.

However, when receiving the file descriptor of an opened tutap device
Cloud Hypervisor is not able to set the flag, leaving the guest without
connectivity.

To avoid such an issue, let's simply add the FFI_NO_PI flag to the
netlink flags and ensure, from our side, that the VMMs don't have to set
it on their side when dealing with an already opened tuntap device.

Note that there's a PR opened[3] just for testing that this change
doesn't cause any breakage.

[0]: e52175c2ab/net_util/src/tap.rs (L129)
[1]: b6d6f71213/src/devices/src/virtio/net/tap.rs (L126)
[2]: 3757b0d08b/net/tap-linux.c (L54)
[3]: https://github.com/kata-containers/kata-containers/pull/4292

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-06-14 10:53:09 +00:00
Fabiano Fidêncio
bf3ddc125d clh: Pass the tuntap fds down to Cloud Hypervisor
This is basically a no-op right now, as:
* netPair.TapInterface.VMFds is nil
* the tap name is still passed to Cloud Hypervisor, which is the Cloud
  Hypervisor's first choice when opening a tap device.

In the very near future we'll stop passing the tap name to Cloud
Hypervisor, and start passing the file descriptors of the opened tap
instead.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-06-14 10:53:09 +00:00
Fabiano Fidêncio
55ed32e924 clh: Take care of the VmAdNetdPut request ourselves
Knowing that VmAddNetPut works as expected, let's switch to manually
building the request and writing it to the appropriate socket.

By doing this it gives us more flexibility to, later on, pass the file
descriptor of the tuntap device to Cloud Hypervisor, as openAPI doesn't
support such operation (it has no notion of SCM Rights).

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-06-14 10:53:09 +00:00