3879 Commits

Author SHA1 Message Date
Fabiano Fidêncio
a606401722 Merge pull request #8803 from jodh-intel/issues-8784-runtime-rs-ch-rm-todo-to-unbreak
runtime-rs: ch: Unbreak CH driver
2024-01-11 19:37:13 -03:00
Fabiano Fidêncio
86a6d133e4 Merge pull request #8248 from microsoft/danmihai1/genpolicy-main
tools: add policy generation tool
2024-01-11 17:02:54 -03:00
James O. D. Hunt
29e0de4e4a runtime-rs: ch: Implement minimal memory hotplug APIs
Replace the `todo!()` calls with a minimal NOP implementation to return
the CH driver to working order since the `todo!()`'s forcibly crash the
driver at runtime. Full implementations for these APIs will be added on
issues #8800, #8801, and #8802.

Fixes: #8784.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2024-01-11 14:11:31 +00:00
James O. D. Hunt
1c0df670af runtime-rs: ch: Add minimal implementation of hypervisor metrics method
Remove the `todo!()` macro which would cause a runtime crash and replace
with a implementation that returns an error as a stop-gap until #8800 is
implemented.

Fixes: #8785.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2024-01-11 14:11:01 +00:00
Hyounggyu Choi
f62ec0a7f5 Merge pull request #8693 from BbolroC/ibm-se-config-validation-fix
runtime: Allow no initrd path for IBM Z Secure Execution
2024-01-11 09:53:51 +01:00
Xuewei Niu
70305fefc5 Merge pull request #8780 from justxuewei/containerd-events
runtime-rs: Forward events to containerd via ttrpc
2024-01-11 14:58:14 +08:00
Xuewei Niu
6fd49f7604 runtime-rs: Forward events to containerd via ttrpc
It is a little bit heavy for the runtime-rs to forwards events via
containerd CLI, contrast to the ttrpc way. Plus, for runtimes that haven't
this mechanism, e.g. CRI-O, we can't get those events anywhere.

This patch introduces two types of forwarders:

- `ContainerdForwarder`: Acquire ttrpc address from environment variables
  and forward events via ttrpc connection.
- `LogForwarder`: Write event info into logs.

Fixes: #7881

Signed-off-by: Xuewei Niu <niuxuewei.nxw@antgroup.com>
2024-01-11 10:32:50 +08:00
Alex.Lyn
695440a431 Merge pull request #8749 from Apokleos/fixup-dragonball-vfio
runtime-rs: fixup vfio device in runtime-rs/dragonball
2024-01-10 15:20:34 +08:00
Greg Kurz
e3611cf27d Merge pull request #8326 from cheriL/8325/fix_method_param
agent: use method params instead of const params in functions
2024-01-09 07:35:19 +01:00
Xuewei Niu
192c6ee9c3 Merge pull request #8773 from justxuewei/dbs-k8s-fragile 2024-01-05 12:54:32 +08:00
Xuewei Niu
0e9d73fe30 agent: Fix an issue reporting OOM events by mistake
The agent registers an event fd in `memory.oom_control`. An OOM event is
forwarded to containerd when the event is emitted, regardless of the
content in that file.

I observed content indicating that events should not be forwarded, as shown
below. When `oom_kill` is set to 0, it means no OOM has occurred. Therefore,
it is important to check the content to avoid mistakenly forwarding OOM
events.

```
oom_kill_disable 0
under_oom 0
oom_kill 0
```

Fixes: #8715

Signed-off-by: Xuewei Niu <niuxuewei.nxw@antgroup.com>
2024-01-05 11:06:37 +08:00
Dan Mihai
7d5336aca3 agent: hold lock while setting new policy
Don't release the lock between is_allowed and set_policy calls,
because the policy might change in between these calls.

Also, move more policy code into policy.rs.

Fixes: #8734

Signed-off-by: Dan Mihai <dmihai@microsoft.com>
2024-01-04 16:45:30 +00:00
Xuewei Niu
b5a6e74cdf Merge pull request #8744 from justxuewei/vhu-net-compile
dragonball: Fix compilation issue without all net features
2024-01-04 19:02:55 +08:00
soup
7c176a62fe agent: use method params instead of const params in functions
Fixes: #8325

Signed-off-by: soup <lqh348659137@outlook.com>
2024-01-04 09:29:29 +01:00
Xuewei Niu
f97f16a44a agent-ctl: Bump ttrpc version
- `ttrpc` from `0.7.1` to `0.8`.

Fixes: #8757

Signed-off-by: Xuewei Niu <niuxuewei.nxw@antgroup.com>
2024-01-04 15:58:34 +08:00
Xuewei Niu
bf59c7b3d4 runtime-rs: Bump ttrpc and containerd-shim-protos versions
- `ttrpc` from `0.7.1` to `0.8`.
- `containerd-shim-protos` from `0.3.0` to `0.6.0`.

Fixes: #8756

Signed-off-by: Xuewei Niu <niuxuewei.nxw@antgroup.com>
2024-01-04 15:58:34 +08:00
Xuewei Niu
cf9a0e21a1 protocols: Bump ttrpc version
- `ttrpc` from `0.7.1` to `0.8`.

Fixes: #8756

Signed-off-by: Xuewei Niu <niuxuewei.nxw@antgroup.com>
2024-01-04 15:58:34 +08:00
Xuewei Niu
91360e7ddb agent: Bump ttrpc version
- `ttrpc` from `0.7.1` to `0.8`.

Fixes: #8756

Signed-off-by: Xuewei Niu <niuxuewei.nxw@antgroup.com>
2024-01-04 15:58:34 +08:00
Chao Wu
f1235ddba3 dbs_virtio_devices: add Cargo.lock
In order to avoid rust-vmm upstream change breaks Dragonball
compilation, we introduce Cargo.lock to dbs crates.

fixes: #8770

Signed-off-by: Chao Wu <chaowu@linux.alibaba.com>
2024-01-04 11:23:30 +08:00
Chao Wu
02cd726bfc dbs-utils: add Cargo.lock
In order to avoid rust-vmm upstream change breaks Dragonball
compilation, we introduce Cargo.lock to dbs crates.

fixes: #8770

Signed-off-by: Chao Wu <chaowu@linux.alibaba.com>
2024-01-04 11:17:45 +08:00
Chao Wu
97bdc1529b dbs-pci: introduce Cargo.lock
As reported in #8767, we have found that the root cause is that rust-vmm's vmm-sys-utils
introduce a new release 0.12.1 and dbs-pci rely on rust-vmm's vfio-ioctls which uses >=
to declare vmm-sys-utils so it automatically upgrade vmm-sys-utils to 0.12.1.
That's how two different versions of vmm-sys-utils is introduced and this breaks the compilation.

In order to fix this and also avoid future problems, we introduce Cargo.lock file to dbs crates.

fixes: #8770

Signed-off-by: Chao Wu <chaowu@linux.alibaba.com>
2024-01-04 11:11:56 +08:00
alex.lyn
d2080fd221 runtime-rs: refactor getting the vfio device guest pci path
Fixes: #8748

Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
2024-01-02 14:28:34 +08:00
alex.lyn
d795fcfc2f runtime-rs: bridge the vfio device between runtime-rs and dragonball
Previously, Dragonball did not support PCI device hot-plugging or
VFIO device passthrough. Therefore, the runtime-rs support for
Dragonball was incomplete. it is time to complete it so that users
can use Dragonball's PCI hot-plugging and VFIO passthrough capabilities.

Fixes: #8748

Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
2024-01-02 14:28:10 +08:00
Chao Wu
67b91c1eb3 Merge pull request #8740 from openanolis/upstream/pci-6-final
Dragonball: add pci vfio passthrough, hot(un)plug support
2023-12-29 01:58:32 +08:00
Chao Wu
71c322c293 runtime-rs: fix ci complains
vfio commits introduce quite a lot change in runtime-rs, this commit is
for all the changes related to ci, including compilation errors and so on.

Signed-off-by: Chao Wu <chaowu@linux.alibaba.com>
2023-12-28 23:34:41 +08:00
Chao Wu
a3f7601f5a dragonball: add pci hotplug / hot-unplug support
Introduce two new vmm action to implement pci hotplug
and pci hot-unplug: PrepareRemoveHostDevice and RemoveHostDevice.

PrepareRemoveHostDevice is to call upcall to unregister the pci device
in the guest kernel.
RemoveHostDevice should be called after PrepareRemoveHostDevice, it is used
to clean the PCI resource in the Dragonball side.

fixes: #8741

Signed-off-by: Gerry Liu <gerry@linux.alibaba.com>
Signed-off-by: Zizheng Bian <zizheng.bian@linux.alibaba.com>
Signed-off-by: Zha Bin <zhabin@linux.alibaba.com>
Signed-off-by: Helin Guo <helinguo@linux.alibaba.com>
Signed-off-by: Chao Wu <chaowu@linux.alibaba.com>
2023-12-28 16:08:31 +08:00
Chao Wu
0f402a14f9 dragonball: add InsertHostDevice vmm action
Introduce a new vmm action InsertHostDevice to passthrough
host pci devices like NIC or GPU devices into guest so that
users could have high performance usage of those devices.

fixes: #8741

Signed-off-by: Gerry Liu <gerry@linux.alibaba.com>
Signed-off-by: Zizheng Bian <zizheng.bian@linux.alibaba.com>
Signed-off-by: Zha Bin <zhabin@linux.alibaba.com>
Signed-off-by: Helin Guo <helinguo@linux.alibaba.com>
Signed-off-by: Chao Wu <chaowu@linux.alibaba.com>
2023-12-28 16:04:22 +08:00
Xuewei Niu
4c023e341c dragonball: Fix compilation issue without all net features
Combinations of network features were tested:

- None
- virtio-net
- vhost-net
- vhost-user-net
- virtio-net,vhost-net
- vhost-net,vhost-user-net
- virtio-net,vhost-user-net
- virtio-net,vhost-net,vhost-user-net

Fixes: #8742

Signed-off-by: Xuewei Niu <niuxuewei.nxw@antgroup.com>
2023-12-28 11:37:26 +08:00
Alex.Lyn
990a3adf39 Merge pull request #8618 from Apokleos/csi-for-directvol
runtime-rs: Add dedicated CSI driver for DirectVolume support in Kata
2023-12-27 21:27:29 +08:00
alex.lyn
ea69c17008 runtime-rs: initialize pcie topology in Device Manager
Add a pcie_topology field to DeviceManager and initialize
pcie_topology when ResourceManager calls DeviceManager's new()
with TopologyConfigInfo.

Fixes: #7218

Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
2023-12-27 15:57:23 +08:00
alex.lyn
b42548b8e1 runtime-rs: do unregister device in Trait Device/detach
Fixes: #7218

Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
2023-12-27 15:53:18 +08:00
alex.lyn
0f0b6d13c9 runtime-rs: do register/update device in Trait Device/attach
Before calling the device driver to attach a device, register
the device to PCIe topology and allocate a PciPath for it.

However, for some hypervisor such as CLH, the allocation is invalid
when plugging devices to VM, they have the ability to return
DeviceInfo containing PciPath. It'll update the PciPath with the
returned pci path in the PCIe topology for them to prevent the
inferred pcipath from being different from the actual value returned.

But the update will not be executed if the pcipath value doesn't change.

Fixes: #7218

Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
2023-12-27 15:49:18 +08:00
alex.lyn
ce7d363695 runtime-rs: Introduce helper macros to simplify PCIe device ops
Introduce helper macros to simplify PCIe device register/unregister
and update, which provides a convenient way to handle devices in
topology.

Fixes: #7218

Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
2023-12-27 15:43:58 +08:00
alex.lyn
0d4992b24d runtime-rs: add one more argument in Device attach/detach
Add one more argument with type &mut Option<&mut PCIeTopology>
in attach and detach to inroduce methods within PCIe Topology.

Fixes: #7218

Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
2023-12-27 15:40:01 +08:00
alex.lyn
b425de6105 runtime-rs: implement Trait PCIeDevice for pcie/pci device
Implement Trait PCIeDevice register/unregister for pcie/pci
device, such as vfio device which needs set/get device's pci
path for kata agent's device handler.

Fixes: #7218

Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
2023-12-27 15:33:08 +08:00
alex.lyn
87e39cd1f6 runtime-rs: introduce Trait PCIeDevice to do [un]register device
Introduce Trait PCIeDevice with register/unregister, which are
used to register or unregister pcie device within the PCIe topology.

Fixes: #7218

Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
2023-12-27 15:29:35 +08:00
alex.lyn
6ebc4884fa runtime-rs: introduce PCIe Topology framework for pcie/pci devices
Due to different ways that different VMMs handle PCI devices,
we expect to provide a general PCIe topology processing framework
that is as compatible as possible with VMMs such as dragonball,
qemu, clh(Though it has its own management method, no conflict).

Currently,it's mainly developed for kinds of PCIe/PCI devices in
dragonball/clh which are attached on the pci/pcie root bus directly.

More will be added when Qemu is ready in runtime-rs.

Fixes: #7218

Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
2023-12-27 15:29:25 +08:00
alex.lyn
88839026b9 runtime-rs: introduce TopologyConfigInfo to initialize pcie topology
A TopologyConfigInfo added to store device config info for PCIe/PCI
devices in the VM from Hypervisor DeviceInfo.

And TopologyConfigInfo::new will be the entry to initialize PCIe
Topology for each VM.

Fixes: #7218

Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
2023-12-27 15:21:53 +08:00
Chao Wu
8895cb82df Merge pull request #8724 from openanolis/chao/add_vfio
dragonball: introduce vfio support
2023-12-27 11:40:53 +08:00
Xuewei Niu
43a627c96f Merge pull request #8632 from adamqqqplay/support-vhost-user-blk
dragonball: introduce vhost-user-blk device
2023-12-27 09:54:21 +08:00
Chao Wu
2f797a6eb7 pci: rename 2 parameters to follow rust naming convention
PciCapabilityID -> PciCapabilityId
PciBarRegionType::IORegion -> PciBarRegionType::IoRegion

Signed-off-by: Chao Wu <chaowu@linux.alibaba.com>
2023-12-26 23:28:47 +08:00
Chao Wu
9c13b2c990 dragonball: introduce vfio support
vfio mod collects lots of information related to the vfio operations, including VfioMsi and VfioMsix capability & state,
vfio interrupt info, pci region infor and vfio pci device info & state.

fixes: #8722

Signed-off-by: Gerry Liu <gerry@linux.alibaba.com>
Signed-off-by: Zizheng Bian <zizheng.bian@linux.alibaba.com>
Signed-off-by: Shifang Feng <fengshifang@linux.alibaba.com>
Signed-off-by: Yang Su <yang.su@linux.alibaba.com>
Signed-off-by: Zha Bin <zhabin@linux.alibaba.com>
Signed-off-by: Xin Lin <jingshan@linux.alibaba.com>
Signed-off-by: Chao Wu <chaowu@linux.alibaba.com>
2023-12-26 23:28:43 +08:00
alex.lyn
ba5437382a runtime-rs: add examples about Kata pod with directvol by CSI.
Fixes: #8602

Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
2023-12-26 20:36:34 +08:00
alex.lyn
c6d2a32146 runtime-rs: add support for directvol csi deploy scripts.
Fixes: #8602

Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
2023-12-26 20:36:34 +08:00
alex.lyn
25d8e83e43 runtime-rs: Add dedicated CSI driver for DirectVolume support in Kata
Bridge the gap between user requirements for direct block device access
and the DirectVolume capabilities provided by Kata runtimes
(kata-runtime/runtime-rs), and facilitate seamless integration with CSI
to improve user experience.

It aims to integrate DirectVolume CSI support into Kata, enabling users
to benefit from its performance and flexibility advantages.

Fixes: #8602

Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
2023-12-26 20:36:22 +08:00
Qinqi Qu
81ab174c16 dragonball: support vhost-user-blk in device manager
This patch introduces a feature of supporting vhost-user-blk device.

Fixes: #8631

Signed-off-by: Qinqi Qu <quqinqi@linux.alibaba.com>
2023-12-26 20:02:38 +08:00
Qinqi Qu
ef8dc3b0ce dragonball: support vhost-user-blk
This patch introduces a feature of supporting vhost-user-blk device.

This device needs to be defined before the VM instance is started,
which can be done through the dbs-cli tool with --virblks option:
--virblks '{
	"drive_id": "8623",
	"device_type": "Spdk",
	"path_on_host": "spdk:///var/tmp/vhost.sock",
	"is_root_device": false,
	"is_read_only": false,
	"is_direct": false,
	"no_drop": false,
	"num_queues": 1,
	"queue_size": 256
}'

Fixes: #8631

Signed-off-by: Eric Ren <renzhen@linux.alibaba.com>
Signed-off-by: fupan <fupan.lfp@antgroup.com>
Signed-off-by: Liu Jiang <gerry@linux.alibaba.com>
Signed-off-by: Qinqi Qu <quqinqi@linux.alibaba.com>
2023-12-26 20:02:32 +08:00
alex.lyn
3b317e69e2 runtime-rs: add README and user guide to deploy directvol CSI Driver
Fixes: #8602

Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
2023-12-26 18:00:35 +08:00
Xuewei Niu
36a4cbccf6 runtime-rs: Expand all DeviceType in match arms
The compiler will give a warning if a developer forget to add an arm for
a new variants defined.

Signed-off-by: Xuewei Niu <niuxuewei.nxw@antgroup.com>
2023-12-26 10:18:59 +08:00
Xuewei Niu
f2d08bc00f runtime-rs: Remove unused index from Endpoints
The affected `Endpoint`s are `VhostUserEndpoint` and `TapEndpoint`.

Signed-off-by: Xuewei Niu <niuxuewei.nxw@antgroup.com>
2023-12-26 10:18:59 +08:00