Commit Graph

445 Commits

Author SHA1 Message Date
Eric Ernst
05680b86c4 Merge pull request #1537 from lifupan/main
cgroups: fix the issue of get wrong online cpus
2021-03-29 15:56:03 -07:00
Eric Ernst
460117a1a6 Merge pull request #1510 from littlejawa/issue_1003
build: remove unused variables from Makefile
2021-03-29 14:54:09 -07:00
James O. D. Hunt
1d448813a1 uevent: Add shutdown channel for task
Allow the uevent task to shutdown on request.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2021-03-29 14:32:12 +01:00
James O. D. Hunt
d8d5b4cd1d signal: Move to a new module
Move the signal handling code into a new module and refactor into the
main handler and a new SIGCHLD handling function to make the code
simpler and easier to understand.

Also added a unit test for shutdown.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2021-03-29 14:32:12 +01:00
James O. D. Hunt
011f7d785a logging: Rework for shutdown
Make changes to logger thread to allow the logger to be replaced with
a NOP logger (required for agent shutdown).

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2021-03-29 14:32:12 +01:00
James O. D. Hunt
7d5f88c0ad agent: Enable clean shutdown
The agent doesn't normally shutdown: it doesn't need to be as it is
killed *after* the workload has finished. However, a clean and ordered
shutdown sequence is required to support agent tracing, since all trace
spans need to be completed to ensure a valid trace transaction.

Enable a controlled shutdown by allowing the main threads (tasks) to be
stopped.

To allow this to happen, each thread is now passed a shutdown channel
which it must listen to asynchronously, and shut down the thread if
activity is detected on that channel.

Since some threads are created for I/O and since the standard `io::copy`
cannot be stopped, added a new `interruptable_io_copier()` function
which shares the same semantics as `io::copy()`, but which is also
passed a shutdown channel to allow asynchronous I/O operations to be
stopped cleanly.

Fixes: #1531.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2021-03-29 14:32:12 +01:00
James O. D. Hunt
dcb39c61f1 main: Create logger task
Encapsulate the logic for handling the task that displays logger output
into a new function to simplify the code and remove another anonymous
async block.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2021-03-29 14:32:11 +01:00
James O. D. Hunt
2cf2897d31 main: Use task list for stopping tasks
Maintain a list of tasks and wait on them all before main returns.

This is preparatory work for the agent shutdown: all tasks that are
started need to be added to the list. This aggregation makes it easier
to identify what needs to stop before the agent can exit cleanly.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2021-03-29 14:32:11 +01:00
James O. D. Hunt
039df1d727 main: Refactor main logic into new async function
Move most of the main logic into a separate async function. This makes
the code clearer and avoids the anonymous async block.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2021-03-29 14:32:11 +01:00
James O. D. Hunt
2a648fa760 logging: Use guard to make threaded logging safe
Return a guard variable from `create_logger()` which the caller can
implicitly drop to guarantee that all threads started by the async log
drain are stopped.

This fixes a long-standing bug [1] whereby the agent could panic with
the following error, generated by the `slog` logging crate:

```
slog::Fuse Drain: Custom { kind: Other, error: "serde serialization error: Bad file descriptor (os error 9)" }
```

[1] - See https://github.com/kata-containers/kata-containers/issues/171.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2021-03-29 14:32:11 +01:00
James O. D. Hunt
38f0d8d3ce config: Fix assert_error testing macro
Fixed the `assert_error!()` test macro so that it correctly handles the
scenario where the test expects an error, but the actual result was `Ok`
(no error).

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2021-03-29 14:32:11 +01:00
fupan.lfp
3f46e6379d cgroups: fix the issue of getting wrong online cpus
It's better to get the online cpus from
"/sys/devices/system/cpu/online" instead of from
cpuset cgroup, cause there would be an latency
between one cpu online and present in the root
cpuset cgroup.

Fixes: #1536

Signed-off-by: fupan.lfp <fupan.lfp@antfin.com>
2021-03-29 15:49:15 +08:00
bin
532ff7c909 runtime: update virtcontainers API documentation
Virtcontainers API documentation is outdated, update documentation from the latest
source.

Fixes: #1455

Signed-off-by: bin <bin@hyper.sh>
2021-03-29 11:50:53 +08:00
Bin Liu
5b5b5cc611 Merge pull request #1539 from bergwolf/ut
fix runtime UTs and enable static check
2021-03-25 16:29:45 +08:00
James O. D. Hunt
2fc7f75724 Merge pull request #1521 from jodh-intel/verify-cid
Verify container ID
2021-03-24 13:27:58 +00:00
Peng Tao
fc0f93aef9 actions: enable unit tests in PR check
Right now we only run UTs for agent. We need to run it for *ALL*
components.

Fixes: #1538
Signed-off-by: Peng Tao <bergwolf@hyper.sh>
2021-03-24 20:12:14 +08:00
Bin Liu
018454be44 Merge pull request #1534 from Tim-Zhang/rework-execute_hook
rustjail: rework execute_hook
2021-03-24 14:09:09 +08:00
Tim Zhang
40861fbab5 Merge pull request #1517 from jodh-intel/agent-server-address-cmdline
agent: Allow server address to be specified on kernel command-line
2021-03-23 19:33:25 +08:00
Tim Zhang
0e4b28e838 rustjail: rework execute_hook
Fixes: #1532

Signed-off-by: Tim Zhang <tim@hyper.sh>
2021-03-22 20:20:30 +08:00
Bin Liu
dd6da22a11 Merge pull request #1525 from Tim-Zhang/disconnect-get-oom
agent: disconnect rpc get_oom_event when destroy_sandbox.
2021-03-22 15:19:31 +08:00
David Gibson
89e5fa7a2f Merge pull request #1506 from dgibson/bug1505
agent: Update Cargo.lock for earlier dependency change
2021-03-20 11:34:32 +11:00
James O. D. Hunt
451b45f9d7 agent: Make use of test consts for error messages
Make use of the `const` values for error messages that were previously
only used for the unit tests. This guarantees consistency.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2021-03-19 09:17:31 +00:00
Tim Zhang
8c4d3346d4 agent: disconnect rpc get_oom_event when destroy_sandbox.
Otherwise it would block the shutdown of ttrpc.

Fixes: #1524

Signed-off-by: Tim Zhang <tim@hyper.sh>
2021-03-19 16:42:37 +08:00
James O. D. Hunt
34e7d5ed97 agent: Validate CID
Validate the container ID as we cannot / should not rely on the
container manager / runtime to do this.

Fixes: #1520.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2021-03-18 11:19:34 +00:00
James O. D. Hunt
ea51c17b64 agent: Allow server address to be specified on kernel command-line
To make debugging and testing easier, allow the ttRPC server address to
be specified via `/proc/cmdline` as `agent.server_addr=`.

Fixes: #1516.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2021-03-17 08:22:03 +00:00
Julien Ropé
3a77e4eb8c build: remove unused variables from Makefile
Some variable are initialized in the Makefile, but never used.
Removing them to clean up the Makefile.

Fixes: #1003

Signed-off-by: Julien Ropé <jrope@redhat.com>
2021-03-16 08:51:16 +01:00
David Gibson
b0e966c3bd agent: Fix unused import warning in unit tests
This unneeded import was accidentally introduced by 81607e34.

fixes #1507

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2021-03-12 17:25:13 +11:00
David Gibson
d5a9d56e79 agent: Update Cargo.lock for earlier dependency change
Commit 81607e34 updated src/agent/rustjail/Cargo.toml, to remove an
unneeded dependency.  That causes cargo to update src/agent/Cargo.lock
on each build.  However, the change to Cargo.lock wasn't checked in
meaning anyone working on the agent code will get bogus diffs with every
build.  Check in the missing file to fix this.

fixes #1505

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2021-03-12 16:51:30 +11:00
Bo Chen
396565fa56 Merge pull request #1489 from jcvenegas/2021-03-03/codecov
agent: makefile: Add codecov target
2021-03-10 11:21:33 -08:00
fupan.lfp
81607e348e rustjail: fix the issue of home_dir function
Since the crate dirs::home_dir function depends on the
libc's api: getpwuid_r, but this api function wouldn't
be static linked on glibc, thus we'd better to figure
out an alternative way to get the home dir from /etc/passwd.
For much more info about this glibc's issue, please see:
https://sourceware.org/bugzilla/show_bug.cgi?id=19341.

This commit read and parse the "/etc/passwd" directly and
fetch the corresponding uid's home dir.

Fixes: #675

Signed-off-by: fupan.lfp <fupan.lfp@antfin.com>
2021-03-08 21:51:23 +08:00
Fupan Li
f6630ddd49 Merge pull request #1478 from lifupan/fix_device
rustjail: fix the issue of bind mount device file from guest
2021-03-08 09:55:00 +08:00
Carlos Venegas
7ae349c511 agent: makefile: Add codecov target
Add target to run codecov report locally.

Useful to identify what are the missing lines
to be covered by unit test.

Fixes: #1487

Signed-off-by: Carlos Venegas <jos.c.venegas.munoz@intel.com>
2021-03-04 16:32:52 +00:00
Eric Ernst
90a18e228b Merge pull request #1457 from mxpv/shared
agent: don't error of virtiofs share is already mounted
2021-03-01 11:16:18 -08:00
fupan.lfp
34dc861cde rustjail: fix the issue of bind mount device file from guest
When do pass guest device files to container, the source
file wouldn't be a regular file, but we also need to create
a corresponding destination file to bind mount source file
to it. Thus it's better to check whether the source file
was a directory instead of regular file.

Fixes: #1477

Signed-off-by: fupan.lfp <fupan.lfp@antfin.com>
2021-03-01 21:20:01 +08:00
Bin Liu
7587d2a8d6 Merge pull request #1462 from Tim-Zhang/fix-clippy-for-rust1.5
agent: fix clippy for rustc 1.5
2021-02-26 15:52:03 +08:00
Fupan Li
b5282fa224 Merge pull request #1305 from Tim-Zhang/upgrade-tokio-to-1.0
agent: Upgrade tokio to 1.2.0
2021-02-26 13:33:24 +08:00
Tim Zhang
6f720761ed agent: fix clippy for rustc 1.5
Fixes: #1461

Signed-off-by: Tim Zhang <tim@hyper.sh>
2021-02-25 17:04:54 +08:00
Bin Liu
735fe3f94a Merge pull request #1444 from ManaSugi/fix-blkio-weight
rustjail: fix blkio conversion
2021-02-25 15:20:20 +08:00
Tim Zhang
4a214720e8 agent: Fix test
Struct TtrpcContext has been changed in ttrpc@0.5

Signed-off-by: Tim Zhang <tim@hyper.sh>
2021-02-25 14:38:38 +08:00
Tim Zhang
02079dbb4f agent: upgrade tokio to 1.0
Fixes: #1257

Signed-off-by: Tim Zhang <tim@hyper.sh>
2021-02-25 14:38:32 +08:00
Maksym Pavlenko
a42dc74898 agent: Agent invokes OCI hooks with wrong PID
Agent sends -1 PID when invoking OCI hooks.

OCI state struct is initialized before obtaining PID, so this PR moves
`oci_state` call down, right after we get the id.

Fixes: #1458

Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
2021-02-24 18:16:17 -08:00
Maksym Pavlenko
17e9a2cff5 agent: don't error of virtiofs share is already mounted
Port kata-containers/agent#883 to the Rust Agent.

In the event that the virtiofs device is already mounted at the
requested destination, don't error out. We'll check before attempting to
mount to see if the destination is already a mount point. If so, skip
doing the mount in the agent.

This facilitates mounting the sharedfs automatically in the guest before
the agent service starts.

Signed-off-by: Eric Ernst eric.g.ernst@gmail.com

Fixes: #1398

Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
2021-02-24 17:05:59 -08:00
Tim Zhang
947913f6e5 agent/protocols: Remove cargo:rerun-if-changed in build.rs
So that the build.rs will be re-runed if any file
within the package is changed.

Signed-off-by: Tim Zhang <tim@hyper.sh>
2021-02-24 21:50:47 +08:00
Manabu Sugimoto
dcea08697a rustjail: fix blkio conversion
BFQ weight controller is using the same BFQ weight scheme (i.e 1->1000).
Therefore, there is no need to do the conversion.

More details here: https://github.com/opencontainers/runc/pull/2786

Fixes: #1440

Signed-off-by: Manabu Sugimoto <Manabu.Sugimoto@sony.com>
2021-02-23 00:26:57 +09:00
bin
bc34cbbce5 agent: Stop receive message from Receiver if got None
If the container has exited, the sender in notifier watching OOM events
will be dropped after the loop exited, and recv() from the according
receiver will get None.

This will lead two problems for get_oom_event rpc all from agent:

- return an wrong OOM event.
- continuously return OOM events.

Fixes: #1369

Signed-off-by: bin <bin@hyper.sh>
2021-02-22 21:56:07 +08:00
Bin Liu
29d4abf23a Merge pull request #1437 from jcvenegas/2021-02-18/self-documented-makefile
makefile: agent: Add self documented help
2021-02-20 10:04:30 +08:00
Carlos Venegas
a494c4de23 makefile: agent: Add self documented help
Add comments that allow self document variables and targets

Fixes: #1436

Signed-off-by: Carlos Venegas <jos.c.venegas.munoz@intel.com>
2021-02-19 15:30:42 +00:00
David Gibson
87c5823c4b agent/device: Add unit test for pcipath_to_sysfs()
Port this test from the Kata 1 Go agent to the Kata 2 Rust agent.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2021-02-19 09:56:08 +11:00
David Gibson
066ce7ab51 agent/device: Pass root bus sysfs path to pcipath_to_sysfs()
Currently pcipath_to_sysfs() generates the path to the root bus node in
sysfs via create_pci_root_bus_path().  This is inconvenient for testing,
though, so instead make it take this as a parameter and generate the path
in the (single) caller.  As a bonus this will make life a bit easier when
we want to support machines with multiple PCI roots.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2021-02-19 09:56:08 +11:00
David Gibson
fda48a9bf0 agent/device: Use pci::Path type, name things consistently
pcipath_to_sysfs takes a PCI path, with a particular format.  A number of
places implicitly need strings in that format, many of them repeat the
description.  To make things safer and briefer use the pci::Path type for
the purpose more widely, and just describe the string formatting of it at
the type definition.

Then, update variable names and comments throughout to call things in
this format "PCI path", rather than "PCI identifier", which is vague,
or "PCI address" which is just plain wrong.  Likewise we change names and
comments which incorrectly refer to sysfs paths as a "PCI address".

This changes the grpc proto definitions, but because it's just
changing the name of a field without changing the field number, it
shouldn't change the actual protocol.

A loose forward port of
da4bc1d184

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2021-02-19 09:56:08 +11:00