Commit Graph

455 Commits

Author SHA1 Message Date
GabyCT
aac852a0bc Merge pull request #1561 from Jakob-Naucke/s390x-statfs-constants
agent: s390x statfs constants
2021-04-06 11:11:40 -05:00
Bin Liu
117c59150d Merge pull request #1613 from Tim-Zhang/pipestream-shutdown-do-nothing
Don't do anything in Pipestream::shutdown
2021-04-06 14:03:00 +08:00
Tim Zhang
ee6a590db1 agent: add test test_pipestream_shutdown
Make sure PipeStream::shutdown() do not close the inner fd.

Signed-off-by: Tim Zhang <tim@hyper.sh>
2021-04-06 11:44:56 +08:00
Tim Zhang
4a2d437043 agent: don't do anything in Pipestream::shutdown
The only right way to shutdown pipe is drop it
Otherwise PipeStream will conflict with its twins
Because they both have the same fd, and both registered.

Fixes: #1614

Signed-off-by: Tim Zhang <tim@hyper.sh>
2021-04-06 11:44:38 +08:00
Peng Tao
d5600641dd Merge pull request #1603 from lifupan/fix_fsgroup
Fix fsgroup
2021-04-06 11:35:03 +08:00
David Gibson
ed08980fc1 agent: Remove many "panic message is not string literal" warnings
Rust 1.51 appears to have added a new warning in anticipation of Rust 2021,
which requires the format string for panic!()s (including via the various
assert!() macros) to be a string literal.  This triggers quite a few times
in the agent code.  This patch fixes them.

fixes #1626

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2021-04-06 11:51:34 +10:00
fupan.lfp
6493942568 mount: fix the issue of missing set fsGroup
For k8s emptyDir volume, a specific fsGroup would
be set for it, thus guest should get this fsGroup
from runtime and set it properly on the emptyDir volume
in guest.

Fixes: #1580

Signed-off-by: fupan.lfp <fupan.lfp@antfin.com>
2021-04-01 11:33:26 +08:00
Fabiano Fidêncio
572aff53e8 build: Only keep one VERSION file
Instead of having different VERSION files spread accross the project,
let's always use the one in the topsrcdir and remove all the others,
keeping only a synlink to the topsrcdir one.

Fixes: #1579

Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
2021-03-31 23:51:20 +02:00
Eric Ernst
05680b86c4 Merge pull request #1537 from lifupan/main
cgroups: fix the issue of get wrong online cpus
2021-03-29 15:56:03 -07:00
Eric Ernst
460117a1a6 Merge pull request #1510 from littlejawa/issue_1003
build: remove unused variables from Makefile
2021-03-29 14:54:09 -07:00
Jakob Naucke
52a276fbdb agent: Fix type for PROC_SUPER_MAGIC on s390x
statfs f_types are long on most architectures, but not on s390x, where
they are uint. Following the fix in rust-lang/libc at
https://github.com/rust-lang/libc/pull/1999, the custom defined
PROC_SUPER_MAGIC must be updated in a similar way.

Fixes: #1204

Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
2021-03-29 17:25:19 +02:00
Jakob Naucke
5b7c8b7d26 agent: Update cgroups-rs to 0.2.5
to pull in the chain of https://github.com/rust-lang/libc/pull/1999,
https://github.com/nix-rust/nix/pull/1372, and
https://github.com/kata-containers/cgroups-rs/pull/38. This adds statfs
constants on s390x. cgroups-rs 0.2.4 also contains this fix, but let's
move to the latest 0.2.5 right away.

Fixes: #1204

Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
2021-03-29 17:25:14 +02:00
James O. D. Hunt
1d448813a1 uevent: Add shutdown channel for task
Allow the uevent task to shutdown on request.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2021-03-29 14:32:12 +01:00
James O. D. Hunt
d8d5b4cd1d signal: Move to a new module
Move the signal handling code into a new module and refactor into the
main handler and a new SIGCHLD handling function to make the code
simpler and easier to understand.

Also added a unit test for shutdown.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2021-03-29 14:32:12 +01:00
James O. D. Hunt
011f7d785a logging: Rework for shutdown
Make changes to logger thread to allow the logger to be replaced with
a NOP logger (required for agent shutdown).

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2021-03-29 14:32:12 +01:00
James O. D. Hunt
7d5f88c0ad agent: Enable clean shutdown
The agent doesn't normally shutdown: it doesn't need to be as it is
killed *after* the workload has finished. However, a clean and ordered
shutdown sequence is required to support agent tracing, since all trace
spans need to be completed to ensure a valid trace transaction.

Enable a controlled shutdown by allowing the main threads (tasks) to be
stopped.

To allow this to happen, each thread is now passed a shutdown channel
which it must listen to asynchronously, and shut down the thread if
activity is detected on that channel.

Since some threads are created for I/O and since the standard `io::copy`
cannot be stopped, added a new `interruptable_io_copier()` function
which shares the same semantics as `io::copy()`, but which is also
passed a shutdown channel to allow asynchronous I/O operations to be
stopped cleanly.

Fixes: #1531.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2021-03-29 14:32:12 +01:00
James O. D. Hunt
dcb39c61f1 main: Create logger task
Encapsulate the logic for handling the task that displays logger output
into a new function to simplify the code and remove another anonymous
async block.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2021-03-29 14:32:11 +01:00
James O. D. Hunt
2cf2897d31 main: Use task list for stopping tasks
Maintain a list of tasks and wait on them all before main returns.

This is preparatory work for the agent shutdown: all tasks that are
started need to be added to the list. This aggregation makes it easier
to identify what needs to stop before the agent can exit cleanly.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2021-03-29 14:32:11 +01:00
James O. D. Hunt
039df1d727 main: Refactor main logic into new async function
Move most of the main logic into a separate async function. This makes
the code clearer and avoids the anonymous async block.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2021-03-29 14:32:11 +01:00
James O. D. Hunt
2a648fa760 logging: Use guard to make threaded logging safe
Return a guard variable from `create_logger()` which the caller can
implicitly drop to guarantee that all threads started by the async log
drain are stopped.

This fixes a long-standing bug [1] whereby the agent could panic with
the following error, generated by the `slog` logging crate:

```
slog::Fuse Drain: Custom { kind: Other, error: "serde serialization error: Bad file descriptor (os error 9)" }
```

[1] - See https://github.com/kata-containers/kata-containers/issues/171.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2021-03-29 14:32:11 +01:00
James O. D. Hunt
38f0d8d3ce config: Fix assert_error testing macro
Fixed the `assert_error!()` test macro so that it correctly handles the
scenario where the test expects an error, but the actual result was `Ok`
(no error).

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2021-03-29 14:32:11 +01:00
fupan.lfp
3f46e6379d cgroups: fix the issue of getting wrong online cpus
It's better to get the online cpus from
"/sys/devices/system/cpu/online" instead of from
cpuset cgroup, cause there would be an latency
between one cpu online and present in the root
cpuset cgroup.

Fixes: #1536

Signed-off-by: fupan.lfp <fupan.lfp@antfin.com>
2021-03-29 15:49:15 +08:00
bin
532ff7c909 runtime: update virtcontainers API documentation
Virtcontainers API documentation is outdated, update documentation from the latest
source.

Fixes: #1455

Signed-off-by: bin <bin@hyper.sh>
2021-03-29 11:50:53 +08:00
Bin Liu
5b5b5cc611 Merge pull request #1539 from bergwolf/ut
fix runtime UTs and enable static check
2021-03-25 16:29:45 +08:00
James O. D. Hunt
2fc7f75724 Merge pull request #1521 from jodh-intel/verify-cid
Verify container ID
2021-03-24 13:27:58 +00:00
Peng Tao
fc0f93aef9 actions: enable unit tests in PR check
Right now we only run UTs for agent. We need to run it for *ALL*
components.

Fixes: #1538
Signed-off-by: Peng Tao <bergwolf@hyper.sh>
2021-03-24 20:12:14 +08:00
Bin Liu
018454be44 Merge pull request #1534 from Tim-Zhang/rework-execute_hook
rustjail: rework execute_hook
2021-03-24 14:09:09 +08:00
Tim Zhang
40861fbab5 Merge pull request #1517 from jodh-intel/agent-server-address-cmdline
agent: Allow server address to be specified on kernel command-line
2021-03-23 19:33:25 +08:00
Tim Zhang
0e4b28e838 rustjail: rework execute_hook
Fixes: #1532

Signed-off-by: Tim Zhang <tim@hyper.sh>
2021-03-22 20:20:30 +08:00
Bin Liu
dd6da22a11 Merge pull request #1525 from Tim-Zhang/disconnect-get-oom
agent: disconnect rpc get_oom_event when destroy_sandbox.
2021-03-22 15:19:31 +08:00
David Gibson
89e5fa7a2f Merge pull request #1506 from dgibson/bug1505
agent: Update Cargo.lock for earlier dependency change
2021-03-20 11:34:32 +11:00
James O. D. Hunt
451b45f9d7 agent: Make use of test consts for error messages
Make use of the `const` values for error messages that were previously
only used for the unit tests. This guarantees consistency.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2021-03-19 09:17:31 +00:00
Tim Zhang
8c4d3346d4 agent: disconnect rpc get_oom_event when destroy_sandbox.
Otherwise it would block the shutdown of ttrpc.

Fixes: #1524

Signed-off-by: Tim Zhang <tim@hyper.sh>
2021-03-19 16:42:37 +08:00
James O. D. Hunt
34e7d5ed97 agent: Validate CID
Validate the container ID as we cannot / should not rely on the
container manager / runtime to do this.

Fixes: #1520.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2021-03-18 11:19:34 +00:00
James O. D. Hunt
ea51c17b64 agent: Allow server address to be specified on kernel command-line
To make debugging and testing easier, allow the ttRPC server address to
be specified via `/proc/cmdline` as `agent.server_addr=`.

Fixes: #1516.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2021-03-17 08:22:03 +00:00
Julien Ropé
3a77e4eb8c build: remove unused variables from Makefile
Some variable are initialized in the Makefile, but never used.
Removing them to clean up the Makefile.

Fixes: #1003

Signed-off-by: Julien Ropé <jrope@redhat.com>
2021-03-16 08:51:16 +01:00
David Gibson
b0e966c3bd agent: Fix unused import warning in unit tests
This unneeded import was accidentally introduced by 81607e34.

fixes #1507

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2021-03-12 17:25:13 +11:00
David Gibson
d5a9d56e79 agent: Update Cargo.lock for earlier dependency change
Commit 81607e34 updated src/agent/rustjail/Cargo.toml, to remove an
unneeded dependency.  That causes cargo to update src/agent/Cargo.lock
on each build.  However, the change to Cargo.lock wasn't checked in
meaning anyone working on the agent code will get bogus diffs with every
build.  Check in the missing file to fix this.

fixes #1505

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2021-03-12 16:51:30 +11:00
Bo Chen
396565fa56 Merge pull request #1489 from jcvenegas/2021-03-03/codecov
agent: makefile: Add codecov target
2021-03-10 11:21:33 -08:00
fupan.lfp
81607e348e rustjail: fix the issue of home_dir function
Since the crate dirs::home_dir function depends on the
libc's api: getpwuid_r, but this api function wouldn't
be static linked on glibc, thus we'd better to figure
out an alternative way to get the home dir from /etc/passwd.
For much more info about this glibc's issue, please see:
https://sourceware.org/bugzilla/show_bug.cgi?id=19341.

This commit read and parse the "/etc/passwd" directly and
fetch the corresponding uid's home dir.

Fixes: #675

Signed-off-by: fupan.lfp <fupan.lfp@antfin.com>
2021-03-08 21:51:23 +08:00
Fupan Li
f6630ddd49 Merge pull request #1478 from lifupan/fix_device
rustjail: fix the issue of bind mount device file from guest
2021-03-08 09:55:00 +08:00
Carlos Venegas
7ae349c511 agent: makefile: Add codecov target
Add target to run codecov report locally.

Useful to identify what are the missing lines
to be covered by unit test.

Fixes: #1487

Signed-off-by: Carlos Venegas <jos.c.venegas.munoz@intel.com>
2021-03-04 16:32:52 +00:00
Eric Ernst
90a18e228b Merge pull request #1457 from mxpv/shared
agent: don't error of virtiofs share is already mounted
2021-03-01 11:16:18 -08:00
fupan.lfp
34dc861cde rustjail: fix the issue of bind mount device file from guest
When do pass guest device files to container, the source
file wouldn't be a regular file, but we also need to create
a corresponding destination file to bind mount source file
to it. Thus it's better to check whether the source file
was a directory instead of regular file.

Fixes: #1477

Signed-off-by: fupan.lfp <fupan.lfp@antfin.com>
2021-03-01 21:20:01 +08:00
Bin Liu
7587d2a8d6 Merge pull request #1462 from Tim-Zhang/fix-clippy-for-rust1.5
agent: fix clippy for rustc 1.5
2021-02-26 15:52:03 +08:00
Fupan Li
b5282fa224 Merge pull request #1305 from Tim-Zhang/upgrade-tokio-to-1.0
agent: Upgrade tokio to 1.2.0
2021-02-26 13:33:24 +08:00
Tim Zhang
6f720761ed agent: fix clippy for rustc 1.5
Fixes: #1461

Signed-off-by: Tim Zhang <tim@hyper.sh>
2021-02-25 17:04:54 +08:00
Bin Liu
735fe3f94a Merge pull request #1444 from ManaSugi/fix-blkio-weight
rustjail: fix blkio conversion
2021-02-25 15:20:20 +08:00
Tim Zhang
4a214720e8 agent: Fix test
Struct TtrpcContext has been changed in ttrpc@0.5

Signed-off-by: Tim Zhang <tim@hyper.sh>
2021-02-25 14:38:38 +08:00
Tim Zhang
02079dbb4f agent: upgrade tokio to 1.0
Fixes: #1257

Signed-off-by: Tim Zhang <tim@hyper.sh>
2021-02-25 14:38:32 +08:00