Commit Graph

66 Commits

Author SHA1 Message Date
Wei Zhang
2ed94cbd9d Config: Remove ConfigJSONKey from annotations
Fixes: #2023

We can get OCI spec config from bundle instead of annotations, so this
field isn't necessary.

Signed-off-by: Wei Zhang <weizhang555.zw@gmail.com>
2019-09-17 11:47:06 +08:00
Wei Zhang
9507f45a0f CompatOCISpec: limit usage of CompatOCISpec
Fixes: #2023

CompatOCISpec is used to gurantee backward compatbility for old runtime
specs, after we convert CompatOCISpec to standard specs.Spec, we should
use specs.Spec instead of CompatOCISpec, and CompatOCISpec should be
useless from then.

Spread usage of CompatOCISpec can make code structure confusing and making
the runtime spec usage non-standard. Besides, this can be the very first
step of removing CompatOCISpec from config's Annotations field.

Signed-off-by: Wei Zhang <weizhang555.zw@gmail.com>
2019-09-05 11:05:55 +08:00
Marco Vedovati
ff8d23d75f tests: allow running unit tests using podman
Refactor unit test so that podman can be used to run tests when docker
is not available.

Fixes: #2006

Signed-off-by: Marco Vedovati <mvedovati@suse.com>
2019-08-29 17:06:50 +02:00
Eric Ernst
e7c785ed19 Merge pull request #2002 from darfux/prevent_killing_cntr_with_failed_exec
v2: Prevent killing all container processes when exec is failed
2019-08-25 13:59:54 +08:00
Peng Tao
3fb872911e Merge pull request #1833 from lifupan/fix_cleanuprace
api: add a CleanupContainer api for VC
2019-08-24 14:34:10 +08:00
lifupan
c91556aa41 api: add a CleanupContainer api for VC
When shimv2 was killed by accident, containerd would try to
launch a new shimv2 binarry to cleanup the container. In order
to avoid race condition, the cleanup should be done serialized
in a sandbox. Thus adding a new api to do this by locking the
sandbox.

Fixes:#1832

Signed-off-by: lifupan <lifupan@gmail.com>
2019-08-24 08:16:02 +08:00
Li Yuxuan
4cf0703c58 v2: Prevent killing all container processes when exec is failed
If an exec is failed(such as executable file not found in $PATH), the
`execs.id` will be empty. This leads to all the container processes
being killed when calling `Kill` on such exec id.

Fixes: #2001
Signed-off-by: Li Yuxuan <liyuxuan04@baidu.com>
2019-08-24 00:25:55 +08:00
Ace-Tang
4a28b52553 test: add test for network metric
add test for metric interface

Signed-off-by: Ace-Tang <aceapril@126.com>
2019-08-19 19:42:54 +08:00
Ace-Tang
6534357925 shim-v2: add network stat in metric
improve metric message, add network stat, base on agent PR: #538 and
containerd/cgroup PR #81

Fixes: #1976

Signed-off-by: ZeroMagic <anthonyliu@zju.edu.cn>
Signed-off-by: Ace-Tang <aceapril@126.com>
2019-08-19 18:15:06 +08:00
Peng Tao
5b50b34df4 shimv2: cancel monitor before stopping sandbox
So that we don't trigger sandbox watcher on our own.

Signed-off-by: Peng Tao <bergwolf@hyper.sh>
2019-08-16 12:58:25 +00:00
Ace-Tang
20587519cd shim-v2: fix shim leak when hypervisor exit unexpectly
Fixes: #1929

in containerd-kata-v2, container can only be deleted in Delete
interface, or other shim operates(like kill/delete) all fails
since can not get container info.

Signed-off-by: Ace-Tang <aceapril@126.com>
2019-08-06 15:48:26 +08:00
Peng Tao
e02f6dc067 shimv2: monitor sandbox liveness
When sandbox quits unexpected, clean things up as much as we can.

Fixes: #1896
Signed-off-by: Peng Tao <bergwolf@hyper.sh>
2019-07-22 19:29:32 -07:00
Peng Tao
bc4460e12f sandbox: support force stop
When force is true, ignore any guest related errors. This can
be used to stop a sandbox when hypervisor process is dead.

Signed-off-by: Peng Tao <bergwolf@hyper.sh>
2019-07-22 19:29:32 -07:00
Odin Ugedal
da7f5f3796 shimv2: Add missing page size to Hugetlb Stat
Fixes: #1856

Signed-off-by: Odin Ugedal <odin@ugedal.com>
2019-07-03 21:07:04 +02:00
Eric Ernst
450402beb5 Merge pull request #1658 from xs3c/use-nonblock-for-stdin
Use O_NONBLOCK for tty stdin.
2019-06-12 08:07:56 -07:00
Eric Ernst
7acdaa2d53 shimv2: Revert shimv2, vendor commits
This reverts:
 - 590ed09 vendor: update gogo/protobuf, containerd and agent vendors
 - eabfd99 shimv2: Improve shim shutdown logic

These introduce a regression for starting pods with k8s 1.14 + contaienr
1.2.6

Fixes: #1781

Signed-off-by: Eric Ernst <eric.ernst@intel.com>
2019-06-10 09:45:23 -07:00
Yang, Wei
efc754f6b1 containerd-shim-kata-v2: Use O_NONBLOCK for tty stdin.
Fixes: #1659

our testing found that the client does not open
stdin with O_WRONLY, so the shim v2 was blocked forever
in open stdin with O_RDONLY. It's better to make it
opened with O_NONBLOCK, and do not block starting process
of container. and the containerd runc shim has done this
by bc1ff514 as well.

Signed-off-by: Yang, Wei <w90p710@gmail.com>
Signed-off-by: Yang, Wei <wei.yang1@linux.alibaba.com>
2019-06-03 14:26:43 +08:00
lifupan
eabfd99734 shimv2: Improve shim shutdown logic
Latest shimv2 publishes an event to containerd used ttrpc instead
of using containerd binary, thus shimv2 shouldn't call `os.Exit` to
terminate the shim's life,  but close the context on shutdown so that
events and other resources have hit the `defer`s.

Fixes:#1731

Signed-off-by: lifupan <lifupan@gmail.com>
2019-05-24 12:18:04 +08:00
Julio Montes
618ae4d03b Merge pull request #1722 from Ace-Tang/rm-ns
shimv2: remove use containerd ns as netns
2019-05-22 09:49:40 -05:00
lifupan
f301c957f6 shimv2: shutdown the sandbox when sandbox container exited
Kubelet would cleanup the pod cgroup resources and kill the processes
in the pod cgroups when it detected all of the containers in a pod exited,
thus shimv2 should close the hypervisor process once the podsandbox container
exited, otherwise, the hypervisor process would be killed by kubelet and
made shimv2 failed to shutdown the sandbox.

Fixes:#1672

Signed-off-by: lifupan <lifupan@gmail.com>
2019-05-22 06:32:27 +00:00
Ace-Tang
d6b3bffad8 shimv2: remove use containerd ns as netns
```
//the network namespace created by cni plugin
netns, err = namespaces.NamespaceRequired(ctx)
if err != nil {
        return nil, errors.Wrap(err, "create namespace")
}
```

the netns is a containerd namespace concept, it not netns, event a cni
set netns for this, this is a tricky way, so remove the logic.

Fixes: #1692

Signed-off-by: Ace-Tang <aceapril@126.com>
2019-05-22 12:20:57 +08:00
lifupan
0d535f56e5 shimv2: kill a container return directly once the container termianted
According to CRI specs, kubelet will call StopPodSandbox()
at least once before calling RemovePodSandbox, and this call
is idempotent, and must not return an error if all relevant
resources have already been reclaimed. And in that call it will
send a SIGKILL signal first to try to stop the container, thus
once the container has terminated, here should ignore this signal
and return directly.

Fixes:#1672

Signed-off-by: lifupan <lifupan@gmail.com>
2019-05-22 04:14:12 +00:00
lifupan
5e1f5ca735 shimv2: fix the issue of passing the wrong container id
It should pass the container id instead of sandbox id.

Fixes:#1672

Signed-off-by: lifupan <lifupan@gmail.com>
2019-05-21 16:40:52 +08:00
Hui Zhu
fb93774d28 Merge pull request #1628 from lifupan/fixctx
shimv2: fix the issue ttrpc server canceled context
2019-05-10 15:28:32 +08:00
lifupan
8eb09dec67 shimv2: fix the issue ttrpc server canceled context
This latest ttrpc vendor supports the feature of request timeout propgation.
this feature will do context cancel after a service call return, and this cancel
will propagated into kata sandbox's agent/hypervisor and resulted in the following
calls canceled. To fix this issue, pass the service's context instead of the service's
call's context to CreateSandbox(), and this context will live until the shim exited.

Fixes:#1627

Signed-off-by: lifupan <lifupan@gmail.com>
2019-05-09 04:58:30 -04:00
lifupan
1b33fe4022 shimv2: create the rootfs dir if it doesn't exist
Latest containerd commit<c0f0b21314b93a1> had moved the
step of creating rootfs dir from creating bundle to container
creation; in order to support both of the old and latest
containerd, check the "rootfs" existed before creating it.

Fixes:#1652

Signed-off-by: lifupan <lifupan@gmail.com>
2019-05-09 04:20:48 -04:00
James O. D. Hunt
23f7cfa9f4 tests: Update test code to use test constraints
Updated the test code to use the new test constraints feature.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2019-05-02 15:42:35 +01:00
Ace-Tang
854cc86e8d shimv2: fix set status when container exit
in wait function, should send msg to exit channel after task status has
updated, since shim.Wait() is running in another goroutine, when it
receive msg from exit channel, it will stop waiting and return, then
someone who hold this Wait() get return, it can delete task, if exit msg
is send first, the container status may still be running.

Fixes: #1600

Signed-off-by: Ace-Tang <aceapril@126.com>
2019-04-29 13:22:26 +08:00
Salvador Fuentes
bc9b9e2af6 vc: Revert "vc: change container rootfs to be a mount"
This reverts commit 196661bc0d.

Reverting because cri-o with devicemapper started
to fail after this commit was merged.

Fixes: #1574.

Signed-off-by: Salvador Fuentes <salvador.fuentes@intel.com>
2019-04-23 08:56:36 -05:00
Peng Tao
196661bc0d vc: change container rootfs to be a mount
We can use the same data structure to describe both of them.
So that we can handle them similarly.

Fixes: #1566

Signed-off-by: Peng Tao <bergwolf@hyper.sh>
2019-04-20 00:42:25 -07:00
Peng Tao
8215a3ce9a shimv2: convert vc errors to grpc errors
containerd checks for the grpc error code to determine
correct recover action upon grpc errors. We need to provide
them properly.

Unfortunately ttrpc doesn't support grpc interceptor so we have
to modify every service function for it.

Fixes: #1527

Signed-off-by: Peng Tao <bergwolf@hyper.sh>
2019-04-12 03:57:01 -07:00
Julio Montes
c08976e1a2 Merge pull request #1494 from lifupan/fixstop
shimv2: fix the issue of stop container failed
2019-04-11 07:10:49 -05:00
lifupan
f7223c6f00 shimv2: fix the issue of stop container failed
There is no need to send another SIGKILL signal following
SIGTERM signal, otherwise, sending the SIGKILL signal would
get failed since the container process would has been terminated
by the previous SIGTERM signal.

Fixes:#1493

Signed-off-by: lifupan <lifupan@gmail.com>
2019-04-11 18:01:23 +08:00
Peng Tao
616f26cfe5 types: split sandbox and container state
Since they do not really share many of the fields.

Fixes: #1434

Signed-off-by: Peng Tao <bergwolf@hyper.sh>
2019-04-09 18:59:56 -07:00
Peng Tao
25d21060e3 Merge pull request #1412 from lifupan/shimv2mount
shimv2: optionally plug rootfs block storage instead of mounting it
2019-04-02 15:30:40 +08:00
lifupan
52c66d20dc shimv2: plugin the block backed rootfs directly instead of mount it
When the container's rootfs is block storage backed such as devmapper,
shimv2 will not mount it on the host, instead it insert it into hypervisor
as a block device directly.

If kata's config set "disable_block_device_use" as true, it will mount
the rootfs onto host as before.

Fixes:#1158

Signed-off-by: lifupan <lifupan@gmail.com>
2019-04-02 10:56:03 +08:00
Ganesh Maharaj Mahalingam
f4428761cb lint: Update go linter from gometalinter to golangci-lint.
gometalinter is deprecated and will be archived April '19. The
suggestion is to switch to golangci-lint which is apparently 5x faster
than gometalinter.

Partially Fixes: #1377

Signed-off-by: Ganesh Maharaj Mahalingam <ganesh.mahalingam@intel.com>
2019-03-25 08:48:13 -07:00
fupan
96e524d2a0 shimv2: Send task events to containerd/cri
The Runtime v2 supports an async event model. In order for the an upstream
caller (such as Docker) to get these events in the correct order a Runtime
v2 shim MUST implement some events.

For much more info, please see:
https://github.com/containerd/containerd/blob/master/runtime/v2/README.md#events

Fixes:#1204

Signed-off-by: fupan <lifupan@gmail.com>
2019-02-11 11:49:10 +08:00
Fupan Li
f1a12ce04c shimv2: use the runtime config file passed from containerd/cri
containerd/cri's different runtime handlers can pass different
config files to shimv2 by a generic runtime options, by this kata
can launch the pods using different VMM for different runtime handlers.

Fixes:#1082

Signed-off-by: Fupan Li <lifupan@gmail.com>
2019-01-28 17:31:09 +08:00
Sebastien Boeuf
744354db56 Merge pull request #1134 from lifupan/marshalany
shimv2: use the UnmarshalAny() to unmarshal Protobuf.Any
2019-01-18 01:33:52 -08:00
Sebastien Boeuf
5329a71b3d runtime-v2: Make sure Shutdown() only shuts the server down
Because the runtime v2 runs as a RPC server, the caller will at some
point use the Shutdown() API to shut down the server. Because this
will cause the server to exit, the caller cannot expect any valid
answer when calling this. That's why we cannot afford stopping and
deleting the sandbox from this function.

Instead, we move sandbox.Stop() and sandbox.Delete() to a more
appropriate API, the Delete() one.

Fixes #1150

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2019-01-17 22:15:29 -08:00
fupan
5ee838d412 shimv2: use the UnmarshalAny() to unmarshal Protobuf.Any
It'll be much clear to unmarshal Protobuf.Any using
UnmarshalAny().

Fixes: #1130

Signed-off-by: fupan <lifupan@gmail.com>
2019-01-17 10:14:37 +08:00
fupan
5ba30fd628 shimv2: multi wait support for wait service
Support multi wait on one container/exec process.

Fixes: #1131

Signed-off-by: fupan <lifupan@gmail.com>
2019-01-16 12:41:59 +08:00
Samuel Ortiz
b05dbe3886 runtime: Convert to the new internal types package
We can now remove all the sandbox shared types and convert the rest of
the code to using the new internal types package.

This commit includes virtcontainers, cli and containerd-shim changes in
one atomic change in order to not break bisect'ibility.

Fixes: #1095

Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
2019-01-08 14:43:33 +01:00
fupan
0b28ab9e57 shimv2: get the kata configure file from env KATA_CONF_FILE
If the env KATA_CONF_FILE was set, shimv2 will use it as the
kata configure file, otherwise, it will try to find it from
/etc and /usr/share directory.

Fixes: #1091

Signed-off-by: fupan <lifupan@gmail.com>
2019-01-05 17:50:47 +08:00
Fupan Li
05be5bf1f9 shimv2: discard log before shim init log output
containerd would like to get the shim's socket
address from shimv2's stdout, thus it's better
to discard the log's output before shimv2 init
it's logger and at the same time add a hook to
log into syslog.

Fixes: #1035

Signed-off-by: Fupan Li <lifupan@gmail.com>
2018-12-17 15:39:44 +00:00
fupan
02f8b29837 containerd-shim-kata-v2: add building of shimv2 into Makefile
Add the Makefile target of building shimv2.

Fixes: #485

Signed-off-by: fupan <lifupan@gmail.com>
2018-11-28 14:32:25 +08:00
fupan
8199d10742 containerd-shim-kata: add unit test cases
Add unit test cases.

Signed-off-by: fupan <lifupan@gmail.com>
2018-11-28 14:32:25 +08:00
ZeroMagic
7951041eb0 containerd-shim-kata-v2: add the service Stats support
Add the Stats api support to get the container's
resouces statistic.

Signed-off-by: ZeroMagic <anthonyliu@zju.edu.cn>
2018-11-28 14:32:25 +08:00
ZeroMagic
5cc016c8a2 containerd-shim-kata-v2: add the service Kill support
Add the Kill api support to send signal to a given
container process.

Signed-off-by: ZeroMagic <anthonyliu@zju.edu.cn>
Signed-off-by: fupan.li <lifupan@gmail.com>
2018-11-28 14:32:25 +08:00