Commit Graph

568 Commits

Author SHA1 Message Date
Sebastien Boeuf
e2c17661b0 Merge pull request #1322 from lifupan/fixconsole
virtcontainer: watch the qemu's console when proxy's debug enabled
2019-03-05 06:50:57 -08:00
fupan
c4957ddd8d virtcontainer: watch the qemu's console when proxy's debug enabled
kata builtin proxy has always watched the qemu's console
whether proxy's debug is set or not, this is not aligned
with kata cli. This patch will change it and watch the
qemu's console only when proxy's debug is set in kata config.

Fixes: #1318

Signed-off-by: fupan <lifupan@gmail.com>
2019-03-05 17:09:26 +08:00
Eric Ernst
5dda0b7eea Merge pull request #1317 from jodh-intel/fix-runtime-tracing
tracing: Fix tracing
2019-03-04 21:11:39 -08:00
James O. D. Hunt
c759cf5f37 tracing: Fix tracing
The store refactor (#1066) inadvertently broke runtime tracing as it
created new contexts containing trace spans.

Reworking the store changes to re-use the existing context resolves the
problem since runtime tracing assumes a single context.

Fixes #1277.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2019-03-04 11:02:31 +00:00
xueshaojia 00464843
03dd780ddd qemu: fix devID value error
reason: When excutes ExecuteNetCCWDeviceAdd, the DevID is always "virtio-".
If add-iface multy times, qemu may report "dumplicated id:virtio-".

Fixes: #1305

Signed-off-by: xueshaojia <xueshaojia@huawei.com>
2019-03-04 09:01:38 +08:00
Jose Carlos Venegas Munoz
6f2597ed11 Merge pull request #1294 from egernst/vCPU-calculations-fix
virtcontainers: fix vCPU calculation errors
2019-02-28 15:53:20 -06:00
Archana Shinde
9b624d5b9b Merge pull request #1255 from caoruidong/ipvlan-ep
Network: add ipvlan unmarshal
2019-02-28 12:18:31 -08:00
Eric Ernst
dc2650889c virtcontainers: fix vCPU calculation errors
We were grabbing a running total of quota and period for each container
and then calculating the number of resulting vCPUs. Summing period
doesn't make sense.  To simplify, let's just calculate mCPU per
container, keep a running total of mCPUs requested, and then translate
to sandbox vCPUs after.

Fixes: #1292

Signed-off-by: Eric Ernst <eric.ernst@intel.com>
2019-02-28 08:13:04 -08:00
James O. D. Hunt
f540a80354 store: Add SetLogger API
Add a `store.SetLogger()` API to allow the store package to log with the
standard set of fields (as expected by the log parser [1].

Fixes #1297.

---

[1] - https://github.com/kata-containers/tests/tree/master/cmd/log-parser#logfile-requirements

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2019-02-28 10:56:26 +00:00
Ruidong Cao
36141d27fc Network: add ipvlan unmarshal
We have 7 types of endpoints, but forget ipvlan in unmarshal funciton.
So add it and refactor for cyclomatic complexity reason.

Fixes #1254

Signed-off-by: Ruidong Cao <caoruidong@huawei.com>
2019-02-23 01:25:26 +08:00
Julio Montes
16bd983387 Merge pull request #1264 from Ace-Tang/master
cgroups: fix failed to remove sandbox cgroup
2019-02-21 13:46:06 -06:00
James O. D. Hunt
b96ca2237f Merge pull request #1260 from stefanha/fix-vhost-user-blk-detach
devices: fix attach count for vhost-user-blk
2019-02-21 16:57:11 +00:00
Ace-Tang
454775fb97 cgroups: fix failed to remove sandbox cgroup
sandbox cgroup use V1NoConstraints, this only create memory subsystem,
but when delete, load parent cgroup always use `cgroups.V1`, so other
subsystem path can not be find, sandbox cgroup can not be deleted.

Fixes: #1263

Signed-off-by: Ace-Tang <aceapril@126.com>
2019-02-21 17:34:34 +08:00
Peng Tao
1d79338a1a Merge pull request #1247 from nitkon/leakyPods
qemu: Cleanup Vm paths irrespective of Sandbox stop pass/fail
2019-02-21 11:56:57 +08:00
GabyCT
60f7c4f401 Merge pull request #1189 from devimc/topic/fixCpuCgroup
virtcontainers: reimplement sandbox cgroup
2019-02-20 10:18:56 -06:00
Stefan Hajnoczi
46e2f885af devices: fix attach count for vhost-user-blk
Commit affd6e3216 ("devices: add reference
count for devices.") introduced an attach count for devices.  The
vhost-user-blk device increments the counter instead of decrementing it
when detaching.

Fixes: #1259
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2019-02-20 14:58:45 +00:00
Nitesh Konkar
6daefdb177 qemu: Cleanup Vm paths irrespective of Sandbox stop pass/fail
Sometimes qemu/qmp commands error out and VM files
get left behind on the host filesystem. Clen them up
irrespective of `stopSandbox` succeeds or fails.

Fixes: #1246

Signed-off-by: Nitesh Konkar niteshkonkar@in.ibm.com
2019-02-20 16:02:48 +05:30
Julio Montes
58d278560e virtcontainers: don't try to talk with the proxy when it's not running
To avoid long timeouts, the runtime shouldn't try to talk with the proxy
when it's not running.

Signed-off-by: Julio Montes <julio.montes@intel.com>
2019-02-19 13:13:45 -06:00
Julio Montes
62c393c119 virtcontainers: change container's state to stop asap
container is killed by force, container's state MUST change its state to stop
immediately to avoid leaving it in a bad state.

fixes #1088

Signed-off-by: Julio Montes <julio.montes@intel.com>
2019-02-19 13:13:44 -06:00
Julio Montes
5201860bb0 virtcontainers: reimplement sandbox cgroup
All containers run in different cgroups even the sandbox, with this new
implementation the sandbox cpu cgroup wil be equal to the sum of all its
containers and the hypervisor process will be placed there impacting to the
containers running in the sandbox (VM). The default number of vcpus is
used when the sandbox has no constraints. For example, if default_vcpus
is 2, then quota will be 200000 and period 100000.

**c-ray test**
http://www.futuretech.blinkenlights.nl/c-ray.html

```
+=============================================+
|         | 6 threads 6cpus | 1 thread 1 cpu  |
+=============================================+
| current |   40 seconds    |   122 seconds   |
+==============================================
|   new   |   37 seconds    |   124 seconds   |
+==============================================
```

current = current cgroups implementation
new = new cgroups implementation

**workload**

```yaml
apiVersion: v1
kind: Pod
metadata:
  name: c-ray
  annotations:
    io.kubernetes.cri.untrusted-workload: "true"
spec:
  restartPolicy: Never
  containers:
  - name: c-ray-1
    image: docker.io/devimc/c-ray:latest
    imagePullPolicy: IfNotPresent
    args: ["-t", "6", "-s", "1600x1200", "-r", "8", "-i",
          "/c-ray-1.1/sphfract", "-o", "/tmp/output.ppm"]
    resources:
      limits:
        cpu: 6
  - name: c-ray-2
    image: docker.io/devimc/c-ray:latest
    imagePullPolicy: IfNotPresent
    args: ["-t", "1", "-s", "1600x1200", "-r", "8", "-i",
          "/c-ray-1.1/sphfract", "-o", "/tmp/output.ppm"]
    resources:
      limits:
        cpu: 1
```

fixes #1153

Signed-off-by: Julio Montes <julio.montes@intel.com>
2019-02-19 13:13:44 -06:00
Julio Montes
9758cdba7c virtcontainers: move cpu cgroup implementation
cpu cgroups are container's specific hence all containers even the sandbox
should be able o create, delete and update their cgroups. The cgroup crated
matches with the cgroup path passed by the containers manager.

fixes #1117
fixes #1118
fixes #1021

Signed-off-by: Julio Montes <julio.montes@intel.com>
2019-02-19 13:13:44 -06:00
Penny Zheng
1b967a4a6a unit-test: add nolint comment to avoid unused warning
since all generic* could bring unused linter warnings, which lead to
CI crash, we add nolint comment to avoid them.

Fixes: #1200

Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
Signed-off-by: Penny Zheng <penny.zheng@arm.com>
2019-02-14 14:56:42 +08:00
Penny Zheng
3ec56eaf9f runtime: add appendBridges for arm64
since generic func genericAppendBridges and genericBridges
is also applied for machine type QemuVirt, we use it as implementation
for appendBridges and bridges on aarch64.
since const defaultPCBridgeBus is used in generic func
genericAppendBridges for pc machine, we should define it once
in generic file, instead of redefining it in different
arch-specific files.

Fixes: #1200

Signed-off-by: Penny Zheng <penny.zheng@arm.com>
2019-02-14 14:56:42 +08:00
Penny Zheng
e93fb0b3a0 unit-test: test func for RunningOnVMM should be arch-dependent
original tests for func RunningOnVMM are sort of amd64-specific,
since all other archs don't support nested VMM for now.

Fixes: #1200

Signed-off-by: Penny Zheng <penny.zheng@arm.com>
2019-02-14 14:56:30 +08:00
Penny Zheng
0679f6fa59 unit-test: refine qemu_arm64_test.go
refine a set of test functions under qemu_arm64_test.go. e.g. test
func for memoryTopology shouldn't be the same one on amd64, since
for now, we don't support nvdimm on arm64.

Fixes: #1200

Signed-off-by: Penny Zheng <penny.zheng@arm.com>
2019-02-14 11:34:05 +08:00
Julio Montes
a1c85902f6 virtcontainers: add method to get hypervisor PID
hypervisor PID can be used to move the whole process and its
threads into a new cgroup.

Signed-off-by: Julio Montes <julio.montes@intel.com>
2019-02-13 18:01:14 -06:00
Nitesh Konkar
a614273af5 virtcontainers: Improve debug when create/delete files
Adding debug messages which state which files
are being created/deleted could be helpful in
analyzing situations like leaky pod issues.

Fixes: #1234

Signed-off-by: Nitesh Konkar niteshkonkar@in.ibm.com
2019-02-13 20:51:33 +05:30
Samuel Ortiz
bb99e4152b virtcontainers: Fix Store related vm factory leak
We are creating Store directories but never removing them.
Calling into a VM factory created vm Stop() will now clean the VM Store
artifacts up.

Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
2019-02-07 00:59:33 +01:00
Samuel Ortiz
7b0376f3d3 virtcontainers: Fix container.go cyclomatic complexity
With the Stores conversion, the newContainer() cyclomatic complexity
went over 15. We fix that by extracting the block devices creation
routine out of newContainer.

Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
2019-02-07 00:59:33 +01:00
Samuel Ortiz
f8e7e308c3 virtcontainers: Remove the resource storage original implementation
Now that we converted the virtcontainers code to the store package, we
can remove all the resource storage old code.

Fixes: #1099

Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
2019-02-07 00:59:33 +01:00
Samuel Ortiz
fad23ea54e virtcontainers: Conversion to Stores
We convert the whole virtcontainers code to use the store package
instead of the resource_storage one. The resource_storage removal will
happen in a separate change for a more logical split.

This change is fairly big but mostly does not change the code logic.
What really changes is when we create a store for a container or a
sandbox. We now need to explictly do so instead of just assigning a
filesystem{} instance. Other than that, the logic is kept intact.

Fixes: #1099

Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
2019-02-07 00:59:29 +01:00
Samuel Ortiz
2ecffda170 virtcontainers: store: Add a ItemLock API
The ItemLock API allows for taking shared and exclusive locks on all
items.
For virtcontainers, this is specialized into taking locks on the Lock
item, and will be used for sandbox locking.

Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
2019-02-06 14:19:18 +01:00
Samuel Ortiz
6e9256f483 virtcontainers: store: Add a Raw API
The Raw API creates a raw item, i.e. an item that must be handled
directly by the caller. A raw item is one that's not defined by the
store.Item enum, i.e. it is a custom, caller defined one.
The caller gets a URL back and is responsible for handling the item
directly from this URL.

Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
2019-02-06 14:19:18 +01:00
Samuel Ortiz
c25c60898b virtcontainers: store: Add a VC specific Store
This is basically a Store dispatcher, for storing items into their right
Store (either configuration or state).
There's very little logic here, except for finding out which store an
item belongs to in the virtcontainers context.

vc.go also provides virtcontainers specific utilities.

Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
2019-02-06 14:19:18 +01:00
Samuel Ortiz
ef11bf52a6 virtcontainers: store: Add a Delete API
It's going to be used to completely clean a Store away.

Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
2019-02-06 14:19:18 +01:00
Samuel Ortiz
f2ab58d841 virtcontainers: store: Implement the filesystem backend
new() only creates the backend and initialized the first layout.

Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
2019-02-06 14:19:18 +01:00
Samuel Ortiz
d22cdf2dd9 virtcontainers: store: Add an internal backend interface
All Store backends will have to implement that simple interface.

Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
2019-02-06 14:19:18 +01:00
Samuel Ortiz
6b87ecfc1b virtcontainers: store: Keep track of newly created Stores
When a component creates a new store from a given root path, we add it
to the store manager and return it back when another component asks for
it.

Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
2019-02-06 14:19:18 +01:00
Samuel Ortiz
efd50ecac9 virtcontainers: Add a Store manager
Each virtcontainers module/component should be able to get a handler on
a Store for loading component specific items. The Store manager is an
internal Store layer for tracking all created Stores.

Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
2019-02-06 14:19:18 +01:00
Samuel Ortiz
4be76e9969 virtcontainers: Initial Store implementation
Store is a replacement for the current resource storage virtcontainers
implementation and the Manager is the front-end for it. The back-ends
will provide actual storage capabilities and the first one will be the
filesystem one, for storing virtcontainers Items on a local filesystem.

The main design goals for Store are the following ones:

- Simplicity: The API should be short and simple.
- Transparency: The core virtcontainers code should not care about
  the storage backend details.
- Extensibility: It should be easily extensible to add non local and in
  memory backends.

Manger provides a very short and simple API for the rest of the virtcontainers
code base to consume:

New: Creates a new Store, if needed.
Load: Loads an Item from a Store
Store: Stores an Item into a Store.

Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
2019-02-06 14:19:18 +01:00
Samuel Ortiz
2affa1fe26 virtcontainers: Reduce hyperstart agent test noise
We need to pass a context to the filesystem handle.

Fixes: #1211

Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
2019-02-05 16:33:06 +01:00
Samuel Ortiz
a3eff87e80 virtcontainers: Make proxy startup sequence less noisy
We only want to know which proxy started when debugging.

Fixes: #1211

Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
2019-02-05 16:33:06 +01:00
Samuel Ortiz
f0312f607b virtcontainers: Reduce filesystem test noise
We need to set the context before calling into the API.

Fixes: #1211

Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
2019-02-05 16:33:06 +01:00
Samuel Ortiz
e402601cf8 virtcontainers: Reduce sandbox test noise
We need to set the sandbox context before calling into its API.

Fixes: #1211

Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
2019-02-05 16:33:06 +01:00
Samuel Ortiz
799ac6edf6 virtcontainers: Reduce qemu test noise
We only need to set the context before calling into qemu's API.

Fixes: #1211

Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
2019-02-05 16:33:06 +01:00
Samuel Ortiz
560902c8f1 virtcontainers: Reduce kata_agent test noise
We only need to set the agent context before calling into its API.

Fixes: #1211

Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
2019-02-05 16:33:05 +01:00
Samuel Ortiz
2093fe6bfd virtcontainers: Reduce cc_proxy mock test noise
We don't need that many logs, especially for the positive path.

Fixes: #1211

Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
2019-02-05 16:33:05 +01:00
Samuel Ortiz
79ed0886c6 virtcontainers: Reduce hyperstart mock test noise
We don't need that many logs, especially for the positive path.

Fixes: #1211

Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
2019-02-05 16:32:58 +01:00
James O. D. Hunt
530360d515 Merge pull request #1186 from nitkon/Hotplug
ppc64le: Fix hotplug issue
2019-01-31 09:26:25 +00:00
Nitesh Konkar
b0986a5f7f ppc64le: Fix vCPU hotplug issue
ppc64le qemu does not need threadID and
socketID parameters when hotplugging.

Fixes: #1155

Signed-off-by: Nitesh Konkar niteshkonkar@in.ibm.com
2019-01-28 23:42:20 +05:30