Commit Graph

362 Commits

Author SHA1 Message Date
Fabiano Fidêncio
bda0354491 ci: cache: Export env vars needed to use ORAS
We do the build of our artefacts inside a container image, and we need
to expose some env vars to the container so ORAS can be used there to
push the artefacts we want to cache to ghcr.io.

The env vars we're exposing are:
* ARTEFACT_REGISTRY: The registry where we're going to save the
  artefacts.
* ARTEFACT_REGISTRY_USERNAME: The username to log in to the registry, as
  ORAS does not use the same json file used by docker.
* ARTEFACT_REGISTRY_PASSWORD: The pasword to log in to the the registry,
  as the ORAS does not use the same json file used by docker.
* TARGET_BRANCH: The target branch, which will be part of the tag of the
  artefact, as we may end up caching the artefacts for both main and
  stable branches.

Fixes: #7834 -- part 0

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
(cherry picked from commit 6bd15a85d5)
2023-09-21 13:58:47 +02:00
Jeremi Piotrowski
a1aed0c78e gha: vfio: Set test timeout to 15m
Sometimes the test gets stuck running commands in the container - need to
investigate why later.

Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
(cherry picked from commit 7211c3dccc)
2023-09-21 13:56:45 +02:00
Fabiano Fidêncio
c7147dabce ci: docker: Run the tests after the kata-static is created
There's no reason to wait till the payload is created to run the tests,
as we rely on the tarball, not on the kata-deploy payload.

That was a mistake on my side, and that's already fixed for the nerdctl
tests.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
(cherry picked from commit c83f167c59)
2023-09-21 13:46:46 +02:00
Fabiano Fidêncio
33430ad60c ci: Add a very basic nerdctl sanity test
Let's add a very basic sanity test to check that we can spawn a
containers using nerdctl + Kata Containers.

This will ensure that, at least, we don't regress to the point where
this feature doesn't work at all.

In the future, we should also test all the VMMs with devmapper, but
that's for a follow-up PR after this test is working as expected.

Fixes: #7911

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
(cherry picked from commit 12d833d07d)
2023-09-21 13:46:37 +02:00
Fabiano Fidêncio
69dd11f459 ci: Add a very basic docker sanity test
Let's add a very basic sanity test to check that we can spawn a
containers using docker + Kata Containers.

This will ensure that, at least, we don't regress to the point where
this feature doesn't work at all.

For now we're running this test against Cloud Hypervisor and QEMU only,
due to an already reported issue with dragonball:
https://github.com/kata-containers/kata-containers/issues/7912

In the future, we should also test all the VMMs with devmapper, but
that's for a follow-up PR after this test is working as expected.

Fixes: #7910

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
(cherry picked from commit 348b8644d6)
2023-09-21 13:46:28 +02:00
Fabiano Fidêncio
fcfa6c6e1a ci: use github.ref_name instead of $GITHUB_REF_NAME
As, regardless of what's mentioned in the documentation, it seems that
$GITHUB_REF_NAME is passed down as a literal string.

Fixes: #7414

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
(cherry picked from commit f811b064ca)
2023-09-21 13:46:00 +02:00
Fabiano Fidêncio
19d9fd9eb1 ci: Add more target-branch related fixes
The ones for the payload-after-push.yamland ci-nightly.yaml are not that
much important right now, but they're needed for when we start running
those on stable branches as well.

The other ones were missed during
bd24afcf73.

Fixes: #7414

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
(cherry picked from commit 6d795c089e)
2023-09-21 13:45:52 +02:00
Fabiano Fidêncio
fe4247a90c ci: Fix target-branch usage
We missed those one as part of bd24afcf73.

Fixes: #7414

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
(cherry picked from commit 8509c31870)
2023-09-21 13:45:43 +02:00
Fabiano Fidêncio
61b1a99fca gha: Manually rebase PR atop of the target branch before testing
We're changing what's been done as part of ac939c458c, as we've
notcied issues using `github.event.pull_request.merge_commit_sha`.

Basically, whenever a force-push would happen, the reference of
merge_commit_sha wouldn't be updated, leading us to test PRs with the
old code. :-/

In order to get the rebase properly working, we need to ensure we pull
the hash of the commit as part of checkout action, and ensure
fetch-depth is set to 0.

Fixes: #7414

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
(cherry picked from commit bd24afcf73)
2023-09-21 13:43:49 +02:00
Fabiano Fidêncio
c723a7d9c8 ci: k8s: devmapper tests should be using ubuntu 20.04
That's what we've been using as part of Jenkins, so let's ensure things
will work as they did before, and only after that consider upgrading the
base OS used for the tests.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
(cherry picked from commit aaec5a09f3)
2023-09-21 13:41:57 +02:00
Fabiano Fidêncio
aee6f36c86 ci: k8s: Add a kata-deploy-garm target
We've been using the `kata-deploy-tdx` target as that also uses k3s as
base, but it's better to just have a specific garm target.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
(cherry picked from commit 27fa7d828d)
2023-09-21 13:41:52 +02:00
Fabiano Fidêncio
5bb77b628d ci: k8s: Export KUBERNETES env var
So we have a better control on which flavour of kubernetes kata-deploy
is expected to be targetting.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
(cherry picked from commit fa62a4c01b)
2023-09-21 13:41:45 +02:00
Fabiano Fidêncio
7ce5c8b3fa ci: k8s: Install bats on GARM runners
GARM runners do not come with the whole set of tools we need, or are
used to when it comes to the GHA runners, so we need to manually install
bats on those.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
(cherry picked from commit 8c9380a798)
2023-09-21 13:41:39 +02:00
Fabiano Fidêncio
1b7ffeac53 ci: k8s: Fix typo in run-k8s-tests-on-garm.yaml
integrations -> integration
integrtion -> integration

Fixes: #6542

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
(cherry picked from commit 8d99972a8a)
2023-09-21 13:39:32 +02:00
Fabiano Fidêncio
79de72592f ci: k8s: Add k8s devmapper tests (part 0)
Let's enable the devmapper kubernetes tests to match exactly what's been
tested as part of the Jenkins CI.

Fixes: #6542

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
(cherry picked from commit 0e8bd50cbb)
2023-09-21 13:39:12 +02:00
Chao Wu
7fd7186780 CI: switch static-checks-dragonball CI machines to Azure
Previously, static-checks-dragonball is using machines from Alibaba
Cloud to run all the CI jobs.

Currently, we are going through an internal process to apply for the new
machines for Dragonball CI. Before the internal process is over, we will
temporarily use Azure VM to run static-checks-dragonball jobs.

fixes: #7838

Signed-off-by: Chao Wu <chaowu@linux.alibaba.com>
(cherry picked from commit 60f733d301)
2023-09-21 13:37:39 +02:00
Fabiano Fidêncio
8c7a4fd121 gha: Rebase atop of the target branch
We have two scenarios we care about this, `pull_request` and
`pull_request_target` events triggered a job.

`pull_request` event:
When using the checkout action, it'll already provide a "rebased atop of
main" repo for us, nothing else is needed, and that's basically what we
already have as part of the jobs in our CI.

`pull_request_target` event:
This one is a little bit tricky, as the checkout action, unless passing
a spsecific repo, give us the PR checked out rebased atop of the HEAD of
the PR branch.  Jeremi Piotrowski nicely pointed out that we could use
github.event.pull_request.merge_commit_sha instead, which is the result
of the PR's branch with the official repo target branch.

Now, the only cases where the contributor's rebase would still be needed
is when the action itself has been changed.

Fixes: #7414

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
(cherry picked from commit ac939c458c)
2023-09-21 13:30:22 +02:00
Fabiano Fidêncio
59e7c3a347 gha: Update to checkout@v3 action
At this point we should always be using the latest checkout action.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
(cherry picked from commit d7a996c686)
2023-09-21 13:29:57 +02:00
Fabiano Fidêncio
6a34bae03d gha: Avoid "fail-fast" in tests that are known to be flaky
Otherwise we'll have to re-run all the tests due to a flaky behaviour in
one of the parts.

Fixes: #7757

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
(cherry picked from commit fb49d5d7ce)
2023-09-21 13:26:26 +02:00
Jeremi Piotrowski
e99a13d26c gha: vfio: Run on Ubuntu 23.04 runner
The vfio test requires nested-nested virtualization:

L0 Azure host
-> L1 Ubuntu VM
  -> L2 Fedora VM
    -> L3 Kata

This hits a kernel bug on v5.15 but works quite nicely on the v6.2 kernel
included in Ubuntu 23.04. We can switch back to Ubuntu 22.04 when they roll out
v6.2.

Fixes: #6555
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
(cherry picked from commit 936e8091a7)
2023-09-21 13:25:35 +02:00
Fabiano Fidêncio
145450544d gha: ci: Start running kata-deploy tests
Let's add the tests as part of the ci.yaml, so they an be triggered as
part of each PR.

For this PR those tests won't be triggered, courtesy to the
`pull_request_target` event we rely on.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
(cherry picked from commit d19a75e80c)
2023-09-21 13:21:46 +02:00
Fabiano Fidêncio
4bb8fcc0c0 tests: kata-deploy: Add placeholder for kata-deploy-tests-on-tdx
This will not be tested as part of the PR, thanks to the
`pull_request_target` event, but we want it to be added so we can build
atop of that in a coming up series.

Fixes: #7642

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
(cherry picked from commit e55fa93db9)
2023-09-21 13:18:42 +02:00
Fabiano Fidêncio
f5e14ef283 tests: kata-deploy: Add placeholder for kata-deploy-tests-on-aks
This will not be tested as part of the PR, thanks to the
`pull_request_target` event, but we want it to be added so we can build
atop of that in a coming up series.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
(cherry picked from commit d9ee17aaec)
2023-09-21 13:18:35 +02:00
Fabiano Fidêncio
058b304455 gha: static-checks: Move to the Azure instances
The GHA runners are not exactly powerful, which makes the static-checks
take way too long (almost an hour).

Let's give a try and move those to the same size of Azure instances used
as part of our CI, and probably have this time reduced.

Fixes: #7446

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
(cherry picked from commit c52d090522)
2023-09-21 13:16:47 +02:00
Fabiano Fidêncio
b8b4ca10e9 ci: unencrypted-image: Fix build context
The build context should be the folder where the Dockerfile is present,
otherwise the files copied into the image won't be found.

Fixes: #7595

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
(cherry picked from commit 03d1fa67b1)
2023-09-21 13:15:15 +02:00
Fabiano Fidêncio
babbd4186c ci: create-confidential-image: Add dependent actions
Following the example on https://github.com/docker/build-push-action,
it's clear that the actions to "Set up QEMU" and "Set up Docker Buildx"
are missing.

Let's add them, and also take the advantage to bump the
build-push-action to its v4, which, by the way, had a typo on its name
(build-and-push-action does **NOT** exist, build-push-action does).

Fixes: #7595

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
(cherry picked from commit a2d731ad26)
2023-09-21 13:14:26 +02:00
Fabiano Fidêncio
a5a3e4124f ci: k8s: tees: Ensure PR_NUMBER is exported
Right now this is not being used, but it'll as the image generated for
the confidential tests have that as part of their tag.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
(cherry picked from commit 43fe5d1b90)
2023-09-21 13:13:52 +02:00
Fabiano Fidêncio
3a21c485bf ci: {{ pr-number }} should be {{ inputs.pr-number }}
One of the joys to rely on the `pull_request_target` is to only be able
to catch those after those are merged.

Fixes: #7595

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
(cherry picked from commit 54f6a78500)
2023-09-21 13:13:45 +02:00
Fabiano Fidêncio
0625d8dfc1 ci: Add build-and-publish-tee-confidential-unencrypted-image
This will be done before running TEE tests, and it's a hard dependency
fr them.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
(cherry picked from commit fac8ccf5cd)
2023-09-21 13:13:23 +02:00
David Esparza
a484666890 metrics: install kata once and run multiple checks
This PR changes the metrics workflow in order to just install
kata once, and run the checks for multiple hypervisor variations.

In this way we save time avoiding installing kata for each
hypervisor to be tested.

Fixes: #7578

Signed-off-by: David Esparza <david.esparza.borquez@intel.com>
(cherry picked from commit e664969862)
2023-09-21 13:12:49 +02:00
Gabriela Cervantes
a75db20167 gha: Add iperf network metrics
This PR adds the iperf network metrics to the github actions
for kata metrics.

Fixes #7535

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
(cherry picked from commit 5b5caf8908)
2023-09-21 13:12:11 +02:00
David Esparza
542012c8be Merge pull request #7503 from GabyCT/topic/ghafio
metrics: Add FIO test to gha for kata metrics CI
2023-08-02 10:05:09 -06:00
Fabiano Fidêncio
025596b289 ci-on-push: Make the CI also run for the stable-* branches
As we only support one stable branch, it'll be used as part of the
stable-3.2 and onwards.

Fixes: #7518

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-08-02 09:26:24 +02:00
Gabriela Cervantes
f764248095 gha: Add FIO test to run metrics yaml
This PR adds FIO test to run metrics yaml.

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2023-08-01 20:29:16 +00:00
Fabiano Fidêncio
743291c6c4 release: Fix upload-versions-yaml
This requires the GITHUB_UPLOAD_TOKEN.  While we're here, let's also fix
the name of the action and remove the "-tarball" suffix, as it's not
really a tarball.

Fixes: #7497

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-31 23:57:33 +02:00
Fabiano Fidêncio
a71d35c764 Merge pull request #7499 from fidencio/topic/gha-release-ensure-stage-is-defined-for-amr64-s300x
gha: release: `stage` must be defined for arm64 / s390x yamls
2023-07-31 22:55:54 +02:00
Fabiano Fidêncio
ff8d7e7e41 Merge pull request #7496 from fidencio/topic/topic/kata-deploy-take-nfd-into-consideration-pre-work
k8s: Rely on the USING_NFD environment variable passed by the jobs
2023-07-31 14:56:15 +02:00
Fabiano Fidêncio
1b111a9aab gha: release: stage must be defined for arm64 / s390x yamls
`stage`  has been added, but only hooked up to the amd64 logic, leaving
arm64 and s390x behind.

Let's fix this right now, and make sure no error occurs when passing
this down to the yaml files.

Fixes: #7497

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-31 14:41:35 +02:00
Fabiano Fidêncio
684a6e1a55 Revert "gha: release: stage must be a string"
This reverts commit 7c857d38c1.

I've misunderstood the error given by github action, let's fix this in
the next commit.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-31 14:37:52 +02:00
Fabiano Fidêncio
7c857d38c1 gha: release: stage must be a string
Otherwise we'll face the following error as part of our GHA:
```
The workflow is not valid.
kata-containers/kata-containers/.github/workflows/release-$foo.yaml
(Line: 13, Col: 14): Invalid input, stage is not defined in the
referenced workflow.
```

Fixes: #7497

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-31 13:39:13 +02:00
Fabiano Fidêncio
91e1e612c3 k8s: Rely on the USING_NFD environment variable passed by the jobs
Let's make sure we can rely on the tests passing down whether they want
to be tested using Node Feataure Discovery or not.

Right now, only the TDX job has this option set to "true", all the other
jobs have this option set to "false".

We can and have to merge this one before merging the NFD related patches
as:
1) It causes no harm in exporting this environment variable, but not
   having it used
2) It will allow us to test the NFD after this one is merged, as changes
   in the yaml file, in the case of the pull_request_target event,  are
   not taken into consideration before they're merged

Fixes: #7495

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-31 13:30:18 +02:00
Fabiano Fidêncio
87568ed985 gha: Test split out runtimeclasses are in sync with all-in-one file
This is needed in order to not lose track of what's been created and
what's been added here and there.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-28 10:04:33 +02:00
Chao Wu
ede1dae65d Merge pull request #7465 from fidencio/topic/fix-dragonball-static-check-runner-selector
gha: dragonball: Run only on the dragonball labeled machine
2023-07-27 10:19:26 +08:00
Fabiano Fidêncio
f28af98ac6 Merge pull request #7453 from sprt/fix-ci-node-debugger
tests: Fix `k8s-job` test
2023-07-26 22:27:21 +02:00
Aurélien Bombo
c5a87eed29 tests: gha: Add timeout to cluster creation
This has been intermittently taking a while lately so let's add a
timeout.

Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
2023-07-26 10:19:07 -07:00
Fabiano Fidêncio
3aa6c77a01 gha: dragonball: Run only on the dragonball labeled machine
Static checks for dragonball are landing on any of the self-hosted
runners, and the reason for that is because "self-hosted" was the label
selector used.

Let's use "dragonball" instead, as the machine has that label as well.

Fixes: #7464

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-26 18:15:04 +02:00
Aurélien Bombo
bdde6aa948 tests: k8s: Split deployment and testing commands
This splits deploying Kata and running the tests into separate commands
to make it possible to rerun tests locally without having to redeploy
Kata each time.

Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
2023-07-25 15:44:46 -07:00
Jeremi Piotrowski
717f775f30 gha: ci: Add skeleton of vfio job
This job will run on a nested virt capable Azure VM (improving test
concurrency). This is just a placeholder while we adapt the test to GHA.

Fixes: #6555
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
2023-07-25 11:13:04 +02:00
Fabiano Fidêncio
6a680e241b gha: ci: Add placeholder for the nydus tests as part of the CI
This will triger the nydus tests, but as they currently are they'll just
return "okay" without actually executing.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-24 13:37:36 +02:00
Fabiano Fidêncio
fb4f7a002c gha: nydus: Add a no-op GHA for nydus
This newly added GHA does nothing, is not even triggered, and it's just
a placeholder that we'll grow in the next commits / PRs, so we can
actually start running the nydus tests as part of our CI.

Fixes: #6543

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-24 13:37:33 +02:00