Run e2e-tests in CI #1210
Draft
Run e2e-tests in CI #1210
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
denisonbarbosa
force-pushed
the
e2e-tests-in-ci
branch
from
f088faa to
e081a57
Compare
adombeck
force-pushed
the
e2e-tests-in-ci
branch
from
1e8856e to
4784536
Compare
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #1210 +/- ##
==========================================
+ Coverage 80.51% 86.67% +6.15%
==========================================
Files 20 111 +91
Lines 970 7203 +6233
Branches 0 111 +111
==========================================
+ Hits 781 6243 +5462
- Misses 189 904 +715
- Partials 0 56 +56 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
denisonbarbosa
force-pushed
the
e2e-tests-in-ci
branch
from
7bbad96 to
b0c9914
Compare
This makes our life easier when trying to upload the test outputs as artifacts on CI.
The syntax used in this execution was flawed.
This is a QoL change to help us fine tune the artifacts in CI.
We were using only the base name to create the image file, which could cause conflicts if we ever try to setup both releases.
The 'await' command on virsh is not available on Noble, which is our runner in github, so we need to use another way to check for the domain status.
Update the commands to also run `sudo apt-get update` and add some dependencies that were missing from the lists
This workflow downloads an Ubuntu image, sets up some of the aplications and configurations that we will need in the VM in order to be able to run the E2E tests and then caches it for later use to save us some time when running the tests. Co-authored-by: Adrian Dombeck <[email protected]>
fix output artifact naming ci/e2e-tests: Use env vars instead of config file The config file is useful when running the tests locally, but it only sets environment variables, so in the CI it's simpler to just set the environment variables directly. ci/e2e-tests: Fix indentation
This caused the provisioning to fail
We were padding the generated code with trailing 0s, but we should have been padding with leading 0s. This commit fixes that.
The code is generated according to the current time and is valid for 30 seconds. This means that if we generate the code just before the time window changes, it might be invalid by the time we use it. To avoid this, we make sure the time is safely within a new window before generating the code.
We see permission denied errors when trying to start the VM both in CI and locally because AppArmor denies access to the qcow2 image (curiously, starting the VM by restoring a running snapshot works). We should investigate the issue and report it upstream. In the meantime, let's work around it by disabling AppArmor confinement for the e2e-test VM. That should be fine since we're not doing anything in the VM which would make them more vulnerable than the host.
We recently changed the error message that is shown when the broker fails to reach the provider, so we need to update this check on the tests as well.
Sometimes machinectl login hangs and it causes the tests to fail. Let's try reopening it a couple of times (in case of failure) to see if we are able to get more consistent behavior
systemd only supports sending the journal via VSOCK since v256. In the GitHub CI the newest Ubuntu release we can use on the runners (both GitHub-hosted or self-hosted) is 24.04, which has systemd 255.
denisonbarbosa
force-pushed
the
e2e-tests-in-ci
branch
from
b0c9914 to
5fd0acc
Compare
Running the tests is quite resource intensive, so let's switch to using our self-hosted runners to see if we can get some improvements in performance and stability.
denisonbarbosa
force-pushed
the
e2e-tests-in-ci
branch
from
5fd0acc to
4201248
Compare
adombeck
commented
Jan 28, 2026
It doesn't only "Build and Cache Ubuntu VM", it runs the e2e-tests and does everything required to do so.
adombeck
force-pushed
the
e2e-tests-in-ci
branch
from
6e51dc9 to
8cfdaef
Compare
adombeck
commented
Jan 28, 2026
Comment on lines
+75
to
+83
| - name: Clean previous cache | ||
| if: always() && steps.provision-vm.outcome == 'success' | ||
| env: | ||
| GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} | ||
| GH_REPO: ${{ github.repository }} | ||
| run: | | ||
| set -eu | ||
| gh cache delete e2e-runner-${{ matrix.release }} || true | ||
| echo "Previous cache (if any) deleted" |
Contributor
Author
There was a problem hiding this comment.
Can we add a comment here explaining why it's necessary to delete the cache even though we had a cache miss above?
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Run our new e2e-tests in the GitHub CI.
UDENG-7426