3783aeab64
# What this PR does - updates the GitHub Actions workflow to move the e2e tests into a "[reusable workflow](https://docs.github.com/en/actions/using-workflows/reusing-workflows#creating-a-reusable-workflow)" which are run in two scenarios: - all tests _except_ those annotated as `@expensive` are run against `grafana/grafana:latest` on all feature branches - all tests _including_ `@expensive` tests are run on weekdays @ 07h00 UTC, against a matrix of 6 grafana versions. Results of these builds will be posted to `#irm-amixr-flux` Slack channel. - local development will now be: ```bash make build-dev-images init-k8s start-k8s ``` - `build-dev-images` - builds the engine and UI docker images (only need to run first time) - `init-k8s` - creates a `kind` cluster and loads the two Docker images onto the cluster nodes (only need to run first time) - `start-k8s` - switches `kubectl` context to the created `kind` cluster, and uses `helm` to deploy everything as defined in `./dev/helm-local.yml` and `./dev/helm-local.dev.yml` (that latter file is `.gitignored` and specific to how _you_ want your setup to look like. Hot reloading works as before. This is the _start_ of #2381. (I've marked these `make` commands as beta, because they've not yet been thoroughly tested for local development). - modifies the `helm` chart to add the concept of `oncall.devMode`, `ui`, and ability to run oncall w/ sqlite - `oncall.devMode` will essentially just add `volumes` and `volumeMounts` to the various engine/migrate containers + - `ui.enabled` + `ui.env` - create a ui container (which is needed for hot reloading locally) - `sqlite` - this was useful for the e2e test environments where Github runner resources are scarce. Running `mariadb` eats up precious resources, instead lets just use sqlite here - fixes an issue that caused sporadic HTTP 502s from the grafana plugin-proxy, which led to flaky tests. See [this comment](https://github.com/grafana/oncall/pull/2751/files#diff-09040e8df192699b9c5742110ebbe8d9d5c3938cb156cc1cb99fa1c3fdee4fefR72-R77) for more context + a link to a relevant Slack conversation. **tldr;** there is a bug with the Grafana plugin proxy in Grafana >= v10.0.3. Let's stop using the `latest`/`main` docker tags in our test and pin to `10.0.2` for now - ~~re-enables the e2e test which validates a phone number via SMS, and asserts that we can receive an alert escalation via SMS (new Mailslurp API Key has been added as a repo secret)~~ update: this is still blocked by procurement, will be done in a future PR ## Checklist - [x] Unit, integration, and e2e (if applicable) tests updated - [x] Documentation added (or `pr:no public docs` PR label added if not required) - [x] `CHANGELOG.md` updated (or `pr:no changelog` PR label added if not required)
61 lines
1.9 KiB
YAML
61 lines
1.9 KiB
YAML
name: Daily e2e tests
|
|
"on":
|
|
# allows manual run on github actions
|
|
workflow_dispatch:
|
|
schedule:
|
|
- cron: "0 6 * * 1-5"
|
|
|
|
jobs:
|
|
end-to-end-tests:
|
|
name: End to end tests
|
|
strategy:
|
|
matrix:
|
|
grafana-image-tag:
|
|
# OnCall doesn't work on the following versions of Grafana
|
|
# - 8.5.22
|
|
# - 9.0.0
|
|
# - 9.1.0
|
|
|
|
# 9.2.0 is the earliest version where things work
|
|
- 9.2.20
|
|
- 9.3.16
|
|
- 9.4.13
|
|
- 9.5.7
|
|
- 10.0.2
|
|
|
|
# NOTE: don't run on > 10.0.3 until this issue is fixed. It appears that something was changed
|
|
# with the plugin-proxy that caused it to start throwing HTTP 502s from time to time
|
|
# https://raintank-corp.slack.com/archives/C01C4K8DETW/p1692279329797149
|
|
# - latest
|
|
fail-fast: false
|
|
uses: ./.github/workflows/e2e-tests.yml
|
|
with:
|
|
grafana-image-tag: ${{ matrix.grafana-image-tag }}
|
|
run-expensive-tests: false
|
|
secrets: inherit
|
|
|
|
post-status-to-slack:
|
|
runs-on: ubuntu-latest
|
|
needs: end-to-end-tests
|
|
if: always()
|
|
steps:
|
|
- uses: slackapi/slack-github-action@v1.24.0
|
|
with:
|
|
channel-id: irm-amixr-flux
|
|
# yamllint disable rule:line-length
|
|
payload: |
|
|
{
|
|
"text": "Daily e2e tests build result: ${{ job.status }}\n${{ github.event.pull_request.html_url || github.event.head_commit.url }}",
|
|
"blocks": [
|
|
{
|
|
"type": "section",
|
|
"text": {
|
|
"type": "mrkdwn",
|
|
"text": "Daily e2e tests build result: ${{ job.status }}\n${{ github.event.pull_request.html_url || github.event.head_commit.url }}"
|
|
}
|
|
}
|
|
]
|
|
}
|
|
# yamllint enable rule:line-length
|
|
env:
|
|
SLACK_BOT_TOKEN: ${{ secrets.SLACK_DAILY_E2E_TESTS_BOT_TOKEN }}
|