# What this PR does
- Use cross-plugin e2e tests setup (cloning ops-devenv, gops-labels)
only on a daily runs and on dev/main branch pipelines (exclude it from
PRs so that community PRs don't rely on secrets)
- Rename "Daily e2e tests" to "Expensive e2e tests" and run them both
daily and when PRs are merged to dev/main
- Post Slack message only if e2e tests fail
## Checklist
- [x] Unit, integration, and e2e (if applicable) tests updated
- [x] Documentation added (or `pr:no public docs` PR label added if not
required)
- [x] Added the relevant release notes label (see labels prefixed w/
`release:`). These labels dictate how your PR will
show up in the autogenerated release notes.
---------
Co-authored-by: Joey Orlando <joey.orlando@grafana.com>
# What this PR does
- Run e2e tests using ops-devenv against environment that includes
OnCall & Labels
- Add first e2e test for Labels (creating new label key and value)
## Which issue(s) this PR closes
Closes https://github.com/grafana/oncall/issues/4083
<!--
*Note*: if you have more than one GitHub issue that this PR closes, be
sure to preface
each issue link with a [closing
keyword](https://docs.github.com/en/get-started/writing-on-github/working-with-advanced-formatting/using-keywords-in-issues-and-pull-requests#linking-a-pull-request-to-an-issue).
This ensures that the issue(s) are auto-closed once the PR has been
merged.
-->
## Checklist
- [x] Unit, integration, and e2e (if applicable) tests updated
- [x] Documentation added (or `pr:no public docs` PR label added if not
required)
- [x] Added the relevant release notes label (see labels prefixed w/
`release:`). These labels dictate how your PR will
show up in the autogenerated release notes.
# What this PR does
Fix daily e2e tests
<!--
*Note*: if you have more than one GitHub issue that this PR closes, be
sure to preface
each issue link with a [closing
keyword](https://docs.github.com/en/get-started/writing-on-github/working-with-advanced-formatting/using-keywords-in-issues-and-pull-requests#linking-a-pull-request-to-an-issue).
This ensures that the issue(s) are auto-closed once the PR has been
merged.
-->
## Checklist
- [x] Unit, integration, and e2e (if applicable) tests updated
- [x] Documentation added (or `pr:no public docs` PR label added if not
required)
- [x] Added the relevant release notes label (see labels prefixed w/
`release:`). These labels dictate how your PR will
show up in the autogenerated release notes.
# What this PR does
- Reuse Tiltfile from local environment and use `tilt ci` to run e2e
tests on Github
- Use Playwright Docker image to get rid of installing Playwright
browsers and system dependencies
- Use ubuntu-latest-16-cores runner for e2e tests job on CI
## Which issue(s) this PR fixes
Closes https://github.com/grafana/oncall/issues/4018
## Checklist
- [x] Unit, integration, and e2e (if applicable) tests updated
- [x] Documentation added (or `pr:no public docs` PR label added if not
required)
- [x] Added the relevant release notes label (see labels prefixed w/
`release:`). These labels dictate how your PR will
show up in the autogenerated release notes.
# What this PR does
Allow e2e tests GitHub Actions job to run for PRs from external forks.
At the moment they are not allowed to run because PRs from external
forks are not allowed to access secrets in this repository. The secrets
accessed by the e2e tests are _really_ only required by the
daily/"expensive" e2e tests.
[Example PR](https://github.com/grafana/oncall/pull/3992) from external
fork where e2e tests fail ([CI
job](https://github.com/grafana/oncall/actions/runs/8175205794?pr=3992)):
<img width="1110" alt="Screenshot 2024-03-06 at 11 58 05"
src="https://github.com/grafana/oncall/assets/9406895/00ca97c2-0740-4e3e-9a03-7e92f20b69e6">
## Which issue(s) this PR closes
N/A
## Checklist
- [ ] Unit, integration, and e2e (if applicable) tests updated
- [ ] Documentation added (or `pr:no public docs` PR label added if not
required)
- [ ] Added the relevant release notes label (see labels prefixed w/
`release:`). These labels dictate how your PR will
show up in the autogenerated release notes.
# What this PR does
- publish Playwright HTML report to GH Pages
- turn off video recordings
## Checklist
- [x] Unit, integration, and e2e (if applicable) tests updated
- [x] Documentation added (or `pr:no public docs` PR label added if not
required)
- [x] `CHANGELOG.md` updated (or `pr:no changelog` PR label added if not
required)
# What this PR does
- fixes our e2e tests to work on all tested versions
- updates Grafana versions that we run the daily e2e tests against (bump
`10.0.2` to `10.0.11` + add `10.1.7` tags)
- updates the Slack status message format + change channel from
#irm-amixr-flux to #gops-oncall-dev
<img width="1479" alt="Screenshot 2024-02-24 at 08 30 06"
src="https://github.com/grafana/oncall/assets/9406895/f5cb91f8-12ce-4978-9c37-c72ee8a01e4b">
## NOTE
It looks like we have some e2e tests that fail under the following
circumstances:
- on Firefox or WebKit
- on Grafana 10.2 and 10.3 (once we fix these, we should [update our e2e
tests that run on all PR
builds](https://github.com/grafana/oncall/blob/dev/.github/workflows/linting-and-tests.yml#L325)
to run against `10.3.3` which is the current latest major version
available)
## Checklist
- [x] Unit, integration, and e2e (if applicable) tests updated
- [x] Documentation added (or `pr:no public docs` PR label added if not
required)
- [x] `CHANGELOG.md` updated (or `pr:no changelog` PR label added if not
required)
# What this PR does
In PR pipelines install dependencies and run e2e tests only in Chromium.
In daily e2e workflow use Chromium, Firefox and Webkit.
## Which issue(s) this PR fixes
## Checklist
- [x] Unit, integration, and e2e (if applicable) tests updated
- [x] Documentation added (or `pr:no public docs` PR label added if not
required)
- [x] `CHANGELOG.md` updated (or `pr:no changelog` PR label added if not
required)
# What this PR does
- updates the GitHub Actions workflow to move the e2e tests into a
"[reusable
workflow](https://docs.github.com/en/actions/using-workflows/reusing-workflows#creating-a-reusable-workflow)"
which are run in two scenarios:
- all tests _except_ those annotated as `@expensive` are run against
`grafana/grafana:latest` on all feature branches
- all tests _including_ `@expensive` tests are run on weekdays @ 07h00
UTC, against a matrix of 6 grafana versions. Results of these builds
will be posted to `#irm-amixr-flux` Slack channel.
- local development will now be:
```bash
make build-dev-images init-k8s start-k8s
```
- `build-dev-images` - builds the engine and UI docker images (only need
to run first time)
- `init-k8s` - creates a `kind` cluster and loads the two Docker images
onto the cluster nodes (only need to run first time)
- `start-k8s` - switches `kubectl` context to the created `kind`
cluster, and uses `helm` to deploy everything as defined in
`./dev/helm-local.yml` and `./dev/helm-local.dev.yml` (that latter file
is `.gitignored` and specific to how _you_ want your setup to look like.
Hot reloading works as before. This is the _start_ of #2381. (I've
marked these `make` commands as beta, because they've not yet been
thoroughly tested for local development).
- modifies the `helm` chart to add the concept of `oncall.devMode`,
`ui`, and ability to run oncall w/ sqlite
- `oncall.devMode` will essentially just add `volumes` and
`volumeMounts` to the various engine/migrate containers +
- `ui.enabled` + `ui.env` - create a ui container (which is needed for
hot reloading locally)
- `sqlite` - this was useful for the e2e test environments where Github
runner resources are scarce. Running `mariadb` eats up precious
resources, instead lets just use sqlite here
- fixes an issue that caused sporadic HTTP 502s from the grafana
plugin-proxy, which led to flaky tests. See [this
comment](https://github.com/grafana/oncall/pull/2751/files#diff-09040e8df192699b9c5742110ebbe8d9d5c3938cb156cc1cb99fa1c3fdee4fefR72-R77)
for more context + a link to a relevant Slack conversation. **tldr;**
there is a bug with the Grafana plugin proxy in Grafana >= v10.0.3.
Let's stop using the `latest`/`main` docker tags in our test and pin to
`10.0.2` for now
- ~~re-enables the e2e test which validates a phone number via SMS, and
asserts that we can receive an alert escalation via SMS (new Mailslurp
API Key has been added as a repo secret)~~ update: this is still blocked
by procurement, will be done in a future PR
## Checklist
- [x] Unit, integration, and e2e (if applicable) tests updated
- [x] Documentation added (or `pr:no public docs` PR label added if not
required)
- [x] `CHANGELOG.md` updated (or `pr:no changelog` PR label added if not
required)
