oncall-engine/dev/helm-local.yml
Joey Orlando 3783aeab64
fix a few flaky e2e tests + allow running project locally via k8s/helm (#2751)
# What this PR does

- updates the GitHub Actions workflow to move the e2e tests into a
"[reusable
workflow](https://docs.github.com/en/actions/using-workflows/reusing-workflows#creating-a-reusable-workflow)"
which are run in two scenarios:
- all tests _except_ those annotated as `@expensive` are run against
`grafana/grafana:latest` on all feature branches
- all tests _including_ `@expensive` tests are run on weekdays @ 07h00
UTC, against a matrix of 6 grafana versions. Results of these builds
will be posted to `#irm-amixr-flux` Slack channel.
- local development will now be:
  ```bash
  make build-dev-images init-k8s start-k8s
  ```
- `build-dev-images` - builds the engine and UI docker images (only need
to run first time)
- `init-k8s` - creates a `kind` cluster and loads the two Docker images
onto the cluster nodes (only need to run first time)
- `start-k8s` - switches `kubectl` context to the created `kind`
cluster, and uses `helm` to deploy everything as defined in
`./dev/helm-local.yml` and `./dev/helm-local.dev.yml` (that latter file
is `.gitignored` and specific to how _you_ want your setup to look like.
Hot reloading works as before. This is the _start_ of #2381. (I've
marked these `make` commands as beta, because they've not yet been
thoroughly tested for local development).
- modifies the `helm` chart to add the concept of `oncall.devMode`,
`ui`, and ability to run oncall w/ sqlite
- `oncall.devMode` will essentially just add `volumes` and
`volumeMounts` to the various engine/migrate containers +
- `ui.enabled` + `ui.env` - create a ui container (which is needed for
hot reloading locally)
- `sqlite` - this was useful for the e2e test environments where Github
runner resources are scarce. Running `mariadb` eats up precious
resources, instead lets just use sqlite here
- fixes an issue that caused sporadic HTTP 502s from the grafana
plugin-proxy, which led to flaky tests. See [this
comment](https://github.com/grafana/oncall/pull/2751/files#diff-09040e8df192699b9c5742110ebbe8d9d5c3938cb156cc1cb99fa1c3fdee4fefR72-R77)
for more context + a link to a relevant Slack conversation. **tldr;**
there is a bug with the Grafana plugin proxy in Grafana >= v10.0.3.
Let's stop using the `latest`/`main` docker tags in our test and pin to
`10.0.2` for now
- ~~re-enables the e2e test which validates a phone number via SMS, and
asserts that we can receive an alert escalation via SMS (new Mailslurp
API Key has been added as a repo secret)~~ update: this is still blocked
by procurement, will be done in a future PR

## Checklist

- [x] Unit, integration, and e2e (if applicable) tests updated
- [x] Documentation added (or `pr:no public docs` PR label added if not
required)
- [x] `CHANGELOG.md` updated (or `pr:no changelog` PR label added if not
required)
2023-08-22 19:03:29 +02:00

126 lines
3.6 KiB
YAML

base_url: localhost:30001
base_url_protocol: http
env:
- name: GRAFANA_CLOUD_NOTIFICATIONS_ENABLED
value: "False"
image:
repository: oncall/engine
tag: dev
pullPolicy: IfNotPresent
broker:
type: redis
redis:
architecture: standalone # don't run replicas, just eats up resources
image:
repository: redis # the default, bitnami/redis does not support ARM64
tag: 7.0.5
rabbitmq:
enabled: false
oncall:
devMode: true
engine:
replicaCount: 1
celery:
replicaCount: 1
ui:
enabled: true
env:
ONCALL_API_URL: http://oncall-dev-engine-external:8080
MOBILE_APP_QR_INTERVAL_QUEUE: 290000 # 4 minutes and 50 seconds
grafana:
replicas: 1
image:
tag: 10.0.2
extraInitContainers:
- name: create-db-if-not-exists
image: mysql:8.0.32
command:
# yamllint disable rule:line-length
[
"bash",
"-c",
'while ! mysqladmin ping -h "$DATABASE_HOST" --silent; do echo ''awaiting mysql db to be available'' && sleep 1; done && mysql -h "$DATABASE_HOST" -u "$DATABASE_USER" -p"$DATABASE_PASSWORD" -e ''CREATE DATABASE IF NOT EXISTS grafana CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;''',
]
# yamllint enable rule:line-length
env:
- name: DATABASE_HOST
value: oncall-dev-mariadb
- name: DATABASE_USER
value: root
- name: DATABASE_PASSWORD
valueFrom:
secretKeyRef:
name: oncall-dev-mariadb
key: mariadb-root-password
env:
GF_FEATURE_TOGGLES_ENABLE: topnav
GF_SECURITY_ADMIN_PASSWORD: oncall
GF_SECURITY_ADMIN_USER: oncall
GF_PLUGINS_ALLOW_LOADING_UNSIGNED_PLUGINS: grafana-oncall-app
GF_DATABASE_TYPE: mysql
GF_DATABASE_HOST: oncall-dev-mariadb:3306
GF_DATABASE_USER: root
GF_DATABASE_SSL_MODE: disable
envValueFrom:
GF_DATABASE_PASSWORD:
secretKeyRef:
name: oncall-dev-mariadb
key: mariadb-root-password
# by settings grafana.plugins to [] and configuring grafana.extraVolumeMounts we are using the locally built
# OnCall plugin rather than the latest published version
plugins: []
# hostPaths are defined in dev/kind.yml
extraVolumeMounts:
- name: plugins
mountPath: /var/lib/grafana/plugins/grafana-plugin
hostPath: /oncall-plugin
readOnly: true
- name: configuration-file
mountPath: /var/lib/grafana/grafana.ini
hostPath: /dev-config/grafana/grafana.dev.ini
readOnly: true
- name: provisioning
mountPath: /var/lib/grafana/provisioning
hostPath: /dev-config/grafana/provisioning
readOnly: true
service:
type: NodePort
nodePort: 30002
database:
type: mysql
mariadb:
enabled: true
primary:
service:
type: NodePort
nodePort: 30003
extraEnvVars:
# See "Passing extra command line flags to mysqld startup" section
# https://hub.docker.com/r/bitnami/mariadb
#
# max_allowed_packet is set to 128mb in bytes
#
# this avoids "Got an error reading communication packets" errors that arise from the grafana container
# apparently sending too much data to mariadb at once
# https://mariadb.com/docs/skysql-dbaas/ref/mdb/system-variables/max_allowed_packet/
- name: MARIADB_EXTRA_FLAGS
value: "--max_allowed_packet=134217728 --max_connections=1024"
- name: MARIADB_CHARACTER_SET
value: utf8mb4
- name: MARIADB_COLLATE
value: utf8mb4_unicode_ci
ingress:
enabled: false
ingress-nginx:
enabled: false
cert-manager:
enabled: false
service:
enabled: true
type: NodePort
port: 8080
nodePort: 30001