# What this PR does
Upgrades the backend to Python 3.11.3 (latest stable release) + update
linting step on Drone builds to run **all** the linting steps, not just
the Python ones.
## Checklist
- [ ] Unit, integration, and e2e (if applicable) tests updated (N/A)
- [ ] Documentation added (or `pr:no public docs` PR label added if not
required) (N/A)
- [x] `CHANGELOG.md` updated (or `pr:no changelog` PR label added if not
required)
# What this PR does
Closes#1651
Plus, add developer instructions on how to run `grafana-enterprise` with
RBAC for OnCall, enabled locally.
## Todo
- [x] add API integration test for new `permission` query param filter
## Checklist
- [x] Unit, integration, and e2e (if applicable) tests updated
- [x] Documentation added (or `pr:no public docs` PR label added if not
required)
- [x] `CHANGELOG.md` updated (or `pr:no changelog` PR label added if not
required)
# What this PR does
## Which issue(s) this PR fixes
## Checklist
- [x] Unit, integration, and e2e (if applicable) tests updated
- [x] Documentation added (or `pr:no public docs` PR label added if not
required)
- [x] `CHANGELOG.md` updated (or `pr:no changelog` PR label added if not
required)
---------
Co-authored-by: Joey Orlando <joey.orlando@grafana.com>
# What this PR does
Moved part of dev/README.md into `make help` command:
```
make help
start start all of the docker containers
init build the frontend plugin code then run make start
restart restart all docker containers
build rebuild images (e.g. when changing requirements.txt)
cleanup this will remove all of the images, containers, volumes, and networks
lint run both frontend and backend linters
test run backend tests
start-celery-beat start celery beat
purge-queues purge celery queues
shell starts an OnCall engine Django shell
dbshell opens a DB shell
engine-manage run Django's `manage.py` script, inside of a docker container, passing `$CMD` as arguments.
exec-engine exec into engine container's bash
_backend-debug-enable enable Django's debug mode and Silk profiling (this is disabled by default for performance reasons)
_backend-debug-disable disable Django's debug mode and Silk profiling
backend-manage-command run Django's `manage.py` script, passing `$CMD` as arguments.
```
## Which issue(s) this PR fixes
## Checklist
- [ ] Tests updated
- [ ] Documentation added
- [ ] `CHANGELOG.md` updated
# What this PR does
This PR:
- modifies the `check_escalation_finished_task` celery task to:
- do stricter escalation validation based on the alert group's
escalation snapshot (see the `audit_alert_group_escalation` method in
`engine/apps/alerts/tasks/check_escalation_finished.py` for the
validation logic)
- use a read-only database for querying alert-groups if one is
configured, otherwise use the "default" one
- ping a configurable heartbeat (new env var
`ALERT_GROUP_ESCALATION_AUDITOR_CELERY_TASK_HEARTBEAT_URL` added)
- increase the task frequency from every 10 to every 13 minutes (this
can be configured via an env variable)
- adds public documentation on how to configure this auditor task
- modifies the local celery startup command to properly take into
consideration all celery related env vars (similar to the ones we use in
`engine/celery_with_exporter.sh`; this made it easier to enable `celery
beat` locally for testing)
- removes the following code:
- removes references to `AlertGroup.estimate_escalation_finish_time` and
marks the model field as deprecated using the [`django-deprecate-fields`
library](https://pypi.org/project/django-deprecate-fields/). This field
was only used for the previous version of this validation task
- `EscalationSnapshotMixin.calculate_eta_for_finish_escalation` was only
used to calculate the value for
`AlertGroup.estimate_escalation_finish_time`
- `calculate_escalation_finish_time` celery task
## Which issue(s) this PR fixes
https://github.com/grafana/oncall-private/issues/1558
## Checklist
- [x] Tests updated
- [x] Documentation added
- [x] `CHANGELOG.md` updated
It's a duplicate of LICENSE env var
**What this PR does**:
Remove OSS_INSTALLATION env var in favour of LICENSE env var. Also, I
refactored features tests a little. From my point of view it makes
little sense to test if all features are disabled or enabled. Better to
test specific use-case (e.g. oss installation).
Also to test that all features are disabled it is needed to set LICENSE
equals cloud license, which makes test confusing.
**Checklist**
- [x] Tests updated
- [ ] Documentation added
- [ ] `CHANGELOG.md` updated
**What this PR does**:
Adds our first UI integration test using
[Playwright](https://playwright.dev/) and runs the test on CI. Right now
the test:
- logs into Grafana
- configures the plugin (if it isn't already)
- creates an OnCall schedule, where the current user will be OnCall
- creates an escalation chain to notify based on the newly created
OnCall schedule
- creates a webhook integration, attached to the created escalation
chain
- sends a demo alert for the new integration
- goes to the alert groups page and validates that the escalation step
to alert the OnCall user actually happened
Currently the Playwright tests are run against the 3 default headless
browsers, chromium, Firefox, and webkit. The CI job that runs these
tests is run as a matrix against 3 tagged versions of `grafana`; `main`,
`latest`, and `9.2.6`.
Secondly, it adds most of the logic for a second test which:
- logs into Grafana
- configures the plugin (if it isn't already)
- goes to the user's settings, verifies their phone number (using a tool
called [MailSlurp](https://www.mailslurp.com/))
- configures the current user's default escalation policy to send alerts
via SMS
- creates an escalation policy and configures it to send alerts to our
current user
- creates an integration and assigns the created escalation policy
- triggers a test alert + verifies that we receive the SMS alert text
(again, using MailSlurp)
**Which issue(s) this PR fixes**:
Closes#873
**Checklist**
- [x] Tests updated
- [ ] Documentation added (N/A)
- [ ] `CHANGELOG.md` updated (N/A)
# What this PR does
This PR adds
[django-migration-linter](https://github.com/3YOURMIND/django-migration-linter)
to keep database migrations
backwards compatible
- we can automatically run migrations and they are zero-downtime, e.g.
old code can work with the migrated database
- we can run and rollback migrations without worrying about data safety
- OnCall is deployed to the multiple environments core team is not able
to control
See [django-migration-linter
checklist](https://github.com/3YOURMIND/django-migration-linter/blob/main/docs/incompatibilities.md)
for the common mistakes and best practices
## Which issue(s) this PR fixes
## Checklist
- [ ] Tests updated
- [ ] Documentation added
- [ ] `CHANGELOG.md` updated
---------
Co-authored-by: Joey Orlando <joey.orlando@grafana.com>
# What this PR does
Remove checks for `mobile_app_settings` DynamicSetting, so changing
`FEATURE_MOBILE_APP_INTEGRATION_ENABLED` is enough for toggling the
mobile app backend (aka remove per-org feature flag)
Co-authored-by: Joey Orlando <joey.orlando@grafana.com>
# What this PR does
Fixes an issue when a local dev setup becomes extremely slow.
- Set `DEBUG` and `SILK_PROFILER_ENABLED` to `False` by default + add
utility make commands to toggle it
- Use `uwsgi` instead of Django's built-in `runserver` for local dev
setup
- Limit Celery concurrency to 3 for local dev setup (previously was 20,
used >1GB RAM on my machine)
---------
Co-authored-by: Joey Orlando <joey.orlando@grafana.com>
# What this PR does
## Main stuff
- add Python script to populate local Grafana/OnCall setup w/ large
amounts of fake data. Right now the data types that can be generated
are:
- teams and Admin users via the Grafana API (must be synced manually by
going into the UI before going onto the next step)
- Calendar Schedules which have three 8h oncall-shifts, via the OnCall
public API
- fixes `django-debug-toolbar` when being run in `docker-compose`
locally
## Other stuff
- documents how to easily modify the Grafana `docker-compose` container
provisioning configuration
- document solutions for two backend setup related issues encountered
when running the engine/celery workers locally, outside of
`docker-compose`, on an Apple silicon Mac
- fixes small bug in `grafana_plugin.helpers.client.APIClient.call_api`
where it would call `response.json()` for all requests, regardless of
whether or not the response actually contained data or not
- in `engine/settings/dev.py`, properly setup `django-silk` and document
the steps to use it locally
- make it possible to log out debug SQL queries by specifying
`DEV_DEBUG_VIEW_SQL_QUERIES` env var, rather than having to uncomment
out a section of `settings/dev.py`
## Which issue(s) this PR fixes
- Some local setup issues when trying to use `django-silk` and
`django-debug-toolbar`
- Makes it much easier to populate your local setup with a lot of fake
data
- Makes it possible to easily modify your local grafana's provisioning
configuration
## Checklist
- [ ] Tests updated (N/A)
- [ ] Documentation added (N/A)
- [ ] `CHANGELOG.md` updated (N/A)
- swaps out `django-push-notifications` for
[`fcm-django`](https://github.com/grafana/fcm-django). Again.. this is a
fork of the parent repo for exactly the same reason.. the migrations
point to `auth_user` without letting us use our own user model, this has
been patched in the `grafana` fork. The reason why we are using
`fcm-django` vs `django-push-notifications` is that the latter does not
support the new FCM API, only the "legacy" API. The legacy FCM API does
not support certain push notification settings that we would like to
use.
- modifies the iOS/Android specific push notification settings
- adds a `flower` pod in the `docker-compose-developer.yml`, useful for
debugging tasks locally
- sets the mobile app verification token TTL to 5 minutes when
developing locally. The default of 1 minute makes working with device
emulators really tricky..
This PR also swaps out the base image in `engine/Dockerfile` from
`python:3.9-alpine3.16` to `python:3.9-slim-buster`.
As to why.. in short, with the introduction of the `fcm-django` library
there is now a peer-dependency on
[`grpcio`](https://github.com/grpc/grpc) (which is used by
`firebase_admin`.. which I am using in this PR to interact directly with
Firebase Cloud Messaging (FCM)). `grpcio` does not publish wheels (read:
compiled binaries) for the Alpine distro. It does publish wheels for
Debian and hence `pip install -r requirements.txt` does not need to
build this library from the source distribution.
This is a [known
"issue"](https://github.com/grpc/grpc/issues/22815#issuecomment-1107874367)
and the recommended solution in the community is to.. not use alpine.
These were the numbers, when building the image locally, in terms of
image size and build time:
| | Local image size (uncompressed | Build time (may differ based on
your network speed) |
| ------------------------- | -------------------------------------- |
---------- |
| `python:3.9-alpine3.16` | 785MB | 320s |
| `python:3.9-slim-buster` | 1.05GB | 90s |
Co-authored-by: Salvatore Giordano <salvatoregiordanoo@gmail.com>
Adds a make command, `enable-mobile-app-feature-flags`, which sets the
backend feature flag in `./dev/.env.dev`, and updates a record in the
`base_dynamicsetting` database table, which are needed to enable the
mobile app backend features.
* Centralize timezone validation into one spot + add serializer validation
for schedules and oncall shifts (both public and internal API)
* add engine-manage make command
* Get rid of installation token (for OSS installations)
This is done by being required to supply the grafana API URL as an
environment variable on the backend. Additionally, optionally an OnCall
API URL environment variable can be passed in to the frontend (this basically
allows completely skipping the need to configure anything).
- deduplicated a lot of the sync logic on the frontend + made
error message more useful and consistent
- Split PluginConfigPage component into several subcomponents
(making it easier to test each individual component)
- Moved RootWithLoader (from plugin/GrafanaPluginRootPage) into its own
subcomponent (making it easier to test)
- Added tests for pre-existing components that were touched:
- PluginConfigPage component (and its new subcomponents)
- state/plugin and state/rootBaseStore functions
- apps.grafana_plugin django app
Helm changes:
- add GRAFANA_API_URL to oncall.env
- some yaml autoformatting changes
- remove reference to python manage.py issue_invite_for_the_frontend --override
Co-authored-by: Joey Orlando <joseph.t.orlando@gmail.com>
Note - you may be wondering "how could your system possibly function if
`.` is not part of `CDPATH`? Testing suggests that behaviour is
inconsistent between shells - `sh` will entirely ignore the current
directory if `.` is absent, but `zsh` will still attempt to search the
current directory even if `.` is not in `CDPATH`
* support enterprise development in docker
* fix flaky mysql healthcheck command
I was getting the mysql_to_create_grafana_db and oncall_db_migration prematurely starting up
this commit changes the healthcheck used here to
be the same as what is used in docker-compose-mysql-rabbitmq.yml
* upgrade docker-compose config files to 3.9
3.8 does not actually support the "long form" version of depends_on
see here for more info https://stackoverflow.com/a/54249757https://docs.docker.com/compose/compose-file/compose-file-v3/#depends_on
* add make init command and update documentation
* cleanup gitignore files
* Modify `docker-compose-developer` configuration files, and `Makefile`
to support running everything in containers for local development
- Make use of the COMPOSE_PROFILES env var that is supported by
docker-compose to allow swapping-out/turning off certain docker-compose
services.
- add makefile cleanup command. Will remove all docker resources related
to running the project locally
- The "restart grafana container" issue, where users would need
to restart their grafana container when setting up the project for the
first time, is now fixed (make command now runs yarn build:dev before docker-compose startup;
this ensures grafana-plugin/dist is available for grafana container before it starts up)
- The DEVELOPER.md has been updated as well to reflect these new changes. It
has been moved to ./dev/README.md (and references to the old file have
been updated).
- The redis image that is referenced in the docker-compose files
has been pinned to v7.0.5 (latest version as of this commit) to avoid
any surprises w/ future releases.
- remove root .dockerignore in favour of individual .dockerignore files
in ./engine and ./grafana-plugin
