# What this PR does
- bumps `uwsgi` to latest version (`2.0.26`), which unblocks us from
bumping Python to 3.12
- bumps Python to 3.12.3
- refactor the Snyk GitHub Actions workflow to use the composable
actions for installed frontend and backend dependencies
- fixes several `AttributeError`s in our tests that went from a warning
to an error in Python 3.12 (see
https://github.com/python/cpython/issues/100690)
# Which issue(s) this PR closes
Closes#4358
Closes https://github.com/grafana/oncall/issues/4387
# What this PR does
- Add `yarn install` to Tiltfile when running locally
- Build prod bundle of the frontend without watch mode on CI
- Remove `engine` from Tilt deps of grafana and oncall provisioning
config map so grafana can be built in parallel to engine to speed up
things
- Update dev/README.md with Node.js requirement
## Which issue(s) this PR closes
Closes https://github.com/grafana/oncall/issues/4295
<!--
*Note*: if you have more than one GitHub issue that this PR closes, be
sure to preface
each issue link with a [closing
keyword](https://docs.github.com/en/get-started/writing-on-github/working-with-advanced-formatting/using-keywords-in-issues-and-pull-requests#linking-a-pull-request-to-an-issue).
This ensures that the issue(s) are auto-closed once the PR has been
merged.
-->
## Checklist
- [ ] Unit, integration, and e2e (if applicable) tests updated
- [x] Documentation added (or `pr:no public docs` PR label added if not
required)
- [x] Added the relevant release notes label (see labels prefixed w/
`release:`). These labels dictate how your PR will
show up in the autogenerated release notes.
[pip-tools](https://github.com/jazzband/pip-tools) helps making builds
deterministic, controlling deps (and indirect deps) upgrades (and
versions consistency) avoiding unexpected (and potentially breaking)
changes.
We keep our direct deps in `requirements.in` from which we generate the
`requirements.txt` (where *all* deps are pinned). We also constrain dev
(and enterprise) deps based on base requirements.
Check how to [update
deps](https://github.com/jazzband/pip-tools?tab=readme-ov-file#updating-requirements).
# What this PR does
- autogenerate new types exposed by backend, remove custom types that
duplicate autogenerated ones
- use autogenerated types for alert receive channels
- in alert_receive_channel model:
- use autogenerate http client (`onCallApi`) for http requests
- extract methods that don't update state into
alert_receive_channel.helpers.ts and make them pure (they accept
AlertReceiveChannelStore as param) to avoid inconsistency and issues
with `this` binding
- use `makeAutoObservable`
- remove unneeded decorators
- rename update* methods to fetch* whenever such methods retrieve data
from backend with GET requests
- in other models use `@action.bound` for actions and arrow functions
for store methods that are not actions (in subsequent PRs we will apply
the same changes as in alert_receive_channel, this is just for now until
we do it)
- refactor http-client so that it shows global notification on http
errors automatically and provide the possibility to opt-out from it when
making a call
- improve type-safety of `GSelect`
- fix bug related to attaching alert group
(https://raintank-corp.slack.com/archives/C04JCU51NF8/p1707476487580579)
## Which issue(s) this PR fixes
https://github.com/grafana/oncall/issues/3331
## Checklist
- [x] Unit, integration, and e2e (if applicable) tests updated
- [x] Documentation added (or `pr:no public docs` PR label added if not
required)
- [x] `CHANGELOG.md` updated (or `pr:no changelog` PR label added if not
required)
---------
Co-authored-by: Vadim Stepanov <vadimkerr@gmail.com>
# What this PR does
## Which issue(s) this PR fixes
## Checklist
- [ ] Unit, integration, and e2e (if applicable) tests updated
- [ ] Documentation added (or `pr:no public docs` PR label added if not
required)
- [ ] `CHANGELOG.md` updated (or `pr:no changelog` PR label added if not
required)
Co-authored-by: Joey Orlando <joey.orlando@grafana.com>
# What this PR does
- introduce e2e tests in Tilt
- support e2e tests commands in Makefile
- stabilize local setup
## Which issue(s) this PR fixes
https://github.com/grafana/oncall/issues/3492
## Checklist
- [x] Unit, integration, and e2e (if applicable) tests updated
- [x] Documentation added (or `pr:no public docs` PR label added if not
required)
- [x] `CHANGELOG.md` updated (or `pr:no changelog` PR label added if not
required)
# What this PR does
Upgrade to Python 3.12 + fix several invalid test assertions that lead
to test failures in the latest version of `pytest`:
```
AttributeError: 'called_once_with' is not a valid assertion. Use a spec for the mock if 'called_once_with' is meant to be an attribute.. Did you mean: 'assert_called_once_with'?
```
## Checklist
- [ ] Unit, integration, and e2e (if applicable) tests updated
- [x] Documentation added (or `pr:no public docs` PR label added if not
required)
- [x] `CHANGELOG.md` updated (or `pr:no changelog` PR label added if not
required)
# What this PR does
https://github.com/grafana/oncall/issues/3330
- add a script that generates TS type definitions based on OnCall API
OpenAPI schemas
- support adding custom properties on the frontend if needed
- add simple example of usage (for `'/labels/keys/'` endpoint)
## Checklist
- [x] Unit, integration, and e2e (if applicable) tests updated
- [x] Documentation added (or `pr:no public docs` PR label added if not
required)
- [x] `CHANGELOG.md` updated (or `pr:no changelog` PR label added if not
required)
# What this PR does
## Which issue(s) this PR fixes
## Checklist
- [ ] Unit, integration, and e2e (if applicable) tests updated
- [ ] Documentation added (or `pr:no public docs` PR label added if not
required)
- [ ] `CHANGELOG.md` updated (or `pr:no changelog` PR label added if not
required)
# What this PR does
Runs Telegram long polling to get updates.
It's enabled by setting `FEATURE_TELEGRAM_LONG_POLLING_ENABLED=True`.
That will disable webhook and run separate deployment for telegram long
polling.
Telegram long polling is not very HA mode, but it does not need to
expose webhook url to internet and simplifies telegram integration.
## Which issue(s) this PR fixes
closes#561
## Checklist
- [x] Unit, integration, and e2e (if applicable) tests updated
- [x] Documentation added (or `pr:no public docs` PR label added if not
required)
- [x] `CHANGELOG.md` updated (or `pr:no changelog` PR label added if not
required)
# What this PR does
There are the following tests added:
- admin is allowed to edit other profiles
- editor is not allowed to edit other profiles
## Which issue(s) this PR fixes
https://github.com/grafana/oncall/issues/1586
## Checklist
- [x] Unit, integration, and e2e (if applicable) tests updated
- [ ] Documentation added (or `pr:no public docs` PR label added if not
required)
- [ ] `CHANGELOG.md` updated (or `pr:no changelog` PR label added if not
required)
---------
Co-authored-by: Rares Mardare <rares.mardare@grafana.com>
# What this PR does
As part of #1953, I wanted to be able to easily locally modify my
Grafana instance's access control provisioning configuration. In the
`grafana` container in `docker-compose-developer.yml`, we already have
this:
```yml
volumes:
- ./provisioning:/etc/grafana/provisioning
```
The `dev` directory has a `.gitignore` setup which is better suited for
what I would like to do. By default, if I add
`./dev/grafana/provisioning/access-control/custom-roles.yml`, it will be
git-ignored. This would've not been the case with the `./provisioning`
directory
*Other stuff*
- create `./dev/grafana` folder. This folder contains resources that are
volume mounted into the `grafana` container in
`docker-compose-developer.yml`.
- We already had a `./dev/grafana.dev.ini` file, relocated this here.
- Relocated `./provisioning` to `./dev/grafana/provisioning`.
- consolidate `./examples/terraform` into `./terraform` directory
## Checklist
- [ ] Unit, integration, and e2e (if applicable) tests updated (N/A)
- [ ] Documentation added (or `pr:no public docs` PR label added if not
required) (N/A)
- [ ] `CHANGELOG.md` updated (or `pr:no changelog` PR label added if not
required) (N/A)
# What this PR does
Lays ground work for #1586. Adds three new fixtures, `adminRolePage`,
`editorRolePage`, and `viewerRolePage`. These fixtures can be easily
accessed in a `test` context and allow the test to be run as a user
authenticated with one of these Grafana basic roles.
The bulk of the changes in the PR are to the "global setup" step. There
is a bit of logic + communication with the Grafana instance's API, in
order to create all the necessary authentication credentials.
Lastly, adds the first basic role authorization test, asserting that
Admin/Editors can view the list of OnCall users, whereas Viewers cannot.
## Checklist
- [x] Unit, integration, and e2e (if applicable) tests updated
- [ ] Documentation added (or `pr:no public docs` PR label added if not
required)
- [ ] `CHANGELOG.md` updated (or `pr:no changelog` PR label added if not
required)
# What this PR does
## Which issue(s) this PR fixes
## Checklist
- [ ] Unit, integration, and e2e (if applicable) tests updated
- [ ] Documentation added (or `pr:no public docs` PR label added if not
required)
- [ ] `CHANGELOG.md` updated (or `pr:no changelog` PR label added if not
required)
# What this PR does
- Improvement to the local development environment for the grafana
plugin
- Run initial yarn build inside the docker container with the same
version that is later used for periodic rebuilds
- Removes the requirement for having yarn/nodejs installed locally
- Using a named volume for storing the node_modules, so they are only
stored once
- Remove the yarn install step from the Dockerfile
- Ideally we store the node_modules only once inside the named volumes.
Currently they are stored times
- on the host system outside of dockerin grafana-plugins/node_modules
- inside the docker image
- inside the anonymous docker volume created at the start of a container
- update `node` to 18.16.0 (14.17.0 has reached end-of-life as of 3
weeks ago)
## Which issue(s) this PR fixes
## Checklist
- [X] ~Unit, integration, and e2e (if applicable) tests updated~ N/A
- [ ] Documentation added (or `pr:no public docs` PR label added if not
required)
- [ ] `CHANGELOG.md` updated (or `pr:no changelog` PR label added if not
required)
---------
Co-authored-by: Joey Orlando <joseph.t.orlando@gmail.com>
# What this PR does
Upgrades the backend to Python 3.11.3 (latest stable release) + update
linting step on Drone builds to run **all** the linting steps, not just
the Python ones.
## Checklist
- [ ] Unit, integration, and e2e (if applicable) tests updated (N/A)
- [ ] Documentation added (or `pr:no public docs` PR label added if not
required) (N/A)
- [x] `CHANGELOG.md` updated (or `pr:no changelog` PR label added if not
required)
# What this PR does
Closes#1651
Plus, add developer instructions on how to run `grafana-enterprise` with
RBAC for OnCall, enabled locally.
## Todo
- [x] add API integration test for new `permission` query param filter
## Checklist
- [x] Unit, integration, and e2e (if applicable) tests updated
- [x] Documentation added (or `pr:no public docs` PR label added if not
required)
- [x] `CHANGELOG.md` updated (or `pr:no changelog` PR label added if not
required)
# What this PR does
## Which issue(s) this PR fixes
## Checklist
- [x] Unit, integration, and e2e (if applicable) tests updated
- [x] Documentation added (or `pr:no public docs` PR label added if not
required)
- [x] `CHANGELOG.md` updated (or `pr:no changelog` PR label added if not
required)
---------
Co-authored-by: Joey Orlando <joey.orlando@grafana.com>
# What this PR does
Moved part of dev/README.md into `make help` command:
```
make help
start start all of the docker containers
init build the frontend plugin code then run make start
restart restart all docker containers
build rebuild images (e.g. when changing requirements.txt)
cleanup this will remove all of the images, containers, volumes, and networks
lint run both frontend and backend linters
test run backend tests
start-celery-beat start celery beat
purge-queues purge celery queues
shell starts an OnCall engine Django shell
dbshell opens a DB shell
engine-manage run Django's `manage.py` script, inside of a docker container, passing `$CMD` as arguments.
exec-engine exec into engine container's bash
_backend-debug-enable enable Django's debug mode and Silk profiling (this is disabled by default for performance reasons)
_backend-debug-disable disable Django's debug mode and Silk profiling
backend-manage-command run Django's `manage.py` script, passing `$CMD` as arguments.
```
## Which issue(s) this PR fixes
## Checklist
- [ ] Tests updated
- [ ] Documentation added
- [ ] `CHANGELOG.md` updated
**What this PR does**:
Adds our first UI integration test using
[Playwright](https://playwright.dev/) and runs the test on CI. Right now
the test:
- logs into Grafana
- configures the plugin (if it isn't already)
- creates an OnCall schedule, where the current user will be OnCall
- creates an escalation chain to notify based on the newly created
OnCall schedule
- creates a webhook integration, attached to the created escalation
chain
- sends a demo alert for the new integration
- goes to the alert groups page and validates that the escalation step
to alert the OnCall user actually happened
Currently the Playwright tests are run against the 3 default headless
browsers, chromium, Firefox, and webkit. The CI job that runs these
tests is run as a matrix against 3 tagged versions of `grafana`; `main`,
`latest`, and `9.2.6`.
Secondly, it adds most of the logic for a second test which:
- logs into Grafana
- configures the plugin (if it isn't already)
- goes to the user's settings, verifies their phone number (using a tool
called [MailSlurp](https://www.mailslurp.com/))
- configures the current user's default escalation policy to send alerts
via SMS
- creates an escalation policy and configures it to send alerts to our
current user
- creates an integration and assigns the created escalation policy
- triggers a test alert + verifies that we receive the SMS alert text
(again, using MailSlurp)
**Which issue(s) this PR fixes**:
Closes#873
**Checklist**
- [x] Tests updated
- [ ] Documentation added (N/A)
- [ ] `CHANGELOG.md` updated (N/A)
# What this PR does
This PR adds
[django-migration-linter](https://github.com/3YOURMIND/django-migration-linter)
to keep database migrations
backwards compatible
- we can automatically run migrations and they are zero-downtime, e.g.
old code can work with the migrated database
- we can run and rollback migrations without worrying about data safety
- OnCall is deployed to the multiple environments core team is not able
to control
See [django-migration-linter
checklist](https://github.com/3YOURMIND/django-migration-linter/blob/main/docs/incompatibilities.md)
for the common mistakes and best practices
## Which issue(s) this PR fixes
## Checklist
- [ ] Tests updated
- [ ] Documentation added
- [ ] `CHANGELOG.md` updated
---------
Co-authored-by: Joey Orlando <joey.orlando@grafana.com>
# What this PR does
Remove checks for `mobile_app_settings` DynamicSetting, so changing
`FEATURE_MOBILE_APP_INTEGRATION_ENABLED` is enough for toggling the
mobile app backend (aka remove per-org feature flag)
Co-authored-by: Joey Orlando <joey.orlando@grafana.com>
# What this PR does
Fixes an issue when a local dev setup becomes extremely slow.
- Set `DEBUG` and `SILK_PROFILER_ENABLED` to `False` by default + add
utility make commands to toggle it
- Use `uwsgi` instead of Django's built-in `runserver` for local dev
setup
- Limit Celery concurrency to 3 for local dev setup (previously was 20,
used >1GB RAM on my machine)
---------
Co-authored-by: Joey Orlando <joey.orlando@grafana.com>
# What this PR does
## Main stuff
- add Python script to populate local Grafana/OnCall setup w/ large
amounts of fake data. Right now the data types that can be generated
are:
- teams and Admin users via the Grafana API (must be synced manually by
going into the UI before going onto the next step)
- Calendar Schedules which have three 8h oncall-shifts, via the OnCall
public API
- fixes `django-debug-toolbar` when being run in `docker-compose`
locally
## Other stuff
- documents how to easily modify the Grafana `docker-compose` container
provisioning configuration
- document solutions for two backend setup related issues encountered
when running the engine/celery workers locally, outside of
`docker-compose`, on an Apple silicon Mac
- fixes small bug in `grafana_plugin.helpers.client.APIClient.call_api`
where it would call `response.json()` for all requests, regardless of
whether or not the response actually contained data or not
- in `engine/settings/dev.py`, properly setup `django-silk` and document
the steps to use it locally
- make it possible to log out debug SQL queries by specifying
`DEV_DEBUG_VIEW_SQL_QUERIES` env var, rather than having to uncomment
out a section of `settings/dev.py`
## Which issue(s) this PR fixes
- Some local setup issues when trying to use `django-silk` and
`django-debug-toolbar`
- Makes it much easier to populate your local setup with a lot of fake
data
- Makes it possible to easily modify your local grafana's provisioning
configuration
## Checklist
- [ ] Tests updated (N/A)
- [ ] Documentation added (N/A)
- [ ] `CHANGELOG.md` updated (N/A)
- swaps out `django-push-notifications` for
[`fcm-django`](https://github.com/grafana/fcm-django). Again.. this is a
fork of the parent repo for exactly the same reason.. the migrations
point to `auth_user` without letting us use our own user model, this has
been patched in the `grafana` fork. The reason why we are using
`fcm-django` vs `django-push-notifications` is that the latter does not
support the new FCM API, only the "legacy" API. The legacy FCM API does
not support certain push notification settings that we would like to
use.
- modifies the iOS/Android specific push notification settings
- adds a `flower` pod in the `docker-compose-developer.yml`, useful for
debugging tasks locally
- sets the mobile app verification token TTL to 5 minutes when
developing locally. The default of 1 minute makes working with device
emulators really tricky..
This PR also swaps out the base image in `engine/Dockerfile` from
`python:3.9-alpine3.16` to `python:3.9-slim-buster`.
As to why.. in short, with the introduction of the `fcm-django` library
there is now a peer-dependency on
[`grpcio`](https://github.com/grpc/grpc) (which is used by
`firebase_admin`.. which I am using in this PR to interact directly with
Firebase Cloud Messaging (FCM)). `grpcio` does not publish wheels (read:
compiled binaries) for the Alpine distro. It does publish wheels for
Debian and hence `pip install -r requirements.txt` does not need to
build this library from the source distribution.
This is a [known
"issue"](https://github.com/grpc/grpc/issues/22815#issuecomment-1107874367)
and the recommended solution in the community is to.. not use alpine.
These were the numbers, when building the image locally, in terms of
image size and build time:
| | Local image size (uncompressed | Build time (may differ based on
your network speed) |
| ------------------------- | -------------------------------------- |
---------- |
| `python:3.9-alpine3.16` | 785MB | 320s |
| `python:3.9-slim-buster` | 1.05GB | 90s |
Co-authored-by: Salvatore Giordano <salvatoregiordanoo@gmail.com>
Adds a make command, `enable-mobile-app-feature-flags`, which sets the
backend feature flag in `./dev/.env.dev`, and updates a record in the
`base_dynamicsetting` database table, which are needed to enable the
mobile app backend features.
* Centralize timezone validation into one spot + add serializer validation
for schedules and oncall shifts (both public and internal API)
* add engine-manage make command
