# What this PR does
- update `make test` to always use `settings.ci-test`. Right now it will
use whatever the value of `DJANGO_SETTINGS_MODULE` is in
`./dev/.env.dev`, which causes ~45 tests to fail
- Fix several Python warnings that we see when running the tests
```bash
RemovedInDjango40Warning: The providing_args argument is deprecated. As it is purely documentational, it has no replacement. If you rely on this argument as documentation, you can move the text to a code comment or docstring.
alert_create_signal = django.dispatch.Signal(
```
```bash
PytestCollectionWarning: cannot collect test class 'TestOnlyBackend' because it has a __init__ constructor (from: apps/api/tests/test_alert_receive_channel_template.py)
class TestOnlyBackend(BaseMessagingBackend):
```
```bash
DeprecationWarning: The parameter 'use_aliases' in emoji.emojize() is deprecated and will be removed in version 2.0.0. Use language='alias' instead.
To hide this warning, pin/downgrade the package to 'emoji~=1.6.3'
return emoji.emojize(self.verbal_name, use_aliases=True)
```
```bash
DateTimeField CustomOnCallShift.start received a naive datetime (2023-06-01 12:53:12) while time zone support is active.
warnings.warn("DateTimeField %s received a naive datetime (%s)"
```
```bash
apps/twilioapp/tests/test_phone_calls.py::test_resolve_by_phone
/etc/app/apps/twilioapp/tests/test_phone_calls.py:173: DeprecationWarning: The 'text' argument to find()-type methods is deprecated. Use 'string' instead.
content = BeautifulSoup(content, features="html.parser").findAll(text=True)
```
```bash
apps/twilioapp/tests/test_phone_calls.py::test_resolve_by_phone
apps/twilioapp/tests/test_phone_calls.py::test_wrong_pressed_digit
/usr/local/lib/python3.11/site-packages/bs4/builder/__init__.py:545: XMLParsedAsHTMLWarning: It looks like you're parsing an XML document using an HTML parser. If this really is an HTML document (maybe it's XHTML?), you can ignore or filter this warning. If it's XML, you should know that using an XML parser will be more reliable. To parse this document as XML, make sure you have the lxml package installed, and pass the keyword argument `features="xml"` into the BeautifulSoup constructor.
```
```bash
apps/twilioapp/tests/test_phone_calls.py::test_forbidden_requests
/usr/local/lib/python3.11/site-packages/social_django/urls.py:15: RemovedInDjango40Warning: django.conf.urls.url() is deprecated in favor of django.urls.re_path().
url(r'^login/(?P<backend>[^/]+){0}$'.format(extra), views.auth,
```
```bash
apps/twilioapp/tests/test_phone_calls.py: 66 warnings
/usr/local/lib/python3.11/site-packages/debug_toolbar/utils.py:255: DeprecationWarning: currentThread() is deprecated, use current_thread() instead
thread = threading.currentThread()
```
## Checklist
- [x] Unit, integration, and e2e (if applicable) tests updated
- [x] Documentation added (or `pr:no public docs` PR label added if not
required)
- [ ] `CHANGELOG.md` updated (or `pr:no changelog` PR label added if not
required)
# What this PR does
Allows fetching inbound email address for an integration using public
API. There's a new field `inbound_email` that's only defined for inbound
email integration and `null` for any other integration types.
## Which issue(s) this PR fixes
https://github.com/grafana/oncall/issues/1989
## Checklist
- [x] Unit, integration, and e2e (if applicable) tests updated
- [x] Documentation added (or `pr:no public docs` PR label added if not
required)
- [x] `CHANGELOG.md` updated (or `pr:no changelog` PR label added if not
required)
When checking current shifts to trigger slack notifications, a stable
value for timezone should be used (to avoid triggering multiple
duplicated notifications; using the first `VTIMEZONE` value available
may change on refresh, besides being potentially incorrect).
Also, `VTIMEZONE` shouldn't be used as the calendar timezone, it just
describes a [timezone
definition](https://icalendar.org/iCalendar-RFC-5545/3-6-5-time-zone-component.html)
which can later be [used when specifying a start/end
date/datetime](https://icalendar.org/iCalendar-RFC-5545/3-2-19-time-zone-identifier.html)
("The parameter MUST be specified on properties with a DATE-TIME value
if the DATE-TIME is not either a UTC or a "floating" time.").
OTOH, Google uses the non-standard[ `X-WR-TIMEZONE`
](https://github.com/collective/icalendar/issues/343)field as fallback
TZ for date-times without a `TZID`. Try to use this when available,
otherwise fallback to `UTC`.
# What this PR does
Fix HTTP 500 when sending demo alert for inbound integration email.
- Make all integration configs have consistent `is_demo_alert_enabled`
and `example_payload` values
- Add tests
## Checklist
- [x] Unit, integration, and e2e (if applicable) tests updated
- [x] Documentation added (or `pr:no public docs` PR label added if not
required)
- [x] `CHANGELOG.md` updated (or `pr:no changelog` PR label added if not
required)
Before this change, a diff ical check (which happens with frequency with
imported ical), particularly with overrides in an API/terraform schedule
would trigger unexpected slack notifications because the prev vs current
ical comparison will flag a diff, but when comparing current and
previous shifts, `current_shifts` will have the shift in progress while
the `prev_shifts` calculated from the overrides-only diff will most of
the time be empty (unless you set/change an override at current time).
Simplified the checks to always compare previous current shifts (ie. the
ones in the schedule from the DB) vs the recalculated ones using the
(refreshed) ical data from the schedule.
# What this PR does
Reworks Slack handlers for buttons and select menus for AG Slack
messages.
<img width="602" alt="Screenshot 2023-05-31 at 19 34 05"
src="https://github.com/grafana/oncall/assets/20116910/857bf096-7bdd-427b-94b6-15aad873a8ac">
## Current implementation
- It's possible to end up with orphaned Slack messages that are posted
to Slack but have no `SlackMessage` instance in the DB. For such
messages, clicking buttons will result in an exception and HTTP 500. See
private repo
[issue](https://github.com/grafana/oncall-private/issues/1841) for more
info.
- Bug in authorization system, which effectively bypasses any permission
checks. For example, it's possible to resolve an alert group while being
a Viewer.
- No tests covering most buttons.
## Changes in this PR
- Make the system more robust, don't use `SlackMessage` model to figure
out the alert group being interacted on, instead embed `alert_group_pk`
to every button and use it when receiving interaction requests from
Slack.
- Existing orphaned Slack messages will be repaired. Clicking buttons
under orphaned messages will work (and missing `SlackMessage` instance
will be created on interaction). This is possible because some buttons
already have `alert_group_pk` embedded, and it's possible to get this
data on button clicks (even if the clicked button itself doesn't have
`alert_group_pk` embedded).
- Fix authorization. Show warning window when unauthorized:
<img width="511" alt="Screenshot 2023-05-31 at 19 40 02"
src="https://github.com/grafana/oncall/assets/20116910/5abeeaa7-1b61-4a47-b3af-0e21d5cd1907">
- Added tests for all the buttons under AG message. Add tests checking
authorization, actual execution of scenario steps, orphan message
repairing, backward compatibility, etc. Also add tests on
`AlertGroupSlackRenderer` checking that correct data is embedded into
buttons.
- Cosmetic changes such as renaming `incident` to `Alert Group`.
## Which issue(s) this PR fixes
Related to https://github.com/grafana/oncall-private/issues/1841
## Checklist
- [x] Unit, integration, and e2e (if applicable) tests updated
- [x] Documentation added (or `pr:no public docs` PR label added if not
required)
- [x] `CHANGELOG.md` updated (or `pr:no changelog` PR label added if not
required)
# What this PR does
## Which issue(s) this PR fixes
## Checklist
- [ ] Unit, integration, and e2e (if applicable) tests updated
- [ ] Documentation added (or `pr:no public docs` PR label added if not
required)
- [ ] `CHANGELOG.md` updated (or `pr:no changelog` PR label added if not
required)
---------
Co-authored-by: Yulia Shanyrova <yulia.shanyrova@grafana.com>
Fixes issue when syncing contact points and there are receiver configs
with no `grafana_managed_receiver_configs` key.
(eg. `{"name": "autogen-contact-point-default"}`)
# What this PR does
This PR moves phone notification logic into separate object PhoneBackend
and introduces PhoneProvider interface to hide actual implementation of
external phone services provider. It should allow add new phone
providers just by implementing one class (See SimplePhoneProvider for
example).
# Why
[Asterisk PR](https://github.com/grafana/oncall/pull/1282) showed that
our phone notification system is not flexible. However this is one of
the most frequent community questions - how to add "X" phone provider.
Also, this refactoring move us one step closer to unifying all
notification backends, since with PhoneBackend all phone notification
logic is collected in one place and independent from concrete
realisation.
# Highligts
1. PhoneBackend object - contains all phone notifications business
logic.
2. PhoneProvider - interface to external phone services provider.
3. TwilioPhoneProvider and SimplePhoneProvider - two examples of
PhoneProvider implementation.
4. PhoneCallRecord and SMSRecord models. I introduced these models to
keep phone notification limits logic decoupled from external providers.
Existing TwilioPhoneCall and TwilioSMS objects will be migrated to the
new table to not to reset limits counter. To be able to receive status
callbacks and gather from Twilio TwilioPhoneCall and TwilioSMS still
exists, but they are linked to PhoneCallRecord and SMSRecord via fk, to
not to leat twilio logic into core code.
---------
Co-authored-by: Yulia Shanyrova <yulia.shanyrova@grafana.com>
# What this PR does
## Which issue(s) this PR fixes
## Checklist
- [ ] Unit, integration, and e2e (if applicable) tests updated
- [ ] Documentation added (or `pr:no public docs` PR label added if not
required)
- [ ] `CHANGELOG.md` updated (or `pr:no changelog` PR label added if not
required)
# What this PR does
`SOURCE_CHOICES` was updated in #1779 but we forgot to include the
accompanying database migration. If you run `make engine-manage
CMD="makemigrations"`, this migration is output. This PR simply adds
that.
## Checklist
- [ ] Unit, integration, and e2e (if applicable) tests updated (N/A)
- [ ] Documentation added (or `pr:no public docs` PR label added if not
required) (N/A)
- [ ] `CHANGELOG.md` updated (or `pr:no changelog` PR label added if not
required) (N/A)
# What this PR does
## Which issue(s) this PR fixes
## Checklist
- [ ] Unit, integration, and e2e (if applicable) tests updated
- [ ] Documentation added (or `pr:no public docs` PR label added if not
required)
- [ ] `CHANGELOG.md` updated (or `pr:no changelog` PR label added if not
required)
# What this PR does
<img width="521" alt="Screenshot 2023-04-28 at 5 36 10 PM"
src="https://user-images.githubusercontent.com/2262529/235112636-56fe0b48-1cda-4ba7-8a09-1cfb0ced2222.png">
## Which issue(s) this PR fixes
## Checklist
- [ ] Unit, integration, and e2e (if applicable) tests updated
- [ ] Documentation added (or `pr:no public docs` PR label added if not
required)
- [ ] `CHANGELOG.md` updated (or `pr:no changelog` PR label added if not
required)
# What this PR does
https://www.loom.com/share/1a6ef0d00c3b46ca80c120579d512dcc
## Checklist
- [ ] Unit, integration, and e2e (if applicable) tests updated (N/A)
- [x] Documentation added (or `pr:no public docs` PR label added if not
required)
- [x] `CHANGELOG.md` updated (or `pr:no changelog` PR label added if not
required)
# What this PR does
This PR adds additional silence options in the UI. Currently we only
have 1 hour, 4 hours and 12 hours silence options. I think it's worth it
to have finer silence options.
## Which issue(s) this PR fixes
No issue ticket but I can create one.
## Checklist
- [ ] Unit, integration, and e2e (if applicable) tests updated
- [x] Documentation added (or `pr:no public docs` PR label added if not
required)
- [x] `CHANGELOG.md` updated (or `pr:no changelog` PR label added if not
required)
# What this PR does
## Which issue(s) this PR fixes
## Checklist
- [ ] Unit, integration, and e2e (if applicable) tests updated
- [ ] Documentation added (or `pr:no public docs` PR label added if not
required)
- [ ] `CHANGELOG.md` updated (or `pr:no changelog` PR label added if not
required)
# What this PR does
This PR set the limit so that workers won't attempt to autoresolve too
big alertmanager alert groups.
## Which issue(s) this PR fixes
## Checklist
- [ ] Unit, integration, and e2e (if applicable) tests updated
- [ ] Documentation added (or `pr:no public docs` PR label added if not
required)
- [ ] `CHANGELOG.md` updated (or `pr:no changelog` PR label added if not
required)
- Change FIRING trigger for webhooks to be sent after escalation
snapshot has been computed
- Extract users from `notify_to_users_queue` and `notify_schedule` from
escalation snapshot to populate `users_to_be_notified` in webhook
payload
# What this PR does
Required for new Integrations page
<img width="674" alt="Screenshot 2023-04-04 at 20 32 03"
src="https://user-images.githubusercontent.com/2262529/229792240-60783f30-00ba-4dfc-bebd-75d6c2c232e3.png">
## Which issue(s) this PR fixes
## Checklist
- [ ] Unit, integration, and e2e (if applicable) tests updated
- [ ] Documentation added (or `pr:no public docs` PR label added if not
required)
- [ ] `CHANGELOG.md` updated (or `pr:no changelog` PR label added if not
required)
---------
Co-authored-by: Joey Orlando <joey.orlando@grafana.com>
# What this PR does
## Which issue(s) this PR fixes
## Checklist
- [ ] Unit, integration, and e2e (if applicable) tests updated
- [ ] Documentation added (or `pr:no public docs` PR label added if not
required)
- [ ] `CHANGELOG.md` updated (or `pr:no changelog` PR label added if not
required)
# What this PR does
- add new columns `gcom_org_contract_type`,
`gcom_org_irm_sku_subscription_start_date`, and
`gcom_org_oldest_admin_with_billing_privileges_user_id` to
`user_management_organization` table + `is_restricted` column to
`alerts_alertgroup` table
- emit two new Django signals
- `org_sync_signal` at the end of the
`engine/apps/user_management/sync.py::sync_organization` method
- `alert_group_created_signal` when a new Alert Group is created
## Checklist
- [ ] Tests updated (N/A)
- [ ] Documentation added (N/A)
- [x] `CHANGELOG.md` updated
---------
Co-authored-by: Rares Mardare <rares.mardare@grafana.com>
The routing of the OnCall plugin has changed and no longer uses URL
params
but instead uses paths. This link is used when declaring an Incident
from
the OnCall Slack alert and needs to match the correct pattern in order
for
Incident to correctly detect it.
# What this PR does
Allows changing team for escalation chains
## Which issue(s) this PR fixes
## Checklist
- [ ] Unit, integration, and e2e (if applicable) tests updated
- [ ] Documentation added (or `pr:no public docs` PR label added if not
required)
- [ ] `CHANGELOG.md` updated (or `pr:no changelog` PR label added if not
required)
---------
Co-authored-by: Ildar Iskhakov <ildar.iskhakov@grafana.com>
Co-authored-by: Vadim Stepanov <vadimkerr@gmail.com>
# What this PR does
* api returns all the resources available to the user by default
* substitutes `team switcher` with `multi-select team filter`
* allow referencing between integrations - escalations chains -
[schedules, outgoing webhooks] across teams
https://user-images.githubusercontent.com/2262529/225634581-2d2e8af2-15ce-4c01-a90e-8267d98f5a23.mov
## Which issue(s) this PR fixes
## Checklist
- [ ] Tests updated
- [ ] Documentation added
- [ ] `CHANGELOG.md` updated
---------
Co-authored-by: Maxim <maxim.mordasov@grafana.com>
Co-authored-by: Joey Orlando <joey.orlando@grafana.com>
# What this PR does
This PR:
- modifies the `check_escalation_finished_task` celery task to:
- do stricter escalation validation based on the alert group's
escalation snapshot (see the `audit_alert_group_escalation` method in
`engine/apps/alerts/tasks/check_escalation_finished.py` for the
validation logic)
- use a read-only database for querying alert-groups if one is
configured, otherwise use the "default" one
- ping a configurable heartbeat (new env var
`ALERT_GROUP_ESCALATION_AUDITOR_CELERY_TASK_HEARTBEAT_URL` added)
- increase the task frequency from every 10 to every 13 minutes (this
can be configured via an env variable)
- adds public documentation on how to configure this auditor task
- modifies the local celery startup command to properly take into
consideration all celery related env vars (similar to the ones we use in
`engine/celery_with_exporter.sh`; this made it easier to enable `celery
beat` locally for testing)
- removes the following code:
- removes references to `AlertGroup.estimate_escalation_finish_time` and
marks the model field as deprecated using the [`django-deprecate-fields`
library](https://pypi.org/project/django-deprecate-fields/). This field
was only used for the previous version of this validation task
- `EscalationSnapshotMixin.calculate_eta_for_finish_escalation` was only
used to calculate the value for
`AlertGroup.estimate_escalation_finish_time`
- `calculate_escalation_finish_time` celery task
## Which issue(s) this PR fixes
https://github.com/grafana/oncall-private/issues/1558
## Checklist
- [x] Tests updated
- [x] Documentation added
- [x] `CHANGELOG.md` updated
This PR add Inbound Email integration.
It designed to support some variety of ESPs, but in prod we will use
Mailgun, so locally I tested it only with mailgun ESP.
**Important:**
To make it work on different clusters I'm planning to provide different
email domains for different regions, like ....@us.oncall.grafana.net,
...@eu.oncall.grafana.net
---------
Co-authored-by: Innokentii Konstantinov <innokenty.konstantinov@grafana.com>
# What this PR does
This PR fixes templates behaviour for public and private api. It fix
"reset to default" for templates from messaging backends and some minor
bugs. Also added acknowledge signal and source link templates
## Checklist
- [x] Tests updated
- [x] Documentation added
- [x] `CHANGELOG.md` updated
This should fix task error as seen in logs, trying to parse an empty
string as ical value:
```
Task apps.schedules.tasks.refresh_ical_files.refresh_ical_file[] raised unexpected: ValueError("Found no components where exactly one is required: ''")
```
# What this PR does
This PR simplifies code of maintenance mode.
1. Perform distribution/escalation maintenance checks in send_signal...
tasks.
2. Use usual alert distribution flow for the maintenance incident.
3. Decouple maintenance mode from slack (all, except
**notify_about_maintenance_action** methods, I don't want to make this
PR too big)
As a bonus from these changes, maintenance mode now mute alert group
delivery in all chatops integrations, not only in slack. (Before,
incidents happened while maintenance were posted to telegram and msteams
anyway)
## Checklist
- [ ] Tests updated
- [ ] Documentation added
- [ ] `CHANGELOG.md` updated
# What this PR does
This PR cleanup ScenarioStep. It's needed to simplify moving Slack to
the messaging backends in future.
1. Introduce AlertGroupSlackService to move logic from ScenarioStep.
Also it allowed to get rid of importing ScenarioSteps in the code not
related to processing of slack callbacks.
2. Remove tags from ScenarioSteps, they are unused.
3. Remove ScenarioStep.dispatch method. It just was calling
ScenarioStep.process_scenario.
4. Remove "action" param from process_scenario, it was unused.
5. Remove creation of SlackActionRecord on handling SlackEvents. We are
not using it, but it generates INSERT query on most of the user-slack
interactions.
6. Remove "random_prefix_for_routing" from ScenarioStep, it was unused.
## Which issue(s) this PR fixes
## Checklist
- [ ] Tests updated
- [ ] Documentation added
- [ ] `CHANGELOG.md` updated
---------
Co-authored-by: Joey Orlando <joey.orlando@grafana.com>
# What this PR does
This PR adds
[django-migration-linter](https://github.com/3YOURMIND/django-migration-linter)
to keep database migrations
backwards compatible
- we can automatically run migrations and they are zero-downtime, e.g.
old code can work with the migrated database
- we can run and rollback migrations without worrying about data safety
- OnCall is deployed to the multiple environments core team is not able
to control
See [django-migration-linter
checklist](https://github.com/3YOURMIND/django-migration-linter/blob/main/docs/incompatibilities.md)
for the common mistakes and best practices
## Which issue(s) this PR fixes
## Checklist
- [ ] Tests updated
- [ ] Documentation added
- [ ] `CHANGELOG.md` updated
---------
Co-authored-by: Joey Orlando <joey.orlando@grafana.com>
# What this PR does
Make direct paging internal API endpoint return an alert group ID.
## Which issue(s) this PR fixes
Related to https://github.com/grafana/oncall/issues/823
## Checklist
- [x] Tests updated
# What this PR does
This PR caches the field `render_for_web` with lifetime 1 day and cache
becomes invalid if it was created before
* last alert received
* template changed
## Which issue(s) this PR fixes
## Checklist
- [ ] Tests updated
- [ ] Documentation added
- [ ] `CHANGELOG.md` updated
**What this PR does**:
- Keep grafana version on create/update contact points to avoid multiple
requests to alerting
- Add retry limit on create contact point async
- Fix bugs related on create contact point
- Update logs on create/update contact point, make them more clear
- Avoid unnecessary requests to Grafana Alerting
