# What this PR does
Fixes for https://github.com/grafana/oncall-private/issues/2423
## Which issue(s) this PR fixes
## Checklist
- [ ] Unit, integration, and e2e (if applicable) tests updated
- [ ] Documentation added (or `pr:no public docs` PR label added if not
required)
---------
Co-authored-by: Joey Orlando <joey.orlando@grafana.com>
Co-authored-by: Dominik <dominik.broj@grafana.com>
# What this PR does
Adds acknowledged and resolved user information to the web hook payload
## Which issue(s) this PR closes
Closes#4126
## Checklist
- [x] Unit, integration, and e2e (if applicable) tests updated
- [x] Documentation added (or `pr:no public docs` PR label added if not
required)
- [x] Added the relevant release notes label (see labels prefixed w/
`release:`). These labels dictate how your PR will
show up in the autogenerated release notes.
---------
Co-authored-by: Joey Orlando <joey.orlando@grafana.com>
# What this PR does
Enables the API to perform updates on the advanced webhooks created via
the UI
## Which issue(s) this PR closes
Closes#3958
## Checklist
- [x] Unit, integration, and e2e (if applicable) tests updated
- [x] Documentation added (or `pr:no public docs` PR label added if not
required)
- [x] Added the relevant release notes label (see labels prefixed w/
`release:`). These labels dictate how your PR will
show up in the autogenerated release notes.
It makes it so webhooks are deleted when a "connected" integration is
deleted. Related to
https://github.com/grafana/oncall-private/issues/2615.
## Checklist
- [x] Unit, integration, and e2e (if applicable) tests updated
- [x] Documentation added (or `pr:no public docs` PR label added if not
required)
- [x] Added the relevant release notes label (see labels prefixed w/
`release:`). These labels dictate how your PR will
show up in the autogenerated release notes.
## Which issue(s) this PR closes
fix incorrect args passed to
`apps.webhooks.tasks.trigger_webhook.execute_webhook`
## Checklist
- [x] Unit, integration, and e2e (if applicable) tests updated
- [x] Documentation added (or `pr:no public docs` PR label added if not
required)
- [x] Added the relevant release notes label (see labels prefixed w/
`release:`). These labels dictate how your PR will
show up in the autogenerated release notes.
Only create a response object if the request is triggered, otherwise log
the reason.
Avoid creating multiple/unneeded responses, which can be also confusing
when displaying response objects. If there is no response, the
webhook/request wasn't triggered (possible reasons: integrations are
filtered, or the trigger template evaluates to False).
Related to https://github.com/grafana/oncall-private/issues/2615
# What this PR does
- removes unused "custom button" backend code now that we've migrated to
outgoing webhooks
- adds new e2e test for webhooks asserting that an `ngrok`/`express`
webhook handler receives the call as expected + payload is as expected
(related to https://github.com/grafana/oncall/issues/2691) - skipped for
now, the test passes locally but fails on GitHub Actions CI, seems to be
networking related
## Checklist
- [x] Unit, integration, and e2e (if applicable) tests updated
- [x] Documentation added (or `pr:no public docs` PR label added if not
required)
---------
Co-authored-by: Michael Derynck <michael.derynck@grafana.com>
# What this PR does
- Adds a new model `AlertGroupExternalID` to keep track of incident IDs
in external systems
- Adds calls to integration config specific functions on alert group
creation and webhook response
## Which issue(s) this PR closes
Related to https://github.com/grafana/oncall-private/issues/2541
## Checklist
- [x] Unit, integration, and e2e (if applicable) tests updated
- [x] Documentation added (or `pr:no public docs` PR label added if not
required)
- [x] Added the relevant release notes label (see labels prefixed w/
`release:`). These labels dictate how your PR will
show up in the autogenerated release notes.
# What this PR does
When a team is deleted webhooks belonging to that team will be marked as
having no team instead of the webhook being deleted.
## Which issue(s) this PR fixes
## Checklist
- [x] Unit, integration, and e2e (if applicable) tests updated
- [x] Documentation added (or `pr:no public docs` PR label added if not
required)
- [x] `CHANGELOG.md` updated (or `pr:no changelog` PR label added if not
required)
# What this PR does
It adds functionality to be able to configure the outgoing webhook
timeout from an environment variable.
## Which issue(s) this PR fixes
Running into timeouts when outgoing webhooks take longer than 4 seconds
(which is exceptional, but can happen) the webhook reports failure,
while it still might have succeeded on the webhook side.
## Checklist
- [x] Unit, integration, and e2e (if applicable) tests updated
- [x] Documentation added (or `pr:no public docs` PR label added if not
required)
- [x] `CHANGELOG.md` updated (or `pr:no changelog` PR label added if not
required)
---------
Co-authored-by: Joey Orlando <joey.orlando@grafana.com>
Co-authored-by: Joey Orlando <joseph.t.orlando@gmail.com>
# What this PR does
Fixes escaping for unicode characters in webhooks.
## Which issue(s) this PR fixes
#3149
## Checklist
- [x] Unit, integration, and e2e (if applicable) tests updated
- [x] Documentation added (or `pr:no public docs` PR label added if not
required)
- [x] `CHANGELOG.md` updated (or `pr:no changelog` PR label added if not
required)
# What this PR does
Adds PATCH method Support for outgoing webhook
## Which issue(s) this PR fixes
Fixes#3564
## Checklist
- [x] Unit, integration, and e2e (if applicable) tests updated
- [x] Documentation added (or `pr:no public docs` PR label added if not
required)
- [x] `CHANGELOG.md` updated (or `pr:no changelog` PR label added if not
required)
---------
Co-authored-by: Joey Orlando <joey.orlando@grafana.com>
# What this PR does
Allows public OnCall API to use Grafana service accounts for
authorization. In cloud requests using a Grafana service account token
also needs to provide headers for `X-Grafana-Org-Slug` and
`X-Grafana-Instance-Slug`
This is **alpha** functionality, it may break or be removed in the
future. Going to use this on one endpoint (resolution notes) before we
consider the implications across all of public API.
## Which issue(s) this PR fixes
## Checklist
- [x] Unit, integration, and e2e (if applicable) tests updated
- [x] Documentation added (or `pr:no public docs` PR label added if not
required)
- [x] `CHANGELOG.md` updated (or `pr:no changelog` PR label added if not
required)
This PR add labels for webhooks.
1. Make webhook "labelable" with ability to filter by labels.
2. Add labels to the webhook payload. It contain new field webhook with
it's name, id and labels. Field integration and alert_group has a
corresponding label field as well. See example of a new payload below:
```
{
"event": {
"type": "escalation"
},
"user": null,
"alert_group": {
"id": "IRFN6ZD31N31B",
"integration_id": "CTWM7U4A2QG97",
"route_id": "RUE7U7Z46SKGY",
"alerts_count": 1,
"state": "firing",
"created_at": "2023-11-22T08:54:55.178243Z",
"resolved_at": null,
"acknowledged_at": null,
"title": "Incident",
"permalinks": {
"slack": null,
"telegram": null,
"web": "http://grafana:3000/a/grafana-oncall-app/alert-groups/IRFN6ZD31N31B"
},
"labels": {
"severity": "critical"
}
},
"alert_group_id": "IRFN6ZD31N31B",
"alert_payload": {
"message": "This alert was sent by user for demonstration purposes"
},
"integration": {
"id": "CTWM7U4A2QG97",
"type": "webhook",
"name": "hi - Webhook",
"team": null,
"labels": {
"hello": "world",
"severity": "critical"
}
},
"notified_users": [],
"users_to_be_notified": [],
"webhook": {
"id": "WHAXK4BTC7TAEQ",
"name": "test",
"labels": {
"hello": "kesha"
}
}
}
```
I feel that there is an opportunity to make code cleaner - remove all
label logic from serializers, views and utils to models or dedicated
LabelerService and introduce Labelable interface with something like
label_verbal, update_labels methods. However, I don't want to tie
webhook labels with a refactoring.
---------
Co-authored-by: Dominik <dominik.broj@grafana.com>
# What this PR does
Add a system similar to how we select integrations when creating
webhooks so that the user has a description of what webhookds do and
does not have to write complex templates for common webhook use cases.
Presets allow us to create the contents of the webhooks in code and
define which fields are controlled by the preset. Some specifics:
- Newly created webhooks must choose between Simple, Advanced or another
predefined system
- Simple is always an escalation step and will post the entire payload
to the given URL
- Advanced is the same as no preset which is our current view where all
fields are available
- There are no changes for all existing webhooks with empty preset
fields
- Once a webhook is created with a preset the preset cannot be changed
- Fields in the webhook that are populated by code will give a
validation error if they are modified
- In the public API webhooks with presets are returned for viewing but
cannot be created or modified. This restriction is in place because the
Web UI provides the context for which fields to use with a preset. The
public API is for interacting with webhooks where all fields are
defined.
To define a preset create a file with metadata and an override function.
The metadata drives validation and what to display in the UI. There are
two functions one is connected to the pre_save hook of the Webhook model
for persistent changes, the other replaces parameters at execution time
for ephemeral changes. See the simple and advanced presets as an
example. The file must be listed in settings in
`INSTALLED_WEBHOOK_PRESETS` to be enabled at runtime..
## Which issue(s) this PR fixes
## Checklist
- [x] Unit, integration, and e2e (if applicable) tests updated
- [x] Documentation added (or `pr:no public docs` PR label added if not
required)
- [x] `CHANGELOG.md` updated (or `pr:no changelog` PR label added if not
required)
---------
Co-authored-by: Joey Orlando <joey.orlando@grafana.com>
# What this PR does
This task does not appear to have been invoked/have any logs associated
within the past month in any of our cloud environments. I'm fairly
certain it is deprecated and can be removed
## Checklist
- [x] Unit, integration, and e2e (if applicable) tests updated
- [x] Documentation added (or `pr:no public docs` PR label added if not
required)
- [x] `CHANGELOG.md` updated (or `pr:no changelog` PR label added if not
required)
# What this PR does
This is a follow up to #2502 which started to remove logic to
"archiving" alert groups. This PR:
- removes all references to `AlertGroup.is_archived` and marks the
column as deprecated. We will remove it in the next release
- removes the `AlertGroup.unarchived_objects` `Manager`
- renames the `AlertGroup.all_objects` `Manager` to `AlertGroup.objects`
## Checklist
- [x] Unit, integration, and e2e (if applicable) tests updated
- [ ] Documentation added (or `pr:no public docs` PR label added if not
required)
- [x] `CHANGELOG.md` updated (or `pr:no changelog` PR label added if not
required)
# What this PR does
- Adds [`mypy` static type checking](https://mypy-lang.org/) to our CI
pipeline. Currently there is still a **ton** of errors being returned by
the tool, as we'll need to fix pre-existing errors. I think we can
slowly chip away at these errors in small PRs, doing them all in one
large PR is likely very risky.
- Also, this PR starts chipping away at one of the main type errors that
we have which is accessing the `datetime` class (from the `datetime`
library) or `timedelta` function on the `django.utils.timezone` module.
Basically we should be instead accessing these two objects from the
native `datetime` module. This makes sense because the [`__all__`
attribute](https://github.com/django/django/blob/main/django/utils/timezone.py#L14-L30)
in `django.utils.timezone` does not re-export `datetime` or `timedelta`.
- splits `engine` dependencies out into `requirements.txt` and
`requirements-dev.txt`
## Checklist
- [ ] Unit, integration, and e2e (if applicable) tests updated (N/A)
- [ ] Documentation added (or `pr:no public docs` PR label added if not
required) (N/A)
- [ ] `CHANGELOG.md` updated (or `pr:no changelog` PR label added if not
required) (N/A)
# What this PR does
- update `make test` to always use `settings.ci-test`. Right now it will
use whatever the value of `DJANGO_SETTINGS_MODULE` is in
`./dev/.env.dev`, which causes ~45 tests to fail
- Fix several Python warnings that we see when running the tests
```bash
RemovedInDjango40Warning: The providing_args argument is deprecated. As it is purely documentational, it has no replacement. If you rely on this argument as documentation, you can move the text to a code comment or docstring.
alert_create_signal = django.dispatch.Signal(
```
```bash
PytestCollectionWarning: cannot collect test class 'TestOnlyBackend' because it has a __init__ constructor (from: apps/api/tests/test_alert_receive_channel_template.py)
class TestOnlyBackend(BaseMessagingBackend):
```
```bash
DeprecationWarning: The parameter 'use_aliases' in emoji.emojize() is deprecated and will be removed in version 2.0.0. Use language='alias' instead.
To hide this warning, pin/downgrade the package to 'emoji~=1.6.3'
return emoji.emojize(self.verbal_name, use_aliases=True)
```
```bash
DateTimeField CustomOnCallShift.start received a naive datetime (2023-06-01 12:53:12) while time zone support is active.
warnings.warn("DateTimeField %s received a naive datetime (%s)"
```
```bash
apps/twilioapp/tests/test_phone_calls.py::test_resolve_by_phone
/etc/app/apps/twilioapp/tests/test_phone_calls.py:173: DeprecationWarning: The 'text' argument to find()-type methods is deprecated. Use 'string' instead.
content = BeautifulSoup(content, features="html.parser").findAll(text=True)
```
```bash
apps/twilioapp/tests/test_phone_calls.py::test_resolve_by_phone
apps/twilioapp/tests/test_phone_calls.py::test_wrong_pressed_digit
/usr/local/lib/python3.11/site-packages/bs4/builder/__init__.py:545: XMLParsedAsHTMLWarning: It looks like you're parsing an XML document using an HTML parser. If this really is an HTML document (maybe it's XHTML?), you can ignore or filter this warning. If it's XML, you should know that using an XML parser will be more reliable. To parse this document as XML, make sure you have the lxml package installed, and pass the keyword argument `features="xml"` into the BeautifulSoup constructor.
```
```bash
apps/twilioapp/tests/test_phone_calls.py::test_forbidden_requests
/usr/local/lib/python3.11/site-packages/social_django/urls.py:15: RemovedInDjango40Warning: django.conf.urls.url() is deprecated in favor of django.urls.re_path().
url(r'^login/(?P<backend>[^/]+){0}$'.format(extra), views.auth,
```
```bash
apps/twilioapp/tests/test_phone_calls.py: 66 warnings
/usr/local/lib/python3.11/site-packages/debug_toolbar/utils.py:255: DeprecationWarning: currentThread() is deprecated, use current_thread() instead
thread = threading.currentThread()
```
## Checklist
- [x] Unit, integration, and e2e (if applicable) tests updated
- [x] Documentation added (or `pr:no public docs` PR label added if not
required)
- [ ] `CHANGELOG.md` updated (or `pr:no changelog` PR label added if not
required)
- Rename Firing to Alert Group Created to reduce confusion as to why the
event only first once and not when unresolve or unacknowledge returns
the alert group to the firing state.
- Increase password field length
- Do not filter webhook execution by team, team is just for filtering
ownership now
- Do not log webhook triggers in alert group escalation log if the
webhook does not trigger (Status/response will still be stored)
- Fix formatting for response content and data fields on the Status page
- Add a content length limit for responses being stored (50000
characters)
- Change FIRING trigger for webhooks to be sent after escalation
snapshot has been computed
- Extract users from `notify_to_users_queue` and `notify_schedule` from
escalation snapshot to populate `users_to_be_notified` in webhook
payload
- Enable/Disable Toggle
- Add Integrations Filter
- Enable log for non-triggered webhook for troubleshooting
- Improve UI to better show enabled, trigger status, request status
- Standardize terms New, Firing, Triggered to Firing
- Mask password field
- Fix bug where UI did not update when webhook is deleted
- Change responses to be referenced by webhook ID rather than trigger
type
Track all webhook responses data, and allow using this between
alertgroup-related webhooks (e.g. use firing webhook response data when
templating the acknowledge webhook request data).
NOTE: dropping the table is not backwards compatible but the feature is
not enabled (and in any case it would drop log entries only used for
status display)
