feat: allow setting direct paging importance for teams (#5379)
## Which issue(s) this PR closes https://www.loom.com/share/e1858db48e8b4fa99014a997af5e3d5e Closes https://github.com/grafana/irm/issues/322 ## Checklist - [x] Unit, integration, and e2e (if applicable) tests updated - [x] Documentation added (or `pr:no public docs` PR label added if not required) - [x] Added the relevant release notes label (see labels prefixed w/ `release:`). These labels dictate how your PR will show up in the autogenerated release notes. --------- Co-authored-by: Vadim Stepanov <vadimkerr@gmail.com>
This commit is contained in:
Joey Orlando
GitHub
parent
f3f7c17f8b
commit
152d5f74fc
15 changed files with 498 additions and 53 deletions
|
|
@ -89,9 +89,16 @@ to the team's ChatOps channels and start an appropriate escalation chain.
|
|||
|
||||
## Set up direct paging for a team
|
||||
|
||||
By default all teams will have a direct paging integration created for them. However, these are not configured by default.
|
||||
If a team does not have their direct paging integration configured, such that it is "contactable" (ie. it has an
|
||||
escalation chain assigned to it, or has at least one Chatops integration connected to send notifications to), you will
|
||||
By default all teams will have a direct paging integration created for them. Each direct paging integration will be
|
||||
created with two routes:
|
||||
|
||||
- a non-default route which has a Jinja2 filtering term of `{{ payload.oncall.important }}`
|
||||
(see [Important Escalations](#important-escalations) below for more details)
|
||||
- a default route to capture all other alerts
|
||||
|
||||
However, these integrations are not configured by default to be "contactable" (ie. their routes will have no
|
||||
escalation chains assigned to them, nor any Chatops integrations connected to send notifications to).
|
||||
If a team does not have their direct paging integration configured, such that it is "contactable" , you will
|
||||
not be able to direct page this team. If this happens, consider following the following steps for the team (or reach out
|
||||
to the relevant team and suggest doing so).
|
||||
|
||||
|
|
@ -99,3 +106,30 @@ Navigate to the **Integrations** page and find the "Direct paging" integration f
|
|||
integration's detail page, you can customize its settings, link it to an escalation chain, and configure associated
|
||||
ChatOps channels. To confirm that the integration is functioning as intended, [create a new alert group](#page-a-team)
|
||||
and select the same team for a test run.
|
||||
|
||||
### Important escalations
|
||||
|
||||
Sometimes you really need to get the attention of a particular team. When directly paging a team, it is possible to
|
||||
page them using an "important escalation". Practically speaking, this will create an alert, using the specified team's
|
||||
direct paging integration as such:
|
||||
|
||||
```json
|
||||
{
|
||||
"oncall": {
|
||||
"title": "IRM is paging Network team to join escalation",
|
||||
"message": "I really need someone from your team to come take a look! The k8s cluster is down!",
|
||||
"uid": "8a20b8d1-56fd-482e-824e-43fbd1bd7b10",
|
||||
"author_username": "irm",
|
||||
"permalink": null,
|
||||
"important": true
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
When you are directly paging a team, either via the web UI, chatops apps, or the API, you can specify that this
|
||||
esclation be "important", which will effectively set the value of `oncall.important` to `true`. As mentioned above in
|
||||
[Set up direct paging for a team](#set-up-direct-paging-for-a-team), direct paging integrations come pre-configured with
|
||||
two routes, with the non-default route having a Jinja2 filtering term of `{{ payload.oncall.important }}`.
|
||||
|
||||
This allows teams to be contacted via different escalation chains, depending on whether or not the user paging them
|
||||
believes that this is an "important escalation".
|
||||
|
|
|
|||
|
|
@ -18,6 +18,11 @@ refs:
|
|||
destination: /docs/oncall/<ONCALL_VERSION>/configure/integrations/references/manual
|
||||
- pattern: /docs/grafana-cloud/
|
||||
destination: /docs/grafana-cloud/configure/integrations/references/manual
|
||||
manual-paging-team-important:
|
||||
- pattern: /docs/oncall/
|
||||
destination: /docs/oncall/<ONCALL_VERSION>/configure/integrations/references/manual#important-escalations
|
||||
- pattern: /docs/grafana-cloud/
|
||||
destination: /docs/grafana-cloud/configure/integrations/references/manual#important-escalations
|
||||
---
|
||||
|
||||
# Escalation HTTP API
|
||||
|
|
@ -90,7 +95,8 @@ curl "{{API_URL}}/api/v1/escalation/" \
|
|||
"title": "We are seeing a network outage in the datacenter",
|
||||
"message": "I need help investigating, can you join the investigation?",
|
||||
"source_url": "https://github.com/myorg/myrepo/issues/123",
|
||||
"team": "TI73TDU19W48J"
|
||||
"team": "TI73TDU19W48J",
|
||||
"important_team_escalation": true
|
||||
}'
|
||||
```
|
||||
|
||||
|
|
@ -176,6 +182,7 @@ The above command returns JSON structured in the following way:
|
|||
| `team` | No | Yes (see [Things to Note](#things-to-note)) | Grafana OnCall team ID. If specified, will use the "Direct Paging" Integration associated with this Grafana OnCall team, to create the Alert Group. |
|
||||
| `users` | No | Yes (see [Things to Note](#things-to-note)) | List of user(s) to escalate to. See above request example for object schema. `id` represents the Grafana OnCall user's ID. `important` is a boolean representing whether to escalate the Alert Group using this user's default or important personal notification policy. |
|
||||
| `alert_group_id` | No | No | If specified, will escalate the specified users for this Alert Group. |
|
||||
| `important_team_escalation` | No | No | Sets the value of `payload.oncall.important` to the value specified here (default is `False`; see [Things to Note](#things-to-note) for more details). |
|
||||
|
||||
## Things to note
|
||||
|
||||
|
|
@ -186,6 +193,10 @@ existing Alert Group
|
|||
if you are trying to escalate to a set of users on an existing Alert Group, you cannot update the `title`, `message`, or
|
||||
`source_url` of that Alert Group
|
||||
- If escalating to a set of users for an existing Alert Group, the Alert Group cannot be in a resolved state
|
||||
- Regarding `important_team_escalation`; this can be useful to send an "important" escalation to the specified team.
|
||||
Teams can configure their Direct Paging Integration to route to different escalation chains based on the value of
|
||||
`payload.oncall.important`. See [Manual paging integration - important escalations](ref:manual-paging-team-important)
|
||||
for more details.
|
||||
|
||||
**HTTP request**
|
||||
|
||||
|
|
|
|||
|
|
@ -0,0 +1,84 @@
|
|||
# Generated by Django 4.2.17 on 2024-12-20 14:19
|
||||
|
||||
import logging
|
||||
|
||||
from django.db import migrations
|
||||
from django.db.models import Count
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
def upsert_direct_paging_integration_routes(apps, schema_editor):
|
||||
AlertReceiveChannel = apps.get_model("alerts", "AlertReceiveChannel")
|
||||
ChannelFilter = apps.get_model("alerts", "ChannelFilter")
|
||||
|
||||
DIRECT_PAGING_INTEGRATION_TYPE = "direct_paging"
|
||||
IMPORTANT_FILTERING_TERM = "{{ payload.oncall.important }}"
|
||||
|
||||
# Fetch all direct paging integrations
|
||||
logger.info("Fetching direct paging integrations which have not had their routes updated.")
|
||||
|
||||
# Ignore updating Direct Paging integrations that have > 1 route, as this means that users have
|
||||
# gone ahead and created their own routes. We don't want to overwrite these.
|
||||
unedited_direct_paging_integrations = (
|
||||
AlertReceiveChannel.objects
|
||||
.filter(integration=DIRECT_PAGING_INTEGRATION_TYPE)
|
||||
.annotate(num_routes=Count("channel_filters"))
|
||||
.filter(num_routes=1)
|
||||
)
|
||||
|
||||
integration_count = unedited_direct_paging_integrations.count()
|
||||
if integration_count == 0:
|
||||
logger.info("No integrations found which meet this criteria. No routes will be upserted.")
|
||||
return
|
||||
|
||||
logger.info(f"Found {integration_count} direct paging integrations that meet this criteria.")
|
||||
|
||||
# Direct Paging Integrations are currently created with a single default route (order=0)
|
||||
# see AlertReceiveChannelManager.create_missing_direct_paging_integrations
|
||||
#
|
||||
# we first need to update this route to be order=1, and then we will subsequently bulk-create the
|
||||
# non-default route (order=0) which will have a filtering term set
|
||||
routes = ChannelFilter.objects.filter(
|
||||
alert_receive_channel__in=unedited_direct_paging_integrations,
|
||||
is_default=True,
|
||||
order=0,
|
||||
)
|
||||
|
||||
logger.info(
|
||||
f"Swapping the order=0 value to order=1 for {routes.count()} Direct Paging Integrations default routes"
|
||||
)
|
||||
|
||||
updated_rows = routes.update(order=1)
|
||||
logger.info(f"Swapped order=0 to order=1 for {updated_rows} Direct Paging Integrations default routes")
|
||||
|
||||
# Bulk create the new non-default routes
|
||||
logger.info(
|
||||
f"Creating new non-default routes for {len(unedited_direct_paging_integrations)} Direct Paging Integrations"
|
||||
)
|
||||
created_objs = ChannelFilter.objects.bulk_create(
|
||||
[
|
||||
ChannelFilter(
|
||||
alert_receive_channel=integration,
|
||||
filtering_term=IMPORTANT_FILTERING_TERM,
|
||||
filtering_term_type=1, # 1 = ChannelFilter.FILTERING_TERM_TYPE_JINJA2
|
||||
is_default=False,
|
||||
order=0,
|
||||
) for integration in unedited_direct_paging_integrations
|
||||
],
|
||||
batch_size=5000,
|
||||
)
|
||||
logger.info(f"Created {len(created_objs)} new non-default routes for Direct Paging Integrations")
|
||||
|
||||
logger.info("Migration for direct paging integration routes completed.")
|
||||
|
||||
|
||||
class Migration(migrations.Migration):
|
||||
|
||||
dependencies = [
|
||||
("alerts", "0071_migrate_labels"),
|
||||
]
|
||||
|
||||
operations = [
|
||||
migrations.RunPython(upsert_direct_paging_integration_routes, migrations.RunPython.noop),
|
||||
]
|
||||
|
|
@ -126,6 +126,8 @@ class AlertReceiveChannelManager(models.Manager):
|
|||
def create_missing_direct_paging_integrations(organization: "Organization") -> None:
|
||||
from apps.alerts.models import ChannelFilter
|
||||
|
||||
logger.info(f"Starting create_missing_direct_paging_integrations for organization: {organization.id}")
|
||||
|
||||
# fetch teams without direct paging integration
|
||||
teams_missing_direct_paging = list(
|
||||
organization.teams.exclude(
|
||||
|
|
@ -134,10 +136,17 @@ class AlertReceiveChannelManager(models.Manager):
|
|||
).values_list("team_id", flat=True)
|
||||
)
|
||||
)
|
||||
number_of_teams_missing_direct_paging = len(teams_missing_direct_paging)
|
||||
logger.info(
|
||||
f"Found {number_of_teams_missing_direct_paging} teams missing direct paging integrations.",
|
||||
)
|
||||
|
||||
if not teams_missing_direct_paging:
|
||||
logger.info("No missing direct paging integrations found. Exiting.")
|
||||
return
|
||||
|
||||
# create missing integrations
|
||||
logger.info(f"Creating missing direct paging integrations for {number_of_teams_missing_direct_paging} teams.")
|
||||
AlertReceiveChannel.objects.bulk_create(
|
||||
[
|
||||
AlertReceiveChannel(
|
||||
|
|
@ -151,29 +160,49 @@ class AlertReceiveChannelManager(models.Manager):
|
|||
batch_size=5000,
|
||||
ignore_conflicts=True, # ignore if direct paging integration already exists for team
|
||||
)
|
||||
logger.info("Missing direct paging integrations creation step completed.")
|
||||
|
||||
# fetch integrations for teams (some of them are created above, but some may already exist previously)
|
||||
alert_receive_channels = organization.alert_receive_channels.filter(
|
||||
team__in=teams_missing_direct_paging, integration=AlertReceiveChannel.INTEGRATION_DIRECT_PAGING
|
||||
)
|
||||
logger.info(f"Fetched {alert_receive_channels.count()} direct paging integrations for the specified teams.")
|
||||
|
||||
# create default routes
|
||||
# we create two routes for each Direct Paging Integration
|
||||
# 1. route for important alerts (using the payload.oncall.important alert field value) - non-default
|
||||
# 2. route for all other alerts - default
|
||||
routes_to_create = []
|
||||
for alert_receive_channel in alert_receive_channels:
|
||||
routes_to_create.extend(
|
||||
[
|
||||
ChannelFilter(
|
||||
alert_receive_channel=alert_receive_channel,
|
||||
filtering_term="{{ payload.oncall.important }}",
|
||||
filtering_term_type=ChannelFilter.FILTERING_TERM_TYPE_JINJA2,
|
||||
is_default=False,
|
||||
order=0,
|
||||
),
|
||||
ChannelFilter(
|
||||
alert_receive_channel=alert_receive_channel,
|
||||
filtering_term=None,
|
||||
is_default=True,
|
||||
order=1,
|
||||
),
|
||||
]
|
||||
)
|
||||
|
||||
logger.info(f"Creating {len(routes_to_create)} channel filter routes.")
|
||||
ChannelFilter.objects.bulk_create(
|
||||
[
|
||||
ChannelFilter(
|
||||
alert_receive_channel=alert_receive_channel,
|
||||
filtering_term=None,
|
||||
is_default=True,
|
||||
order=0,
|
||||
)
|
||||
for alert_receive_channel in alert_receive_channels
|
||||
],
|
||||
routes_to_create,
|
||||
batch_size=5000,
|
||||
ignore_conflicts=True, # ignore if default route already exists for integration
|
||||
ignore_conflicts=True, # ignore if routes already exist for integration
|
||||
)
|
||||
logger.info("Direct paging routes creation completed.")
|
||||
|
||||
# add integrations to metrics cache
|
||||
logger.info("Adding integrations to metrics cache.")
|
||||
metrics_add_integrations_to_cache(list(alert_receive_channels), organization)
|
||||
logger.info("Integrations have been added to the metrics cache.")
|
||||
|
||||
def get_queryset(self):
|
||||
return AlertReceiveChannelQueryset(self.model, using=self._db).filter(
|
||||
|
|
|
|||
|
|
@ -48,6 +48,7 @@ class DirectPagingAlertPayload(typing.TypedDict):
|
|||
def _trigger_alert(
|
||||
organization: Organization,
|
||||
team: Team | None,
|
||||
important_team_escalation: bool,
|
||||
message: str,
|
||||
title: str,
|
||||
permalink: str | None,
|
||||
|
|
@ -82,6 +83,13 @@ def _trigger_alert(
|
|||
"uid": str(uuid4()), # avoid grouping
|
||||
"author_username": from_user.username,
|
||||
"permalink": permalink,
|
||||
# NOTE: this field is mostly being added for purposes of escalating to a team
|
||||
# this field is provided via the web UI/API/slack as a checkbox, indicating that the user doing the paging
|
||||
# would like to send an "important" page to the team.
|
||||
#
|
||||
# Teams can configure routing in their Direct Paging Integration to route based on this field to different
|
||||
# escalation chains
|
||||
"important": important_team_escalation,
|
||||
},
|
||||
}
|
||||
|
||||
|
|
@ -128,6 +136,7 @@ def direct_paging(
|
|||
source_url: str | None = None,
|
||||
grafana_incident_id: str | None = None,
|
||||
team: Team | None = None,
|
||||
important_team_escalation: bool = False,
|
||||
users: UserNotifications | None = None,
|
||||
alert_group: AlertGroup | None = None,
|
||||
) -> AlertGroup | None:
|
||||
|
|
@ -156,7 +165,16 @@ def direct_paging(
|
|||
# create alert group if needed
|
||||
with transaction.atomic():
|
||||
if alert_group is None:
|
||||
alert_group = _trigger_alert(organization, team, message, title, source_url, grafana_incident_id, from_user)
|
||||
alert_group = _trigger_alert(
|
||||
organization,
|
||||
team,
|
||||
important_team_escalation,
|
||||
message,
|
||||
title,
|
||||
source_url,
|
||||
grafana_incident_id,
|
||||
from_user,
|
||||
)
|
||||
|
||||
for u, important in users:
|
||||
alert_group.log_records.create(
|
||||
|
|
|
|||
|
|
@ -259,27 +259,47 @@ def test_create_missing_direct_paging_integrations(
|
|||
):
|
||||
organization = make_organization()
|
||||
|
||||
# team with no direct paging integration
|
||||
# two teams with no direct paging integration
|
||||
team1 = make_team(organization)
|
||||
team2 = make_team(organization)
|
||||
|
||||
# team with direct paging integration
|
||||
team2 = make_team(organization)
|
||||
team3 = make_team(organization)
|
||||
alert_receive_channel = make_alert_receive_channel(
|
||||
organization, team=team2, integration=AlertReceiveChannel.INTEGRATION_DIRECT_PAGING
|
||||
organization, team=team3, integration=AlertReceiveChannel.INTEGRATION_DIRECT_PAGING
|
||||
)
|
||||
make_channel_filter(alert_receive_channel, is_default=True, order=0)
|
||||
|
||||
# create missing direct paging integration for organization
|
||||
AlertReceiveChannel.objects.create_missing_direct_paging_integrations(organization)
|
||||
|
||||
assert organization.alert_receive_channels.count() == 3
|
||||
|
||||
# check that missing integrations and default routes were created
|
||||
assert organization.alert_receive_channels.count() == 2
|
||||
mock_metrics_add_integrations_to_cache.assert_called_once()
|
||||
#
|
||||
# NOTE: we explicitly don't test team3, it already has a Direct Paging integraiton associated with it
|
||||
# and AlertReceiveChannel.objects.create_missing_direct_paging_integrations is not responsible for filling
|
||||
# in missing routes.
|
||||
#
|
||||
# See apps/alerts/migrations/0072_upsert_direct_paging_integration_routes.py which is a data migration that does
|
||||
# exactly this.
|
||||
for team in [team1, team2]:
|
||||
alert_receive_channel = organization.alert_receive_channels.get(
|
||||
team=team, integration=AlertReceiveChannel.INTEGRATION_DIRECT_PAGING
|
||||
)
|
||||
assert alert_receive_channel.channel_filters.get().is_default
|
||||
alert_receive_channel = organization.alert_receive_channels.get(team=team)
|
||||
|
||||
direct_paging_integration_routes = alert_receive_channel.channel_filters.all()
|
||||
|
||||
assert direct_paging_integration_routes.count() == 2
|
||||
|
||||
for route in direct_paging_integration_routes:
|
||||
if route.is_default:
|
||||
assert route.order == 1
|
||||
assert route.filtering_term is None
|
||||
else:
|
||||
assert route.order == 0
|
||||
assert route.filtering_term == "{{ payload.oncall.important }}"
|
||||
assert route.filtering_term_type == route.FILTERING_TERM_TYPE_JINJA2
|
||||
|
||||
mock_metrics_add_integrations_to_cache.assert_called_once()
|
||||
|
||||
|
||||
@pytest.mark.django_db
|
||||
|
|
|
|||
|
|
@ -1,4 +1,4 @@
|
|||
from unittest.mock import call, patch
|
||||
from unittest.mock import ANY, call, patch
|
||||
|
||||
import pytest
|
||||
from django.utils import timezone
|
||||
|
|
@ -86,23 +86,46 @@ def test_direct_paging_user(make_organization, make_user_for_organization, djang
|
|||
assert_log_record(ag, f"{from_user.username} paged user {u.username}", expected_info=expected_info)
|
||||
|
||||
|
||||
@pytest.mark.parametrize("important_team_escalation", [True, False])
|
||||
@pytest.mark.django_db
|
||||
def test_direct_paging_team(make_organization, make_team, make_user_for_organization):
|
||||
def test_direct_paging_team(make_organization, make_team, make_user_for_organization, important_team_escalation):
|
||||
organization = make_organization()
|
||||
from_user = make_user_for_organization(organization)
|
||||
team = make_team(organization)
|
||||
|
||||
from_author_username = from_user.username
|
||||
source_url = "https://www.example.com"
|
||||
title = f"{from_author_username} is paging {team.name} to join escalation"
|
||||
msg = "Fire"
|
||||
|
||||
direct_paging(organization, from_user, msg, team=team)
|
||||
direct_paging(
|
||||
organization,
|
||||
from_user,
|
||||
msg,
|
||||
source_url=source_url,
|
||||
team=team,
|
||||
important_team_escalation=important_team_escalation,
|
||||
)
|
||||
|
||||
# alert group created
|
||||
alert_groups = AlertGroup.objects.all()
|
||||
assert alert_groups.count() == 1
|
||||
ag = alert_groups.get()
|
||||
alert = ag.alerts.get()
|
||||
assert alert.title == f"{from_user.username} is paging {team.name} to join escalation"
|
||||
assert alert.title == title
|
||||
assert alert.message == msg
|
||||
|
||||
assert alert.raw_request_data == {
|
||||
"oncall": {
|
||||
"title": title,
|
||||
"message": msg,
|
||||
"uid": ANY,
|
||||
"author_username": from_author_username,
|
||||
"permalink": source_url,
|
||||
"important": important_team_escalation,
|
||||
},
|
||||
}
|
||||
|
||||
assert ag.channel.verbal_name == f"Direct paging ({team.name} team)"
|
||||
assert ag.channel.team == team
|
||||
|
||||
|
|
|
|||
|
|
@ -39,6 +39,7 @@ class BasePagingSerializer(serializers.Serializer):
|
|||
|
||||
users = UserReferenceSerializer(many=True, required=False, default=list)
|
||||
team = TeamPrimaryKeyRelatedField(allow_null=True, default=CurrentTeamDefault())
|
||||
important_team_escalation = serializers.BooleanField(required=False, default=False)
|
||||
|
||||
alert_group_id = serializers.CharField(required=False, default=None)
|
||||
alert_group = serializers.HiddenField(default=None) # set in DirectPagingSerializer.validate
|
||||
|
|
|
|||
|
|
@ -1,3 +1,5 @@
|
|||
from unittest.mock import ANY
|
||||
|
||||
import pytest
|
||||
from django.urls import reverse
|
||||
from rest_framework import status
|
||||
|
|
@ -59,11 +61,13 @@ def test_direct_paging_new_alert_group(
|
|||
assert alert.message == message
|
||||
|
||||
|
||||
@pytest.mark.parametrize("important_team_escalation", [True, False])
|
||||
@pytest.mark.django_db
|
||||
def test_direct_paging_page_team(
|
||||
make_organization_and_user_with_plugin_token,
|
||||
make_team,
|
||||
make_user_auth_headers,
|
||||
important_team_escalation,
|
||||
):
|
||||
organization, user, token = make_organization_and_user_with_plugin_token(role=LegacyAccessControlRole.EDITOR)
|
||||
team = make_team(organization=organization)
|
||||
|
|
@ -81,6 +85,7 @@ def test_direct_paging_page_team(
|
|||
"message": message,
|
||||
"source_url": source_url,
|
||||
"grafana_incident_id": grafana_incident_id,
|
||||
"important_team_escalation": important_team_escalation,
|
||||
},
|
||||
format="json",
|
||||
**make_user_auth_headers(user, token),
|
||||
|
|
@ -92,7 +97,16 @@ def test_direct_paging_page_team(
|
|||
alert = alert_group.alerts.first()
|
||||
|
||||
assert alert_group.grafana_incident_id == grafana_incident_id
|
||||
assert alert.raw_request_data["oncall"]["permalink"] == source_url
|
||||
assert alert.raw_request_data == {
|
||||
"oncall": {
|
||||
"title": ANY,
|
||||
"message": message,
|
||||
"uid": ANY,
|
||||
"author_username": ANY,
|
||||
"permalink": source_url,
|
||||
"important": important_team_escalation,
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
@pytest.mark.django_db
|
||||
|
|
|
|||
|
|
@ -40,6 +40,7 @@ class DirectPagingAPIView(APIView):
|
|||
source_url=validated_data["source_url"],
|
||||
grafana_incident_id=validated_data["grafana_incident_id"],
|
||||
team=validated_data["team"],
|
||||
important_team_escalation=validated_data["important_team_escalation"],
|
||||
users=[(user["instance"], user["important"]) for user in validated_data["users"]],
|
||||
alert_group=validated_data["alert_group"],
|
||||
)
|
||||
|
|
|
|||
|
|
@ -89,11 +89,13 @@ def test_escalation_new_alert_group(
|
|||
assert alert.message == message
|
||||
|
||||
|
||||
@pytest.mark.parametrize("important_team_escalation", [True, False])
|
||||
@pytest.mark.django_db
|
||||
def test_escalation_team(
|
||||
make_organization_and_user_with_token,
|
||||
make_team,
|
||||
make_user_auth_headers,
|
||||
important_team_escalation,
|
||||
):
|
||||
organization, user, token = make_organization_and_user_with_token()
|
||||
team = make_team(organization=organization)
|
||||
|
|
@ -110,6 +112,7 @@ def test_escalation_team(
|
|||
"team": team.public_primary_key,
|
||||
"message": message,
|
||||
"source_url": source_url,
|
||||
"important_team_escalation": important_team_escalation,
|
||||
},
|
||||
format="json",
|
||||
**make_user_auth_headers(user, token),
|
||||
|
|
@ -120,7 +123,16 @@ def test_escalation_team(
|
|||
alert_group = AlertGroup.objects.get(public_primary_key=response.json()["id"])
|
||||
alert = alert_group.alerts.first()
|
||||
|
||||
assert alert.raw_request_data["oncall"]["permalink"] == source_url
|
||||
assert alert.raw_request_data == {
|
||||
"oncall": {
|
||||
"title": mock.ANY,
|
||||
"message": message,
|
||||
"uid": mock.ANY,
|
||||
"author_username": mock.ANY,
|
||||
"permalink": source_url,
|
||||
"important": important_team_escalation,
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
@pytest.mark.django_db
|
||||
|
|
|
|||
|
|
@ -41,6 +41,7 @@ class EscalationView(APIView):
|
|||
title=validated_data["title"],
|
||||
source_url=validated_data["source_url"],
|
||||
team=validated_data["team"],
|
||||
important_team_escalation=validated_data["important_team_escalation"],
|
||||
users=[(user["instance"], user["important"]) for user in validated_data["users"]],
|
||||
alert_group=validated_data["alert_group"],
|
||||
)
|
||||
|
|
|
|||
|
|
@ -37,12 +37,14 @@ if typing.TYPE_CHECKING:
|
|||
from apps.slack.models import SlackTeamIdentity, SlackUserIdentity
|
||||
from apps.user_management.models import Organization, Team, User
|
||||
|
||||
|
||||
DIRECT_PAGING_TEAM_SELECT_ID = "paging_team_select"
|
||||
DIRECT_PAGING_TEAM_SEVERITY_CHECKBOXES_ID = "paging_team_severity_checkboxes"
|
||||
DIRECT_PAGING_ORG_SELECT_ID = "paging_org_select"
|
||||
DIRECT_PAGING_USER_SELECT_ID = "paging_user_select"
|
||||
DIRECT_PAGING_MESSAGE_INPUT_ID = "paging_message_input"
|
||||
|
||||
DIRECT_PAGING_TEAM_SEVERITY_CHECKBOX_VALUE = "important"
|
||||
|
||||
DEFAULT_TEAM_VALUE = "default_team"
|
||||
|
||||
|
||||
|
|
@ -248,6 +250,7 @@ class FinishDirectPaging(scenario_step.ScenarioStep):
|
|||
from_user=user,
|
||||
message=message,
|
||||
team=selected_team,
|
||||
important_team_escalation=_get_team_escalation_severity_from_payload(payload, input_id_prefix),
|
||||
users=selected_users,
|
||||
)
|
||||
except DirectPagingUserTeamValidationError:
|
||||
|
|
@ -331,6 +334,14 @@ class OnPagingTeamChange(scenario_step.ScenarioStep):
|
|||
)
|
||||
|
||||
|
||||
class OnPagingTeamSeverityCheckboxChange(OnPagingTeamChange):
|
||||
"""
|
||||
Specify alert severity when escalating to a team.
|
||||
|
||||
NOTE: we simply reuse `OnPagingTeamChange` step, since the behavior is the same.
|
||||
"""
|
||||
|
||||
|
||||
class OnPagingUserChange(scenario_step.ScenarioStep):
|
||||
"""Add selected to user to the list.
|
||||
|
||||
|
|
@ -491,6 +502,7 @@ def render_dialog(
|
|||
new_private_metadata["input_id_prefix"] = new_input_id_prefix
|
||||
selected_organization = predefined_org if predefined_org else available_organizations.first()
|
||||
is_team_selected, selected_team = False, None
|
||||
is_team_escalation_important = False
|
||||
else:
|
||||
# setup form using data/state
|
||||
old_input_id_prefix, new_input_id_prefix, new_private_metadata = _get_and_change_input_id_prefix_from_metadata(
|
||||
|
|
@ -502,6 +514,7 @@ def render_dialog(
|
|||
else _get_selected_org_from_payload(payload, old_input_id_prefix, slack_team_identity, slack_user_identity)
|
||||
)
|
||||
is_team_selected, selected_team = _get_selected_team_from_payload(payload, old_input_id_prefix)
|
||||
is_team_escalation_important = _get_team_escalation_severity_from_payload(payload, old_input_id_prefix)
|
||||
|
||||
blocks: Block.AnyBlocks = []
|
||||
|
||||
|
|
@ -523,9 +536,14 @@ def render_dialog(
|
|||
)
|
||||
blocks.append(organization_select)
|
||||
|
||||
# Add team select and additional responders blocks
|
||||
# Add team select/severity and additional responders blocks
|
||||
blocks += _get_team_select_blocks(
|
||||
slack_user_identity, selected_organization, is_team_selected, selected_team, new_input_id_prefix
|
||||
slack_user_identity,
|
||||
selected_organization,
|
||||
is_team_selected,
|
||||
selected_team,
|
||||
is_team_escalation_important,
|
||||
new_input_id_prefix,
|
||||
)
|
||||
blocks += _get_user_select_blocks(payload, selected_organization, new_input_id_prefix, error_msg)
|
||||
|
||||
|
|
@ -629,6 +647,25 @@ def _get_select_field_value(payload: EventPayload, prefix_id: str, routing_uid:
|
|||
return json.loads(field["value"])["id"] if field else None
|
||||
|
||||
|
||||
def _get_first_selected_checkbox_option_value(
|
||||
payload: EventPayload,
|
||||
prefix_id: str,
|
||||
routing_uid: str,
|
||||
field_id: str,
|
||||
) -> str | None:
|
||||
"""
|
||||
NOTE: if reusing this for other logic outside of the team severity checkboxes, note that this function
|
||||
will only return the value of the first checkbox option...
|
||||
"""
|
||||
try:
|
||||
selected_options = payload["view"]["state"]["values"][prefix_id + field_id][routing_uid]["selected_options"]
|
||||
if not selected_options:
|
||||
return None
|
||||
return selected_options[0]["value"]
|
||||
except KeyError:
|
||||
return None
|
||||
|
||||
|
||||
def _get_selected_org_from_payload(
|
||||
payload: EventPayload,
|
||||
input_id_prefix: str,
|
||||
|
|
@ -676,6 +713,7 @@ def _get_team_select_blocks(
|
|||
organization: "Organization",
|
||||
is_selected: bool,
|
||||
value: typing.Optional["Team"],
|
||||
is_team_escalation_important: bool,
|
||||
input_id_prefix: str,
|
||||
) -> Block.AnyBlocks:
|
||||
blocks: Block.AnyBlocks = []
|
||||
|
|
@ -702,7 +740,7 @@ def _get_team_select_blocks(
|
|||
if not teams:
|
||||
direct_paging_info_msg["elements"][0][
|
||||
"text"
|
||||
] += ". There are currently no teams which have a Direct Paging integration that is configured."
|
||||
] += ".\n\nThere are currently no teams which have a Direct Paging integration that is configured."
|
||||
blocks.append(direct_paging_info_msg)
|
||||
return blocks
|
||||
|
||||
|
|
@ -769,6 +807,62 @@ def _get_team_select_blocks(
|
|||
}
|
||||
)
|
||||
|
||||
team_severity_important_checkbox_option: CompositionObjectOption = {
|
||||
"text": {
|
||||
"type": "mrkdwn",
|
||||
"text": "Important escalation",
|
||||
},
|
||||
"value": DIRECT_PAGING_TEAM_SEVERITY_CHECKBOX_VALUE,
|
||||
}
|
||||
|
||||
team_severity_checkboxes_element: Block.Section = {
|
||||
"type": "section",
|
||||
"block_id": input_id_prefix + DIRECT_PAGING_TEAM_SEVERITY_CHECKBOXES_ID,
|
||||
"text": {
|
||||
"type": "plain_text",
|
||||
# NOTE: this is a bit of a hack. Slack requires us to specify this text object, and it cannot be empty
|
||||
# hence the empty space. We do this so that we can render the text instead in a context block below
|
||||
# (which allows us to render it in a slightly smaller font size)
|
||||
# https://api.slack.com/reference/block-kit/blocks#section
|
||||
"text": " ",
|
||||
},
|
||||
"accessory": {
|
||||
"type": "checkboxes",
|
||||
"options": [team_severity_important_checkbox_option],
|
||||
"action_id": OnPagingTeamSeverityCheckboxChange.routing_uid(),
|
||||
},
|
||||
}
|
||||
|
||||
if is_team_escalation_important:
|
||||
# From the docs https://api.slack.com/reference/block-kit/block-elements#checkboxes__fields
|
||||
# An array of option objects that EXACTLY matches one or more of the options within options
|
||||
team_severity_checkboxes_element["accessory"]["initial_options"] = [team_severity_important_checkbox_option]
|
||||
|
||||
blocks.extend(
|
||||
[
|
||||
team_severity_checkboxes_element,
|
||||
typing.cast(
|
||||
Block.Context,
|
||||
{
|
||||
# NOTE: we add this here instead of as a checkbox option description because those can only
|
||||
# be defined as plain text (ie. not markdown where links are supported)
|
||||
"type": "context",
|
||||
"elements": [
|
||||
{
|
||||
"type": "mrkdwn",
|
||||
"text": (
|
||||
"Check the above box if you would like to escalate to this team as an 'important' "
|
||||
"escalation. Teams can configure their Direct Paging Integration to route to different "
|
||||
"escalation chains based on this. "
|
||||
"<https://grafana.com/docs/oncall/latest/integrations/manual/#important-escalations|Learn more>"
|
||||
),
|
||||
},
|
||||
],
|
||||
},
|
||||
),
|
||||
]
|
||||
)
|
||||
|
||||
return blocks
|
||||
|
||||
|
||||
|
|
@ -951,6 +1045,16 @@ def _get_selected_team_from_payload(
|
|||
return selected_team_id, Team.objects.filter(pk=selected_team_id).first()
|
||||
|
||||
|
||||
def _get_team_escalation_severity_from_payload(payload: EventPayload, input_id_prefix: str) -> bool:
|
||||
checkbox_value = _get_first_selected_checkbox_option_value(
|
||||
payload,
|
||||
input_id_prefix,
|
||||
OnPagingTeamSeverityCheckboxChange.routing_uid(),
|
||||
DIRECT_PAGING_TEAM_SEVERITY_CHECKBOXES_ID,
|
||||
)
|
||||
return checkbox_value == DIRECT_PAGING_TEAM_SEVERITY_CHECKBOX_VALUE
|
||||
|
||||
|
||||
def _get_selected_user_from_payload(payload: EventPayload, input_id_prefix: str) -> typing.Optional["User"]:
|
||||
from apps.user_management.models import User
|
||||
|
||||
|
|
@ -1035,6 +1139,12 @@ STEPS_ROUTING: ScenarioRoute.RoutingSteps = [
|
|||
"block_action_id": OnPagingTeamChange.routing_uid(),
|
||||
"step": OnPagingTeamChange,
|
||||
},
|
||||
{
|
||||
"payload_type": PayloadType.BLOCK_ACTIONS,
|
||||
"block_action_type": BlockActionType.CHECKBOXES,
|
||||
"block_action_id": OnPagingTeamSeverityCheckboxChange.routing_uid(),
|
||||
"step": OnPagingTeamSeverityCheckboxChange,
|
||||
},
|
||||
{
|
||||
"payload_type": PayloadType.BLOCK_ACTIONS,
|
||||
"block_action_type": BlockActionType.STATIC_SELECT,
|
||||
|
|
|
|||
|
|
@ -12,12 +12,14 @@ from apps.slack.scenarios.paging import (
|
|||
DIRECT_PAGING_MESSAGE_INPUT_ID,
|
||||
DIRECT_PAGING_ORG_SELECT_ID,
|
||||
DIRECT_PAGING_TEAM_SELECT_ID,
|
||||
DIRECT_PAGING_TEAM_SEVERITY_CHECKBOXES_ID,
|
||||
DIRECT_PAGING_USER_SELECT_ID,
|
||||
DataKey,
|
||||
FinishDirectPaging,
|
||||
OnPagingItemActionChange,
|
||||
OnPagingOrgChange,
|
||||
OnPagingTeamChange,
|
||||
OnPagingTeamSeverityCheckboxChange,
|
||||
OnPagingUserChange,
|
||||
Policy,
|
||||
StartDirectPaging,
|
||||
|
|
@ -28,7 +30,13 @@ from apps.user_management.models import Organization
|
|||
|
||||
|
||||
def make_paging_view_slack_payload(
|
||||
selected_org=None, predefined_org=None, team=None, user=None, current_users=None, actions=None
|
||||
selected_org=None,
|
||||
predefined_org=None,
|
||||
team=None,
|
||||
important_team_escalation=False,
|
||||
user=None,
|
||||
current_users=None,
|
||||
actions=None,
|
||||
):
|
||||
"""
|
||||
Helper function to create a payload for paging view.
|
||||
|
|
@ -66,6 +74,15 @@ def make_paging_view_slack_payload(
|
|||
}
|
||||
}
|
||||
},
|
||||
DIRECT_PAGING_TEAM_SEVERITY_CHECKBOXES_ID: {
|
||||
OnPagingTeamSeverityCheckboxChange.routing_uid(): {
|
||||
"selected_options": [
|
||||
{"value": "important"},
|
||||
]
|
||||
if important_team_escalation
|
||||
else []
|
||||
},
|
||||
},
|
||||
DIRECT_PAGING_TEAM_SELECT_ID: {
|
||||
OnPagingTeamChange.routing_uid(): {
|
||||
"selected_option": {"value": make_value({"id": team.pk if team else None}, organization)}
|
||||
|
|
@ -141,6 +158,7 @@ def test_page_team_with_predefined_org(make_organization_and_user_with_slack_ide
|
|||
from_user=user,
|
||||
message="The Message",
|
||||
team=team,
|
||||
important_team_escalation=False,
|
||||
users=[],
|
||||
)
|
||||
|
||||
|
|
@ -385,15 +403,21 @@ def test_trigger_paging_additional_responders(make_organization_and_user_with_sl
|
|||
from_user=user,
|
||||
message="The Message",
|
||||
team=team,
|
||||
important_team_escalation=False,
|
||||
users=[(user, True)],
|
||||
)
|
||||
|
||||
|
||||
@pytest.mark.parametrize("important_team_escalation", [True, False])
|
||||
@pytest.mark.django_db
|
||||
def test_page_team(make_organization_and_user_with_slack_identities, make_team):
|
||||
def test_page_team(make_organization_and_user_with_slack_identities, make_team, important_team_escalation):
|
||||
organization, user, slack_team_identity, slack_user_identity = make_organization_and_user_with_slack_identities()
|
||||
team = make_team(organization)
|
||||
payload = make_paging_view_slack_payload(selected_org=organization, team=team)
|
||||
payload = make_paging_view_slack_payload(
|
||||
selected_org=organization,
|
||||
team=team,
|
||||
important_team_escalation=important_team_escalation,
|
||||
)
|
||||
|
||||
step = FinishDirectPaging(slack_team_identity)
|
||||
with patch("apps.slack.scenarios.paging.direct_paging") as mock_direct_paging:
|
||||
|
|
@ -405,6 +429,7 @@ def test_page_team(make_organization_and_user_with_slack_identities, make_team):
|
|||
from_user=user,
|
||||
message="The Message",
|
||||
team=team,
|
||||
important_team_escalation=important_team_escalation,
|
||||
users=[],
|
||||
)
|
||||
|
||||
|
|
@ -421,6 +446,7 @@ def test_get_organization_select(make_organization):
|
|||
assert select["element"]["options"][0]["text"]["text"] == "Organization (stack_slug)"
|
||||
|
||||
|
||||
@pytest.mark.parametrize("is_team_escalation_important", [True, False])
|
||||
@pytest.mark.django_db
|
||||
def test_get_team_select_blocks(
|
||||
make_organization_and_user_with_slack_identities,
|
||||
|
|
@ -428,6 +454,7 @@ def test_get_team_select_blocks(
|
|||
make_alert_receive_channel,
|
||||
make_escalation_chain,
|
||||
make_channel_filter,
|
||||
is_team_escalation_important,
|
||||
):
|
||||
info_msg = (
|
||||
"*Note*: You can only page teams which have a Direct Paging integration that is configured. "
|
||||
|
|
@ -444,7 +471,14 @@ def test_get_team_select_blocks(
|
|||
|
||||
# no team selected - no team direct paging integrations available
|
||||
organization, _, _, slack_user_identity = make_organization_and_user_with_slack_identities()
|
||||
blocks = _get_team_select_blocks(slack_user_identity, organization, False, None, input_id_prefix)
|
||||
blocks = _get_team_select_blocks(
|
||||
slack_user_identity,
|
||||
organization,
|
||||
False,
|
||||
None,
|
||||
is_team_escalation_important,
|
||||
input_id_prefix,
|
||||
)
|
||||
|
||||
assert len(blocks) == 1
|
||||
|
||||
|
|
@ -452,7 +486,7 @@ def test_get_team_select_blocks(
|
|||
assert context_block["type"] == "context"
|
||||
assert (
|
||||
context_block["elements"][0]["text"]
|
||||
== info_msg + ". There are currently no teams which have a Direct Paging integration that is configured."
|
||||
== info_msg + ".\n\nThere are currently no teams which have a Direct Paging integration that is configured."
|
||||
)
|
||||
|
||||
# no team selected - 1 team direct paging integration available
|
||||
|
|
@ -462,7 +496,14 @@ def test_get_team_select_blocks(
|
|||
escalation_chain = make_escalation_chain(organization)
|
||||
make_channel_filter(arc, is_default=True, escalation_chain=escalation_chain)
|
||||
|
||||
blocks = _get_team_select_blocks(slack_user_identity, organization, False, None, input_id_prefix)
|
||||
blocks = _get_team_select_blocks(
|
||||
slack_user_identity,
|
||||
organization,
|
||||
False,
|
||||
None,
|
||||
is_team_escalation_important,
|
||||
input_id_prefix,
|
||||
)
|
||||
|
||||
assert len(blocks) == 2
|
||||
input_block, context_block = blocks
|
||||
|
|
@ -472,7 +513,7 @@ def test_get_team_select_blocks(
|
|||
assert input_block["element"]["options"] == [_contstruct_team_option(team)]
|
||||
assert context_block["elements"][0]["text"] == info_msg
|
||||
|
||||
# team selected
|
||||
# team selected - team severity checkbox should also now appear
|
||||
organization, _, _, slack_user_identity = make_organization_and_user_with_slack_identities()
|
||||
team1 = make_team(organization)
|
||||
team2 = make_team(organization)
|
||||
|
|
@ -488,10 +529,25 @@ def test_get_team_select_blocks(
|
|||
_setup_direct_paging_integration(team1)
|
||||
team2_direct_paging_arc = _setup_direct_paging_integration(team2)
|
||||
|
||||
blocks = _get_team_select_blocks(slack_user_identity, organization, True, team2, input_id_prefix)
|
||||
blocks = _get_team_select_blocks(
|
||||
slack_user_identity,
|
||||
organization,
|
||||
True,
|
||||
team2,
|
||||
is_team_escalation_important,
|
||||
input_id_prefix,
|
||||
)
|
||||
|
||||
assert len(blocks) == 2
|
||||
input_block, context_block = blocks
|
||||
assert len(blocks) == 4
|
||||
input_block, context_block, team_severity_checkboxes, team_severity_context_block = blocks
|
||||
|
||||
team_severity_important_checkbox_option = {
|
||||
"text": {
|
||||
"type": "mrkdwn",
|
||||
"text": "Important escalation",
|
||||
},
|
||||
"value": "important",
|
||||
}
|
||||
|
||||
team1_option = _contstruct_team_option(team1)
|
||||
team2_option = _contstruct_team_option(team2)
|
||||
|
|
@ -509,6 +565,20 @@ def test_get_team_select_blocks(
|
|||
== f"Integration <{team2_direct_paging_arc.web_link}|{team2_direct_paging_arc.verbal_name}> will be used for notification."
|
||||
)
|
||||
|
||||
assert team_severity_context_block["elements"][0]["text"] == (
|
||||
"Check the above box if you would like to escalate to this team as an 'important' "
|
||||
"escalation. Teams can configure their Direct Paging Integration to route to different "
|
||||
"escalation chains based on this. "
|
||||
"<https://grafana.com/docs/oncall/latest/integrations/manual/#important-escalations|Learn more>"
|
||||
)
|
||||
assert team_severity_checkboxes["accessory"]["type"] == "checkboxes"
|
||||
assert team_severity_checkboxes["accessory"]["options"] == [team_severity_important_checkbox_option]
|
||||
|
||||
if is_team_escalation_important:
|
||||
assert team_severity_checkboxes["accessory"]["initial_options"] == [team_severity_important_checkbox_option]
|
||||
else:
|
||||
assert "initial_options" not in team_severity_checkboxes["accessory"]
|
||||
|
||||
# team's direct paging integration has two routes associated with it
|
||||
# the team should only be displayed once
|
||||
organization, _, _, slack_user_identity = make_organization_and_user_with_slack_identities()
|
||||
|
|
@ -519,7 +589,14 @@ def test_get_team_select_blocks(
|
|||
make_channel_filter(arc, is_default=True, escalation_chain=escalation_chain)
|
||||
make_channel_filter(arc, escalation_chain=escalation_chain)
|
||||
|
||||
blocks = _get_team_select_blocks(slack_user_identity, organization, False, None, input_id_prefix)
|
||||
blocks = _get_team_select_blocks(
|
||||
slack_user_identity,
|
||||
organization,
|
||||
False,
|
||||
None,
|
||||
is_team_escalation_important,
|
||||
input_id_prefix,
|
||||
)
|
||||
|
||||
assert len(blocks) == 2
|
||||
input_block, context_block = blocks
|
||||
|
|
|
|||
|
|
@ -203,23 +203,33 @@ def test_sync_teams_for_organization(make_organization, make_team, make_alert_re
|
|||
assert created_team.team_id == api_teams[2]["id"]
|
||||
assert created_team.name == api_teams[2]["name"]
|
||||
|
||||
def _assert_teams_direct_paging_integration_is_configured_properly(integration):
|
||||
assert integration.channel_filters.count() == 2
|
||||
|
||||
for route in integration.channel_filters.all():
|
||||
if route.is_default:
|
||||
assert route.order == 1
|
||||
assert route.filtering_term is None
|
||||
else:
|
||||
assert route.order == 0
|
||||
assert route.filtering_term == "{{ payload.oncall.important }}"
|
||||
assert route.filtering_term_type == route.FILTERING_TERM_TYPE_JINJA2
|
||||
|
||||
# check that direct paging is created for created team
|
||||
direct_paging_integration = AlertReceiveChannel.objects.get(
|
||||
organization=organization,
|
||||
integration=AlertReceiveChannel.INTEGRATION_DIRECT_PAGING,
|
||||
team=created_team,
|
||||
)
|
||||
assert direct_paging_integration.channel_filters.count() == 1
|
||||
assert direct_paging_integration.channel_filters.first().order == 0
|
||||
assert direct_paging_integration.channel_filters.first().is_default
|
||||
_assert_teams_direct_paging_integration_is_configured_properly(direct_paging_integration)
|
||||
|
||||
# check that direct paging is created for existing team
|
||||
direct_paging_integration = AlertReceiveChannel.objects.get(
|
||||
organization=organization, integration=AlertReceiveChannel.INTEGRATION_DIRECT_PAGING, team=teams[2]
|
||||
organization=organization,
|
||||
integration=AlertReceiveChannel.INTEGRATION_DIRECT_PAGING,
|
||||
team=teams[2],
|
||||
)
|
||||
assert direct_paging_integration.channel_filters.count() == 1
|
||||
assert direct_paging_integration.channel_filters.first().order == 0
|
||||
assert direct_paging_integration.channel_filters.first().is_default
|
||||
_assert_teams_direct_paging_integration_is_configured_properly(direct_paging_integration)
|
||||
|
||||
|
||||
@pytest.mark.django_db
|
||||
|
|
|
|||
Loading…
Add table
Reference in a new issue