From 23fd02d50de8d93fcf2711e44d85d713d35a5c31 Mon Sep 17 00:00:00 2001 From: Robby Milo Date: Thu, 25 Aug 2022 12:24:11 +0200 Subject: [PATCH 01/17] fix docs breadcrumbs --- .../_index.md} | 0 docs/sources/{getting-started.md => getting-started/_index.md} | 0 docs/sources/{open-source.md => open-source/_index.md} | 0 3 files changed, 0 insertions(+), 0 deletions(-) rename docs/sources/{configure-user-settings.md => configure-user-settings/_index.md} (100%) rename docs/sources/{getting-started.md => getting-started/_index.md} (100%) rename docs/sources/{open-source.md => open-source/_index.md} (100%) diff --git a/docs/sources/configure-user-settings.md b/docs/sources/configure-user-settings/_index.md similarity index 100% rename from docs/sources/configure-user-settings.md rename to docs/sources/configure-user-settings/_index.md diff --git a/docs/sources/getting-started.md b/docs/sources/getting-started/_index.md similarity index 100% rename from docs/sources/getting-started.md rename to docs/sources/getting-started/_index.md diff --git a/docs/sources/open-source.md b/docs/sources/open-source/_index.md similarity index 100% rename from docs/sources/open-source.md rename to docs/sources/open-source/_index.md From b45a420a302ea1bdaa0f8842ba2ca6051c49a4fd Mon Sep 17 00:00:00 2001 From: Robby Milo Date: Thu, 25 Aug 2022 12:34:49 +0200 Subject: [PATCH 02/17] fix ref errors --- docs/sources/chat-options/configure-slack.md | 14 +++++----- docs/sources/getting-started/_index.md | 28 ++++++++++---------- 2 files changed, 21 insertions(+), 21 deletions(-) diff --git a/docs/sources/chat-options/configure-slack.md b/docs/sources/chat-options/configure-slack.md index 25a5d27a..77126bb9 100644 --- a/docs/sources/chat-options/configure-slack.md +++ b/docs/sources/chat-options/configure-slack.md @@ -16,16 +16,16 @@ weight: 100 # Slack integration for Grafana OnCall -The Slack integration for Grafana OnCall incorporates your Slack workspace directly into your incident response workflow to help your team focus on alert resolution with less friction. +The Slack integration for Grafana OnCall incorporates your Slack workspace directly into your incident response workflow to help your team focus on alert resolution with less friction. Integrating your Slack workspace with Grafana OnCall allows users and teams to be notified of alerts directly in Slack with automated alert escalation steps and user notification preferences. There are a number of alert actions that users can take directly from Slack, including acknowledge, resolve, add resolution notes, and more. ## Before you begin -To install the Slack integration, you must have Admin permissions in your Grafana instance as well as the Slack workspace that you’d like to integrate with. +To install the Slack integration, you must have Admin permissions in your Grafana instance as well as the Slack workspace that you’d like to integrate with. -For Open Source Grafana OnCall Slack installation guidance, refer to [Open Source Grafana OnCall]({{< relref "../open-source.md" >}}). +For Open Source Grafana OnCall Slack installation guidance, refer to [Open Source Grafana OnCall]({{< relref "../open-source" >}}). ## Install Slack integration for Grafana OnCall @@ -41,23 +41,23 @@ For Open Source Grafana OnCall Slack installation guidance, refer to [Open Sourc Configure the following additional settings to ensure Grafana OnCall alerts are routed to the intended Slack channels and users: 1. From your **Slack integration** settings, select a default slack channel in the first dropdown menu. This is where alerts will be sent unless otherwise specified in escalation chains. -2. In **Additional Settings**, configure alert reminders for alerts to retrigger after being acknowledged for some amount of time. +2. In **Additional Settings**, configure alert reminders for alerts to retrigger after being acknowledged for some amount of time. 3. Ensure all users verify their slack account in their Grafana OnCall **users info**. ### Configure Escalation Chains with Slack notifications -Once your Slack integration is configured you can configure Escalation Chains to notify via Slack messages for alerts in Grafana OnCall. +Once your Slack integration is configured you can configure Escalation Chains to notify via Slack messages for alerts in Grafana OnCall. There are two Slack notification options that you can configure into escalation chains, notify whole Slack channel and notify Slack user group: 1. In Grafana OnCall, navigate to the **Escalation Chains** tab then select an existing escalation chain or click **+ New escalation chain**. 2. Click the dropdown for **Add escalation step**. -3. Configure your escalation chain with automated Slack notifications. +3. Configure your escalation chain with automated Slack notifications. ### Configure user notifications with Slack mentions To be notified of alerts in Grafana OnCall via Slack mentions: 1. Navigate to the **Users** tab in Grafana OnCall, click **Edit** next to a user. -2. In the **User Info** tab, edit or configure notification steps by clicking + Add Notification step +2. In the **User Info** tab, edit or configure notification steps by clicking + Add Notification step 3. select **Notify by** in the first dropdown and select **Slack mentions** in the second dropdown to receive alert notifications via Slack mentions. ### Configure on-call notifications in Slack diff --git a/docs/sources/getting-started/_index.md b/docs/sources/getting-started/_index.md index 38726bb5..e84ffdd2 100644 --- a/docs/sources/getting-started/_index.md +++ b/docs/sources/getting-started/_index.md @@ -24,16 +24,16 @@ The following diagram details an example alert workflow with Grafana OnCall: These procedures introduce you to initial Grafana OnCall configuration steps, including monitoring system integration, how to set up escalation chains, and how to use your calendar service for on-call scheduling. -## Before you begin +## Before you begin -Grafana OnCall is available for Grafana Cloud as well as Grafana open source users. You must have a Grafana Cloud account or [Open Source Grafana OnCall]({{< relref "open-source.md" >}}) +Grafana OnCall is available for Grafana Cloud as well as Grafana open source users. You must have a Grafana Cloud account or [Open Source Grafana OnCall]({{< relref "../open-source" >}}) For more information, see [Grafana Pricing](https://grafana.com/pricing/) for details. ## Install Open Source Grafana OnCall -For Open Source Grafana OnCall installation guidance, refer to [Open Source Grafana OnCall]({{< relref "open-source.md" >}}) +For Open Source Grafana OnCall installation guidance, refer to [Open Source Grafana OnCall]({{< relref "../open-source" >}}) >**Note:** If you are using Grafana OnCall with your Grafana Cloud instance there are no install steps. Access Grafana OnCall from your Grafana Cloud account and skip ahead to “Get alerts into Grafana OnCall” @@ -53,13 +53,13 @@ Regardless of where your alerts originate, you can send them to Grafana OnCall v 4. Complete any necessary configurations in your monitoring system to send alerts to Grafana OnCall. -#### Send a demo alert +#### Send a demo alert 1. In the integration tab, click **Send demo alert** then navigate to the **Alert Groups** tab to see your test alert firing. 2. Explore the alert by clicking on the title of the alert. 3. Acknowledge and resolve the test alert. -For more information on Grafana OnCall integrations and further configuration guidance, refer to, [Connect to Grafana OnCall]({{< relref "integrations/" >}}) +For more information on Grafana OnCall integrations and further configuration guidance, refer to, [Connect to Grafana OnCall]({{< relref "../integrations" >}}) ### Configure Escalation Chains @@ -72,18 +72,18 @@ To configure Escalation Chains: 1. Navigate to the **Escalation Chains** tab and click **+ New Escalation Chain** 2. Give your Escalation Chain a useful name and click **Create** 3. Add a series of escalation steps from the available dropdown options. -4. To link your Escalation Chain to your integration, navigate back to the **Integrations tab**, Select your newly created Escalation Chain from the “**Escalate to**” dropdown. +4. To link your Escalation Chain to your integration, navigate back to the **Integrations tab**, Select your newly created Escalation Chain from the “**Escalate to**” dropdown. Alerts from this integration will now follow the escalation steps configured in your Escalation Chain. -For more information on Escalation Chains and more ways to customize them, refer to [Configure and manage Escalation Chains]({{< relref "escalation-policies/configure-escalation-chains/" >}}) +For more information on Escalation Chains and more ways to customize them, refer to [Configure and manage Escalation Chains]({{< relref "../escalation-policies/configure-escalation-chains" >}}) ## Get notified of an alert In order for Grafana OnCall to notify you of an alert, you must configure how you want to be notified. Personal notification policies, chatops integrations, and on-call schedules allow you to automate how users are notified of alerts. -### Configure personal notification policies -Personal notification policies determine how a user is notified for a certain type of alert. Get notified by SMS, phone call, or Slack mentions. Administrators can configure how users receive notification for certain types of alerts. For more information on personal notification policies, refer to [Manage users and teams for Grafana OnCall]({{< relref "configure-user-settings/" >}}) +### Configure personal notification policies +Personal notification policies determine how a user is notified for a certain type of alert. Get notified by SMS, phone call, or Slack mentions. Administrators can configure how users receive notification for certain types of alerts. For more information on personal notification policies, refer to [Manage users and teams for Grafana OnCall]({{< relref "../configure-user-settings" >}}) To configure users personal notification policies: @@ -94,7 +94,7 @@ To configure users personal notification policies: ### Configure Slack for Grafana OnCall -Grafana OnCall integrates closely with your Slack workspace to deliver alert notifications to individuals, user groups, and channels. Slack notifications can be triggered by steps in an escalation chain or as a step in users personal notification policies. +Grafana OnCall integrates closely with your Slack workspace to deliver alert notifications to individuals, user groups, and channels. Slack notifications can be triggered by steps in an escalation chain or as a step in users personal notification policies. To configure Slack for Grafana OnCall: @@ -105,20 +105,20 @@ To configure Slack for Grafana OnCall: 5. Click Allow to allow Grafana OnCall to access Slack. 6. Ensure users verify their Slack accounts in their user profile in Grafana OnCall. -For further instruction on connecting to your Slack workspace, refer to [Connect Slack to Grafana OnCall]({{< relref "chat-options/configure-slack/" >}}) +For further instruction on connecting to your Slack workspace, refer to [Connect Slack to Grafana OnCall]({{< relref "../chat-options/configure-slack" >}}) ### Add your on-call schedule -Grafana OnCall allows you to manage your on-call schedule in your preferred calendar app such as Google Calendar or Microsoft Outlook. +Grafana OnCall allows you to manage your on-call schedule in your preferred calendar app such as Google Calendar or Microsoft Outlook. To integrate your on-call calendar with Grafana OnCall: 1. In the **Schedules** tab of Grafana OnCall, click **+ Add team schedule for on-call rotation**. 2. Provide a schedule name. -3. Copy the iCal URL associated with your on-call calendar from your calendar integration settings. +3. Copy the iCal URL associated with your on-call calendar from your calendar integration settings. 4. Configure the rest of the schedule settings and click Create Schedule -For more information on on-call schedules, refer to [Configure and manage on-call schedules]({{< relref "calendar-schedules/" >}}) +For more information on on-call schedules, refer to [Configure and manage on-call schedules]({{< relref "../calendar-schedules" >}}) From 50dc7d62404c780e9dd9ef6e2ef87a527f49fc16 Mon Sep 17 00:00:00 2001 From: Michael Derynck Date: Thu, 1 Sep 2022 11:39:47 -0600 Subject: [PATCH 03/17] Handle a exceptions on some retrying tasks --- .../escalation_snapshot/escalation_snapshot_mixin.py | 6 +++++- .../base/models/user_notification_policy_log_record.py | 8 +++++++- engine/apps/slack/scenarios/notification_delivery.py | 9 --------- engine/apps/telegram/models/connectors/personal.py | 7 +++++++ engine/apps/telegram/tasks.py | 9 ++++++++- 5 files changed, 27 insertions(+), 12 deletions(-) diff --git a/engine/apps/alerts/escalation_snapshot/escalation_snapshot_mixin.py b/engine/apps/alerts/escalation_snapshot/escalation_snapshot_mixin.py index 0ec5e67c..40f983e5 100644 --- a/engine/apps/alerts/escalation_snapshot/escalation_snapshot_mixin.py +++ b/engine/apps/alerts/escalation_snapshot/escalation_snapshot_mixin.py @@ -7,6 +7,7 @@ from dateutil.parser import parse from django.apps import apps from django.utils import timezone from django.utils.functional import cached_property +from rest_framework.exceptions import ValidationError from apps.alerts.constants import NEXT_ESCALATION_DELAY from apps.alerts.escalation_snapshot.snapshot_classes import ( @@ -189,7 +190,10 @@ class EscalationSnapshotMixin: escalation_snapshot_object = None raw_escalation_snapshot = self.raw_escalation_snapshot if raw_escalation_snapshot is not None: - escalation_snapshot_object = self._deserialize_escalation_snapshot(raw_escalation_snapshot) + try: + escalation_snapshot_object = self._deserialize_escalation_snapshot(raw_escalation_snapshot) + except ValidationError as e: + logger.error(f"Error trying to deserialize raw escalation snapshot: {e}") return escalation_snapshot_object def _deserialize_escalation_snapshot(self, raw_escalation_snapshot) -> EscalationSnapshot: diff --git a/engine/apps/base/models/user_notification_policy_log_record.py b/engine/apps/base/models/user_notification_policy_log_record.py index ed261b2b..e29a9ec4 100644 --- a/engine/apps/base/models/user_notification_policy_log_record.py +++ b/engine/apps/base/models/user_notification_policy_log_record.py @@ -69,7 +69,8 @@ class UserNotificationPolicyLogRecord(models.Model): ERROR_NOTIFICATION_IN_SLACK_RATELIMIT, ERROR_NOTIFICATION_MESSAGING_BACKEND_ERROR, ERROR_NOTIFICATION_NOT_ALLOWED_USER_ROLE, - ) = range(26) + ERROR_NOTIFICATION_TELEGRAM_USER_IS_DEACTIVATED, + ) = range(27) # for this errors we want to send message to general log channel ERRORS_TO_SEND_IN_SLACK_CHANNEL = [ @@ -272,6 +273,11 @@ class UserNotificationPolicyLogRecord(models.Model): self.notification_error_code == UserNotificationPolicyLogRecord.ERROR_NOTIFICATION_NOT_ALLOWED_USER_ROLE ): result += f"failed to notify {user_verbal}, not allowed role" + elif ( + self.notification_error_code + == UserNotificationPolicyLogRecord.ERROR_NOTIFICATION_TELEGRAM_USER_IS_DEACTIVATED + ): + result += f"failed to send telegram message to {user_verbal} because user has been deactivated" else: # TODO: handle specific backend errors try: diff --git a/engine/apps/slack/scenarios/notification_delivery.py b/engine/apps/slack/scenarios/notification_delivery.py index 4b9805da..73ed7d05 100644 --- a/engine/apps/slack/scenarios/notification_delivery.py +++ b/engine/apps/slack/scenarios/notification_delivery.py @@ -87,12 +87,3 @@ class NotificationDeliveryStep(scenario_step.ScenarioStep): print(e) else: raise e - - def get_color_id(self, color): - if color == "red": - color_id = "#FF0000" - elif color == "yellow": - color_id = "#c6c000" - else: - color_id = color - return color_id diff --git a/engine/apps/telegram/models/connectors/personal.py b/engine/apps/telegram/models/connectors/personal.py index 895c6a62..8ac440ac 100644 --- a/engine/apps/telegram/models/connectors/personal.py +++ b/engine/apps/telegram/models/connectors/personal.py @@ -111,6 +111,13 @@ class TelegramToUserConnector(models.Model): notification_policy, UserNotificationPolicyLogRecord.ERROR_NOTIFICATION_TELEGRAM_TOKEN_ERROR, ) + elif e.message == "Forbidden: user is deactivated": + TelegramToUserConnector.create_telegram_notification_error( + alert_group, + self.user, + notification_policy, + UserNotificationPolicyLogRecord.ERROR_NOTIFICATION_TELEGRAM_USER_IS_DEACTIVATED, + ) else: raise e else: diff --git a/engine/apps/telegram/tasks.py b/engine/apps/telegram/tasks.py index 9f6cfd97..cd55aa27 100644 --- a/engine/apps/telegram/tasks.py +++ b/engine/apps/telegram/tasks.py @@ -119,7 +119,14 @@ def send_link_to_channel_message_or_fallback_to_full_incident( @ignore_bot_deleted def send_log_and_actions_message(self, channel_chat_id, group_chat_id, channel_message_id, reply_to_message_id): with OkToRetry(task=self, exc=TelegramMessage.DoesNotExist, num_retries=5): - channel_message = TelegramMessage.objects.get(chat_id=channel_chat_id, message_id=channel_message_id) + try: + channel_message = TelegramMessage.objects.get(chat_id=channel_chat_id, message_id=channel_message_id) + except TelegramMessage.DoesNotExist: + logger.warning( + f"Could not send log and actions message, telegram message does not exit " + f" chat_id={channel_chat_id} message_id={channel_message_id}" + ) + return if channel_message.discussion_group_message_id is None: channel_message.discussion_group_message_id = reply_to_message_id From 0ef2cbf77b440894b455218ce9dc669118d0e7d0 Mon Sep 17 00:00:00 2001 From: Michael Derynck Date: Thu, 1 Sep 2022 11:41:52 -0600 Subject: [PATCH 04/17] Fix message typo --- engine/apps/telegram/tasks.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/engine/apps/telegram/tasks.py b/engine/apps/telegram/tasks.py index cd55aa27..8d8c0e0b 100644 --- a/engine/apps/telegram/tasks.py +++ b/engine/apps/telegram/tasks.py @@ -123,7 +123,7 @@ def send_log_and_actions_message(self, channel_chat_id, group_chat_id, channel_m channel_message = TelegramMessage.objects.get(chat_id=channel_chat_id, message_id=channel_message_id) except TelegramMessage.DoesNotExist: logger.warning( - f"Could not send log and actions message, telegram message does not exit " + f"Could not send log and actions message, telegram message does not exist " f" chat_id={channel_chat_id} message_id={channel_message_id}" ) return From ecbc3ea778b9a376ee9035bdf2b6a960e1e80937 Mon Sep 17 00:00:00 2001 From: Matias Bordese Date: Fri, 2 Sep 2022 14:26:47 -0300 Subject: [PATCH 05/17] Fix channel filter updates when there are multiple backends --- engine/apps/api/serializers/channel_filter.py | 5 +++-- engine/apps/api/tests/test_channel_filter.py | 13 +++++++++++-- 2 files changed, 14 insertions(+), 4 deletions(-) diff --git a/engine/apps/api/serializers/channel_filter.py b/engine/apps/api/serializers/channel_filter.py index c8b58ef2..d739ff1f 100644 --- a/engine/apps/api/serializers/channel_filter.py +++ b/engine/apps/api/serializers/channel_filter.py @@ -96,7 +96,7 @@ class ChannelFilterSerializer(OrderedModelSerializerMixin, EagerLoadingMixin, se organization = self.context["request"].auth.organization if not isinstance(notification_backends, dict): raise serializers.ValidationError(["Invalid messaging backend data"]) - current = self.instance.notification_backends or {} + updated = self.instance.notification_backends or {} for backend_id in notification_backends: backend = get_messaging_backend_from_id(backend_id) if backend is None: @@ -106,7 +106,8 @@ class ChannelFilterSerializer(OrderedModelSerializerMixin, EagerLoadingMixin, se notification_backends[backend_id], ) # update existing backend data - notification_backends[backend_id] = current.get(backend_id, {}) | updated_data + updated[backend_id] = updated.get(backend_id, {}) | updated_data + notification_backends = updated return notification_backends diff --git a/engine/apps/api/tests/test_channel_filter.py b/engine/apps/api/tests/test_channel_filter.py index 8a608eb8..f70c8956 100644 --- a/engine/apps/api/tests/test_channel_filter.py +++ b/engine/apps/api/tests/test_channel_filter.py @@ -437,7 +437,10 @@ def test_channel_filter_update_notification_backends_updates_existing_data( ): organization, user, token = make_organization_and_user_with_plugin_token() alert_receive_channel = make_alert_receive_channel(organization) - existing_notification_backends = {"TESTONLY": {"enabled": True, "channel": "ABCDEF"}} + existing_notification_backends = { + "TESTONLY": {"enabled": True, "channel": "ABCDEF"}, + "ANOTHERONE": {"enabled": False, "channel": "123456"}, + } channel_filter = make_channel_filter(alert_receive_channel, notification_backends=existing_notification_backends) client = APIClient() @@ -448,7 +451,13 @@ def test_channel_filter_update_notification_backends_updates_existing_data( "notification_backends": notification_backends_update, } - response = client.put(url, data=data_for_update, format="json", **make_user_auth_headers(user, token)) + class FakeBackend: + def validate_channel_filter_data(self, organization, data): + return data + + with patch("apps.api.serializers.channel_filter.get_messaging_backend_from_id") as mock_get_backend: + mock_get_backend.return_value = FakeBackend() + response = client.put(url, data=data_for_update, format="json", **make_user_auth_headers(user, token)) channel_filter.refresh_from_db() From c3b054ae610651ad08cddbaddaf209cd10ee1045 Mon Sep 17 00:00:00 2001 From: Michael Derynck Date: Fri, 2 Sep 2022 15:03:08 -0600 Subject: [PATCH 06/17] Retry on exception first 5 times --- engine/apps/telegram/tasks.py | 13 ++++++++----- 1 file changed, 8 insertions(+), 5 deletions(-) diff --git a/engine/apps/telegram/tasks.py b/engine/apps/telegram/tasks.py index 8d8c0e0b..fff14616 100644 --- a/engine/apps/telegram/tasks.py +++ b/engine/apps/telegram/tasks.py @@ -122,11 +122,14 @@ def send_log_and_actions_message(self, channel_chat_id, group_chat_id, channel_m try: channel_message = TelegramMessage.objects.get(chat_id=channel_chat_id, message_id=channel_message_id) except TelegramMessage.DoesNotExist: - logger.warning( - f"Could not send log and actions message, telegram message does not exist " - f" chat_id={channel_chat_id} message_id={channel_message_id}" - ) - return + if self.request.retries <= 5: + raise + else: + logger.warning( + f"Could not send log and actions message, telegram message does not exist " + f" chat_id={channel_chat_id} message_id={channel_message_id}" + ) + return if channel_message.discussion_group_message_id is None: channel_message.discussion_group_message_id = reply_to_message_id From d74f4cc2be2063f339672a2ae05a598a1e8183f4 Mon Sep 17 00:00:00 2001 From: Matias Bordese Date: Mon, 5 Sep 2022 17:23:35 -0300 Subject: [PATCH 07/17] Update next shifts per user to list all (web) schedule users --- engine/apps/api/tests/test_schedules.py | 22 ++++++-- engine/apps/api/views/schedule.py | 4 +- .../apps/schedules/models/on_call_schedule.py | 20 +++++++ .../schedules/tests/test_on_call_schedule.py | 54 ++++++++++++++++++- 4 files changed, 93 insertions(+), 7 deletions(-) diff --git a/engine/apps/api/tests/test_schedules.py b/engine/apps/api/tests/test_schedules.py index d2f90957..ab29ef5f 100644 --- a/engine/apps/api/tests/test_schedules.py +++ b/engine/apps/api/tests/test_schedules.py @@ -817,7 +817,7 @@ def test_next_shifts_per_user( ) tomorrow = timezone.now().replace(hour=0, minute=0, second=0, microsecond=0) + timezone.timedelta(days=1) - user_a, user_b, user_c = (make_user_for_organization(organization, username=i) for i in "ABC") + user_a, user_b, user_c, user_d = (make_user_for_organization(organization, username=i) for i in "ABCD") shifts = ( # user, priority, start time (h), duration (hs) @@ -841,6 +841,19 @@ def test_next_shifts_per_user( ) on_call_shift.users.add(user) + # override in the past: 17-18 / D + # won't be listed, but user D will still be included in the response + override_data = { + "start": tomorrow - timezone.timedelta(days=3), + "rotation_start": tomorrow - timezone.timedelta(days=3), + "duration": timezone.timedelta(hours=1), + "schedule": schedule, + } + override = make_on_call_shift( + organization=organization, shift_type=CustomOnCallShift.TYPE_OVERRIDE, **override_data + ) + override.add_rolling_users([[user_d]]) + # override: 17-18 / C override_data = { "start": tomorrow + timezone.timedelta(hours=17), @@ -853,7 +866,7 @@ def test_next_shifts_per_user( ) override.add_rolling_users([[user_c]]) - # final schedule: 7-12: B, 15-16: A, 16-17: B, 17-18: C (override), 18-20: C + # final sdhedule: 7-12: B, 15-16: A, 16-17: B, 17-18: C (override), 18-20: C url = reverse("api-internal:schedule-next-shifts-per-user", kwargs={"pk": schedule.public_primary_key}) response = client.get(url, format="json", **make_user_auth_headers(user, token)) @@ -863,8 +876,11 @@ def test_next_shifts_per_user( user_a.public_primary_key: (tomorrow + timezone.timedelta(hours=15), tomorrow + timezone.timedelta(hours=16)), user_b.public_primary_key: (tomorrow + timezone.timedelta(hours=7), tomorrow + timezone.timedelta(hours=12)), user_c.public_primary_key: (tomorrow + timezone.timedelta(hours=17), tomorrow + timezone.timedelta(hours=18)), + user_d.public_primary_key: None, + } + returned_data = { + u: (ev["start"], ev["end"]) if ev is not None else None for u, ev in response.data["users"].items() } - returned_data = {u: (ev["start"], ev["end"]) for u, ev in response.data["users"].items()} assert returned_data == expected diff --git a/engine/apps/api/views/schedule.py b/engine/apps/api/views/schedule.py index 7ef4792b..40b9bffd 100644 --- a/engine/apps/api/views/schedule.py +++ b/engine/apps/api/views/schedule.py @@ -251,10 +251,10 @@ class ScheduleView( schedule = self.original_get_object() events = schedule.final_events(user_tz, starting_date, days=30) - users = {} + users = {u: None for u in schedule.related_users()} for e in events: user = e["users"][0]["pk"] if e["users"] else None - if user is not None and user not in users and e["end"] > now: + if user is not None and users.get(user) is None and e["end"] > now: users[user] = e result = {"users": users} diff --git a/engine/apps/schedules/models/on_call_schedule.py b/engine/apps/schedules/models/on_call_schedule.py index ad26893f..424ebe39 100644 --- a/engine/apps/schedules/models/on_call_schedule.py +++ b/engine/apps/schedules/models/on_call_schedule.py @@ -1,4 +1,5 @@ import datetime +import functools import itertools import icalendar @@ -196,6 +197,10 @@ class OnCallSchedule(PolymorphicModel): self.cached_ical_file_overrides = None self.save(update_fields=["cached_ical_file_overrides", "prev_ical_file_overrides"]) + def related_users(self): + """Return public primary keys for all users referenced in the schedule.""" + return set() + def filter_events(self, user_timezone, starting_date, days, with_empty=False, with_gap=False, filter_by=None): """Return filtered events from schedule.""" shifts = ( @@ -628,6 +633,21 @@ class OnCallScheduleWeb(OnCallSchedule): self.cached_ical_file_overrides = self._generate_ical_file_overrides() self.save(update_fields=["cached_ical_file_overrides", "prev_ical_file_overrides"]) + def related_users(self): + """Return public primary keys for all users referenced in the schedule.""" + rolling_users = self.custom_shifts.values_list("rolling_users", flat=True) + users = functools.reduce( + set.union, + ( + set(g.values()) + for rolling_groups in rolling_users + if rolling_groups is not None + for g in rolling_groups + if g is not None + ), + ) + return users + def preview_shift(self, custom_shift, user_tz, starting_date, days, updated_shift_pk=None): """Return unsaved rotation and final schedule preview events.""" if custom_shift.type == CustomOnCallShift.TYPE_OVERRIDE: diff --git a/engine/apps/schedules/tests/test_on_call_schedule.py b/engine/apps/schedules/tests/test_on_call_schedule.py index 766b44bb..f46bb4b2 100644 --- a/engine/apps/schedules/tests/test_on_call_schedule.py +++ b/engine/apps/schedules/tests/test_on_call_schedule.py @@ -261,9 +261,9 @@ def test_final_schedule_events(make_organization, make_user_for_organization, ma "schedule": schedule, } on_call_shift = make_on_call_shift( - organization=organization, shift_type=CustomOnCallShift.TYPE_RECURRENT_EVENT, **data + organization=organization, shift_type=CustomOnCallShift.TYPE_ROLLING_USERS_EVENT, **data ) - on_call_shift.users.add(user) + on_call_shift.add_rolling_users([[user]]) # override: 22-23 / E override_data = { @@ -710,3 +710,53 @@ def test_preview_override_shift(make_organization, make_user_for_organization, m # final ical schedule didn't change assert schedule._ical_file_overrides == schedule_overrides_ical + + +@pytest.mark.django_db +def test_schedule_related_users(make_organization, make_user_for_organization, make_on_call_shift, make_schedule): + organization = make_organization() + schedule = make_schedule( + organization, + schedule_class=OnCallScheduleWeb, + name="test_web_schedule", + ) + + now = timezone.now().replace(hour=0, minute=0, second=0, microsecond=0) + start_date = now - timezone.timedelta(days=7) + + user_a, _, _, user_d, user_e = (make_user_for_organization(organization, username=i) for i in "ABCDE") + + shifts = ( + # user, priority, start time (h), duration (hs) + (user_a, 1, 10, 5), # r1-1: 10-15 / A + (user_d, 2, 20, 3), # r2-4: 20-23 / D + ) + for user, priority, start_h, duration in shifts: + data = { + "start": start_date + timezone.timedelta(hours=start_h), + "rotation_start": start_date + timezone.timedelta(hours=start_h), + "duration": timezone.timedelta(hours=duration), + "priority_level": priority, + "frequency": CustomOnCallShift.FREQUENCY_DAILY, + "schedule": schedule, + } + on_call_shift = make_on_call_shift( + organization=organization, shift_type=CustomOnCallShift.TYPE_RECURRENT_EVENT, **data + ) + on_call_shift.add_rolling_users([[user]]) + + # override: 22-23 / E + override_data = { + "start": start_date - timezone.timedelta(hours=22), + "rotation_start": start_date - timezone.timedelta(hours=22), + "duration": timezone.timedelta(hours=1), + "schedule": schedule, + } + override = make_on_call_shift( + organization=organization, shift_type=CustomOnCallShift.TYPE_OVERRIDE, **override_data + ) + override.add_rolling_users([[user_e]]) + + schedule.refresh_from_db() + users = schedule.related_users() + assert users == set(u.public_primary_key for u in [user_a, user_d, user_e]) From 4e8727d1a1e927d2c4ddaa8a5f828f4e33dc2b3a Mon Sep 17 00:00:00 2001 From: Innokentii Konstantinov Date: Tue, 6 Sep 2022 19:12:10 +0500 Subject: [PATCH 08/17] New ical comparision func (#480) * New ical comparision func * Add support for field `sequence` for custom on-call shifts * Fix ical comparison * New ical comparision func 2 * fix * Revert "Add support for field `sequence` for custom on-call shifts" This reverts commit b7b18d5a Co-authored-by: Julia --- .../tasks/notify_ical_schedule_shift.py | 2 +- engine/apps/schedules/ical_utils.py | 28 +++++++++++++++++-- .../schedules/tasks/refresh_ical_files.py | 6 ++-- 3 files changed, 31 insertions(+), 5 deletions(-) diff --git a/engine/apps/alerts/tasks/notify_ical_schedule_shift.py b/engine/apps/alerts/tasks/notify_ical_schedule_shift.py index ce7049b0..9ec0f7e6 100644 --- a/engine/apps/alerts/tasks/notify_ical_schedule_shift.py +++ b/engine/apps/alerts/tasks/notify_ical_schedule_shift.py @@ -265,7 +265,7 @@ def notify_ical_schedule_shift(schedule_pk): for prev_ical_file, current_ical_file in prev_and_current_ical_files: if prev_ical_file is not None and ( - current_ical_file is None or not is_icals_equal(current_ical_file, prev_ical_file) + current_ical_file is None or not is_icals_equal(current_ical_file, prev_ical_file, schedule) ): # If icals are not equal then compare current_events from them is_prev_ical_diff = True diff --git a/engine/apps/schedules/ical_utils.py b/engine/apps/schedules/ical_utils.py index d78b99af..aa9ab809 100644 --- a/engine/apps/schedules/ical_utils.py +++ b/engine/apps/schedules/ical_utils.py @@ -33,7 +33,7 @@ This is a hack to allow us to load models for type checking without circular dep This module likely needs to refactored to be part of the OnCallSchedule module. """ if TYPE_CHECKING: - from apps.schedules.models import OnCallSchedule + from apps.schedules.models import OnCallSchedule, OnCallScheduleICal # noqa from apps.user_management.models import User @@ -408,7 +408,7 @@ def get_users_from_ical_event(event, organization): return users -def is_icals_equal(first, second): +def is_icals_equal_line_by_line(first, second): first = first.split("\n") second = second.split("\n") if len(first) != len(second): @@ -425,6 +425,30 @@ def is_icals_equal(first, second): return True +def is_icals_equal(first, second, schedule): + from apps.schedules.models import OnCallScheduleICal # noqa + + if isinstance(schedule, OnCallScheduleICal): + first_cal = Calendar.from_ical(first) + second_cal = Calendar.from_ical(second) + first_subcomponents = first_cal.subcomponents + second_subcomponents = second_cal.subcomponents + if len(first_subcomponents) != len(second_subcomponents): + return False + for idx, first_cmp in enumerate(first_cal.subcomponents): + second_cmp = second_subcomponents[idx] + if first_cmp.name == second_cmp.name == "VEVENT": + first_uid, first_seq = first_cmp.get("UID", None), first_cmp.get("SEQUENCE", None) + second_uid, second_seq = second_cmp.get("UID", None), second_cmp.get("SEQUENCE", None) + if first_uid != second_uid: + return False + elif first_seq != second_seq: + return False + return True + else: + return is_icals_equal_line_by_line(first, second) + + def ical_date_to_datetime(date, tz, start): datetime_to_combine = datetime.time.min all_day = False diff --git a/engine/apps/schedules/tasks/refresh_ical_files.py b/engine/apps/schedules/tasks/refresh_ical_files.py index 5797c668..cad0baee 100644 --- a/engine/apps/schedules/tasks/refresh_ical_files.py +++ b/engine/apps/schedules/tasks/refresh_ical_files.py @@ -46,7 +46,9 @@ def refresh_ical_file(schedule_pk): run_task_primary = True task_logger.info(f"run_task_primary {schedule_pk} {run_task_primary} prev_ical_file_primary is None") else: - run_task_primary = not is_icals_equal(schedule.cached_ical_file_primary, schedule.prev_ical_file_primary) + run_task_primary = not is_icals_equal( + schedule.cached_ical_file_primary, schedule.prev_ical_file_primary, schedule + ) task_logger.info(f"run_task_primary {schedule_pk} {run_task_primary} icals not equal") run_task_overrides = False if schedule.cached_ical_file_overrides is not None: @@ -55,7 +57,7 @@ def refresh_ical_file(schedule_pk): task_logger.info(f"run_task_overrides {schedule_pk} {run_task_primary} prev_ical_file_overrides is None") else: run_task_overrides = not is_icals_equal( - schedule.cached_ical_file_overrides, schedule.prev_ical_file_overrides + schedule.cached_ical_file_overrides, schedule.prev_ical_file_overrides, schedule ) task_logger.info(f"run_task_overrides {schedule_pk} {run_task_primary} icals not equal") run_task = run_task_primary or run_task_overrides From 66be6378f3e80348735a05ffe8e03393954616e7 Mon Sep 17 00:00:00 2001 From: Innokentii Konstantinov Date: Tue, 6 Sep 2022 19:43:48 +0500 Subject: [PATCH 09/17] Style fix --- engine/apps/schedules/ical_utils.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/engine/apps/schedules/ical_utils.py b/engine/apps/schedules/ical_utils.py index aa9ab809..b9594ad9 100644 --- a/engine/apps/schedules/ical_utils.py +++ b/engine/apps/schedules/ical_utils.py @@ -33,7 +33,7 @@ This is a hack to allow us to load models for type checking without circular dep This module likely needs to refactored to be part of the OnCallSchedule module. """ if TYPE_CHECKING: - from apps.schedules.models import OnCallSchedule, OnCallScheduleICal # noqa + from apps.schedules.models import OnCallSchedule from apps.user_management.models import User From 0432f6f72a401d76dddaf104560a80a9880a3148 Mon Sep 17 00:00:00 2001 From: Innokentii Konstantinov Date: Tue, 6 Sep 2022 20:18:03 +0500 Subject: [PATCH 10/17] Update CHANGELOG.md --- CHANGELOG.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index 1ee5efd1..41b3098c 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,8 @@ # Change Log +## v1.0.34 (2022-09-06) +- Fix schedule notification spam + ## v1.0.33 (2022-09-06) - Add raw alert view - Add GitHub star button for OSS installations From 1a133d33ff15bcbf83729dff5f79a25da8fb540c Mon Sep 17 00:00:00 2001 From: Matias Bordese Date: Tue, 6 Sep 2022 15:33:58 -0300 Subject: [PATCH 11/17] Add rotation ID fallback for ical events --- engine/apps/schedules/ical_utils.py | 3 +++ engine/apps/schedules/tests/test_ical_utils.py | 8 ++++++++ 2 files changed, 11 insertions(+) diff --git a/engine/apps/schedules/ical_utils.py b/engine/apps/schedules/ical_utils.py index b9594ad9..8055558f 100644 --- a/engine/apps/schedules/ical_utils.py +++ b/engine/apps/schedules/ical_utils.py @@ -367,6 +367,9 @@ def parse_event_uid(string): match = RE_EVENT_UID_V1.match(string) if match: _, _, _, source = match.groups() + else: + # fallback to use the UID string as the rotation ID + pk = string if source is not None: source = int(source) diff --git a/engine/apps/schedules/tests/test_ical_utils.py b/engine/apps/schedules/tests/test_ical_utils.py index e7946617..38213be8 100644 --- a/engine/apps/schedules/tests/test_ical_utils.py +++ b/engine/apps/schedules/tests/test_ical_utils.py @@ -78,3 +78,11 @@ def test_parse_event_uid_v2(): pk, source = parse_event_uid(event_uid) assert pk == pk_value assert source == "slack" + + +def test_parse_event_uid_fallback(): + # use ical existing UID for imported events + event_uid = "someid@google.com" + pk, source = parse_event_uid(event_uid) + assert pk == event_uid + assert source is None From 981fdf54a1c887a23b5506267682fae8a8e617e5 Mon Sep 17 00:00:00 2001 From: Yulia Shanyrova Date: Wed, 7 Sep 2022 11:39:25 +0200 Subject: [PATCH 12/17] error message added to failed request on resolving incident (#499) * error message added to failed request on resolving incident * changed the order of condition --- grafana-plugin/src/models/alertgroup/alertgroup.ts | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/grafana-plugin/src/models/alertgroup/alertgroup.ts b/grafana-plugin/src/models/alertgroup/alertgroup.ts index 8c035c4b..8bc67fca 100644 --- a/grafana-plugin/src/models/alertgroup/alertgroup.ts +++ b/grafana-plugin/src/models/alertgroup/alertgroup.ts @@ -414,8 +414,7 @@ export class AlertGroupStore extends BaseStore { console.log('undoAction', undoAction); } catch (e) { this.updateAlert(alertId, { loading: false }); - - openErrorNotification(e.response.data?.detail); + openErrorNotification(e.response.data?.detail || e.response.data); } } From b68dd9bc92a00b9962526076da8eb2bb1e0031a7 Mon Sep 17 00:00:00 2001 From: Vadim Stepanov Date: Wed, 7 Sep 2022 10:58:31 +0100 Subject: [PATCH 13/17] fix deleting the last notification rule for pagerduty migrator (#484) --- .../migrator/resources/notification_rules.py | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/tools/pagerduty-migrator/migrator/resources/notification_rules.py b/tools/pagerduty-migrator/migrator/resources/notification_rules.py index e3efb2f5..39b7a39a 100644 --- a/tools/pagerduty-migrator/migrator/resources/notification_rules.py +++ b/tools/pagerduty-migrator/migrator/resources/notification_rules.py @@ -31,12 +31,16 @@ def migrate_notification_rules(user: dict) -> None: notification_rules, user["oncall_user"]["id"] ) - for rule in user["oncall_user"]["notification_rules"]: - oncall_api_client.delete("personal_notification_rules/{}".format(rule["id"])) - for rule in oncall_rules: oncall_api_client.create("personal_notification_rules", rule) + if oncall_rules: + # delete old notification rules if any new rules were created + for rule in user["oncall_user"]["notification_rules"]: + oncall_api_client.delete( + "personal_notification_rules/{}".format(rule["id"]) + ) + def transform_notification_rules( notification_rules: list[dict], user_id: str From e63374db1907917c45488901a360f606f657596a Mon Sep 17 00:00:00 2001 From: Vadim Stepanov Date: Wed, 7 Sep 2022 11:05:46 +0100 Subject: [PATCH 14/17] Update README.md --- tools/pagerduty-migrator/README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/tools/pagerduty-migrator/README.md b/tools/pagerduty-migrator/README.md index a59c39b5..b83f550d 100644 --- a/tools/pagerduty-migrator/README.md +++ b/tools/pagerduty-migrator/README.md @@ -20,7 +20,7 @@ Resources that can be migrated using this tool: 1. Make sure you have `docker` installed 2. Build the docker image: `docker build -t pd-oncall-migrator .` -3. Obtain a PagerDuty API token: https://support.pagerduty.com/docs/api-access-keys +3. Obtain a PagerDuty API user token: https://support.pagerduty.com/docs/api-access-keys#generate-a-user-token-rest-api-key 4. Obtain a Grafana OnCall API token and API URL on the "Settings" page of your Grafana OnCall instance ## Migration plan @@ -84,4 +84,4 @@ It's possible to specify a default contact method type for user notification rul * Connect integrations (press the "How to connect" button on the integration page) * Make sure users connect their phone numbers, Slack accounts, etc. in their user settings * At some point you would probably want to recreate schedules using Google Calendar or Terraform to be able to modify migrated on-call schedules in Grafana OnCall - \ No newline at end of file + From f5a8d859c70c2a60423096763269493ab5a30345 Mon Sep 17 00:00:00 2001 From: Vadim Stepanov Date: Wed, 7 Sep 2022 11:35:41 +0100 Subject: [PATCH 15/17] Update CHANGELOG.md --- CHANGELOG.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index 41b3098c..ffaeb731 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,8 @@ # Change Log +## v1.0.35 (2022-09-07) +- Bug fixes + ## v1.0.34 (2022-09-06) - Fix schedule notification spam From 3b27d8a3612bdca55dcf503e7053525113a0b3cf Mon Sep 17 00:00:00 2001 From: Vadim Stepanov Date: Wed, 7 Sep 2022 11:36:41 +0100 Subject: [PATCH 16/17] Update CHANGELOG.md --- grafana-plugin/CHANGELOG.md | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/grafana-plugin/CHANGELOG.md b/grafana-plugin/CHANGELOG.md index 74ec49b2..ffaeb731 100644 --- a/grafana-plugin/CHANGELOG.md +++ b/grafana-plugin/CHANGELOG.md @@ -1,5 +1,17 @@ # Change Log +## v1.0.35 (2022-09-07) +- Bug fixes + +## v1.0.34 (2022-09-06) +- Fix schedule notification spam + +## v1.0.33 (2022-09-06) +- Add raw alert view +- Add GitHub star button for OSS installations +- Restore alert group search functionality +- Bug fixes + ## v1.0.32 (2022-09-01) - Bug fixes From bd5dd39f8657bc7b4a2f253bc2a477fbbd6f40bc Mon Sep 17 00:00:00 2001 From: Vadim Stepanov Date: Wed, 7 Sep 2022 12:32:38 +0100 Subject: [PATCH 17/17] Update Chart.yaml --- helm/oncall/Chart.yaml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/helm/oncall/Chart.yaml b/helm/oncall/Chart.yaml index 2f98dd4c..a4849952 100644 --- a/helm/oncall/Chart.yaml +++ b/helm/oncall/Chart.yaml @@ -8,13 +8,13 @@ type: application # This is the chart version. This version number should be incremented each time you make changes # to the chart and its templates, including the app version. # Versions are expected to follow Semantic Versioning (https://semver.org/) -version: 1.0.4 +version: 1.0.5 # This is the version number of the application being deployed. This version number should be # incremented each time you make changes to the application. Versions are not expected to # follow Semantic Versioning. They should reflect the version the application is using. # It is recommended to use it with quotes. -appVersion: "v1.0.32" +appVersion: "v1.0.35" dependencies: - name: cert-manager version: v1.8.0