oncall-engine/engine/apps/google/tasks.py

144 lines
6.2 KiB
Python
Raw Permalink Normal View History

Google OAuth2 flow + fetch Google Calendar OOO events (#4067) # What this PR does The following is deployed under a feature flag. **How it works** 1. The user clicks on the "Connect using your Google account" button in the user profile settings modal 2. The UI makes a call to `GET /api/internal/v1/login/google-oauth2`. The backend has now been configured to add `apps.social_auth.backends.GoogleOAuth2` as a "`social_auth` backend". 3. The backend will respond w/ a URL which points to the Google OAuth2 consent screen. The frontend then proceeds by sending the user to this page. This URL includes the following query parameters (amongst others): - `redirect_uri` - this will send the user back to `/api/internal/v1/complete/google-oauth2` (ie. make another API call to the OnCall backend to finalize the Google OAuth2 flow) - `state` - this represents an `apps.auth_token.models.GoogleOAuth2Token` token. This allows us to identify the OnCall user once they've linked their Google account. 4. Once redirected back to `/api/internal/v1/complete/google-oauth2`, this will complete the OAuth2 flow. At this point, the backend has access to several pieces of information about the Google user, including their `access_token` and `refresh_token`. We persist these (encrypted) for future use to fetch the user's out-of-office calendar events 5. The response from the API call in 4 above ☝️ is HTTP 302 (redirect) to `/a/grafana-oncall-app/users/me` (ie. open the user profile settings modal). At this point the user will see that their account has been connected and they can further configure the settings ![image](https://github.com/grafana/oncall/assets/9406895/c7673055-8485-4f9a-98df-b4f7347229ce) ## Which issue(s) this PR closes Closes https://github.com/grafana/oncall-private/issues/2584 ## Checklist - [x] Unit, integration, and e2e (if applicable) tests updated - [x] Documentation added (or `pr:no public docs` PR label added if not required) - will be done in https://github.com/grafana/oncall-private/issues/2591 - [x] Added the relevant release notes label (see labels prefixed w/ `release:`). These labels dictate how your PR will show up in the autogenerated release notes. - will be done in https://github.com/grafana/oncall-private/issues/2591 --------- Co-authored-by: Dominik <dominik.broj@grafana.com> Co-authored-by: Maxim Mordasov <maxim.mordasov@grafana.com>
2024-04-02 14:59:03 -04:00
import logging
from celery.utils.log import get_task_logger
address Google OAuth2 issues where user didn't grant us the `https://www.googleapis.com/auth/calendar.events.readonly` scope (#4802) # What this PR does Follow up PR to https://github.com/grafana/oncall/pull/4792 Basically if when communicating with Google Calendar's API we encounter an HTTP 403, or the Google client throws a `google.auth.exceptions.RefreshError` this means one of three things: 1. the refresh token we have persisted for the user is missing the `https://www.googleapis.com/auth/calendar.events.readonly` scope (HTTP 403) 2. the Google user has been deleted (`google.auth.exceptions.RefreshError`) 3. the refresh token has expired (`google.auth.exceptions.RefreshError`) To prevent scenario 1 above from happening in the future we now will check that the token has been granted the required scopes. If the user doesn't grant us all the necessary scopes, we will show them an error message in the UI: https://www.loom.com/share/0055ef03192b4154b894c2221cecbd5f For tokens that were granted prior to this PR and which are missing the required scope, we will show the user a dismissible warning banner in the UI letting them know that they will need to reconnect their account and grant us the missing permissions (see [this second demo video](https://www.loom.com/share/bf2ee8b840864a64893165370a892bcd) showing this). ## Checklist - [x] Unit, integration, and e2e (if applicable) tests updated - [x] Documentation added (or `pr:no public docs` PR label added if not required) - [x] Added the relevant release notes label (see labels prefixed w/ `release:`). These labels dictate how your PR will show up in the autogenerated release notes. --------- Co-authored-by: Dominik <dominik.broj@grafana.com>
2024-08-14 18:02:34 -04:00
from django.db.models import Q
Google OAuth2 flow + fetch Google Calendar OOO events (#4067) # What this PR does The following is deployed under a feature flag. **How it works** 1. The user clicks on the "Connect using your Google account" button in the user profile settings modal 2. The UI makes a call to `GET /api/internal/v1/login/google-oauth2`. The backend has now been configured to add `apps.social_auth.backends.GoogleOAuth2` as a "`social_auth` backend". 3. The backend will respond w/ a URL which points to the Google OAuth2 consent screen. The frontend then proceeds by sending the user to this page. This URL includes the following query parameters (amongst others): - `redirect_uri` - this will send the user back to `/api/internal/v1/complete/google-oauth2` (ie. make another API call to the OnCall backend to finalize the Google OAuth2 flow) - `state` - this represents an `apps.auth_token.models.GoogleOAuth2Token` token. This allows us to identify the OnCall user once they've linked their Google account. 4. Once redirected back to `/api/internal/v1/complete/google-oauth2`, this will complete the OAuth2 flow. At this point, the backend has access to several pieces of information about the Google user, including their `access_token` and `refresh_token`. We persist these (encrypted) for future use to fetch the user's out-of-office calendar events 5. The response from the API call in 4 above ☝️ is HTTP 302 (redirect) to `/a/grafana-oncall-app/users/me` (ie. open the user profile settings modal). At this point the user will see that their account has been connected and they can further configure the settings ![image](https://github.com/grafana/oncall/assets/9406895/c7673055-8485-4f9a-98df-b4f7347229ce) ## Which issue(s) this PR closes Closes https://github.com/grafana/oncall-private/issues/2584 ## Checklist - [x] Unit, integration, and e2e (if applicable) tests updated - [x] Documentation added (or `pr:no public docs` PR label added if not required) - will be done in https://github.com/grafana/oncall-private/issues/2591 - [x] Added the relevant release notes label (see labels prefixed w/ `release:`). These labels dictate how your PR will show up in the autogenerated release notes. - will be done in https://github.com/grafana/oncall-private/issues/2591 --------- Co-authored-by: Dominik <dominik.broj@grafana.com> Co-authored-by: Maxim Mordasov <maxim.mordasov@grafana.com>
2024-04-02 14:59:03 -04:00
from apps.google import constants
from apps.google.client import (
GoogleCalendarAPIClient,
GoogleCalendarGenericHTTPError,
GoogleCalendarRefreshError,
GoogleCalendarUnauthorizedHTTPError,
)
Google OAuth2 flow + fetch Google Calendar OOO events (#4067) # What this PR does The following is deployed under a feature flag. **How it works** 1. The user clicks on the "Connect using your Google account" button in the user profile settings modal 2. The UI makes a call to `GET /api/internal/v1/login/google-oauth2`. The backend has now been configured to add `apps.social_auth.backends.GoogleOAuth2` as a "`social_auth` backend". 3. The backend will respond w/ a URL which points to the Google OAuth2 consent screen. The frontend then proceeds by sending the user to this page. This URL includes the following query parameters (amongst others): - `redirect_uri` - this will send the user back to `/api/internal/v1/complete/google-oauth2` (ie. make another API call to the OnCall backend to finalize the Google OAuth2 flow) - `state` - this represents an `apps.auth_token.models.GoogleOAuth2Token` token. This allows us to identify the OnCall user once they've linked their Google account. 4. Once redirected back to `/api/internal/v1/complete/google-oauth2`, this will complete the OAuth2 flow. At this point, the backend has access to several pieces of information about the Google user, including their `access_token` and `refresh_token`. We persist these (encrypted) for future use to fetch the user's out-of-office calendar events 5. The response from the API call in 4 above ☝️ is HTTP 302 (redirect) to `/a/grafana-oncall-app/users/me` (ie. open the user profile settings modal). At this point the user will see that their account has been connected and they can further configure the settings ![image](https://github.com/grafana/oncall/assets/9406895/c7673055-8485-4f9a-98df-b4f7347229ce) ## Which issue(s) this PR closes Closes https://github.com/grafana/oncall-private/issues/2584 ## Checklist - [x] Unit, integration, and e2e (if applicable) tests updated - [x] Documentation added (or `pr:no public docs` PR label added if not required) - will be done in https://github.com/grafana/oncall-private/issues/2591 - [x] Added the relevant release notes label (see labels prefixed w/ `release:`). These labels dictate how your PR will show up in the autogenerated release notes. - will be done in https://github.com/grafana/oncall-private/issues/2591 --------- Co-authored-by: Dominik <dominik.broj@grafana.com> Co-authored-by: Maxim Mordasov <maxim.mordasov@grafana.com>
2024-04-02 14:59:03 -04:00
from apps.google.models import GoogleOAuth2User
from apps.schedules.models import OnCallSchedule, ShiftSwapRequest
Google OAuth2 flow + fetch Google Calendar OOO events (#4067) # What this PR does The following is deployed under a feature flag. **How it works** 1. The user clicks on the "Connect using your Google account" button in the user profile settings modal 2. The UI makes a call to `GET /api/internal/v1/login/google-oauth2`. The backend has now been configured to add `apps.social_auth.backends.GoogleOAuth2` as a "`social_auth` backend". 3. The backend will respond w/ a URL which points to the Google OAuth2 consent screen. The frontend then proceeds by sending the user to this page. This URL includes the following query parameters (amongst others): - `redirect_uri` - this will send the user back to `/api/internal/v1/complete/google-oauth2` (ie. make another API call to the OnCall backend to finalize the Google OAuth2 flow) - `state` - this represents an `apps.auth_token.models.GoogleOAuth2Token` token. This allows us to identify the OnCall user once they've linked their Google account. 4. Once redirected back to `/api/internal/v1/complete/google-oauth2`, this will complete the OAuth2 flow. At this point, the backend has access to several pieces of information about the Google user, including their `access_token` and `refresh_token`. We persist these (encrypted) for future use to fetch the user's out-of-office calendar events 5. The response from the API call in 4 above ☝️ is HTTP 302 (redirect) to `/a/grafana-oncall-app/users/me` (ie. open the user profile settings modal). At this point the user will see that their account has been connected and they can further configure the settings ![image](https://github.com/grafana/oncall/assets/9406895/c7673055-8485-4f9a-98df-b4f7347229ce) ## Which issue(s) this PR closes Closes https://github.com/grafana/oncall-private/issues/2584 ## Checklist - [x] Unit, integration, and e2e (if applicable) tests updated - [x] Documentation added (or `pr:no public docs` PR label added if not required) - will be done in https://github.com/grafana/oncall-private/issues/2591 - [x] Added the relevant release notes label (see labels prefixed w/ `release:`). These labels dictate how your PR will show up in the autogenerated release notes. - will be done in https://github.com/grafana/oncall-private/issues/2591 --------- Co-authored-by: Dominik <dominik.broj@grafana.com> Co-authored-by: Maxim Mordasov <maxim.mordasov@grafana.com>
2024-04-02 14:59:03 -04:00
from common.custom_celery_tasks import shared_dedicated_queue_retry_task
logger = get_task_logger(__name__)
logger.setLevel(logging.DEBUG)
@shared_dedicated_queue_retry_task(autoretry_for=(Exception,), retry_backoff=True)
def sync_out_of_office_calendar_events_for_user(google_oauth2_user_pk: int) -> None:
google_oauth2_user = GoogleOAuth2User.objects.get(pk=google_oauth2_user_pk)
google_api_client = GoogleCalendarAPIClient(google_oauth2_user.access_token, google_oauth2_user.refresh_token)
user = google_oauth2_user.user
user_id = user.pk
logger.info(f"Syncing out of office Google Calendar events for user {user_id}")
users_schedules = OnCallSchedule.objects.related_to_user(user)
user_google_calendar_settings = user.google_calendar_settings
oncall_schedules_to_consider_for_shift_swaps = user_google_calendar_settings[
"oncall_schedules_to_consider_for_shift_swaps"
]
if oncall_schedules_to_consider_for_shift_swaps:
users_schedules = users_schedules.filter(public_primary_key__in=oncall_schedules_to_consider_for_shift_swaps)
try:
out_of_office_events = google_api_client.fetch_out_of_office_events()
except GoogleCalendarUnauthorizedHTTPError:
address Google OAuth2 issues where user didn't grant us the `https://www.googleapis.com/auth/calendar.events.readonly` scope (#4802) # What this PR does Follow up PR to https://github.com/grafana/oncall/pull/4792 Basically if when communicating with Google Calendar's API we encounter an HTTP 403, or the Google client throws a `google.auth.exceptions.RefreshError` this means one of three things: 1. the refresh token we have persisted for the user is missing the `https://www.googleapis.com/auth/calendar.events.readonly` scope (HTTP 403) 2. the Google user has been deleted (`google.auth.exceptions.RefreshError`) 3. the refresh token has expired (`google.auth.exceptions.RefreshError`) To prevent scenario 1 above from happening in the future we now will check that the token has been granted the required scopes. If the user doesn't grant us all the necessary scopes, we will show them an error message in the UI: https://www.loom.com/share/0055ef03192b4154b894c2221cecbd5f For tokens that were granted prior to this PR and which are missing the required scope, we will show the user a dismissible warning banner in the UI letting them know that they will need to reconnect their account and grant us the missing permissions (see [this second demo video](https://www.loom.com/share/bf2ee8b840864a64893165370a892bcd) showing this). ## Checklist - [x] Unit, integration, and e2e (if applicable) tests updated - [x] Documentation added (or `pr:no public docs` PR label added if not required) - [x] Added the relevant release notes label (see labels prefixed w/ `release:`). These labels dictate how your PR will show up in the autogenerated release notes. --------- Co-authored-by: Dominik <dominik.broj@grafana.com>
2024-08-14 18:02:34 -04:00
# this means the user's current token is missing the required scopes, don't delete their token for now
# we'll notify them via the plugin UI and ask them to reauth and grant us the missing scope
logger.warning(
f"Failed to fetch out of office events for user {user_id} due to missing required scopes. "
"Safe to skip for now"
)
return
except GoogleCalendarRefreshError:
# in this scenarios there's really not much we can do with the refresh/access token that we
# have available. The user will need to re-connect with Google so lets delete their persisted token
logger.exception(
f"Failed to fetch out of office events for user {user_id} due to an invalid access and/or refresh token"
)
user.reset_google_oauth2_settings()
return
address Google OAuth2 issues where user didn't grant us the `https://www.googleapis.com/auth/calendar.events.readonly` scope (#4802) # What this PR does Follow up PR to https://github.com/grafana/oncall/pull/4792 Basically if when communicating with Google Calendar's API we encounter an HTTP 403, or the Google client throws a `google.auth.exceptions.RefreshError` this means one of three things: 1. the refresh token we have persisted for the user is missing the `https://www.googleapis.com/auth/calendar.events.readonly` scope (HTTP 403) 2. the Google user has been deleted (`google.auth.exceptions.RefreshError`) 3. the refresh token has expired (`google.auth.exceptions.RefreshError`) To prevent scenario 1 above from happening in the future we now will check that the token has been granted the required scopes. If the user doesn't grant us all the necessary scopes, we will show them an error message in the UI: https://www.loom.com/share/0055ef03192b4154b894c2221cecbd5f For tokens that were granted prior to this PR and which are missing the required scope, we will show the user a dismissible warning banner in the UI letting them know that they will need to reconnect their account and grant us the missing permissions (see [this second demo video](https://www.loom.com/share/bf2ee8b840864a64893165370a892bcd) showing this). ## Checklist - [x] Unit, integration, and e2e (if applicable) tests updated - [x] Documentation added (or `pr:no public docs` PR label added if not required) - [x] Added the relevant release notes label (see labels prefixed w/ `release:`). These labels dictate how your PR will show up in the autogenerated release notes. --------- Co-authored-by: Dominik <dominik.broj@grafana.com>
2024-08-14 18:02:34 -04:00
except GoogleCalendarGenericHTTPError:
logger.exception(f"Failed to fetch out of office events for user {user_id} due to a generic HTTP error")
return
for out_of_office_event in out_of_office_events:
raw_event = out_of_office_event.raw_event
event_title = raw_event["summary"]
event_id = raw_event["id"]
start_time_utc = out_of_office_event.start_time_utc
end_time_utc = out_of_office_event.end_time_utc
logger.info(
f"Processing out of office event {event_id} starting at {start_time_utc} and ending at "
f"{end_time_utc} for user {user_id}"
)
if constants.EVENT_SUMMARY_IGNORE_KEYWORD in event_title.lower():
logger.info(
f"Skipping out of office event {event_id} because it contains the ignore keyword "
f"'{constants.EVENT_SUMMARY_IGNORE_KEYWORD}'"
)
continue
for schedule in users_schedules:
_, current_shifts, upcoming_shifts = schedule.shifts_for_user(
user,
start_time_utc,
datetime_end=end_time_utc,
)
if current_shifts or upcoming_shifts:
logger.info(
f"Found {len(current_shifts)} current shift(s) and {len(upcoming_shifts)} upcoming shift(s) "
f"for user {user_id} during the out of office event {event_id}"
)
# also consider deleted shift swap requests.. this can be useful in the event
# that we autogenerated a shift swap request for the user but the user decided to delete it
# in this case, we shouldn't recreate a new one
shift_swap_request_exists = ShiftSwapRequest.objects_with_deleted.filter(
beneficiary=user,
schedule=schedule,
swap_start=start_time_utc,
swap_end=end_time_utc,
).exists()
if not shift_swap_request_exists:
logger.info(
f"Creating shift swap request for user {user_id} schedule {schedule.pk} "
f"due to the out of office event {event_id}"
)
ssr = ShiftSwapRequest.objects.create(
beneficiary=user,
schedule=schedule,
swap_start=start_time_utc,
swap_end=end_time_utc,
description=f"{user.name or user.email} will be out of office during this time according to Google Calendar",
)
logger.info(f"Created shift swap request {ssr.pk}")
else:
logger.info(f"Shift swap request already exists for user {user_id} schedule {schedule.pk}")
else:
logger.info(
f"No current or upcoming shifts found for user {user_id} during the out of office event {event_id}"
)
Google OAuth2 flow + fetch Google Calendar OOO events (#4067) # What this PR does The following is deployed under a feature flag. **How it works** 1. The user clicks on the "Connect using your Google account" button in the user profile settings modal 2. The UI makes a call to `GET /api/internal/v1/login/google-oauth2`. The backend has now been configured to add `apps.social_auth.backends.GoogleOAuth2` as a "`social_auth` backend". 3. The backend will respond w/ a URL which points to the Google OAuth2 consent screen. The frontend then proceeds by sending the user to this page. This URL includes the following query parameters (amongst others): - `redirect_uri` - this will send the user back to `/api/internal/v1/complete/google-oauth2` (ie. make another API call to the OnCall backend to finalize the Google OAuth2 flow) - `state` - this represents an `apps.auth_token.models.GoogleOAuth2Token` token. This allows us to identify the OnCall user once they've linked their Google account. 4. Once redirected back to `/api/internal/v1/complete/google-oauth2`, this will complete the OAuth2 flow. At this point, the backend has access to several pieces of information about the Google user, including their `access_token` and `refresh_token`. We persist these (encrypted) for future use to fetch the user's out-of-office calendar events 5. The response from the API call in 4 above ☝️ is HTTP 302 (redirect) to `/a/grafana-oncall-app/users/me` (ie. open the user profile settings modal). At this point the user will see that their account has been connected and they can further configure the settings ![image](https://github.com/grafana/oncall/assets/9406895/c7673055-8485-4f9a-98df-b4f7347229ce) ## Which issue(s) this PR closes Closes https://github.com/grafana/oncall-private/issues/2584 ## Checklist - [x] Unit, integration, and e2e (if applicable) tests updated - [x] Documentation added (or `pr:no public docs` PR label added if not required) - will be done in https://github.com/grafana/oncall-private/issues/2591 - [x] Added the relevant release notes label (see labels prefixed w/ `release:`). These labels dictate how your PR will show up in the autogenerated release notes. - will be done in https://github.com/grafana/oncall-private/issues/2591 --------- Co-authored-by: Dominik <dominik.broj@grafana.com> Co-authored-by: Maxim Mordasov <maxim.mordasov@grafana.com>
2024-04-02 14:59:03 -04:00
@shared_dedicated_queue_retry_task(autoretry_for=(Exception,), retry_backoff=True)
def sync_out_of_office_calendar_events_for_all_users() -> None:
address Google OAuth2 issues where user didn't grant us the `https://www.googleapis.com/auth/calendar.events.readonly` scope (#4802) # What this PR does Follow up PR to https://github.com/grafana/oncall/pull/4792 Basically if when communicating with Google Calendar's API we encounter an HTTP 403, or the Google client throws a `google.auth.exceptions.RefreshError` this means one of three things: 1. the refresh token we have persisted for the user is missing the `https://www.googleapis.com/auth/calendar.events.readonly` scope (HTTP 403) 2. the Google user has been deleted (`google.auth.exceptions.RefreshError`) 3. the refresh token has expired (`google.auth.exceptions.RefreshError`) To prevent scenario 1 above from happening in the future we now will check that the token has been granted the required scopes. If the user doesn't grant us all the necessary scopes, we will show them an error message in the UI: https://www.loom.com/share/0055ef03192b4154b894c2221cecbd5f For tokens that were granted prior to this PR and which are missing the required scope, we will show the user a dismissible warning banner in the UI letting them know that they will need to reconnect their account and grant us the missing permissions (see [this second demo video](https://www.loom.com/share/bf2ee8b840864a64893165370a892bcd) showing this). ## Checklist - [x] Unit, integration, and e2e (if applicable) tests updated - [x] Documentation added (or `pr:no public docs` PR label added if not required) - [x] Added the relevant release notes label (see labels prefixed w/ `release:`). These labels dictate how your PR will show up in the autogenerated release notes. --------- Co-authored-by: Dominik <dominik.broj@grafana.com>
2024-08-14 18:02:34 -04:00
# some existing tokens may not have all the required scopes, lets skip these
tokens_containing_required_scopes = GoogleOAuth2User.objects.filter(
*[Q(oauth_scope__contains=scope) for scope in constants.REQUIRED_OAUTH_SCOPES],
user__organization__deleted_at__isnull=True,
)
logger.info(
f"Google OAuth2 tokens with the required scopes - "
f"{tokens_containing_required_scopes.count()}/{GoogleOAuth2User.objects.count()}"
)
for google_oauth2_user in tokens_containing_required_scopes:
Google OAuth2 flow + fetch Google Calendar OOO events (#4067) # What this PR does The following is deployed under a feature flag. **How it works** 1. The user clicks on the "Connect using your Google account" button in the user profile settings modal 2. The UI makes a call to `GET /api/internal/v1/login/google-oauth2`. The backend has now been configured to add `apps.social_auth.backends.GoogleOAuth2` as a "`social_auth` backend". 3. The backend will respond w/ a URL which points to the Google OAuth2 consent screen. The frontend then proceeds by sending the user to this page. This URL includes the following query parameters (amongst others): - `redirect_uri` - this will send the user back to `/api/internal/v1/complete/google-oauth2` (ie. make another API call to the OnCall backend to finalize the Google OAuth2 flow) - `state` - this represents an `apps.auth_token.models.GoogleOAuth2Token` token. This allows us to identify the OnCall user once they've linked their Google account. 4. Once redirected back to `/api/internal/v1/complete/google-oauth2`, this will complete the OAuth2 flow. At this point, the backend has access to several pieces of information about the Google user, including their `access_token` and `refresh_token`. We persist these (encrypted) for future use to fetch the user's out-of-office calendar events 5. The response from the API call in 4 above ☝️ is HTTP 302 (redirect) to `/a/grafana-oncall-app/users/me` (ie. open the user profile settings modal). At this point the user will see that their account has been connected and they can further configure the settings ![image](https://github.com/grafana/oncall/assets/9406895/c7673055-8485-4f9a-98df-b4f7347229ce) ## Which issue(s) this PR closes Closes https://github.com/grafana/oncall-private/issues/2584 ## Checklist - [x] Unit, integration, and e2e (if applicable) tests updated - [x] Documentation added (or `pr:no public docs` PR label added if not required) - will be done in https://github.com/grafana/oncall-private/issues/2591 - [x] Added the relevant release notes label (see labels prefixed w/ `release:`). These labels dictate how your PR will show up in the autogenerated release notes. - will be done in https://github.com/grafana/oncall-private/issues/2591 --------- Co-authored-by: Dominik <dominik.broj@grafana.com> Co-authored-by: Maxim Mordasov <maxim.mordasov@grafana.com>
2024-04-02 14:59:03 -04:00
sync_out_of_office_calendar_events_for_user.apply_async(args=(google_oauth2_user.pk,))