oncall-engine/engine/common/api_helpers/utils.py

149 lines
4.6 KiB
Python
Raw Permalink Normal View History

import datetime
import re
import typing
2022-07-12 15:42:20 -06:00
from urllib.parse import urljoin
import requests
from django.conf import settings
from django.utils import dateparse, timezone
from icalendar import Calendar
from rest_framework import serializers
from rest_framework.request import Request
from apps.schedules.ical_utils import fetch_ical_file
from common.api_helpers.exceptions import BadRequest
from common.jinja_templater import apply_jinja_template
from common.jinja_templater.apply_jinja_template import JinjaTemplateWarning
from common.timezones import raise_exception_if_not_valid_timezone
class CurrentOrganizationDefault:
"""
Utility class to get the current organization right from the serializer field.
In pair with serializers.HiddenField gives an ability to create objects
without overriding perform_create on the model, while respecting unique_together constraints.
Example: organization = serializers.HiddenField(default=CurrentOrganizationDefault())
"""
Fix warnings when running backend tests (#2079) # What this PR does - update `make test` to always use `settings.ci-test`. Right now it will use whatever the value of `DJANGO_SETTINGS_MODULE` is in `./dev/.env.dev`, which causes ~45 tests to fail - Fix several Python warnings that we see when running the tests ```bash RemovedInDjango40Warning: The providing_args argument is deprecated. As it is purely documentational, it has no replacement. If you rely on this argument as documentation, you can move the text to a code comment or docstring. alert_create_signal = django.dispatch.Signal( ``` ```bash PytestCollectionWarning: cannot collect test class 'TestOnlyBackend' because it has a __init__ constructor (from: apps/api/tests/test_alert_receive_channel_template.py) class TestOnlyBackend(BaseMessagingBackend): ``` ```bash DeprecationWarning: The parameter 'use_aliases' in emoji.emojize() is deprecated and will be removed in version 2.0.0. Use language='alias' instead. To hide this warning, pin/downgrade the package to 'emoji~=1.6.3' return emoji.emojize(self.verbal_name, use_aliases=True) ``` ```bash DateTimeField CustomOnCallShift.start received a naive datetime (2023-06-01 12:53:12) while time zone support is active. warnings.warn("DateTimeField %s received a naive datetime (%s)" ``` ```bash apps/twilioapp/tests/test_phone_calls.py::test_resolve_by_phone /etc/app/apps/twilioapp/tests/test_phone_calls.py:173: DeprecationWarning: The 'text' argument to find()-type methods is deprecated. Use 'string' instead. content = BeautifulSoup(content, features="html.parser").findAll(text=True) ``` ```bash apps/twilioapp/tests/test_phone_calls.py::test_resolve_by_phone apps/twilioapp/tests/test_phone_calls.py::test_wrong_pressed_digit /usr/local/lib/python3.11/site-packages/bs4/builder/__init__.py:545: XMLParsedAsHTMLWarning: It looks like you're parsing an XML document using an HTML parser. If this really is an HTML document (maybe it's XHTML?), you can ignore or filter this warning. If it's XML, you should know that using an XML parser will be more reliable. To parse this document as XML, make sure you have the lxml package installed, and pass the keyword argument `features="xml"` into the BeautifulSoup constructor. ``` ```bash apps/twilioapp/tests/test_phone_calls.py::test_forbidden_requests /usr/local/lib/python3.11/site-packages/social_django/urls.py:15: RemovedInDjango40Warning: django.conf.urls.url() is deprecated in favor of django.urls.re_path(). url(r'^login/(?P<backend>[^/]+){0}$'.format(extra), views.auth, ``` ```bash apps/twilioapp/tests/test_phone_calls.py: 66 warnings /usr/local/lib/python3.11/site-packages/debug_toolbar/utils.py:255: DeprecationWarning: currentThread() is deprecated, use current_thread() instead thread = threading.currentThread() ``` ## Checklist - [x] Unit, integration, and e2e (if applicable) tests updated - [x] Documentation added (or `pr:no public docs` PR label added if not required) - [ ] `CHANGELOG.md` updated (or `pr:no changelog` PR label added if not required)
2023-06-06 20:38:00 +02:00
requires_context = True
Fix warnings when running backend tests (#2079) # What this PR does - update `make test` to always use `settings.ci-test`. Right now it will use whatever the value of `DJANGO_SETTINGS_MODULE` is in `./dev/.env.dev`, which causes ~45 tests to fail - Fix several Python warnings that we see when running the tests ```bash RemovedInDjango40Warning: The providing_args argument is deprecated. As it is purely documentational, it has no replacement. If you rely on this argument as documentation, you can move the text to a code comment or docstring. alert_create_signal = django.dispatch.Signal( ``` ```bash PytestCollectionWarning: cannot collect test class 'TestOnlyBackend' because it has a __init__ constructor (from: apps/api/tests/test_alert_receive_channel_template.py) class TestOnlyBackend(BaseMessagingBackend): ``` ```bash DeprecationWarning: The parameter 'use_aliases' in emoji.emojize() is deprecated and will be removed in version 2.0.0. Use language='alias' instead. To hide this warning, pin/downgrade the package to 'emoji~=1.6.3' return emoji.emojize(self.verbal_name, use_aliases=True) ``` ```bash DateTimeField CustomOnCallShift.start received a naive datetime (2023-06-01 12:53:12) while time zone support is active. warnings.warn("DateTimeField %s received a naive datetime (%s)" ``` ```bash apps/twilioapp/tests/test_phone_calls.py::test_resolve_by_phone /etc/app/apps/twilioapp/tests/test_phone_calls.py:173: DeprecationWarning: The 'text' argument to find()-type methods is deprecated. Use 'string' instead. content = BeautifulSoup(content, features="html.parser").findAll(text=True) ``` ```bash apps/twilioapp/tests/test_phone_calls.py::test_resolve_by_phone apps/twilioapp/tests/test_phone_calls.py::test_wrong_pressed_digit /usr/local/lib/python3.11/site-packages/bs4/builder/__init__.py:545: XMLParsedAsHTMLWarning: It looks like you're parsing an XML document using an HTML parser. If this really is an HTML document (maybe it's XHTML?), you can ignore or filter this warning. If it's XML, you should know that using an XML parser will be more reliable. To parse this document as XML, make sure you have the lxml package installed, and pass the keyword argument `features="xml"` into the BeautifulSoup constructor. ``` ```bash apps/twilioapp/tests/test_phone_calls.py::test_forbidden_requests /usr/local/lib/python3.11/site-packages/social_django/urls.py:15: RemovedInDjango40Warning: django.conf.urls.url() is deprecated in favor of django.urls.re_path(). url(r'^login/(?P<backend>[^/]+){0}$'.format(extra), views.auth, ``` ```bash apps/twilioapp/tests/test_phone_calls.py: 66 warnings /usr/local/lib/python3.11/site-packages/debug_toolbar/utils.py:255: DeprecationWarning: currentThread() is deprecated, use current_thread() instead thread = threading.currentThread() ``` ## Checklist - [x] Unit, integration, and e2e (if applicable) tests updated - [x] Documentation added (or `pr:no public docs` PR label added if not required) - [ ] `CHANGELOG.md` updated (or `pr:no changelog` PR label added if not required)
2023-06-06 20:38:00 +02:00
def __call__(self, serializer_field):
self.organization = serializer_field.context["request"].auth.organization
return self.organization
def __repr__(self):
return "%s()" % self.__class__.__name__
class CurrentTeamDefault:
"""
Utility class to get the current team right from the serializer field.
"""
Fix warnings when running backend tests (#2079) # What this PR does - update `make test` to always use `settings.ci-test`. Right now it will use whatever the value of `DJANGO_SETTINGS_MODULE` is in `./dev/.env.dev`, which causes ~45 tests to fail - Fix several Python warnings that we see when running the tests ```bash RemovedInDjango40Warning: The providing_args argument is deprecated. As it is purely documentational, it has no replacement. If you rely on this argument as documentation, you can move the text to a code comment or docstring. alert_create_signal = django.dispatch.Signal( ``` ```bash PytestCollectionWarning: cannot collect test class 'TestOnlyBackend' because it has a __init__ constructor (from: apps/api/tests/test_alert_receive_channel_template.py) class TestOnlyBackend(BaseMessagingBackend): ``` ```bash DeprecationWarning: The parameter 'use_aliases' in emoji.emojize() is deprecated and will be removed in version 2.0.0. Use language='alias' instead. To hide this warning, pin/downgrade the package to 'emoji~=1.6.3' return emoji.emojize(self.verbal_name, use_aliases=True) ``` ```bash DateTimeField CustomOnCallShift.start received a naive datetime (2023-06-01 12:53:12) while time zone support is active. warnings.warn("DateTimeField %s received a naive datetime (%s)" ``` ```bash apps/twilioapp/tests/test_phone_calls.py::test_resolve_by_phone /etc/app/apps/twilioapp/tests/test_phone_calls.py:173: DeprecationWarning: The 'text' argument to find()-type methods is deprecated. Use 'string' instead. content = BeautifulSoup(content, features="html.parser").findAll(text=True) ``` ```bash apps/twilioapp/tests/test_phone_calls.py::test_resolve_by_phone apps/twilioapp/tests/test_phone_calls.py::test_wrong_pressed_digit /usr/local/lib/python3.11/site-packages/bs4/builder/__init__.py:545: XMLParsedAsHTMLWarning: It looks like you're parsing an XML document using an HTML parser. If this really is an HTML document (maybe it's XHTML?), you can ignore or filter this warning. If it's XML, you should know that using an XML parser will be more reliable. To parse this document as XML, make sure you have the lxml package installed, and pass the keyword argument `features="xml"` into the BeautifulSoup constructor. ``` ```bash apps/twilioapp/tests/test_phone_calls.py::test_forbidden_requests /usr/local/lib/python3.11/site-packages/social_django/urls.py:15: RemovedInDjango40Warning: django.conf.urls.url() is deprecated in favor of django.urls.re_path(). url(r'^login/(?P<backend>[^/]+){0}$'.format(extra), views.auth, ``` ```bash apps/twilioapp/tests/test_phone_calls.py: 66 warnings /usr/local/lib/python3.11/site-packages/debug_toolbar/utils.py:255: DeprecationWarning: currentThread() is deprecated, use current_thread() instead thread = threading.currentThread() ``` ## Checklist - [x] Unit, integration, and e2e (if applicable) tests updated - [x] Documentation added (or `pr:no public docs` PR label added if not required) - [ ] `CHANGELOG.md` updated (or `pr:no changelog` PR label added if not required)
2023-06-06 20:38:00 +02:00
requires_context = True
Fix warnings when running backend tests (#2079) # What this PR does - update `make test` to always use `settings.ci-test`. Right now it will use whatever the value of `DJANGO_SETTINGS_MODULE` is in `./dev/.env.dev`, which causes ~45 tests to fail - Fix several Python warnings that we see when running the tests ```bash RemovedInDjango40Warning: The providing_args argument is deprecated. As it is purely documentational, it has no replacement. If you rely on this argument as documentation, you can move the text to a code comment or docstring. alert_create_signal = django.dispatch.Signal( ``` ```bash PytestCollectionWarning: cannot collect test class 'TestOnlyBackend' because it has a __init__ constructor (from: apps/api/tests/test_alert_receive_channel_template.py) class TestOnlyBackend(BaseMessagingBackend): ``` ```bash DeprecationWarning: The parameter 'use_aliases' in emoji.emojize() is deprecated and will be removed in version 2.0.0. Use language='alias' instead. To hide this warning, pin/downgrade the package to 'emoji~=1.6.3' return emoji.emojize(self.verbal_name, use_aliases=True) ``` ```bash DateTimeField CustomOnCallShift.start received a naive datetime (2023-06-01 12:53:12) while time zone support is active. warnings.warn("DateTimeField %s received a naive datetime (%s)" ``` ```bash apps/twilioapp/tests/test_phone_calls.py::test_resolve_by_phone /etc/app/apps/twilioapp/tests/test_phone_calls.py:173: DeprecationWarning: The 'text' argument to find()-type methods is deprecated. Use 'string' instead. content = BeautifulSoup(content, features="html.parser").findAll(text=True) ``` ```bash apps/twilioapp/tests/test_phone_calls.py::test_resolve_by_phone apps/twilioapp/tests/test_phone_calls.py::test_wrong_pressed_digit /usr/local/lib/python3.11/site-packages/bs4/builder/__init__.py:545: XMLParsedAsHTMLWarning: It looks like you're parsing an XML document using an HTML parser. If this really is an HTML document (maybe it's XHTML?), you can ignore or filter this warning. If it's XML, you should know that using an XML parser will be more reliable. To parse this document as XML, make sure you have the lxml package installed, and pass the keyword argument `features="xml"` into the BeautifulSoup constructor. ``` ```bash apps/twilioapp/tests/test_phone_calls.py::test_forbidden_requests /usr/local/lib/python3.11/site-packages/social_django/urls.py:15: RemovedInDjango40Warning: django.conf.urls.url() is deprecated in favor of django.urls.re_path(). url(r'^login/(?P<backend>[^/]+){0}$'.format(extra), views.auth, ``` ```bash apps/twilioapp/tests/test_phone_calls.py: 66 warnings /usr/local/lib/python3.11/site-packages/debug_toolbar/utils.py:255: DeprecationWarning: currentThread() is deprecated, use current_thread() instead thread = threading.currentThread() ``` ## Checklist - [x] Unit, integration, and e2e (if applicable) tests updated - [x] Documentation added (or `pr:no public docs` PR label added if not required) - [ ] `CHANGELOG.md` updated (or `pr:no changelog` PR label added if not required)
2023-06-06 20:38:00 +02:00
def __call__(self, serializer_field):
self.team = serializer_field.context["request"].user.current_team
return self.team
def __repr__(self):
return "%s()" % self.__class__.__name__
class CurrentUserDefault:
"""
Utility class to get the current user right from the serializer field.
"""
Fix warnings when running backend tests (#2079) # What this PR does - update `make test` to always use `settings.ci-test`. Right now it will use whatever the value of `DJANGO_SETTINGS_MODULE` is in `./dev/.env.dev`, which causes ~45 tests to fail - Fix several Python warnings that we see when running the tests ```bash RemovedInDjango40Warning: The providing_args argument is deprecated. As it is purely documentational, it has no replacement. If you rely on this argument as documentation, you can move the text to a code comment or docstring. alert_create_signal = django.dispatch.Signal( ``` ```bash PytestCollectionWarning: cannot collect test class 'TestOnlyBackend' because it has a __init__ constructor (from: apps/api/tests/test_alert_receive_channel_template.py) class TestOnlyBackend(BaseMessagingBackend): ``` ```bash DeprecationWarning: The parameter 'use_aliases' in emoji.emojize() is deprecated and will be removed in version 2.0.0. Use language='alias' instead. To hide this warning, pin/downgrade the package to 'emoji~=1.6.3' return emoji.emojize(self.verbal_name, use_aliases=True) ``` ```bash DateTimeField CustomOnCallShift.start received a naive datetime (2023-06-01 12:53:12) while time zone support is active. warnings.warn("DateTimeField %s received a naive datetime (%s)" ``` ```bash apps/twilioapp/tests/test_phone_calls.py::test_resolve_by_phone /etc/app/apps/twilioapp/tests/test_phone_calls.py:173: DeprecationWarning: The 'text' argument to find()-type methods is deprecated. Use 'string' instead. content = BeautifulSoup(content, features="html.parser").findAll(text=True) ``` ```bash apps/twilioapp/tests/test_phone_calls.py::test_resolve_by_phone apps/twilioapp/tests/test_phone_calls.py::test_wrong_pressed_digit /usr/local/lib/python3.11/site-packages/bs4/builder/__init__.py:545: XMLParsedAsHTMLWarning: It looks like you're parsing an XML document using an HTML parser. If this really is an HTML document (maybe it's XHTML?), you can ignore or filter this warning. If it's XML, you should know that using an XML parser will be more reliable. To parse this document as XML, make sure you have the lxml package installed, and pass the keyword argument `features="xml"` into the BeautifulSoup constructor. ``` ```bash apps/twilioapp/tests/test_phone_calls.py::test_forbidden_requests /usr/local/lib/python3.11/site-packages/social_django/urls.py:15: RemovedInDjango40Warning: django.conf.urls.url() is deprecated in favor of django.urls.re_path(). url(r'^login/(?P<backend>[^/]+){0}$'.format(extra), views.auth, ``` ```bash apps/twilioapp/tests/test_phone_calls.py: 66 warnings /usr/local/lib/python3.11/site-packages/debug_toolbar/utils.py:255: DeprecationWarning: currentThread() is deprecated, use current_thread() instead thread = threading.currentThread() ``` ## Checklist - [x] Unit, integration, and e2e (if applicable) tests updated - [x] Documentation added (or `pr:no public docs` PR label added if not required) - [ ] `CHANGELOG.md` updated (or `pr:no changelog` PR label added if not required)
2023-06-06 20:38:00 +02:00
requires_context = True
Fix warnings when running backend tests (#2079) # What this PR does - update `make test` to always use `settings.ci-test`. Right now it will use whatever the value of `DJANGO_SETTINGS_MODULE` is in `./dev/.env.dev`, which causes ~45 tests to fail - Fix several Python warnings that we see when running the tests ```bash RemovedInDjango40Warning: The providing_args argument is deprecated. As it is purely documentational, it has no replacement. If you rely on this argument as documentation, you can move the text to a code comment or docstring. alert_create_signal = django.dispatch.Signal( ``` ```bash PytestCollectionWarning: cannot collect test class 'TestOnlyBackend' because it has a __init__ constructor (from: apps/api/tests/test_alert_receive_channel_template.py) class TestOnlyBackend(BaseMessagingBackend): ``` ```bash DeprecationWarning: The parameter 'use_aliases' in emoji.emojize() is deprecated and will be removed in version 2.0.0. Use language='alias' instead. To hide this warning, pin/downgrade the package to 'emoji~=1.6.3' return emoji.emojize(self.verbal_name, use_aliases=True) ``` ```bash DateTimeField CustomOnCallShift.start received a naive datetime (2023-06-01 12:53:12) while time zone support is active. warnings.warn("DateTimeField %s received a naive datetime (%s)" ``` ```bash apps/twilioapp/tests/test_phone_calls.py::test_resolve_by_phone /etc/app/apps/twilioapp/tests/test_phone_calls.py:173: DeprecationWarning: The 'text' argument to find()-type methods is deprecated. Use 'string' instead. content = BeautifulSoup(content, features="html.parser").findAll(text=True) ``` ```bash apps/twilioapp/tests/test_phone_calls.py::test_resolve_by_phone apps/twilioapp/tests/test_phone_calls.py::test_wrong_pressed_digit /usr/local/lib/python3.11/site-packages/bs4/builder/__init__.py:545: XMLParsedAsHTMLWarning: It looks like you're parsing an XML document using an HTML parser. If this really is an HTML document (maybe it's XHTML?), you can ignore or filter this warning. If it's XML, you should know that using an XML parser will be more reliable. To parse this document as XML, make sure you have the lxml package installed, and pass the keyword argument `features="xml"` into the BeautifulSoup constructor. ``` ```bash apps/twilioapp/tests/test_phone_calls.py::test_forbidden_requests /usr/local/lib/python3.11/site-packages/social_django/urls.py:15: RemovedInDjango40Warning: django.conf.urls.url() is deprecated in favor of django.urls.re_path(). url(r'^login/(?P<backend>[^/]+){0}$'.format(extra), views.auth, ``` ```bash apps/twilioapp/tests/test_phone_calls.py: 66 warnings /usr/local/lib/python3.11/site-packages/debug_toolbar/utils.py:255: DeprecationWarning: currentThread() is deprecated, use current_thread() instead thread = threading.currentThread() ``` ## Checklist - [x] Unit, integration, and e2e (if applicable) tests updated - [x] Documentation added (or `pr:no public docs` PR label added if not required) - [ ] `CHANGELOG.md` updated (or `pr:no changelog` PR label added if not required)
2023-06-06 20:38:00 +02:00
def __call__(self, serializer_field):
self.user = serializer_field.context["request"].user
return self.user
def __repr__(self):
return "%s()" % self.__class__.__name__
def validate_ical_url(url):
if url:
if settings.BASE_URL in url:
raise serializers.ValidationError("Potential self-reference")
try:
ical_file = fetch_ical_file(url)
Calendar.from_ical(ical_file)
except requests.exceptions.RequestException:
raise serializers.ValidationError("Ical download failed")
except ValueError:
raise serializers.ValidationError("Ical parse failed")
return url
return None
2022-07-12 15:42:20 -06:00
"""
This utility function is for building a URL when we don't know if the base URL
has been given a trailing / such as reading from environment variable or user
input. If the base URL is coming from a validated model field urljoin can be used
instead. Do not use this function to append query parameters since a / is added
to the end of the base URL if there isn't one.
"""
def create_engine_url(path, override_base=None):
2022-07-12 15:42:20 -06:00
base = settings.BASE_URL
if override_base:
base = override_base
2022-07-12 15:42:20 -06:00
if not base.endswith("/"):
base += "/"
trimmed_path = path.lstrip("/")
return urljoin(base, trimmed_path)
def get_date_range_from_request(request: Request) -> typing.Tuple[str, datetime.date, int]:
"""Extract timezone, starting date and number of days params from request.
Used mainly for schedules and shifts API.
"""
user_tz: str = request.query_params.get("user_tz", "UTC")
raise_exception_if_not_valid_timezone(user_tz)
date = timezone.now().date()
date_param = request.query_params.get("date")
if date_param is not None:
try:
date = dateparse.parse_date(date_param)
except ValueError:
raise BadRequest(detail="Invalid date format")
else:
if date is None:
raise BadRequest(detail="Invalid date format")
starting_date = date if request.query_params.get("date") else None
if starting_date is None:
# default to current week start
starting_date = date - datetime.timedelta(days=date.weekday())
try:
days = int(request.query_params.get("days", 7)) # fallback to a week
except ValueError:
raise BadRequest(detail="Invalid days format")
return user_tz, starting_date, days
Phone provider refactoring (#1713) # What this PR does This PR moves phone notification logic into separate object PhoneBackend and introduces PhoneProvider interface to hide actual implementation of external phone services provider. It should allow add new phone providers just by implementing one class (See SimplePhoneProvider for example). # Why [Asterisk PR](https://github.com/grafana/oncall/pull/1282) showed that our phone notification system is not flexible. However this is one of the most frequent community questions - how to add "X" phone provider. Also, this refactoring move us one step closer to unifying all notification backends, since with PhoneBackend all phone notification logic is collected in one place and independent from concrete realisation. # Highligts 1. PhoneBackend object - contains all phone notifications business logic. 2. PhoneProvider - interface to external phone services provider. 3. TwilioPhoneProvider and SimplePhoneProvider - two examples of PhoneProvider implementation. 4. PhoneCallRecord and SMSRecord models. I introduced these models to keep phone notification limits logic decoupled from external providers. Existing TwilioPhoneCall and TwilioSMS objects will be migrated to the new table to not to reset limits counter. To be able to receive status callbacks and gather from Twilio TwilioPhoneCall and TwilioSMS still exists, but they are linked to PhoneCallRecord and SMSRecord via fk, to not to leat twilio logic into core code. --------- Co-authored-by: Yulia Shanyrova <yulia.shanyrova@grafana.com>
2023-05-24 14:27:48 +08:00
def check_phone_number_is_valid(phone_number):
return re.match(r"^\+\d{8,15}$", phone_number) is not None
def serialize_datetime_as_utc_timestamp(dt: datetime.datetime) -> str:
return dt.strftime("%Y-%m-%dT%H:%M:%S.%fZ")
def valid_jinja_template_for_serializer_method_field(template):
for _, val in template.items():
try:
apply_jinja_template(val, payload={})
except JinjaTemplateWarning:
# Suppress warnings, template may be valid with payload
pass