upgrade to python 3.11.3 (#1849)

# What this PR does Upgrades the backend to Python 3.11.3 (latest stable release) + update linting step on Drone builds to run **all** the linting steps, not just the Python ones. ## Checklist - [ ] Unit, integration, and e2e (if applicable) tests updated (N/A) - [ ] Documentation added (or `pr:no public docs` PR label added if not required) (N/A) - [x] `CHANGELOG.md` updated (or `pr:no changelog` PR label added if not required)
2023-05-05 11:32:40 -04:00 · 2023-05-05 11:32:40 -04:00 · bb3521b879
commit bb3521b879
parent a61b0d8b3d
11 changed files with 35 additions and 37 deletions
--- a/.drone.yml
+++ b/.drone.yml
@ -61,18 +61,16 @@ steps:
      ref:
        - refs/tags/v*.*.*

-  - name: Lint Backend
-    image: python:3.9
+  - name: Lint Everything
+    image: python:3.11.3
    environment:
      DJANGO_SETTINGS_MODULE: settings.ci-test
    commands:
      - pip install $(grep "pre-commit" engine/requirements.txt)
-      - pre-commit run isort --all-files
-      - pre-commit run black --all-files
-      - pre-commit run flake8 --all-files
+      - pre-commit run --all-files

  - name: Unit Test Backend
-    image: python:3.9
+    image: python:3.11.3
    environment:
      RABBITMQ_URI: amqp://rabbitmq:rabbitmq@rabbit_test:5672
      DJANGO_SETTINGS_MODULE: settings.ci-test
@ -418,6 +416,4 @@ kind: secret
 name: drone_token
 ---
 kind: signature
-hmac: 321dd85cbf586980c1e00e6cb962da1e7a270c2cef170c789e1f3a8f64647ef5
-
-...
+hmac: a428b546ec4557971dfe3c7eeeac70ddf08faea7336607647ad3f146dd4cb46a
--- a/.github/workflows/linting-and-tests.yml
+++ b/.github/workflows/linting-and-tests.yml
@ -26,7 +26,7 @@ jobs:
      - uses: actions/checkout@v3
      - uses: actions/setup-python@v4
        with:
-          python-version: "3.9.12"
+          python-version: "3.11.3"
          cache: "pip"
          cache-dependency-path: engine/requirements.txt
      # following 2 steps - need to install the frontend dependencies for the eslint/prettier/stylelint steps
@ -111,7 +111,7 @@ jobs:
      - uses: actions/checkout@v3
      - uses: actions/setup-python@v4
        with:
-          python-version: "3.9.12"
+          python-version: "3.11.3"
          cache: "pip"
          cache-dependency-path: engine/requirements.txt
      - name: Lint migrations
@ -151,7 +151,7 @@ jobs:
      - uses: actions/checkout@v3
      - uses: actions/setup-python@v4
        with:
-          python-version: "3.9.12"
+          python-version: "3.11.3"
          cache: "pip"
          cache-dependency-path: engine/requirements.txt
      - name: Unit Test Backend
@ -199,7 +199,7 @@ jobs:
      - uses: actions/checkout@v3
      - uses: actions/setup-python@v4
        with:
-          python-version: "3.9.12"
+          python-version: "3.11.3"
          cache: "pip"
          cache-dependency-path: engine/requirements.txt
      - name: Unit Test Backend
@ -235,7 +235,7 @@ jobs:
      - uses: actions/checkout@v3
      - uses: actions/setup-python@v4
        with:
-          python-version: "3.9.12"
+          python-version: "3.11.3"
          cache: "pip"
          cache-dependency-path: engine/requirements.txt
      - name: Unit Test Backend
@ -252,7 +252,7 @@ jobs:
      - uses: actions/checkout@v3
      - uses: actions/setup-python@v4
        with:
-          python-version: "3.9.12"
+          python-version: "3.11.3"
          cache: "pip"
          cache-dependency-path: tools/pagerduty-migrator/requirements.txt
      - name: Unit Test PD Migrator
--- a/.github/workflows/snyk.yml
+++ b/.github/workflows/snyk.yml
@ -17,7 +17,7 @@ jobs:
      - uses: actions/checkout@v3
      - uses: actions/setup-python@v4
        with:
-          python-version: "3.9.12"
+          python-version: "3.11.3"
          cache: "pip"
          cache-dependency-path: engine/requirements.txt
      - uses: actions/setup-node@v3
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@ -14,6 +14,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 ### Changed

 - Improve ical comparison when checking for imported ical updates ([1870](https://github.com/grafana/oncall/pull/1870))
+- Upgrade to Python 3.11.3 by @joeyorlando ([#1849](https://github.com/grafana/oncall/pull/1849))

 ### Fixed

--- a/dev/README.md
+++ b/dev/README.md
@ -155,8 +155,8 @@ See the `django-silk` documentation [here](https://github.com/jazzband/django-si
 By default everything runs inside Docker. If you would like to run the backend services outside of Docker
 (for integrating w/ PyCharm for example), follow these instructions:

-1. Create a Python 3.9 virtual environment using a method of your choosing (ex.
-   [venv](https://docs.python.org/3.9/library/venv.html) or [pyenv-virtualenv](https://github.com/pyenv/pyenv-virtualenv)).
+1. Create a Python 3.11 virtual environment using a method of your choosing (ex.
+   [venv](https://docs.python.org/3.11/library/venv.html) or [pyenv-virtualenv](https://github.com/pyenv/pyenv-virtualenv)).
   Make sure the virtualenv is "activated".
 2. `postgres` is a dependency on some of our Python dependencies (notably `psycopg2`
   ([docs](https://www.psycopg.org/docs/install.html#prerequisites))). Please visit
--- a/dev/scripts/generate-fake-data/README.md
+++ b/dev/scripts/generate-fake-data/README.md
@ -10,7 +10,7 @@ capable of generating the following objects:

 ## Prerequisites

-1. Create/active a Python 3.9 virtual environment
+1. Create/active a Python 3.11 virtual environment
 2. `pip install -r requirements.txt`
 3. Must have a local version of Grafana and OnCall up and running
 4. Generate an API key inside of Grafana OnCall
--- a/engine/Dockerfile
+++ b/engine/Dockerfile
@ -1,4 +1,4 @@
-FROM python:3.9-slim-buster AS base
+FROM python:3.11.3-slim-buster AS base
 RUN apt-get update && apt-get install -y \
    python3-dev \
    gcc \
@ -6,7 +6,8 @@ RUN apt-get update && apt-get install -y \
    libpq-dev \
    netcat \
    curl \
-    bash
+    bash \
+    git

 WORKDIR /etc/app
 COPY ./requirements.txt ./
--- a/engine/apps/api/tests/test_alert_group.py
+++ b/engine/apps/api/tests/test_alert_group.py
@ -1346,8 +1346,8 @@ def test_invalid_bulk_action(
    assert response.status_code == status.HTTP_400_BAD_REQUEST


-@patch("apps.alerts.tasks.send_alert_group_signal.apply_async", return_value=None)
-@patch("apps.alerts.tasks.send_update_log_report_signal.apply_async", return_value=None)
+@patch("apps.alerts.tasks.send_alert_group_signal.send_alert_group_signal.apply_async", return_value=None)
+@patch("apps.alerts.tasks.send_update_log_report_signal.send_update_log_report_signal.apply_async", return_value=None)
@patch("apps.alerts.models.AlertGroup.start_escalation_if_needed", return_value=None)
@pytest.mark.django_db
 def test_bulk_action_restart(
@ -1411,8 +1411,8 @@ def test_bulk_action_restart(
    assert mocked_start_escalate_alert.called


-@patch("apps.alerts.tasks.send_alert_group_signal.apply_async", return_value=None)
-@patch("apps.alerts.tasks.send_update_log_report_signal.apply_async", return_value=None)
+@patch("apps.alerts.tasks.send_alert_group_signal.send_alert_group_signal.apply_async", return_value=None)
+@patch("apps.alerts.tasks.send_update_log_report_signal.send_update_log_report_signal.apply_async", return_value=None)
@pytest.mark.django_db
 def test_bulk_action_acknowledge(
    mocked_alert_group_signal_task,
@ -1468,8 +1468,8 @@ def test_bulk_action_acknowledge(
    assert mocked_log_report_signal_task.called


-@patch("apps.alerts.tasks.send_alert_group_signal.apply_async", return_value=None)
-@patch("apps.alerts.tasks.send_update_log_report_signal.apply_async", return_value=None)
+@patch("apps.alerts.tasks.send_alert_group_signal.send_alert_group_signal.apply_async", return_value=None)
+@patch("apps.alerts.tasks.send_update_log_report_signal.send_update_log_report_signal.apply_async", return_value=None)
@pytest.mark.django_db
 def test_bulk_action_resolve(
    mocked_alert_group_signal_task,
@ -1520,8 +1520,8 @@ def test_bulk_action_resolve(
    assert mocked_log_report_signal_task.called


-@patch("apps.alerts.tasks.send_alert_group_signal.apply_async", return_value=None)
-@patch("apps.alerts.tasks.send_update_log_report_signal.apply_async", return_value=None)
+@patch("apps.alerts.tasks.send_alert_group_signal.send_alert_group_signal.apply_async", return_value=None)
+@patch("apps.alerts.tasks.send_update_log_report_signal.send_update_log_report_signal.apply_async", return_value=None)
@patch("apps.alerts.models.AlertGroup.start_unsilence_task", return_value=None)
@pytest.mark.django_db
 def test_bulk_action_silence(
--- a/engine/apps/public_api/tests/test_incidents.py
+++ b/engine/apps/public_api/tests/test_incidents.py
@ -240,7 +240,7 @@ def test_get_incidents_filter_by_route_no_result(
    assert response.json()["results"] == []


-@mock.patch("apps.alerts.tasks.delete_alert_group.apply_async", return_value=None)
+@mock.patch("apps.public_api.views.incidents.delete_alert_group", return_value=None)
@pytest.mark.django_db
 def test_delete_incident_success_response(mocked_task, incident_public_api_setup):
    token, incidents, _, _ = incident_public_api_setup
@ -251,7 +251,7 @@ def test_delete_incident_success_response(mocked_task, incident_public_api_setup
    data = {"mode": "delete"}
    response = client.delete(url, data=data, format="json", HTTP_AUTHORIZATION=f"{token}")
    assert response.status_code == status.HTTP_204_NO_CONTENT
-    assert mocked_task.call_count == 1
+    assert mocked_task.apply_async.call_count == 1


@pytest.mark.django_db
--- a/engine/requirements.txt
+++ b/engine/requirements.txt
@ -8,7 +8,7 @@ django-ordered-model==3.1.1
 celery[amqp,redis]==5.2.7
 redis==3.4.1
 humanize==0.5.1
-uwsgi==2.0.20
+uwsgi==2.0.21
 django-cors-headers==3.7.0
 django-debug-toolbar==3.2.1
 django-sns-view==0.1.2
@ -21,12 +21,12 @@ django-filter==2.4.0
 icalendar==4.0.7
 recurring-ical-events==0.1.16b0
 slack-export-viewer==1.0.0
-beautifulsoup4==4.8.1
+beautifulsoup4==4.12.2
 social-auth-app-django==3.1.0
 cryptography==38.0.4 # version 39.0.0 introduced an issue - https://stackoverflow.com/a/75053968/3902555
-pytest==5.4.3
-pytest-django==3.9.0
-pytest_factoryboy==2.0.3
+pytest==7.1.3
+pytest-django==4.5.2
+pytest_factoryboy==2.5.1
 factory-boy<3.0
 python-json-logger==2.0.1
 django-log-request-id==1.6.0
--- a/tools/pagerduty-migrator/Dockerfile
+++ b/tools/pagerduty-migrator/Dockerfile
@ -1,4 +1,4 @@
-FROM python:3.9-alpine
+FROM python:3.11.3-alpine

 ENV PYTHONUNBUFFERED=1
 WORKDIR /app