From 535baf7fc8bfd207c5df895dff3ce65017c907d7 Mon Sep 17 00:00:00 2001 From: Joey Orlando Date: Fri, 9 Aug 2024 16:09:47 -0400 Subject: [PATCH] Fix missing `setuptools` dep (#4799) # What this PR does _tldr;_ I think we should install `setuptools` into our engine `Dockerfile` + in our CI env because Python 3.12 no longer installs `distutils` by default. This should unblock us from being able to merge #4656 and #4555. **More details** I would like to be able to merge #4656 and #4555. _However_, in both of these PRs `setuptools` is being removed from `requirements-dev.txt` ([here](https://github.com/grafana/oncall/pull/4555/files#diff-d8146d0816a943b0fa69a20399d7bbdb58e1c84c8b7933b2ba6dea7c10c410f5L113-L116) and [here](https://github.com/grafana/oncall/pull/4656/files#diff-d8146d0816a943b0fa69a20399d7bbdb58e1c84c8b7933b2ba6dea7c10c410f5L113-L116)). This leads to things breaking because of: ```bash File "/opt/hostedtoolcache/Python/3.12.3/x64/lib/python3.12/site-packages/polymorphic/__init__.py", line 9, in import pkg_resources ModuleNotFoundError: No module named 'pkg_resources' ``` - https://github.com/grafana/oncall/actions/runs/9865348392/job/27242117474?pr=4555#step:5:98 - https://github.com/grafana/oncall/actions/runs/10078898966/job/27864920455?pr=4656#step:5:100 Python 3.12 made a change to no longer pre-install `distutils` ([relevant release notes](https://docs.python.org/3/whatsnew/3.12.html#:~:text=The%20third%2Dparty%20Setuptools%20package%20continues%20to%20provide%20distutils%2C%20if%20you%20still%20require%20it%20in%20Python%203.12%20and%20beyond)): > [PEP 632](https://peps.python.org/pep-0632/): Remove the distutils package. See [the migration guide](https://peps.python.org/pep-0632/#migration-advice) for advice replacing the APIs it provided. The third-party [Setuptools](https://setuptools.pypa.io/en/latest/deprecated/distutils-legacy.html) package continues to provide distutils, if you still require it in Python 3.12 and beyond. > > [gh-95299](https://github.com/python/cpython/issues/95299): Do not pre-install setuptools in virtual environments created with [venv](https://docs.python.org/3/library/venv.html#module-venv). This means that distutils, setuptools, pkg_resources, and easy_install will no longer available by default; to access these run pip install setuptools in the [activated](https://docs.python.org/3/library/venv.html#venv-explanation) virtual environment. Additionally, `setuptools` is in `pip-tools` `UNSAFE_PACKAGES` list ([related GitHub issue](https://github.com/pypa/pipenv/issues/1417#issuecomment-364795745)), hence why I think Dependabot is removing it in #4656 and #4555. ## Checklist - [x] Unit, integration, and e2e (if applicable) tests updated - [x] Documentation added (or `pr:no public docs` PR label added if not required) - [x] Added the relevant release notes label (see labels prefixed w/ `release:`). These labels dictate how your PR will show up in the autogenerated release notes. --- .github/actions/setup-python/action.yml | 2 +- engine/Dockerfile | 2 +- engine/requirements-dev.txt | 4 ++-- engine/requirements.in | 11 +---------- engine/requirements.txt | 10 +++++++--- 5 files changed, 12 insertions(+), 17 deletions(-) diff --git a/.github/actions/setup-python/action.yml b/.github/actions/setup-python/action.yml index 1a9b2df6..6d46a6bd 100644 --- a/.github/actions/setup-python/action.yml +++ b/.github/actions/setup-python/action.yml @@ -23,5 +23,5 @@ runs: if: ${{ inputs.install-dependencies == 'true' }} shell: bash run: | - pip install uv + pip install uv setuptools uv pip sync --system ${{ inputs.python-requirements-paths }} diff --git a/engine/Dockerfile b/engine/Dockerfile index f3ab2e7b..f2f134cd 100644 --- a/engine/Dockerfile +++ b/engine/Dockerfile @@ -27,7 +27,7 @@ RUN if [ "$TARGETPLATFORM" = "linux/arm64" ]; then \ && rm grpcio-1.64.1-cp312-cp312-linux_aarch64.whl; \ fi -RUN pip install uv +RUN pip install uv setuptools # TODO: figure out how to get this to work.. see comment in .github/workflows/e2e-tests.yml # https://stackoverflow.com/a/71846527 diff --git a/engine/requirements-dev.txt b/engine/requirements-dev.txt index 532c5d10..4608e48c 100644 --- a/engine/requirements-dev.txt +++ b/engine/requirements-dev.txt @@ -106,11 +106,11 @@ pyyaml==6.0.1 # via # -c requirements.txt # pre-commit -requests==2.32.0 +requests==2.32.3 # via # -c requirements.txt # djangorestframework-stubs -setuptools==70.0.0 +setuptools==72.1.0 # via # -c requirements.txt # nodeenv diff --git a/engine/requirements.in b/engine/requirements.in index fb9f4b65..c57ca1ac 100644 --- a/engine/requirements.in +++ b/engine/requirements.in @@ -53,7 +53,7 @@ python-telegram-bot==13.13 recurring-ical-events==2.1.0 redis==5.0.1 regex==2021.11.2 -requests==2.32.0 +requests==2.32.3 slack-export-viewer==1.1.4 slack_sdk==3.21.3 social-auth-app-django==5.4.1 @@ -64,12 +64,3 @@ whitenoise==5.3.0 google-api-python-client==2.122.0 google-auth-httplib2==0.2.0 google-auth-oauthlib==1.2.0 -# see the following resources as to why we need to install setuptools manually -# -# Python 3.12 release notes https://docs.python.org/3/whatsnew/3.12.html -# -# python/cpython#95299: Do not pre-install setuptools in virtual environments -# created with venv. This means that distutils, setuptools, pkg_resources, and -# easy_install will no longer available by default; to access these run pip -# install setuptools in the activated virtual environment. -setuptools==70.0.0 diff --git a/engine/requirements.txt b/engine/requirements.txt index e3584de8..10488803 100644 --- a/engine/requirements.txt +++ b/engine/requirements.txt @@ -34,7 +34,7 @@ cachetools==4.2.2 # via # google-auth # python-telegram-bot -celery[redis]==5.3.1 +celery==5.3.1 # via -r requirements.in certifi==2024.2.2 # via @@ -157,7 +157,7 @@ firebase-admin==5.4.0 # via fcm-django flask==3.0.2 # via slack-export-viewer -google-api-core[grpc]==2.17.0 +google-api-core==2.17.0 # via # firebase-admin # google-api-python-client @@ -392,7 +392,7 @@ referencing==0.33.0 # jsonschema-specifications regex==2021.11.2 # via -r requirements.in -requests==2.32.0 +requests==2.32.3 # via # -r requirements.in # cachecontrol @@ -415,6 +415,10 @@ rsa==4.9 # via google-auth s3transfer==0.10.0 # via boto3 +setuptools==72.1.0 + # via + # apscheduler + # opentelemetry-instrumentation six==1.16.0 # via # apscheduler