Commit graph

94 commits

Author SHA1 Message Date
Matias Bordese
2a87bea6ed
feat: add filter affected services internal endpoint (#5415)
Related to https://github.com/grafana/oncall-private/issues/2977

e.g.

`GET
/api/plugins/grafana-oncall-app/resources/alertgroups/filter_affected_services?service=service-a&service=service-b`

```
[
  {"name": "service-a",
   "service_url": "http://localhost:3000/a/grafana-slo-app/service/service-a",
   "alert_groups_url": "http://localhost:3000/a/grafana-oncall-app/alert-groups?status=0&status=1&started_at=now-7d_now&label=service_name:service-a"}
]
```
2025-01-16 12:19:32 +00:00
Matias Bordese
132bdf235b
feat: update service account auth not to require rbac enabled org (#5360)
Related to https://github.com/grafana/oncall-private/issues/2826

RBAC enabled or not (OSS or cloud), it is possible to get service
account permissions, enabling perm check (for service account tokens) in
public API.

Also allow empty value for users in sync (instead of returning a 400
response).
2024-12-12 22:11:59 +00:00
Joey Orlando
fa071bcd6e
chore: add pytest-socket library + disable network calls in tests (#5315)
# What this PR does

Inspired by [this
discussion](https://github.com/grafana/oncall/pull/5307#discussion_r1862449480).
_tldr;_ ensures that if any of our tests try making an external network
call, they will fail.

Setup an example test:

```python
def test_external_network_call():
    import requests

    response = requests.get('https://www.example.com')
    assert response.status_code == 200
```

and it worked (failed; [example CI test
run](https://github.com/grafana/oncall/actions/runs/12106416991/job/33752144727?pr=5315#step:6:389))
as expected:

```bash
__________________________ test_external_network_call __________________________
    def test_external_network_call():
        import requests
    
>       response = requests.get('https://www.example.com')
requests   = <module 'requests' from '/opt/hostedtoolcache/Python/3.12.3/x64/lib/python3.12/site-packages/requests/__init__.py'>
apps/test_joey.py:4: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
/opt/hostedtoolcache/Python/3.12.3/x64/lib/python3.12/site-packages/requests/api.py:73: in get
    return request("get", url, params=params, **kwargs)
        kwargs     = {}
        params     = None
        url        = 'https://www.example.com'
/opt/hostedtoolcache/Python/3.12.3/x64/lib/python3.12/site-packages/requests/api.py:59: in request
    return session.request(method=method, url=url, **kwargs)
        kwargs     = {'params': None}
        method     = 'get'
        session    = <requests.sessions.Session object at 0x7f10ebaada90>
        url        = 'https://www.example.com'
/opt/hostedtoolcache/Python/3.12.3/x64/lib/python3.12/site-packages/requests/sessions.py:589: in request
    resp = self.send(prep, **send_kwargs)
        allow_redirects = True
        auth       = None
        cert       = None
        cookies    = None
        data       = None
        files      = None
        headers    = None
        hooks      = None
        json       = None
        method     = 'get'
        params     = None
        prep       = <PreparedRequest [GET]>
        proxies    = {}
        req        = <Request [GET]>
        self       = <requests.sessions.Session object at 0x7f10ebaada90>
        send_kwargs = {'allow_redirects': True, 'cert': None, 'proxies': OrderedDict(), 'stream': False, ...}
        settings   = {'cert': None, 'proxies': OrderedDict(), 'stream': False, 'verify': True}
        stream     = None
        timeout    = None
        url        = 'https://www.example.com'
        verify     = None
/opt/hostedtoolcache/Python/3.12.3/x64/lib/python3.12/site-packages/requests/sessions.py:703: in send
    r = adapter.send(request, **kwargs)
        adapter    = <requests.adapters.HTTPAdapter object at 0x7f10ebaada30>
        allow_redirects = True
        hooks      = {'response': []}
        kwargs     = {'cert': None, 'proxies': OrderedDict(), 'stream': False, 'timeout': None, ...}
        request    = <PreparedRequest [GET]>
        self       = <requests.sessions.Session object at 0x7f10ebaada90>
        start      = 1733064371.649901
        stream     = False
/opt/hostedtoolcache/Python/3.12.3/x64/lib/python3.12/site-packages/requests/adapters.py:667: in send
    resp = conn.urlopen(
        cert       = None
        chunked    = False
        conn       = <urllib3.connectionpool.HTTPSConnectionPool object at 0x7f10ebaadd30>
        proxies    = OrderedDict()
        request    = <PreparedRequest [GET]>
        self       = <requests.adapters.HTTPAdapter object at 0x7f10ebaada30>
        stream     = False
        timeout    = Timeout(connect=None, read=None, total=None)
        url        = '/'
        verify     = True
/opt/hostedtoolcache/Python/3.12.3/x64/lib/python3.12/site-packages/urllib3/connectionpool.py:715: in urlopen
    httplib_response = self._make_request(
        assert_same_host = False
        body       = None
        body_pos   = None
        chunked    = False
        clean_exit = False
        conn       = None
        destination_scheme = None
        err        = None
        headers    = {'User-Agent': 'python-requests/2.32.3', 'Accept-Encoding': 'gzip, deflate', 'Accept': '*/*', 'Connection': 'keep-alive'}
        http_tunnel_required = False
        is_new_proxy_conn = False
        method     = 'GET'
        parsed_url = Url(scheme=None, auth=None, host=None, port=None, path='/', query=None, fragment=None)
        pool_timeout = None
        redirect   = False
        release_conn = False
        release_this_conn = True
        response_kw = {'decode_content': False, 'preload_content': False}
        retries    = Retry(total=0, connect=None, read=False, redirect=None, status=None)
        self       = <urllib3.connectionpool.HTTPSConnectionPool object at 0x7f10ebaadd30>
        timeout    = Timeout(connect=None, read=None, total=None)
        timeout_obj = Timeout(connect=None, read=None, total=None)
        url        = '/'
/opt/hostedtoolcache/Python/3.12.3/x64/lib/python3.12/site-packages/urllib3/connectionpool.py:404: in _make_request
    self._validate_conn(conn)
        chunked    = False
        conn       = <urllib3.connection.HTTPSConnection object at 0x7f10ebaadd60>
        httplib_request_kw = {'body': None, 'headers': {'User-Agent': 'python-requests/2.32.3', 'Accept-Encoding': 'gzip, deflate', 'Accept': '*/*', 'Connection': 'keep-alive'}}
        method     = 'GET'
        self       = <urllib3.connectionpool.HTTPSConnectionPool object at 0x7f10ebaadd30>
        timeout    = Timeout(connect=None, read=None, total=None)
        timeout_obj = Timeout(connect=None, read=None, total=None)
        url        = '/'
/opt/hostedtoolcache/Python/3.12.3/x64/lib/python3.12/site-packages/urllib3/connectionpool.py:1060: in _validate_conn
    conn.connect()
        __class__  = <class 'urllib3.connectionpool.HTTPSConnectionPool'>
        conn       = <urllib3.connection.HTTPSConnection object at 0x7f10ebaadd60>
        self       = <urllib3.connectionpool.HTTPSConnectionPool object at 0x7f10ebaadd30>
/opt/hostedtoolcache/Python/3.12.3/x64/lib/python3.12/site-packages/urllib3/connection.py:363: in connect
    self.sock = conn = self._new_conn()
        self       = <urllib3.connection.HTTPSConnection object at 0x7f10ebaadd60>
/opt/hostedtoolcache/Python/3.12.3/x64/lib/python3.12/site-packages/urllib3/connection.py:174: in _new_conn
    conn = connection.create_connection(
        extra_kw   = {'socket_options': [(6, 1, 1)]}
        self       = <urllib3.connection.HTTPSConnection object at 0x7f10ebaadd60>
/opt/hostedtoolcache/Python/3.12.3/x64/lib/python3.12/site-packages/urllib3/util/connection.py:85: in create_connection
    sock.connect(sa)
        address    = ('www.example.com', 443)
        af         = <AddressFamily.AF_INET: 2>
        canonname  = ''
        err        = None
        family     = <AddressFamily.AF_UNSPEC: 0>
        host       = 'www.example.com'
        port       = 443
        proto      = 6
        res        = (<AddressFamily.AF_INET: 2>, <SocketKind.SOCK_STREAM: 1>, 6, '', ('93.184.215.14', 443))
        sa         = ('93.184.215.14', 443)
        sock       = <socket.socket fd=12, family=2, type=1, proto=6, laddr=('0.0.0.0', 0)>
        socket_options = [(6, 1, 1)]
        socktype   = <SocketKind.SOCK_STREAM: 1>
        source_address = None
        timeout    = None
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
inst = <socket.socket fd=12, family=2, type=1, proto=6, laddr=('0.0.0.0', 0)>
args = (('93.184.215.14', 443),), host = '93.184.215.14'
    def guarded_connect(inst, *args):
        host = host_from_connect_args(args)
        if host in allowed_ip_hosts_and_hostnames or (
            _is_unix_socket(inst.family) and allow_unix_socket
        ):
            return _true_connect(inst, *args)
    
>       raise SocketConnectBlockedError(allowed_list, host)
E       pytest_socket.SocketConnectBlockedError: A test tried to use socket.socket.connect() with host "93.184.215.14" (allowed: "calendar.google.com (142.251.167.100,142.251.167.101,142.251.167.102,142.251.167.113,142.251.167.138,142.251.167.139,2607:f8b0:4004:c09::65,2607:f8b0:4004:c09::66,2607:f8b0:4004:c09::71,2607:f8b0:4004:c09::8b),localhost (127.0.0.1,::1),oncall-dev-mariadb ()").
allow_unix_socket = False
allowed_ip_hosts_and_hostnames = {'127.0.0.1', '142.251.167.100', '142.251.167.101', '142.251.167.102', '142.251.167.113', '142.251.167.138', ...}
allowed_list = ['calendar.google.com (142.251.167.100,142.251.167.101,142.251.167.102,142.251.167.113,142.251.167.138,142.251.167.139...8b0:4004:c09::66,2607:f8b0:4004:c09::71,2607:f8b0:4004:c09::8b)', 'localhost (127.0.0.1,::1)', 'oncall-dev-mariadb ()']
args       = (('93.184.215.14', 443),)
host       = '93.184.215.14'
inst       = <socket.socket fd=12, family=2, type=1, proto=6, laddr=('0.0.0.0', 0)>
/opt/hostedtoolcache/Python/3.12.3/x64/lib/python3.12/site-packages/pytest_socket.py:252: SocketConnectBlockedError
```

## Checklist

- [x] Unit, integration, and e2e (if applicable) tests updated
- [x] Documentation added (or `pr:no public docs` PR label added if not
required)
- [x] Added the relevant release notes label (see labels prefixed w/
`release:`). These labels dictate how your PR will
    show up in the autogenerated release notes.
2024-12-02 10:53:18 -05:00
Matias Bordese
bb4875f8a5
feat: add service account checks in plugin auth (#5305)
Related to https://github.com/grafana/oncall-private/issues/2826
Related to https://github.com/grafana/irm/pull/459

Allow org sync requests from service account users. Also trigger a sync
during public API requests if the org wasn't yet setup.
2024-11-28 19:03:07 +00:00
Matias Bordese
2bcbac8454
Enable service account token auth for public API (#5254)
Related to https://github.com/grafana/oncall-private/issues/2826

Continuing work started in https://github.com/grafana/oncall/pull/5211,
this adds support for Grafana service accounts tokens for API
authentication (except alert group actions which will still require a
user behind). Next steps would be updating the go client and the
terraform provider to allow service account token auth for OnCall
resources.

Following proposal 1.1 from
[doc](https://docs.google.com/document/d/1I3nFbsUEkiNPphBXT-kWefIeramTY71qqZ1OA06Kmls/edit?usp=sharing).
2024-11-19 12:52:23 +00:00
Joey Orlando
673d2e9595
feat: persist is_grafana_irm_enabled from backend plugin sync data (#5171)
# What this PR does

Will start persisting the `organization.is_grafana_irm_enabled` flag
from the backend plugin's sync data that is sent to the oncall backend.
The implications of this are that when `is_grafana_irm_enabled` is set
to True, we will:
- start using `grafana-irm-app` prefixed RBAC permissions (RBAC
permissions for `grafana-irm-app`, as well as `grafana-oncall-app`, are
already being synced to the OnCall backend since
https://github.com/grafana/irm/pull/200 was merged/deployed)
- start building UI URLs w/ `grafana-irm-app` instead of
`grafana-oncall-app`

## Which issue(s) this PR closes

Closes https://github.com/grafana/irm/issues/242

## Checklist

- [x] Unit, integration, and e2e (if applicable) tests updated
- [x] Documentation added (or `pr:no public docs` PR label added if not
required)
- [x] Added the relevant release notes label (see labels prefixed w/
`release:`). These labels dictate how your PR will
    show up in the autogenerated release notes.
2024-10-11 14:57:59 -04:00
Joey Orlando
04ab676463
feat: update RBAC permissioning to support grafana-irm-app (#5149)
# What this PR does

Closes https://github.com/grafana/irm/issues/31 (and supersedes
https://github.com/grafana/oncall/pull/4784)

Main changes:
- updates `apps.api.permissions.user_is_authorized` to check the value
of `organization.is_grafana_irm_enabled`. If it is, we check for the
presence of `grafana-irm-app` prefixed RBAC permissions rather than
`grafana-oncall-app`
- cleans-up `engine/apps/api/tests/test_permissions.py` (bulk of the
changes in the PR)
- converts `apps.user_management.models.User.build_permissions_query` to
a `UserQuerySet` method instead
  - means we can now do things like this instead:
  ```python3

User.objects.filter_by_permission(RBACPermission.Permissions.NOTIFICATIONS_READ,
organization)
  ```

## Checklist

- [x] Unit, integration, and e2e (if applicable) tests updated
- [x] Documentation added (or `pr:no public docs` PR label added if not
required)
- [x] Added the relevant release notes label (see labels prefixed w/
`release:`). These labels dictate how your PR will
    show up in the autogenerated release notes.
2024-10-10 19:02:21 +00:00
Joey Orlando
bfcc0b9f29
update URLs constructed by the backend to support IRM plugin (#5137)
# What this PR does

Introduces a new class,
`apps.grafana_plugin.ui_url_builder.UIURLBuilder`, which is responsible
for... building UI URLs (😄). The class mainly does two things:
- it will decide if the URL should point to `grafana-oncall-app` or
`grafana-irm-app` based on the value of
`organization.is_grafana_irm_enabled` (**NOTE**: this value isn't yet
being set + defaults to `False`; logic for setting this value will be
done in a subsequent PR)
- Adds `enum`s, `OnCallPage` and `IncidentPage` to DRYify hardcoded UI
URLs (in case we decide to change these slightly in the near future)

## Checklist

- [x] Unit, integration, and e2e (if applicable) tests updated
- [x] Documentation added (or `pr:no public docs` PR label added if not
required)
- [x] Added the relevant release notes label (see labels prefixed w/
`release:`). These labels dictate how your PR will
    show up in the autogenerated release notes.
2024-10-09 08:55:10 -04:00
Michael Derynck
5cf382a2e3
Fix error encoding on install_v2 endpoint (#5133)
# What this PR does
Fixes incorrect encoding when SyncException contains one of the
predefined dataclass errors in install_v2 endpoint.

## Which issue(s) this PR closes

Related to #5124 

<!--
*Note*: If you want the issue to be auto-closed once the PR is merged,
change "Related to" to "Closes" in the line above.
If you have more than one GitHub issue that this PR closes, be sure to
preface
each issue link with a [closing
keyword](https://docs.github.com/en/get-started/writing-on-github/working-with-advanced-formatting/using-keywords-in-issues-and-pull-requests#linking-a-pull-request-to-an-issue).
This ensures that the issue(s) are auto-closed once the PR has been
merged.
-->

## Checklist

- [x] Unit, integration, and e2e (if applicable) tests updated
- [x] Documentation added (or `pr:no public docs` PR label added if not
required)
- [x] Added the relevant release notes label (see labels prefixed w/
`release:`). These labels dictate how your PR will
    show up in the autogenerated release notes.
2024-10-07 20:08:53 +00:00
Joey Orlando
f39a755942
fix: update internal labels endpoints to work with RBAC (#5099)
# What this PR does

Related to:
- https://github.com/grafana/oncall-private/issues/2943

## Checklist

- [x] Unit, integration, and e2e (if applicable) tests updated
- [x] Documentation added (or `pr:no public docs` PR label added if not
required)
- [x] Added the relevant release notes label (see labels prefixed w/
`release:`). These labels dictate how your PR will
    show up in the autogenerated release notes.

---------

Co-authored-by: Vadim Stepanov <vadimkerr@gmail.com>
2024-10-02 17:39:49 +00:00
Michael Derynck
d1cb862125
Make sync settings configurable (#5002)
# What this PR does
Add settings for how sync jobs get split up to control throughput of
requests.
- `SYNC_V2_MAX_TASKS ` controls how many tasks can run concurrently
- `SYNC_V2_PERIOD_SECONDS` controls the time offset before starting
another set of tasks each time `SYNC_V2_MAX_TASKS` is reached
- `SYNC_V2_BATCH_SIZE` controls how many organizations will be sync'd
per task

## Which issue(s) this PR closes

Related to [issue link here]

<!--
*Note*: If you want the issue to be auto-closed once the PR is merged,
change "Related to" to "Closes" in the line above.
If you have more than one GitHub issue that this PR closes, be sure to
preface
each issue link with a [closing
keyword](https://docs.github.com/en/get-started/writing-on-github/working-with-advanced-formatting/using-keywords-in-issues-and-pull-requests#linking-a-pull-request-to-an-issue).
This ensures that the issue(s) are auto-closed once the PR has been
merged.
-->

## Checklist

- [x] Unit, integration, and e2e (if applicable) tests updated
- [x] Documentation added (or `pr:no public docs` PR label added if not
required)
- [x] Added the relevant release notes label (see labels prefixed w/
`release:`). These labels dictate how your PR will
    show up in the autogenerated release notes.
2024-09-10 14:17:46 +00:00
Michael Derynck
cf69289888
Relax sync team serializer validation to match underlying model (#4976)
# What this PR does
- Teams do not require valid email addresses to be sync'd
- AvatarURL can be blank

## Which issue(s) this PR closes

Related to [issue link here]

<!--
*Note*: If you want the issue to be auto-closed once the PR is merged,
change "Related to" to "Closes" in the line above.
If you have more than one GitHub issue that this PR closes, be sure to
preface
each issue link with a [closing
keyword](https://docs.github.com/en/get-started/writing-on-github/working-with-advanced-formatting/using-keywords-in-issues-and-pull-requests#linking-a-pull-request-to-an-issue).
This ensures that the issue(s) are auto-closed once the PR has been
merged.
-->

## Checklist

- [x] Unit, integration, and e2e (if applicable) tests updated
- [x] Documentation added (or `pr:no public docs` PR label added if not
required)
- [x] Added the relevant release notes label (see labels prefixed w/
`release:`). These labels dictate how your PR will
    show up in the autogenerated release notes.
2024-09-03 18:47:54 +00:00
Michael Derynck
962cc34432
Compress sync data (#4951)
# What this PR does
- Compresses sync data being sent to engine
- Minor fix to log messages when JSON parse errors occur

## Which issue(s) this PR closes

Related to [issue link here]

<!--
*Note*: If you want the issue to be auto-closed once the PR is merged,
change "Related to" to "Closes" in the line above.
If you have more than one GitHub issue that this PR closes, be sure to
preface
each issue link with a [closing
keyword](https://docs.github.com/en/get-started/writing-on-github/working-with-advanced-formatting/using-keywords-in-issues-and-pull-requests#linking-a-pull-request-to-an-issue).
This ensures that the issue(s) are auto-closed once the PR has been
merged.
-->

## Checklist

- [x] Unit, integration, and e2e (if applicable) tests updated
- [x] Documentation added (or `pr:no public docs` PR label added if not
required)
- [x] Added the relevant release notes label (see labels prefixed w/
`release:`). These labels dictate how your PR will
    show up in the autogenerated release notes.
2024-08-29 18:36:35 +00:00
Michael Derynck
586205368e
Sync process and backend plugin tweaks (#4939)
- Split up sync requests into batches to run concurrently
- Add more logging for when API calls to Grafana fail to parse
- Call sync from backend plugin when status is called
- Lock sync from backend plugin to only run every 5 mins
- Add timer display for API call to sync to return remaining time before
sync can execute
- Remove locks from celery task since it's work is low cost and we lock
in the backend plugin anyways.
2024-08-28 16:34:30 +00:00
Matias Bordese
d3f034b218
Handle a deleted organization triggering auth (#4937)
Related to https://github.com/grafana/oncall-private/issues/2203
2024-08-28 16:32:24 +00:00
Michael Derynck
9655a90f23
Fix plugin sync issues (#4930)
# What this PR does
- Fix incorrect response for error message on sync
- Remove sleep delay from sync (natural latency provides enough delay)

## Which issue(s) this PR closes

Related to [issue link here]

<!--
*Note*: If you want the issue to be auto-closed once the PR is merged,
change "Related to" to "Closes" in the line above.
If you have more than one GitHub issue that this PR closes, be sure to
preface
each issue link with a [closing
keyword](https://docs.github.com/en/get-started/writing-on-github/working-with-advanced-formatting/using-keywords-in-issues-and-pull-requests#linking-a-pull-request-to-an-issue).
This ensures that the issue(s) are auto-closed once the PR has been
merged.
-->

## Checklist

- [ ] Unit, integration, and e2e (if applicable) tests updated
- [ ] Documentation added (or `pr:no public docs` PR label added if not
required)
- [ ] Added the relevant release notes label (see labels prefixed w/
`release:`). These labels dictate how your PR will
    show up in the autogenerated release notes.
2024-08-26 19:12:18 +00:00
Matias Bordese
0ac7c40671
Update org creation to use DB uniqueness constraint (#4926)
Fix issue related to
[logs](https://ops.grafana-ops.net/explore?schemaVersion=1&panes=%7B%2257p%22:%7B%22datasource%22:%22000000193%22,%22queries%22:%5B%7B%22refId%22:%22A%22,%22expr%22:%22%7Bnamespace%3D%5C%22amixr-prod%5C%22,%20cluster%3D%5C%22prod-us-central-0%5C%22,%20job%3D%5C%22amixr-prod%2Famixr-engine%5C%22%7D%20%7C%3D%20%5C%22user_management_organization.user_management_organization_stack_id_org_id_727b4929_uniq%5C%22%22,%22queryType%22:%22range%22,%22datasource%22:%7B%22type%22:%22loki%22,%22uid%22:%22000000193%22%7D,%22editorMode%22:%22code%22%7D%5D,%22range%22:%7B%22from%22:%22now-2d%22,%22to%22:%22now%22%7D%7D%7D&orgId=1)
(check for existing org using the unique DB index)
2024-08-26 15:03:26 +00:00
Michael Derynck
a25d44da1a
Move validate_grafana_token_format to common location, use in sync_v2 (#4919)
# What this PR does
Moves validate_grafana_token_format to GrafanaAPIClient, use it in
sync_v2 to improve logging and skip requests that would not work.

## Which issue(s) this PR closes

Related to [issue link here]

<!--
*Note*: If you want the issue to be auto-closed once the PR is merged,
change "Related to" to "Closes" in the line above.
If you have more than one GitHub issue that this PR closes, be sure to
preface
each issue link with a [closing
keyword](https://docs.github.com/en/get-started/writing-on-github/working-with-advanced-formatting/using-keywords-in-issues-and-pull-requests#linking-a-pull-request-to-an-issue).
This ensures that the issue(s) are auto-closed once the PR has been
merged.
-->

## Checklist

- [x] Unit, integration, and e2e (if applicable) tests updated
- [x] Documentation added (or `pr:no public docs` PR label added if not
required)
- [x] Added the relevant release notes label (see labels prefixed w/
`release:`). These labels dictate how your PR will
    show up in the autogenerated release notes.
2024-08-23 19:52:53 +00:00
Michael Derynck
a5770309d9
Add more validation when updating api_token (#4918)
# What this PR does
Skip updating stored api_token for Grafana if it does not look like one.
Note: Exact format is not checked (prefix) since there are some
differences between versions for what API tokens might look like and
this should tolerate those differences.

## Which issue(s) this PR closes

Related to [issue link here]

<!--
*Note*: If you want the issue to be auto-closed once the PR is merged,
change "Related to" to "Closes" in the line above.
If you have more than one GitHub issue that this PR closes, be sure to
preface
each issue link with a [closing
keyword](https://docs.github.com/en/get-started/writing-on-github/working-with-advanced-formatting/using-keywords-in-issues-and-pull-requests#linking-a-pull-request-to-an-issue).
This ensures that the issue(s) are auto-closed once the PR has been
merged.
-->

## Checklist

- [x] Unit, integration, and e2e (if applicable) tests updated
- [x] Documentation added (or `pr:no public docs` PR label added if not
required)
- [x] Added the relevant release notes label (see labels prefixed w/
`release:`). These labels dictate how your PR will
    show up in the autogenerated release notes.
2024-08-23 18:21:57 +00:00
Michael Derynck
b2b64da86c
Fix api_token not being updated (#4912)
# What this PR does
Fixes organization api_token not being updated when it differs from what
is stored in the DB.

## Which issue(s) this PR closes

Related to [issue link here]

<!--
*Note*: If you want the issue to be auto-closed once the PR is merged,
change "Related to" to "Closes" in the line above.
If you have more than one GitHub issue that this PR closes, be sure to
preface
each issue link with a [closing
keyword](https://docs.github.com/en/get-started/writing-on-github/working-with-advanced-formatting/using-keywords-in-issues-and-pull-requests#linking-a-pull-request-to-an-issue).
This ensures that the issue(s) are auto-closed once the PR has been
merged.
-->

## Checklist

- [x] Unit, integration, and e2e (if applicable) tests updated
- [x] Documentation added (or `pr:no public docs` PR label added if not
required)
- [x] Added the relevant release notes label (see labels prefixed w/
`release:`). These labels dictate how your PR will
    show up in the autogenerated release notes.
2024-08-23 13:22:54 +00:00
Michael Derynck
042fb49aaf
Add logging for invalid api_tokens during sync (#4905)
# What this PR does
Add logging for when we skip an organization for sync if it is missing
its api token.

## Which issue(s) this PR closes

Related to [issue link here]

<!--
*Note*: If you want the issue to be auto-closed once the PR is merged,
change "Related to" to "Closes" in the line above.
If you have more than one GitHub issue that this PR closes, be sure to
preface
each issue link with a [closing
keyword](https://docs.github.com/en/get-started/writing-on-github/working-with-advanced-formatting/using-keywords-in-issues-and-pull-requests#linking-a-pull-request-to-an-issue).
This ensures that the issue(s) are auto-closed once the PR has been
merged.
-->

## Checklist

- [x] Unit, integration, and e2e (if applicable) tests updated
- [x] Documentation added (or `pr:no public docs` PR label added if not
required)
- [x] Added the relevant release notes label (see labels prefixed w/
`release:`). These labels dictate how your PR will
    show up in the autogenerated release notes.
2024-08-22 20:40:18 +00:00
Matias Bordese
66925366a6
Fix flaky test depending on DB ordering (#4907)
See
https://github.com/grafana/oncall/actions/runs/10513279782/job/29128524241
2024-08-22 18:56:59 +00:00
Matias Bordese
87dd5c6623
Update sync task to consider never synced orgs (#4901) 2024-08-22 15:09:28 +00:00
Michael Derynck
a16525ea45
Push sync from backend plugins (#4861)
# What this PR does
Change push sync to have a lock to ensure only 1 instance runs at a
time. Also adds argument to optionally specify which orgs to sync. Added
some more log messages.

## Which issue(s) this PR closes

Related to [issue link here]

<!--
*Note*: If you want the issue to be auto-closed once the PR is merged,
change "Related to" to "Closes" in the line above.
If you have more than one GitHub issue that this PR closes, be sure to
preface
each issue link with a [closing
keyword](https://docs.github.com/en/get-started/writing-on-github/working-with-advanced-formatting/using-keywords-in-issues-and-pull-requests#linking-a-pull-request-to-an-issue).
This ensures that the issue(s) are auto-closed once the PR has been
merged.
-->

## Checklist

- [x] Unit, integration, and e2e (if applicable) tests updated
- [x] Documentation added (or `pr:no public docs` PR label added if not
required)
- [x] Added the relevant release notes label (see labels prefixed w/
`release:`). These labels dictate how your PR will
    show up in the autogenerated release notes.
2024-08-20 16:20:33 +00:00
Dominik Broj
06d19bf6e9
New OnCall plugin initialization process (#4657)
# What this PR does

New OnCall plugin initialization process

## Checklist

- [x] Unit, integration, and e2e (if applicable) tests updated
- [x] Documentation added (or `pr:no public docs` PR label added if not
required)
- [x] Added the relevant release notes label (see labels prefixed w/
`release:`). These labels dictate how your PR will
    show up in the autogenerated release notes.

---------

Co-authored-by: Michael Derynck <michael.derynck@grafana.com>
Co-authored-by: Matias Bordese <mbordese@gmail.com>
2024-08-16 16:43:52 +00:00
Joey Orlando
92ac1d884f
address occasional missing nextCursor data key from GCOM GET /instances (#4794)
# What this PR does

Addressing this task exception
([logs](https://ops.grafana-ops.net/goto/rkVAurrSR?orgId=1)):
```
2024-08-08 16:30:49,455 source=engine:celery worker=ForkPoolWorker-18 task_id=969226be-64a8-4616-ac32-3909d1f0cb60 task_name=apps.grafana_plugin.tasks.sync.start_sync_organizations name=celery.app.trace level=ERROR Task apps.grafana_plugin.tasks.sync.start_sync_organizations[969226be-64a8-4616-ac32-3909d1f0cb60] raised unexpected: KeyError('nextCursor')
```

[This
conversation](https://raintank-corp.slack.com/archives/C0K031RP1/p1723158123932529)
in `#grafana-com-dev` has more context. The _tldr;_ takeaway after
chatting w/ the GCOM team was lets simply retry requests in this case.

## Checklist

- [x] Unit, integration, and e2e (if applicable) tests updated
- [x] Documentation added (or `pr:no public docs` PR label added if not
required)
- [x] Added the relevant release notes label (see labels prefixed w/
`release:`). These labels dictate how your PR will
    show up in the autogenerated release notes.
2024-08-09 15:42:10 +00:00
Matias Bordese
35f23cdcc6
Rework organization sync and grafana plugin engine backend (#4756)
Related to
https://github.com/grafana/oncall-private/issues/2806#issuecomment-2246286918.

Prepare engine for the backend plugin enablement/migration:

 - Refactor sync code
- Improve plugin user authentication to set up user on-the-fly (when
missing)
- Implement v2 endpoints for install, sync and status (to be used via
the backend plugin)

(most of the changes come from
https://github.com/grafana/oncall/pull/4657; backport all engine changes
that keep backwards compatibility)
2024-07-31 16:12:56 +00:00
Vadim Stepanov
7a2fc923df
Don't update RBAC status on Grafana server error (#4753)
# What this PR does

Fixes a bug when RBAC permissions are getting erased when Grafana's API
returns a 5xx server error on organization sync.

## Which issue(s) this PR closes

Closes https://github.com/grafana/oncall-private/issues/2834

## Checklist

- [x] Unit, integration, and e2e (if applicable) tests updated
- [x] Documentation added (or `pr:no public docs` PR label added if not
required)
- [x] Added the relevant release notes label (see labels prefixed w/
`release:`). These labels dictate how your PR will
    show up in the autogenerated release notes.
2024-07-29 16:28:35 +00:00
Michael Derynck
ec360f08dd
Fix bug for incorrect arguments used to invoke cleanup_empty_deleted_integrations (#4707)
# What this PR does
- Fixes a bug where cleanup_empty_deleted_integrations was not being
passed the correct arguments
- Removes start_cleanup_organizations now that it is no longer in use

## Which issue(s) this PR closes

Related to [issue link here]

<!--
*Note*: If you want the issue to be auto-closed once the PR is merged,
change "Related to" to "Closes" in the line above.
If you have more than one GitHub issue that this PR closes, be sure to
preface
each issue link with a [closing
keyword](https://docs.github.com/en/get-started/writing-on-github/working-with-advanced-formatting/using-keywords-in-issues-and-pull-requests#linking-a-pull-request-to-an-issue).
This ensures that the issue(s) are auto-closed once the PR has been
merged.
-->

## Checklist

- [x] Unit, integration, and e2e (if applicable) tests updated
- [x] Documentation added (or `pr:no public docs` PR label added if not
required)
- [x] Added the relevant release notes label (see labels prefixed w/
`release:`). These labels dictate how your PR will
    show up in the autogenerated release notes.
2024-07-19 14:32:22 +00:00
Michael Derynck
f7e406ca6f
Only run cleanup for integrations deleted recently (#4677)
# What this PR does
Changes operations to cleanup deleted empty integrations so that they
are only performed on organizations that have deleted integrations
recently. The previous task checked everything because we were not
performing the cleanup on a regular basis, now that it has been
scheduled regularly we can operate on recently deleted integrations
instead.

The existing task has been removed from the schedule, the code for it
can be removed after the other tasks have cleared.

## Which issue(s) this PR closes

Closes [issue link here]

<!--
*Note*: if you have more than one GitHub issue that this PR closes, be
sure to preface
each issue link with a [closing
keyword](https://docs.github.com/en/get-started/writing-on-github/working-with-advanced-formatting/using-keywords-in-issues-and-pull-requests#linking-a-pull-request-to-an-issue).
This ensures that the issue(s) are auto-closed once the PR has been
merged.
-->

## Checklist

- [x] Unit, integration, and e2e (if applicable) tests updated
- [x] Documentation added (or `pr:no public docs` PR label added if not
required)
- [x] Added the relevant release notes label (see labels prefixed w/
`release:`). These labels dictate how your PR will
    show up in the autogenerated release notes.
2024-07-16 13:59:08 +00:00
Michael Derynck
ecd87782a3
Don't retry cleanup tasks (#4633)
# What this PR does
Disable retry on cleanup tasks. Since these will be scheduled to occur
again on a regular interval minimize the tasks left on queue. These are
not critical tasks.

## Which issue(s) this PR closes

Closes [issue link here]

<!--
*Note*: if you have more than one GitHub issue that this PR closes, be
sure to preface
each issue link with a [closing
keyword](https://docs.github.com/en/get-started/writing-on-github/working-with-advanced-formatting/using-keywords-in-issues-and-pull-requests#linking-a-pull-request-to-an-issue).
This ensures that the issue(s) are auto-closed once the PR has been
merged.
-->

## Checklist

- [ ] Unit, integration, and e2e (if applicable) tests updated
- [ ] Documentation added (or `pr:no public docs` PR label added if not
required)
- [ ] Added the relevant release notes label (see labels prefixed w/
`release:`). These labels dictate how your PR will
    show up in the autogenerated release notes.
2024-07-08 20:04:46 +00:00
Joey Orlando
49d20f1a7e
bump uwsgi to 2.0.26 + Python to 3.12.3 (#4495)
# What this PR does

- bumps `uwsgi` to latest version (`2.0.26`), which unblocks us from
bumping Python to 3.12
- bumps Python to 3.12.3
- refactor the Snyk GitHub Actions workflow to use the composable
actions for installed frontend and backend dependencies
- fixes several `AttributeError`s in our tests that went from a warning
to an error in Python 3.12 (see
https://github.com/python/cpython/issues/100690)

# Which issue(s) this PR closes

Closes #4358
Closes https://github.com/grafana/oncall/issues/4387
2024-06-10 15:33:37 -04:00
Joey Orlando
a92203e71c
remove unnecessary CI step bash command (#4397)
# What this PR does

This command is no longer necessary. Currently it doesn't seem to be
working (see screenshot 👇). Presumably `netcat-traditional` is installed
on the `ubuntu-latest-8-cores` runner.

![Screenshot 2024-05-24 at 14 42
23](https://github.com/grafana/oncall/assets/9406895/dcb80711-6168-4885-a7b8-a05e323c03bf)
2024-05-24 15:13:38 -04:00
Joey Orlando
bd6d8f1aaf
fix typo 2024-05-15 11:10:04 -04:00
Joey Orlando
2582a1b1dc
Refactor how RBAC enabled/disabled status is determined for Grafana Cloud stacks (#4279)
# What this PR does

In cloud we are currently (somewhat) improperly determining whether or
not a Grafana stack had the `accessControlOnCall` feature flag enabled.
At first things worked fine. We would enable this feature toggle via the
Grafana Admin UI, and then the OnCall backend would read this value from
GCOM's `GET /instance/<stack_id>` endpoint (via
`config.feature_toggles`), and everything worked as expected.

There was a recent change made in `grafana/deployment_tools` to set this
feature flag to True for all stacks. However, for some reason, the GCOM
endpoint above doesn't return the `accessControlOnCall` feature toggle
value in `config.feature_toggles` if it is set in this manner (it only
returns the value if it is set via the Grafana Admin UI).

So what we should instead be doing is such instead of asking GCOM for
this feature toggle, infer whether RBAC is enabled on the stack by doing
a `HEAD /api/access-control/users/permissions/search` (this endpoint _is
only_ available on a Grafana stack if `accessControlOnCall` is enabled).

**Few caveats to this ☝️**
1. we first have to make sure that the cloud stack is in an `active`
state (ie. not paused). This is because, no matter if the
`accessControlOnCall` is enabled or not, if the stack is in a `paused`
state it will ALWAYS return `HTTP 200` which can be misleading and lead
to bugs (this feels like a bug on the Grafana API, will follow up with
core grafana team)
2. Once we roll out this change we will effectively **actually** be
enabling RBAC for OnCall for all orgs. The Identity Access team would
prefer a progressive rollout, which is why I decided to introduce the
concept of
[`settings.CLOUD_RBAC_ROLLOUT_PERCENTAGE`](https://github.com/grafana/oncall/pull/4279/files#diff-3383aef931e41e44d95829ad971641eeb98fe001be2f5da92217446d300ea1b3R918)
(see also [`Organization.
should_be_considered_for_rbac_permissioning`](https://github.com/grafana/oncall/pull/4279/files#diff-2ca9917f4f56349be39545ee8abd459be5076295d02ca3a7ec545152fcddccdfR348-R362))

## Which issue(s) this PR closes

Related to https://github.com/grafana/identity-access-team/issues/667

## Checklist

- [x] Unit, integration, and e2e (if applicable) tests updated
- [x] Documentation added (or `pr:no public docs` PR label added if not
required)
- [x] Added the relevant release notes label (see labels prefixed w/
`release:`). These labels dictate how your PR will
    show up in the autogenerated release notes.
2024-05-14 16:30:16 +00:00
Michael Derynck
d938b52d80
Add scheduled task to start cleanup tasks (#3976)
# What this PR does
Add scheduled task to start cleanup tasks. Currently purpose is to run
the task every 12 hours and for all active orgs cleanup empty & deleted
integrations. For deleted orgs we can run this manually. It will also
run if an org moves from active to deleted. It is expected to add more
to cleanup_organization task over time.

## Which issue(s) this PR fixes

## Checklist

- [x] Unit, integration, and e2e (if applicable) tests updated
- [x] Documentation added (or `pr:no public docs` PR label added if not
required)
- [x] `CHANGELOG.md` updated (or `pr:no changelog` PR label added if not
required)
2024-03-04 19:45:01 +00:00
Michael Derynck
2c7ebcf87c
Add task to delete empty deleted integrations from the database (#3941)
# What this PR does
Add task which will cleanup deleted integrations from the database if
they don't have any alert groups. This is to help address an issue where
queries are slowing down due to having a large numbers of ids that do
not contribute to the result.

This will be connected as part of sync organization task once it tests
out ok.

## Which issue(s) this PR fixes

## Checklist

- [x] Unit, integration, and e2e (if applicable) tests updated
- [x] Documentation added (or `pr:no public docs` PR label added if not
required)
- [x] `CHANGELOG.md` updated (or `pr:no changelog` PR label added if not
required)
2024-02-27 17:34:41 +00:00
Innokentii Konstantinov
c58a81bbdf
Enable labels feature only if labels plugin is enabled (#3769)
# What this PR does
Adds a check to enable labels feature only if plugin provisioned. It's
needed to be protected from reconciliation delays and etc.

## Checklist

- [x] Unit, integration, and e2e (if applicable) tests updated
- [x] Documentation added (or `pr:no public docs` PR label added if not
required)
- [x] `CHANGELOG.md` updated (or `pr:no changelog` PR label added if not
required)
2024-01-30 07:29:16 +00:00
Michael Derynck
032ced6fd0
Add more logging to plugin sync and install (#3730)
# What this PR does
Add logging to process for syncing OnCall backend with Grafana to help
troubleshoot issues in self-hosted setups.


## Which issue(s) this PR fixes

## Checklist

- [ ] Unit, integration, and e2e (if applicable) tests updated
- [x] Documentation added (or `pr:no public docs` PR label added if not
required)
- [x] `CHANGELOG.md` updated (or `pr:no changelog` PR label added if not
required)
2024-01-23 22:59:33 +00:00
Joey Orlando
da7f07ffd6
Fix occasional AttributeError in apps.grafana_plugin.tasks.sync.sync_organization_async task (#3687)
# Which issue(s) this PR fixes

Fix this issue I came across in a celery task retry exception log:
![Screenshot 2024-01-15 at 11 21
13](https://github.com/grafana/oncall/assets/9406895/ed08f2f1-dc7d-4ad3-88a0-dc02cd740582)


## Checklist

- [x] Unit, integration, and e2e (if applicable) tests updated
- [x] Documentation added (or `pr:no public docs` PR label added if not
required)
- [x] `CHANGELOG.md` updated (or `pr:no changelog` PR label added if not
required)
2024-01-15 11:34:40 -05:00
Joey Orlando
382b18b052
Mobile app proxy gateway (#3449)
# What this PR does

Closes https://github.com/grafana/oncall-private/issues/2324

## Checklist

- [x] Unit, integration, and e2e (if applicable) tests updated
- [x] Documentation added (or `pr:no public docs` PR label added if not
required)
- [x] `CHANGELOG.md` updated (or `pr:no changelog` PR label added if not
required)
2023-12-05 19:58:05 +00:00
Matias Bordese
e39baa6bbe
Revert "Refactor gcom api calls when syncing org" (#3498)
Reverts grafana/oncall#3489

Reviewing logs, it seems something broke related to [token
auth](https://ops.grafana-ops.net/explore?schemaVersion=1&panes=%7B%22ffS%22:%7B%22datasource%22:%22OP27Xzxnk%22,%22queries%22:%5B%7B%22refId%22:%22A%22,%22expr%22:%22%7Bcluster%3D~%5C%22dev-.%2A%5C%22,%20namespace%3D%5C%22grafana-com%5C%22,%20job%3D%5C%22grafana-com%2Fgrafana-com-api%5C%22%7D%20%7C~%20%5C%22%2Finstances%2F%5Ba-z0-9%5D%2B.config%3Dtrue%5C%22%20%7C%3D%20%5C%22Grafana%20OnCall%5C%22%22,%22editorMode%22:%22code%22,%22queryType%22:%22range%22,%22datasource%22:%7B%22type%22:%22loki%22,%22uid%22:%22OP27Xzxnk%22%7D%7D%5D,%22range%22:%7B%22from%22:%22now-1h%22,%22to%22:%22now%22%7D%7D%7D&orgId=1).
Reverting for now, will revisit in a later PR.
2023-12-04 18:02:58 +00:00
Matias Bordese
9eb09c0272
Refactor gcom api calls when syncing org (#3489)
Make one API call instead of two.
2023-12-04 13:08:59 +00:00
Joey Orlando
76a88bc0c1
Revert "upgrade to Python 3.12 (#3456)" and "bump uwsgi version to latest #3466" (#3483)
# What this PR does

This reverts commits 7c4b40a046 and
cdb22285db.

See https://github.com/grafana/oncall-private/pull/2361 for more
details.
2023-12-01 09:56:26 -05:00
Joey Orlando
7c4b40a046
upgrade to Python 3.12 (#3456)
# What this PR does

Upgrade to Python 3.12 + fix several invalid test assertions that lead
to test failures in the latest version of `pytest`:
```
AttributeError: 'called_once_with' is not a valid assertion. Use a spec for the mock if 'called_once_with' is meant to be an attribute.. Did you mean: 'assert_called_once_with'?
```

## Checklist

- [ ] Unit, integration, and e2e (if applicable) tests updated
- [x] Documentation added (or `pr:no public docs` PR label added if not
required)
- [x] `CHANGELOG.md` updated (or `pr:no changelog` PR label added if not
required)
2023-11-30 13:47:41 +00:00
Matias Bordese
7aa78f5f73
Enable flake8-bugbear, fix issues (#3454)
Enables [flake8-bugbear](https://github.com/PyCQA/flake8-bugbear),
checking for bugs/design problems, and [fixes the issues
found](https://pastebin.com/fEDBz6Ta) (some interesting ones,
particularly with mutable args).

Related to https://github.com/grafana/oncall/pull/3448
2023-11-29 15:04:48 +00:00
Michael Derynck
60ef4348f5
Allow OnCall API to use Grafana Service Accounts (#3189)
# What this PR does
Allows public OnCall API to use Grafana service accounts for
authorization. In cloud requests using a Grafana service account token
also needs to provide headers for `X-Grafana-Org-Slug` and
`X-Grafana-Instance-Slug`

This is **alpha** functionality, it may break or be removed in the
future. Going to use this on one endpoint (resolution notes) before we
consider the implications across all of public API.

## Which issue(s) this PR fixes

## Checklist

- [x] Unit, integration, and e2e (if applicable) tests updated
- [x] Documentation added (or `pr:no public docs` PR label added if not
required)
- [x] `CHANGELOG.md` updated (or `pr:no changelog` PR label added if not
required)
2023-11-23 16:42:27 +00:00
Yulya Artyukhina
24f4969f61
Add labels implementation for integration (#3014)
# What this PR does
Adds labels implementation for integrations:
- ability to create/update labels on creating/updating integration
- ability to associate labels to integrations
- cache for label reprs on OnCall side
- feature flag to enable/disable labels

## Which issue(s) this PR fixes
https://github.com/grafana/oncall-private/issues/2157

## Checklist

- [x] Unit, integration, and e2e (if applicable) tests updated
- [x] Documentation added (or `pr:no public docs` PR label added if not
required)
- [x] `CHANGELOG.md` updated (or `pr:no changelog` PR label added if not
required)

---------

Co-authored-by: Maxim <maxim.mordasov@grafana.com>
Co-authored-by: Rares Mardare <rares.mardare@grafana.com>
2023-10-20 07:30:11 +00:00
Michael Derynck
f40e0463d2
Fix inconsistency with API URL returned by plugin status (#3122)
# What this PR does
Change status to return backend URL in the same way (trailing /) that is
used to build URLs throughout by using create_engine_url utility
function.

## Which issue(s) this PR fixes

## Checklist

- [x] Unit, integration, and e2e (if applicable) tests updated
- [x] Documentation added (or `pr:no public docs` PR label added if not
required)
- [x] `CHANGELOG.md` updated (or `pr:no changelog` PR label added if not
required)
2023-10-16 17:38:08 +00:00
Michael Derynck
5a5fb6ea8e
Status to not access grafana headers from mobile request (#2870)
# What this PR does
When accessing the status endpoint using a mobile authentication token
do not use headers that would only be available if the request came from
Grafana.

## Which issue(s) this PR fixes

## Checklist

- [x] Unit, integration, and e2e (if applicable) tests updated
- [x] Documentation added (or `pr:no public docs` PR label added if not
required)
- [x] `CHANGELOG.md` updated (or `pr:no changelog` PR label added if not
required)
2023-08-23 19:39:02 +00:00