Grafana OnCall engine fork — self-hosted on-call scheduler and alert router
Find a file
Yulya Artyukhina adfb496a81
Fix slack channels sync (#2571)
# What this PR does
- Fixes issue with slack channels sync periodic tasks when we get slack
rate limit exception.
- Adds check for active task id to avoid starting multiple tasks for one
slack team.

Collecting channels for slack for some teams causes rate limit
exception, which causes the task to restart and start collecting slack
channels from the beginning. This PR adds new paginated api call and
refactors the slack channel sync task to continue collect data after
rate limit from the step before it was raised using `cursor` value from
the slack response.


## Checklist

- [x] Unit, integration, and e2e (if applicable) tests updated
- [ ] Documentation added (or `pr:no public docs` PR label added if not
required)
- [x] `CHANGELOG.md` updated (or `pr:no changelog` PR label added if not
required)

---------

Co-authored-by: Joey Orlando <joey.orlando@grafana.com>
2023-07-19 07:17:21 +00:00
.github Update all make docs CI to match (#2538) 2023-07-17 13:15:34 +00:00
dev move local developer resources to ./dev folder (#2479) 2023-07-10 06:09:22 -04:00
docs Outgoing webhooks docs minor updates (#2572) 2023-07-18 18:49:55 +01:00
engine Fix slack channels sync (#2571) 2023-07-19 07:17:21 +00:00
grafana-plugin Fix duplicate orders on routes and escalation policies (#2568) 2023-07-18 17:17:53 +00:00
helm Added resources limits definition for wait-for-db container (#2501) 2023-07-12 15:30:20 +00:00
terraform move local developer resources to ./dev folder (#2479) 2023-07-10 06:09:22 -04:00
tools don't enforce line-length rule for markdownlint for code-blocks or tables (#2145) 2023-06-09 06:57:19 +00:00
.drone.yml Remove v prefix from plugin version on build (#2540) 2023-07-17 13:59:16 +00:00
.gitignore ignore .http file extensions (#1762) 2023-04-17 10:52:03 +02:00
.markdownlint.json don't enforce line-length rule for markdownlint for code-blocks or tables (#2145) 2023-06-09 06:57:19 +00:00
.markdownlintignore Add tracing support 2022-12-19 17:15:06 +08:00
.pre-commit-config.yaml Update all links to use docs/reference style (#2456) 2023-07-13 14:38:35 +00:00
CHANGELOG.md Fix slack channels sync (#2571) 2023-07-19 07:17:21 +00:00
CODE_OF_CONDUCT.md add precommit rules for markdown/json files (#915) 2022-12-01 14:26:54 +01:00
docker-compose-developer.yml move local developer resources to ./dev folder (#2479) 2023-07-10 06:09:22 -04:00
docker-compose-mysql-rabbitmq.yml update rabbitmq docker containers to v3.12.0 (#2359) 2023-06-29 06:49:00 -04:00
docker-compose.yml Add instructions for prometheus exporter setup (#2103) 2023-06-12 13:04:07 +00:00
GOVERNANCE.md add precommit rules for markdown/json files (#915) 2022-12-01 14:26:54 +01:00
LICENSE World, meet OnCall! 2022-06-03 08:09:47 -06:00
LICENSING.md add precommit rules for markdown/json files (#915) 2022-12-01 14:26:54 +01:00
MAINTAINERS.md add precommit rules for markdown/json files (#915) 2022-12-01 14:26:54 +01:00
Makefile add mypy static type checker to backend codebase (#2151) 2023-06-12 12:50:33 -04:00
README.md [README]fix prometheus yml indentation error (#2327) 2023-06-26 13:44:33 +00:00
screenshot.png Merge dev to main (#54) 2022-06-13 16:39:58 -06:00
screenshot_mobile.png Readme updates 2023-04-11 15:43:52 +03:00

Grafana OnCall

Latest Release License Docker Pulls Slack Discussion Build Status

Developer-friendly incident response with brilliant Slack integration.

  • Collect and analyze alerts from multiple monitoring systems
  • On-call rotations based on schedules
  • Automatic escalations
  • Phone calls, SMS, Slack, Telegram notifications

Getting Started

We prepared multiple environments:

  1. Download docker-compose.yml:

    curl -fsSL https://raw.githubusercontent.com/grafana/oncall/dev/docker-compose.yml -o docker-compose.yml
    
  2. Set variables:

    echo "DOMAIN=http://localhost:8080
    # Remove 'with_grafana' below if you want to use existing grafana
    # Add 'with_prometheus' below to optionally enable a local prometheus for oncall metrics
    # e.g. COMPOSE_PROFILES=with_grafana,with_prometheus
    COMPOSE_PROFILES=with_grafana
    # to setup an auth token for prometheus exporter metrics:
    # PROMETHEUS_EXPORTER_SECRET=my_random_prometheus_secret
    # also, make sure to enable the /metrics endpoint:
    # FEATURE_PROMETHEUS_EXPORTER_ENABLED=True
    SECRET_KEY=my_random_secret_must_be_more_than_32_characters_long" > .env
    
  3. (Optional) If you want to enable/setup the prometheus metrics exporter (besides the changes above), create a prometheus.yml file (replacing my_random_prometheus_secret accordingly), next to your docker-compose.yml:

    echo "global:
      scrape_interval:     15s
      evaluation_interval: 15s
    
    scrape_configs:
      - job_name: prometheus
        metrics_path: /metrics/
        authorization:
          credentials: my_random_prometheus_secret
        static_configs:
          - targets: [\"host.docker.internal:8080\"]" > prometheus.yml
    

    NOTE: you will need to setup a Prometheus datasource using http://prometheus:9090 as the URL in the Grafana UI.

  4. Launch services:

    docker-compose pull && docker-compose up -d
    
  5. Go to OnCall Plugin Configuration, using log in credentials as defined above: admin/admin (or find OnCall plugin in configuration->plugins) and connect OnCall plugin with OnCall backend:

    OnCall backend URL: http://engine:8080
    
  6. Enjoy! Check our OSS docs if you want to set up Slack, Telegram, Twilio or SMS/calls through Grafana Cloud.

Update version

To update your Grafana OnCall hobby environment:

# Update Docker image
docker-compose pull engine

# Re-deploy
docker-compose up -d

After updating the engine, you'll also need to click the "Update" button on the plugin version page. See Grafana docs for more info on updating Grafana plugins.

Join community

Stargazers over time

Stargazers over time

Further Reading