Grafana OnCall

Developer-friendly incident response with brilliant Slack integration.

Android & iOS:

• Collect and analyze alerts from multiple monitoring systems
• On-call rotations based on schedules
• Automatic escalations
• Phone calls, SMS, Slack, Telegram notifications

Getting Started

Important

These instructions are for using Grafana 11 or newer. You must enable the feature toggle for externalServiceAccounts. This is already done for the docker files and helm charts. If you are running Grafana separately see the Grafana documentation on how to enable this.

We prepared multiple environments:

• production
• developer
• hobby (described in the following steps)

1. Download docker-compose.yml:

   curl -fsSL https://raw.githubusercontent.com/grafana/oncall/dev/docker-compose.yml -o docker-compose.yml
   

2. Set variables:

   echo "DOMAIN=http://localhost:8080
   # Remove 'with_grafana' below if you want to use existing grafana
   # Add 'with_prometheus' below to optionally enable a local prometheus for oncall metrics
   # e.g. COMPOSE_PROFILES=with_grafana,with_prometheus
   COMPOSE_PROFILES=with_grafana
   # to setup an auth token for prometheus exporter metrics:
   # PROMETHEUS_EXPORTER_SECRET=my_random_prometheus_secret
   # also, make sure to enable the /metrics endpoint:
   # FEATURE_PROMETHEUS_EXPORTER_ENABLED=True
   SECRET_KEY=my_random_secret_must_be_more_than_32_characters_long" > .env
   

3. (Optional) If you want to enable/setup the prometheus metrics exporter (besides the changes above), create a prometheus.yml file (replacing my_random_prometheus_secret accordingly), next to your docker-compose.yml:

   echo "global:
     scrape_interval:     15s
     evaluation_interval: 15s
   
   scrape_configs:
     - job_name: prometheus
       metrics_path: /metrics/
       authorization:
         credentials: my_random_prometheus_secret
       static_configs:
         - targets: [\"host.docker.internal:8080\"]" > prometheus.yml
   

   NOTE: you will need to setup a Prometheus datasource using http://prometheus:9090 as the URL in the Grafana UI.

4. Launch services:

   docker-compose pull && docker-compose up -d
   

5. Provision the plugin (If you run Grafana outside the included docker files install the plugin before these steps):

   If you are using the included docker compose file use admin/admin credentials and localhost:3000 to perform this task. If you have configured Grafana differently adjust your credentials and hostnames accordingly.

   # Note: onCallApiUrl 'engine' and grafanaUrl 'grafana' use the name from the docker compose file.  If you are 
   # running your grafana or oncall engine instance with another hostname adjust accordingly. 
   curl -X POST 'http://admin:admin@localhost:3000/api/plugins/grafana-oncall-app/settings' -H "Content-Type: application/json" -d '{"enabled":true, "jsonData":{"stackId":5, "orgId":100, "onCallApiUrl":"http://engine:8080", "grafanaUrl":"http://grafana:3000"}}'
   curl -X POST 'http://admin:admin@localhost:3000/api/plugins/grafana-oncall-app/resources/plugin/install'
   

6. Start using OnCall, log in to Grafana with credentials as defined above: admin/admin

7. Enjoy! Check our OSS docs if you want to set up Slack, Telegram, Twilio or SMS/calls through Grafana Cloud.

Troubleshooting

Here are some API calls that can be made to help if you are having difficulty connecting Grafana and OnCall. (Modify parameters to match your credentials and environment)

# Use this to get more information about the connection between Grafana and OnCall
curl -X GET 'http://admin:admin@localhost:3000/api/plugins/grafana-oncall-app/resources/plugin/status'

# If you added a user or changed permissions and don't see it show up in OnCall you can manually trigger sync.
# Note: This is called automatically when the app is loaded (page load/refresh) but there is a 5 min timeout so 
# that it does not generate unnecessary activity.
curl -X POST 'http://admin:admin@localhost:3000/api/plugins/grafana-oncall-app/resources/plugin/sync'

Update version

To update your Grafana OnCall hobby environment:

# Update Docker image
docker-compose pull engine

# Re-deploy
docker-compose up -d

After updating the engine, you'll also need to click the "Update" button on the plugin version page. See Grafana docs for more info on updating Grafana plugins.

Join community

Have a question, comment or feedback? Don't be afraid to open an issue!

Stargazers over time

Further Reading

• Automated migration from other on-call tools - Migrator
• Documentation - Grafana OnCall
• Overview Webinar - YouTube
• How To Add Integration - How to Add Integration
• Blog Post - Announcing Grafana OnCall, the easiest way to do on-call management
• Presentation - Deep dive into the Grafana, Prometheus, and Alertmanager stack for alerting and on-call management