This PR applies the diff described in
https://github.com/grafana/irm/issues/1863 to the PagerDuty migrator
tool.
Specifically, it:
- Fixes the target type check for user references in service filtering
- Ensures escalation policies are included in the PagerDuty API request
- Corrects the logging output for service migration errors
## Summary of Changes
### Improved Filtering Logic
- Changed filtering logic to use OR operations between filter types
(team, users, regex)
- Resources matching ANY filter are now included in the migration
- Made filtering more intuitive and aligned with user expectations
### New `PAGERDUTY_FILTER_USERS` option for `add_users_to_grafana.py`
script
- This new config (environment variable) allows importing only a subset
of users from your PagerDuty instance.
- Added full test coverage for `add_users_to_grafana.py`
- Updated documentation to reflect latest changes
### Added Verbose Logging Option
- Added `PAGERDUTY_VERBOSE_LOGGING` environment variable to control
output verbosity
- When disabled, only shows summary counts without detailed per-resource
output
- Significantly reduces output for large PagerDuty instances
### Fixed User Handling
- Properly skips user fetching and processing when `MIGRATE_USERS=false`
- Marks schedules and escalation policies properly when not migrating
users
- When `MIGRATE_USERS=true` and `PAGERDUTY_FILTER_USERS` is set, only
those specific users are migrated
### Added Migration Progress Summary
- Shows counts of filtered resources and those eligible for migration
- Provides better visibility into the migration process
### Updated Tests
- Added comprehensive tests for the new OR-based filtering logic
- Added tests for user filtering
- Added tests for verbose and non-verbose logging modes
### Updated Documentation
- Clearly documented the new filtering behavior
- Explained the verbose logging option
- Updated configuration descriptions to be more accurate
These changes address issues with filtering behavior and user handling,
making the PagerDuty migrator more intuitive, efficient, and flexible.
# What this PR does
Adds Service and Business Service migration to the Pager Duty Migrator.
To test, in addition to the OnCall configs, you need to crate a Grafana
Service Account with `Admin` permission and generate a token. You will
set `GRAFANA_SERVICE_ACCOUNT_URL`, per the README, to
`https://<namespace>:<token>@<server>` The namespace is the stack id, in
the format of `stacks-<stack id>`
Service migration is configurable, filterable, and idempotent.
## Which issue(s) this PR closes
Related to [issue link here]
<!--
*Note*: If you want the issue to be auto-closed once the PR is merged,
change "Related to" to "Closes" in the line above.
If you have more than one GitHub issue that this PR closes, be sure to
preface
each issue link with a [closing
keyword](https://docs.github.com/en/get-started/writing-on-github/working-with-advanced-formatting/using-keywords-in-issues-and-pull-requests#linking-a-pull-request-to-an-issue).
This ensures that the issue(s) are auto-closed once the PR has been
merged.
-->
## Checklist
- [x] Unit, integration, and e2e (if applicable) tests updated
- [x] Documentation added (or `pr:no public docs` PR label added if not
required)
- [ ] Added the relevant release notes label (see labels prefixed w/
`release:`). These labels dictate how your PR will
show up in the autogenerated release notes.
---------
Co-authored-by: Joey Orlando <joey.orlando@grafana.com>
Co-authored-by: GitHub Actions <actions@github.com>
Co-authored-by: grafana-irm-app[bot] <165293418+grafana-irm-app[bot]@users.noreply.github.com>
Co-authored-by: Joey Orlando <joseph.t.orlando@gmail.com>
This PR adds filtering capabilities to the PagerDuty migrator tool and
fixes user notification rule preservation behavior.
Closes https://github.com/grafana/irm/issues/612
## Changes
### 1. Added Resource Filtering
Added the ability to filter PagerDuty resources during migration based
on:
- Team membership
- User association
- Name patterns (using regex)
New environment variables for filtering:
```
PAGERDUTY_FILTER_TEAM
PAGERDUTY_FILTER_USERS
PAGERDUTY_FILTER_SCHEDULE_REGEX
PAGERDUTY_FILTER_ESCALATION_POLICY_REGEX
PAGERDUTY_FILTER_INTEGRATION_REGEX
```
#### Example Usage
Filter by team:
```bash
docker run --rm \
-e MIGRATING_FROM="pagerduty" \
-e MODE="plan" \
-e ONCALL_API_URL="<your-oncall-api-url>" \
-e ONCALL_API_TOKEN="<your-oncall-api-token>" \
-e PAGERDUTY_API_TOKEN="<your-pd-api-token>" \
-e PAGERDUTY_FILTER_TEAM="SRE Team" \
oncall-migrator
```
Filter by specific users:
```bash
docker run --rm \
-e MIGRATING_FROM="pagerduty" \
-e MODE="plan" \
-e ONCALL_API_URL="<your-oncall-api-url>" \
-e ONCALL_API_TOKEN="<your-oncall-api-token>" \
-e PAGERDUTY_API_TOKEN="<your-pd-api-token>" \
-e PAGERDUTY_FILTER_USERS="P123ABC,P456DEF" \
oncall-migrator
```
Filter schedules by name pattern:
```bash
docker run --rm \
-e MIGRATING_FROM="pagerduty" \
-e MODE="plan" \
-e ONCALL_API_URL="<your-oncall-api-url>" \
-e ONCALL_API_TOKEN="<your-oncall-api-token>" \
-e PAGERDUTY_API_TOKEN="<your-pd-api-token>" \
-e PAGERDUTY_FILTER_SCHEDULE_REGEX="^(Primary|Secondary)" \
oncall-migrator
```
Filter escalation policies by name pattern:
```bash
docker run --rm \
-e MIGRATING_FROM="pagerduty" \
-e MODE="plan" \
-e ONCALL_API_URL="<your-oncall-api-url>" \
-e ONCALL_API_TOKEN="<your-oncall-api-token>" \
-e PAGERDUTY_API_TOKEN="<your-pd-api-token>" \
-e PAGERDUTY_FILTER_ESCALATION_POLICY_REGEX="^Prod" \
oncall-migrator
```
Filter integrations by name pattern:
```bash
docker run --rm \
-e MIGRATING_FROM="pagerduty" \
-e MODE="plan" \
-e ONCALL_API_URL="<your-oncall-api-url>" \
-e ONCALL_API_TOKEN="<your-oncall-api-token>" \
-e PAGERDUTY_API_TOKEN="<your-pd-api-token>" \
-e PAGERDUTY_FILTER_INTEGRATION_REGEX="Prometheus$" \
oncall-migrator
```
### 2. Fixed User Notification Rule Preservation
Introduces a `PRESERVE_EXISTING_USER_NOTIFICATION_RULES` config (default
of `true`). The migrator now:
- does not delete user notification rules in Grafana OnCall, if the
Grafana user already has some defined, AND
`PRESERVE_EXISTING_USER_NOTIFICATION_RULES` is True
- if the Grafana user has no personal notification rules defined in
OnCall, we will create them
- deletes existing user notification rules, and creates new ones, in
Grafana OnCall, if `PRESERVE_EXISTING_USER_NOTIFICATION_RULES` is False
- basically make sure that the state in Grafana OnCall matches the
_latest_ state in PagerDuty
- Improves logging to clearly indicate when rules are being preserved
#### Example Usage
Preserve existing notification policies (default):
```bash
docker run --rm \
-e MIGRATING_FROM="pagerduty" \
-e MODE="migrate" \
-e ONCALL_API_URL="<your-oncall-api-url>" \
-e ONCALL_API_TOKEN="<your-oncall-api-token>" \
-e PAGERDUTY_API_TOKEN="<your-pd-api-token>" \
oncall-migrator
```
Replace existing notification policies:
```bash
docker run --rm \
-e MIGRATING_FROM="pagerduty" \
-e MODE="migrate" \
-e ONCALL_API_URL="<your-oncall-api-url>" \
-e ONCALL_API_TOKEN="<your-oncall-api-token>" \
-e PAGERDUTY_API_TOKEN="<your-pd-api-token>" \
-e PRESERVE_EXISTING_USER_NOTIFICATION_RULES="false" \
oncall-migrator
```
### 3. Improved Testing
Added comprehensive test coverage for filtering functionality and
updated user notification rule preservation tests
## Testing Done
- Manual testing of filtering capabilities in both plan and migrate
modes
- Verified notification policy preservation behavior
# What this PR does
Refactors the PagerDuty migration script to be a bit more generic + adds
a migration script to migrate from Splunk OnCall (VictorOps)
tldr;
```bash
❯ docker build -t oncall-migrator .
[+] Building 0.4s (10/10) FINISHED
❯ docker run --rm \
-e MIGRATING_FROM="pagerduty" \
-e MODE="plan" \
-e ONCALL_API_URL="http://localhost:8080" \
-e ONCALL_API_TOKEN="<ONCALL_API_TOKEN>" \
-e PAGERDUTY_API_TOKEN="<PAGERDUTY_API_TOKEN>" \
oncall-migrator
running pagerduty migration script...
❯ docker run --rm \
-e MIGRATING_FROM="splunk" \
-e MODE="plan" \
-e ONCALL_API_URL="http://localhost:8080" \
-e ONCALL_API_TOKEN="<ONCALL_API_TOKEN>" \
-e SPLUNK_API_ID="<SPLUNK_API_ID>" \
-e SPLUNK_API_KEY="<SPLUNK_API_KEY>" \
oncall-migrator
migrating from splunk oncall...
```
https://www.loom.com/share/a855062d436a4ef79f030e22528d8c71
## Checklist
- [x] Unit, integration, and e2e (if applicable) tests updated
- [x] Documentation added (or `pr:no public docs` PR label added if not
required)
- [x] Added the relevant release notes label (see labels prefixed w/
`release:`). These labels dictate how your PR will
show up in the autogenerated release notes.
