Skip to main content

Send Kubernetes incident alerts to PagerDuty with kwatch

· 2 min read
Amgad Ramses
Amgad Ramses
Maintainer of kwatch

PagerDuty is a good destination when a Kubernetes failure needs an on-call response. kwatch adds the Kubernetes context before it sends the incident: the affected workload, likely cause, recent evidence, and what to investigate next.

1. Create a PagerDuty integration​

Create a service, choose an escalation policy, and add the Events API v2 integration. PagerDuty's escalation policy guide and Events API reference show the current UI and integration options.

Create a PagerDuty service Configure the service Choose an escalation policy Add the Events API v2 integration

Copy the service's integration key. It is a credential and must be stored in a Kubernetes Secret, never as a plain value in a ConfigMap or Git repository.

2. Install kwatch with kwatch.sh​

Run the supported interactive installer:

/bin/bash -c "$(curl -fsSL https://kwatch.dev/kwatch.sh)"

Choose Configure notification, select PagerDuty, and provide the integration key. The generated configuration uses a Secret-backed file:

alert:
  pagerduty:
    integrationKey: "${file:/config/pagerduty-integration-key}"

The manager installs the matching release and waits for readiness. It also handles future configuration changes and upgrades; do not apply old release manifests from this historical article.

3. Verify delivery​

Run kwatch lint --check where PagerDuty credential checks are supported, or use the diagnostics /test-alert endpoint for an end-to-end notification. The PagerDuty channel guide covers routing, retry, fallback, and compact delivery options.

Copy the PagerDuty integration key

You should now receive a PagerDuty incident when kwatch detects a configured Kubernetes failure:

PagerDuty incident from kwatch

For more destinations, browse the complete provider reference.