Worker Sentinel Documentation

Installation

Worker Sentinel can be installed in two ways:

Option 1: Upload HPI File

1. Download the latest worker-sentinel.hpi file from your dashboard
2. Navigate to Manage Jenkins → Plugins → Advanced Settings
3. Under "Deploy Plugin", click "Choose File" and select the HPI file
4. Click "Deploy" to install the plugin

Option 2: Manual Installation

1. Copy the worker-sentinel.hpi file to your Jenkins plugins directory:

   cp worker-sentinel.hpi $JENKINS_HOME/plugins/

2. Restart Jenkins to load the plugin

Quick Start

Get up and running with Worker Sentinel in minutes:

1. Enable the Plugin

Navigate to Manage Jenkins → System and find the "Worker Sentinel" section. Check "Enable Worker Sentinel".

2. Add a Notification Destination

Click "Add Destination" and select your preferred channel (Slack, PagerDuty, etc.). Enter the required credentials.

3. Test the Connection

Click "Test Connection" to verify your configuration is working.

4. Activate Your License

Enter your license key in the "License" section. The plugin will validate and activate automatically.

5. Save Configuration

Click "Save" at the bottom of the page. Worker Sentinel will immediately begin monitoring your agents.

Requirements

• Jenkins Version: 2.426.x LTS or later
• Java Version: Java 17 or later
• Network: HTTPS connectivity for license validation and external notifications

Global Settings

Configure Worker Sentinel's global behavior in Manage Jenkins → System → Worker Sentinel.

Enable Worker Sentinel

Master toggle to enable/disable all monitoring and alerting.

Default Grace Period

Time to wait (in seconds) before sending an alert after a node goes offline. Range: 0-3600 seconds. Default: 60 seconds.

This prevents alerts for brief network hiccups or temporary disconnections.

Default Deduplication Window

Time window (in seconds) to prevent duplicate alerts for the same node. Default: 300 seconds (5 minutes).

Send Recovery Notifications

When enabled, sends notifications when nodes come back online after being offline.

A recovery notification goes only to the destinations that actually received that outage's original alert, still filtered by current routing and this setting. If the outage never alerted anyone (for example it recovered within its grace period), no recovery notification is sent either.

Flapping Detection

• Threshold: Number of state changes that triggers flapping detection (default: 5)
• Window: Time window for counting state changes (default: 300 seconds)

If a flapping node settles while still down, any destination the open outage never reached is topped up with a deferred alert, each after its own grace period, instead of staying unalerted.

Notification Destinations

Add one or more notification destinations to receive alerts. Each destination can be configured independently.

Common Settings

• Name: Identifier for this destination
• Enabled: Toggle to enable/disable this destination
• Grace Period Override: Wait time before alerting this destination after a node goes offline. Leave blank to use the Default Grace Period.
• Deduplication Window Override: Override the global deduplication window

Routing Policies

Route alerts to specific destinations based on node characteristics.

Policy Configuration

• Name: Descriptive name for the policy
• Priority: Lower numbers are evaluated first
• Node Name Pattern: Glob pattern (e.g., prod-*) or regex
• Label Expression: Jenkins label expression (e.g., linux && docker)
• Minimum Severity: OFFLINE, RECOVERY, or FLAPPING
• Destinations: Which destinations to route matching alerts to
• Stop on Match: If true, stop evaluating further policies after this match

Example: Route Production Nodes to PagerDuty

Name: Production Alerts
Priority: 1
Node Name Pattern: prod-*
Destinations: PagerDuty
Stop on Match: false

Routing Failures

Routing failures can have different outcomes depending on when they occur:

• If routing can't be evaluated when an outage starts, every destination is timed on the Default Grace Period instead, and a WARNING is logged.
• If routing or dispatch still fails when a destination's grace timer expires, that alert is simply not sent and not retried, also logged at WARNING -- it is dropped, not deferred.

Alert Templates

Customize alert messages using template variables:

Slack Integration

Setup

1. Create a Slack App or use an existing webhook URL
2. In Jenkins, add a "Secret text" credential with your webhook URL
3. In Worker Sentinel, add a Slack destination and select the credential

Configuration Options

• Webhook Credential: Jenkins credential containing the webhook URL
• Channel Override: Post to a specific channel (optional)
• Username: Bot username to display
• Icon: Emoji or URL for bot icon
• Message Template: Custom message format

PagerDuty Integration

Setup

1. In PagerDuty, create a new service or use an existing one
2. Add an "Events API v2" integration to get a routing key
3. In Jenkins, add a "Secret text" credential with the routing key
4. In Worker Sentinel, add a PagerDuty destination

Configuration Options

• Routing Key Credential: Integration key from PagerDuty
• Severity: Alert severity (critical, error, warning, info)
• Auto-Resolve: Automatically resolve incidents when nodes recover. Incident state survives a Jenkins restart, and a failed resolve call retries up to 5 attempts total over about 3 hours (at 5, 15, 45 and 120 minute intervals) before giving up with a WARNING and an audit entry.
• Component/Group/Class: Optional metadata fields

Twilio SMS Integration

Setup

1. Create a Twilio account and get a phone number
2. Find your Account SID and Auth Token in the Twilio console
3. In Jenkins, add credentials for Account SID and Auth Token
4. In Worker Sentinel, add a Twilio destination

Configuration Options

• Account SID: Your Twilio account identifier
• Auth Token: Your Twilio authentication token
• From Number: Your Twilio phone number (E.164 format)
• To Numbers: Recipient phone numbers (E.164 format)
• Max SMS/Hour: Rate limit to prevent unexpected charges

Email Integration

Setup

Email can use the Jenkins mailer configuration or custom SMTP settings.

Configuration Options

• Recipients: Comma-separated email addresses
• Subject Template: Email subject line template
• Body Template: Email body template
• HTML Format: Send as HTML email
• Use Custom SMTP: Override Jenkins mailer settings

API Reference

Worker Sentinel provides a JSON API for integration with other tools.

Endpoint

GET /worker-sentinel/api

Response

{
  "enabled": true,
  "licenseStatus": "valid",
  "nodeCount": 47,
  "offlineNodes": [
    {
      "name": "build-agent-07",
      "offlineSince": "2024-01-15T10:30:00Z",
      "isFlapping": false
    }
  ],
  "recentEvents": [...]
}

Licensing

Activating Your License

1. Purchase a subscription from the product page
2. Copy your license key from the dashboard
3. In Jenkins, navigate to Manage Jenkins → System → Worker Sentinel
4. Enter your license key in the "License Key" field
5. Click "Activate License"

License Validation

Worker Sentinel validates your license periodically (every 24 hours). If the license server is unreachable, a 72-hour grace period allows continued operation.

Moving Your License

To move your license to a different Jenkins controller, regenerate your API key in the dashboard. The old key will be invalidated automatically.

FAQ

Why am I not receiving alerts?

• Verify Worker Sentinel is enabled in the global configuration
• Check that your notification destination is enabled and configured correctly
• Use "Test Connection" to verify connectivity
• Check the grace period - alerts are delayed by default
• Review the audit log for suppressed alerts

How do I reduce alert noise?

• Increase the grace period to ignore brief disconnections
• Enable flapping detection to suppress alerts for unstable nodes
• Use routing policies to send different alert types to different channels
• Increase the deduplication window

Does Worker Sentinel work with cloud agents?

Yes, Worker Sentinel monitors all Jenkins agents including cloud-provisioned agents from EC2, Kubernetes, Docker, etc.

What data does Worker Sentinel collect?

Worker Sentinel only collects:

• Node names and states
• Timestamps of state changes
• License validation data (license key, pseudonymous controller identifier, or the controller URL when a request does not include one as with older plugin versions, plugin version)

We never collect job names, build logs, or user information.

Changelog

Recent Updates (2026-10-23)

• Per-destination grace period overrides are now honored. A recovery notification goes only to the destinations an outage actually alerted, and a flapping node that settles while still down tops up any destination the open outage never reached, each after its own grace period.
• PagerDuty incident state now survives a Jenkins restart, and a failed resolve call retries automatically (up to 5 attempts total, over about 3 hours) before giving up.
• Upgrade notes: the PagerDuty dedup_key format changed to stop a delete-and-recreate collision. If you have a PagerDuty Event Orchestration rule keyed on the old worker-sentinel-<node> format, update it. An incident already open across the upgrade is not auto-resolved and needs one manual resolve in PagerDuty.

Version 1.0.0

Initial Release

• Real-time node monitoring with event-driven detection
• Four notification channels: Slack, PagerDuty, Twilio SMS, Email
• Grace period system to reduce false positives
• Flapping detection for unstable nodes
• Alert deduplication
• Flexible routing policies
• Comprehensive audit logging
• Built-in status dashboard
• JSON API for integrations