What is an Alert Agent?
Every paging alert should matter. Most don’t. TierZero Alert Agent investigates every alert by pulling in telemetry data, code, deployments, relevant runbooks, and past incidents to produce a likely root cause and recommended next actions. Noisy alerts get flagged, related alerts get grouped, and known issues get rediscovered.Key Capabilities
Auto-Investigates Every Alert
When an alert fires, TierZero pulls logs, traces, metrics, recent deploys, and past incidents to build a complete picture. By the time an engineer sees it, the investigation is already done.- Cross-stack correlation: Connects signals across your observability tools, code repos, and deployment pipelines automatically.
- Known-issue matching: Checks memory for similar past alerts and applies known fixes without human intervention.
- Full context on escalation: When a human is needed, they get the investigation summary, not a raw alert.
Trend Analysis
TierZero tracks alert frequency, timing, and co-occurrence across your stack. It surfaces trends that humans miss, like an alert that fires 3x more often after Thursday deploys, or two services that always fail together.- Noisy alert detection: Identifies alerts that fire frequently but never lead to action, so you can tune or suppress them.
- Correlated failure patterns: Discovers which alerts tend to fire together, revealing shared root causes across services.
Severity Classification
Determines blast radius and severity based on historical patterns, affected services, and downstream impact. Prioritizes alerts by SLO impact and error budget burn rate.Smart Escalation
Integrates with your IDP and escalates with the full investigation context already attached. Service dependency mapping calculates blast radius across your infrastructure so the right team is paged first.Alert Grouping
Related alerts become one thread, not ten. Your channel stays clean while the AI handles grouping behind the scenes.Supported Alert Sources
TierZero can respond to alerts from:- Slack
- OpsGenie
- PagerDuty
- FireHydrant
- Rootly
- Datadog
- New Relic
Use Cases
Accelerate Incident Response
- Challenge: Engineers lose time context-switching, gathering telemetry, and triaging severity.
- How it works: Slack and webhook alerts trigger an immediate investigation using your investigation runbook; the details page shows a concise summary, tags (category, services), and a link to the full investigation.
- Outcome: Faster triage with pre-populated likely causes and relevant evidence before responders join.
Reduce Alert Fatigue
- Challenge: High-volume channels and duplicate alerts obscure what needs action.
- How it works: Configure alert agents to focus on relevant alerts, use debounce to prevent frequent repeated investigations during noisy spikes, and use alert insights to extract significant alerts and reduce noise.
- Outcome: Higher signal-to-noise, fewer duplicate investigations, and clearer prioritization.
Scale Team Knowledge
- Challenge: Investigation know-how lives in tribal knowledge and inconsistent practices.
- How it works: Codify investigation steps in your Investigation Runbook, then iterate in Prompt Studio using real alert inputs. Every run produces a summarized, searchable history.
- Outcome: Consistent investigations across rotations and faster onboarding for new responders.
How to Create an Alert Agent
Step 1: Alert Source
Connect your alert source to automatically respond to alerts.- Go to Alert Agents and click “Create Alert Agent”
- Choose a source: Slack, OpsGenie, PagerDuty, FireHydrant, or Rootly
- Slack: Select the channel to monitor
- Webhooks: Select one or more webhook subscriptions
Step 2: Filtering & Notifications
Define which alert events to respond to and where to send notifications.- Text Matches: Specify text patterns to match against (e.g., alert name or monitor ID)
- Sender ID (Slack only): Optionally filter by Slack bot, app, or user ID
- Notifications: Configure where to send investigation results
Step 3: Advanced Instructions (Optional)
Customize how your agent investigates alerts. These settings are optional but can improve investigation quality.- Investigation Runbook: Provide custom instructions that TierZero should follow when investigating alerts. This helps the agent focus on what matters most for your specific alerts.
- Impact & Severity Analysis Runbook: Define how TierZero should assess the impact and severity of alerts for better prioritization.
Step 4: Review
Review your configuration and create the alert agent.- Set a name for your alert agent
- Review all settings
- Click “Create Agent”
Best Practices
1. Configure Investigation Runbooks- Custom runbooks instruct the agent to focus on what matters and produce consistent results
- Begin with alerts that already have documented steps or predictable diagnostics
- Keep these prompts shorter than investigation runbooks for quick triage
- Focus on identifying blast radius: which users, orgs, or services are affected
- Define severity criteria specific to your business (e.g., revenue impact, customer-facing vs internal)
- Helps responders prioritize when multiple alerts fire simultaneously
- Test prompts against real alert inputs in Prompt Studio, compare versions, and refine quickly
- Save improved prompts back to the agent once satisfied
- Filter alerts to specific monitors, services, or alert types
- Reduce noise by only investigating relevant alerts