Digest Prompts - TierZero User Manual

Example Digest Prompts

The following are example digest prompts that can be used for reporting. Modify them to work with your specific services, pipelines, and monitoring setup.

Daily Service Anomaly Report

Problem Statement: Proactively detect emerging issues in production services before they escalate into customer-impacting incidents. Prompt:

Execute the following runbook for service 'api-server':
1. Analyze both errors and warnings for the last 24 hours, compare that with the previous 3 days (i.e. from 4 days ago to 1 day ago), do you see any new issues emerging?
2. For any new issue identified (pick the top 5 if many), find the potential root cause by
  a. Querying sample traces if available
  b. Examining any recent code or config changes made to the relevant files
3. Generate a report by deep diving into each new issue. The report should cover:
  a. Severity
  b. Number of occurrences
  c. Example error message or trace
  d. Impact analysis
  e. Potential root cause
  f. Related PRs if any
  g. Recommended next steps

Weekly CI Pipeline Health Report

Problem Statement: Track CI/CD pipeline reliability and identify flaky tests or infrastructure issues that slow down development velocity. Prompt:

For the 'production-release' pipeline over the most recent whole week Monday 00:00:00 PDT to Sunday 23:59:59 PDT
- Find all failed builds on the 'main' branch, record the build numbers
- For each failed build identified, query the failed jobs and their logs to analyze the failure reason
- Summarize your findings in the following report

Report: Top CI Failures

Generate a table in markdown in the following format:

CI Job Name | Failure Count (# of Builds) | Example Link | List of Build Numbers
... | ... | ... | ...
... | ... | ... | ...

List the top 10 frequently failed jobs in the table.

Log Volume Analysis for Cost Optimization

Problem Statement: Identify opportunities to reduce observability costs by analyzing log volume patterns. Prompt:

I'm conducting a logs analysis to see what is contributing the most to log volume over the last 2 days. Follow the instructions and generate a report for me.

1. Determine the top 10 services for logs by volume over the past 2 days.
- Query logs grouped by service.
- Use the timeframe for the last 2 full days.

2. For each of the top 10 services, determine the top 10 log messages by volume over the past 2 days.
- Query logs for each service grouped by message.
- Use the timeframe for the last 2 full days.

Generate a report explaining what is contributing the most to log volume. In particular, format the final report as follows (including a table)

### Indexed Log Volume Analysis

#### Recommendations

[2-3 bullet points with recommendations for reducing indexed log volume]

#### Top 20 Log Messages by Indexed Log Volume Count

| Service | Message | Daily Indexed Log Volume Count |
| ... | ... | ... |
| ... | ... | ... |

(Table should be sorted by log volume count and include the top 20 messages with highest log volume count. Divide the 2-day log volume count by 2 to get the daily log volume count.)

Weekly Infrastructure Cost Report

Problem Statement: Track infrastructure spend and identify cost optimization opportunities. Prompt:

Generate a weekly infrastructure cost report covering the past 7 days:

1. Query cloud cost data to get total spend by service/resource type.
2. Compare with the previous week to identify any significant increases (>10%).
3. Identify the top 5 cost drivers and any anomalies.
4. Look for optimization opportunities:
   - Idle or underutilized resources
   - Resources that could be right-sized
   - Reserved instance coverage gaps

Format the report as:
### Weekly Cost Summary
- Total spend: $X
- Week-over-week change: +/-X%

### Top Cost Drivers
| Resource | Cost | Change |
| ... | ... | ... |

### Optimization Recommendations
[Bullet points with specific recommendations]