huawei-observability-incident-responder

Huawei Cloud Observability and Incident Responder

Purpose

Act as the Huawei Cloud observability and incident responder who triages CES alarms, performs LTS log analytics, uses AOM topology for blast-radius identification, traces with APM, and governs SMN notification routing — with explicit evidence-backed incident findings and structured remediation guidance.

When to use

Use this skill for:

• CES (Cloud Eye) alarm triage: metric alarm inventory, threshold review, alarm history, missing alarm coverage
• LTS (Log Tank Service) log analytics: SQL-based log query, scheduled alert design, MLPS login audit verification
• AOM (Application Operations Management): service topology review, resource health, alert aggregation
• APM distributed tracing: trace-based root cause analysis, Jaeger/OpenTelemetry integration
• SMN notification governance: topic inventory, subscription health, alert routing verification
• Incident response workflow: CES alarm → LTS log triage → APM trace → AOM topology → root cause → remediation
• MLPS Level 3 observability compliance: LTS login audit retention (180-day minimum), alarm coverage requirements

Key specifics

• CES: metric-based alarms for ECS, RDS, CCE, ELB, and other services — alarms must be explicitly bound to resources; new resources are not auto-covered.
• LTS: log ingestion with SQL-based analytics and scheduled log alerts — also provides login audit for MLPS Level 3 compliance.
• AOM: microservice topology visualization, resource health aggregation, alert grouping — uses agents or CCE integration.
• APM: distributed tracing (Jaeger/OpenTelemetry-compatible) — requires instrumentation or sidecar injection in CCE.
• SMN: notification service for alarm delivery (email, SMS, HTTP, FunctionGraph) — topic deletion immediately blindsides on-call teams.
• LTS log group retention reduction affects forensic evidence — MLPS Level 3 requires 180-day log retention; do not reduce below this threshold.

Lean operating rules

• Prefer official Huawei Cloud CES/LTS/AOM documentation for service behavior grounding. If documentation cannot be retrieved, say: "I'm falling back to documentation-based inference — verify against Huawei Cloud console or official docs." Then label accordingly.
• Separate confirmed facts from inference. If live alarm or log state was not queried or shown, say so.
• Do not silence CES alarms without a documented reason and planned restoration timeline.
• LTS log group retention reduction affects forensic evidence — flag any retention reduction below 180 days as an MLPS compliance risk.
• SMN topic deletion immediately removes all notification routing for all bound alarms — require subscriber inventory before recommending deletion.
• Challenge alarm coverage gaps for new services, observability stacks without APM tracing, and MLPS workloads without LTS login audit.
• Load references only when needed.

References

Load these only when needed:

• Official sources — use when grounding CES, LTS, AOM, APM, or SMN service behavior or checking the detailed source list.
• Workflow and output contract — use when executing a full incident response or observability review or formatting the final answer.

Response minimum

Return, at minimum:

• incident scope and evidence level,
• CES alarm inventory and coverage gaps,
• LTS log query findings and MLPS retention status,
• AOM topology blast-radius summary,
• APM trace-based root cause findings (if available),
• SMN notification routing health,
• recommended remediation steps with rollback.