From edgedelta-pipelines
This skill should be used when users want to create EdgeDelta pipelines, validate pipeline YAML configurations, deploy pipelines to EdgeDelta, or ask about EdgeDelta monitoring and observability. Recognizes phrases like "create a pipeline", "EdgeDelta config", "validate my pipeline", "deploy to EdgeDelta", "what can I monitor", and "help with OTLP/telemetry collection". Provides 7 production-tested templates, validation tools, and direct API deployment.
How this skill is triggered — by the user, by Claude, or both
Slash command
/edgedelta-pipelines:edgedelta-pipelinesThe summary Claude sees in its skill listing — used to decide when to auto-load this skill
Creates, validates, and deploys EdgeDelta pipeline v3 configurations. Provides production-tested templates, validation tools, environment inspection, and interactive pipeline building.
CHANGELOG.mdassets/examples/templates-summary.mdassets/references/best-practices.mdassets/references/sequence-processors.mdassets/references/validation-rules.mdassets/scripts/deploy_pipeline.pyassets/scripts/inspect_environment.pyassets/scripts/pipeline_builder.pyassets/scripts/validate_pipeline.pyassets/templates/template-1-log-pii-masking.yamlassets/templates/template-2-otlp-dual.yamlassets/templates/template-3-mixed-telemetry.yamlassets/templates/template-4-api-json.yamlassets/templates/template-5-multi-api.yamlassets/templates/template-6-prometheus-scraper.yamlassets/templates/template-7-lookup-enrichment.yamlCreates, validates, and deploys EdgeDelta pipeline v3 configurations. Provides production-tested templates, validation tools, environment inspection, and interactive pipeline building.
Activate this skill when the user:
For fast lookups without loading full skill context:
# Find template by use case
grep -n "Use case:" SKILL.md
# Find template architecture
grep -n "Architecture:" SKILL.md
# Find validation rules
grep -n "validation" assets/scripts/validate_pipeline.py
# Find processor examples
grep -rn "type: generic_mask" assets/templates/
# Find API deployment script
ls assets/scripts/deploy_pipeline.py
# See all templates
ls assets/templates/template-*.yaml
When: User wants to quickly deploy a pipeline
Steps:
Ask for EdgeDelta credentials:
~/.edgedelta.env or project .env file firstPresent template options:
Template 1: Log Ingestion with PII Masking
Template 2: OTLP Dual Receiver
Template 3: Mixed Telemetry Processing
Template 4: API Pull with JSON Processing
Template 5: Multi-API with Aggregation
Template 6: Prometheus Metrics Scraper
Template 7: Lookup Enrichment
Ask customization questions based on selected template:
Read the template from assets/templates/template-{number}-{name}.yaml
Customize the template with user's values
Run validation: python3 assets/scripts/validate_pipeline.py /tmp/customized.yaml
If valid, deploy: python3 assets/scripts/deploy_pipeline.py /tmp/customized.yaml <org_id> <api_token>
Provide deployment details:
When: User wants to build based on their environment OR asks "what can I monitor"
Steps:
Ask for credentials (same as Workflow 1)
Run environment inspection:
python3 assets/scripts/inspect_environment.py --verbose
Present findings:
Ask user which sources to monitor
Ask processing requirements:
Build configuration spec (JSON):
{
"tag": "user-specified-name",
"inputs": [
{"type": "file_input", "path": "/var/log/app/*.log"}
],
"processing": {
"pii_masking": ["passwords", "emails"],
"extract_metrics": true
},
"output": "edgedelta"
}
Generate pipeline:
python3 assets/scripts/pipeline_builder.py --config /tmp/spec.json --output /tmp/pipeline.yaml
Validate and deploy (same as Workflow 1 steps 6-8)
When: User provides a YAML file or asks to validate
Steps:
Read the YAML file
Run validation with verbose output:
python3 assets/scripts/validate_pipeline.py <file.yaml> --verbose
Report results:
If errors found, suggest fixes:
assets/references/validation-rules.mdOffer to auto-fix common issues:
ed_self_telemetry_inputjson_field_path starting with "."final: true to last processorIf user wants, apply fixes and re-validate
When: User asks "what can I monitor" without wanting to create pipeline yet
Steps:
Run inspection (no credentials needed):
python3 assets/scripts/inspect_environment.py --format markdown
Present report with categories:
For each finding, suggest EdgeDelta input configuration
Ask if user wants to proceed with pipeline creation
When: User wants to modify an existing pipeline (add processors, fix bugs, change configuration)
Steps:
Get the pipeline ID (conf_id) - user provides or retrieve via EdgeDelta MCP tools:
mcp__edgedelta__get_pipelines # List all pipelines
mcp__edgedelta__get_pipeline_config(conf_id) # Get current config
Get current pipeline configuration to understand existing structure
Create modified YAML with changes:
settings.tag as the existing pipelineValidate the modified YAML:
python3 assets/scripts/validate_pipeline.py /tmp/updated-pipeline.yaml --verbose
Deploy to existing pipeline using --pipeline-id:
python3 assets/scripts/deploy_pipeline.py /tmp/updated-pipeline.yaml --pipeline-id <conf_id> --env-file <path>
Verify deployment via MCP tools:
mcp__edgedelta__get_pipeline_history(conf_id) # Confirm new version is deployed
Alternative: Direct API Deployment When MCP tools are available, update pipelines directly via API calls:
/orgs/{org_id}/pipelines/{conf_id}/save with {"content": yaml_content}/orgs/{org_id}/pipelines/{conf_id}/history → extract timestamp from first entry/orgs/{org_id}/pipelines/{conf_id}/deploy/{timestamp}When: User asks about sequences, processors, patterns, or "how do I..."
Steps:
Identify the question type:
assets/references/sequence-processors.mdassets/references/validation-rules.mdassets/references/best-practices.mdRead relevant reference document
Provide explanation with examples from templates
Link to EdgeDelta documentation when appropriate:
Offer to create example configuration
Check for credentials in this order:
~/.edgedelta.env or project .env fileED_ORG_ID, ED_ORG_API_TOKENAll pipelines use sequence architecture:
Every pipeline MUST have:
version: v3ed_self_telemetry_input nodeed_output)- at column 0 (NO indentation)processors: keyPII Masking:
- type: generic_mask
capture_group_masks:
- capture_group: "(?i)(password|passwd|pwd)[:=]\\S+"
enabled: true
mask: "***PASSWORD***"
name: "password"
Metric Extraction:
- type: extract_metric
extract_metric_rules:
- name: "errors_total"
unit: "1"
conditions:
- 'IsMatch(body, "(?i)ERROR")'
sum:
aggregation_temporality: delta
is_monotonic: true
value: 1
interval: 1m
final: true
OTTL Transform:
- type: ottl_transform
statements: |
set(attributes["processed"], "true")
set(attributes["timestamp"], Now())
❌ persisting_cursor_settings - causes API 500 errors
❌ Unicode in YAML comments (→, ✓, ✗) - API rejects
❌ json_field_path: "." - must use "$" instead
❌ Multiple final: true flags - only last processor
❌ Non-sequence processors in sequences - see compatibility list
python3 assets/scripts/validate_pipeline.py <pipeline.yaml> [--verbose]
# With credentials directly
python3 assets/scripts/deploy_pipeline.py <pipeline.yaml> <org_id> <api_token> [Linux|Windows|Kubernetes]
# With .env file
python3 assets/scripts/deploy_pipeline.py <pipeline.yaml> --env-file <path> [environment]
# With credentials directly
python3 assets/scripts/deploy_pipeline.py <pipeline.yaml> --pipeline-id <conf_id> <org_id> <api_token>
# With .env file
python3 assets/scripts/deploy_pipeline.py <pipeline.yaml> --pipeline-id <conf_id> --env-file <path>
Important: When updating an existing pipeline, use the --pipeline-id flag with the pipeline's conf_id (UUID). The tag in the YAML should match the existing pipeline's tag.
python3 assets/scripts/inspect_environment.py [--verbose] [--format json|markdown]
python3 assets/scripts/pipeline_builder.py --config <spec.json> --output <pipeline.yaml>
User: "I need to collect application logs and mask sensitive data" Assistant: Uses Workflow 1, suggests Template 1, customizes for user's paths
User: "What can I monitor in my Kubernetes cluster?" Assistant: Uses Workflow 4, runs inspect_environment.py, presents K8s findings
User: "This pipeline YAML isn't working" Assistant: Uses Workflow 3, validates, explains errors, offers fixes
User: "How do I extract metrics from logs?" Assistant: Uses Workflow 5, shows extract_metric examples, references docs
When: User asks about specific processors, needs processor syntax, or wants processor specifications
For processor syntax and documentation, consult https://docs.edgedelta.com/processors.
Example 1: User asks about a processor during pipeline building
User: "I need to mask credit cards in the logs"
Assistant: Looks up generic_mask syntax from EdgeDelta processor docs
→ Returns Quick Copy snippet with credit card regex
→ Incorporates into pipeline sequence
Example 2: User needs processor specifications
User: "What parameters does extract_metric support?"
Assistant: Consults https://docs.edgedelta.com/processors for extract_metric
→ Provides parameter table and examples
→ Helps user build extract_metric configuration
Example 3: Troubleshooting processor issues
User: "My json_unroll is failing with 'path cannot start with dot'"
Assistant: Consults https://docs.edgedelta.com/processors for json_unroll
→ Identifies issue and provides fix
→ Updates pipeline configuration
Quick Reference Pattern:
Complementary Workflow:
edgedelta-ottl skilledgedelta-ottl skillYou have comprehensive templates, validation tools, and automation scripts. Guide users confidently through pipeline creation, always validating before deployment, and leveraging the tested templates whenever possible. For detailed processor specifications and syntax, consult https://docs.edgedelta.com/processors. Use the edgedelta-ottl skill for OTTL function reference when building ottl_transform or ottl_filter processors.
npx claudepluginhub edgedelta/claude-code-plugins --plugin edgedelta-pipelinesCreates and repairs Redpanda Connect pipeline configurations from natural language descriptions or broken configs. Delegates to component-search and bloblang-authoring skills.
Automates test-driven Grafana Cloud observability setup: SLOs, alerting, synthetic monitoring, k6 load testing, IRM on-call, dashboards, cost optimization, GitOps export.
Creates a complete monitoring setup guide covering golden signals, alerts, dashboards, logs, and tracing. Use when asked to set up monitoring or define alerting strategy.