Stats
Actions
Tags
From code-abyss
Provides reference knowledge on data engineering, covering orchestration (Airflow, Dagster), stream processing (Kafka Streams, Flink), and data quality patterns. Use for building data pipelines, ETL workflows, and stream processing jobs.
How this skill is triggered — by the user, by Claude, or both
Slash command
/code-abyss:engineering-data-pipelinesThe summary Claude sees in its skill listing — used to decide when to auto-load this skill
```
编排：Airflow(调度) | Dagster(资产) | Prefect(现代流)
流处理：Kafka Streams(嵌入式) | Flink(集群) | Spark Streaming
质量：Great Expectations | dbt tests | Soda Core
幂等(UPSERT/分区覆盖) | 增量(WHERE updated_at > last_run) | 事件驱动触发 | 跨 DAG 依赖 | 数据血缘(ref()/Asset deps)
时间语义选择 | Watermark 乱序容忍 | 状态 TTL 防膨胀 | Checkpoint 间隔 | 端到端 Exactly-Once | 背压监控
分层验证(源→转换→目标) | 完整性+准确性+一致性 | 及时性阈值 | 加权评分 | 告警(Slack/PagerDuty)
工具对比、API 用法、质量维度详见 references/details.md
npx claudepluginhub telagod/code-abyss --plugin code-abyssRoutes users to specialized agents for data pipeline design, schema modeling, data quality, SQL optimization, lakehouse, AI pipelines, and data contracts. Provides access to 23 knowledge domains.
Designs scalable data pipelines for batch and streaming processing. Covers ETL/ELT, Lambda, Kappa, Lakehouse architectures, orchestration (Airflow/Prefect), dbt transformations, and data quality frameworks.
Design batch and streaming data pipelines. Plan ingestion, transformation, quality checks, and failure recovery. Use when building ETL/ELT systems or data infrastructure.