managing-sources

Managing Sources

Operational guide for managing changelog sources.

Tool Reference

Operations can be performed via CLI commands or typed MCP/agent tools. Use whichever interface is available in your context.

Operation	CLI	Typed tool
List sources	releases list [slug] --json [--org <org>] [--query <text>] [--has-feed] [--category <c>] [--compact] [--limit <n>] [--page <n>]	list_catalog (filter kind: "source" to exclude products); list_sources is a deprecated alias
Add source	releases admin source create <name> --url <url> [--type <type>] [--org <org>] [--feed-url <url>] [--primary]	manage_source action "add" with name, url, type, organization, feed_url, is_primary (type auto-detected if omitted; only pass is_primary=true when the source is the org's primary changelog — see "Primary Sources")
Edit source	releases admin source update <identifier> [--primary] [--priority <p>]	manage_source action "edit" with identifier, is_primary, fetch_priority, name, url, type (use only when changing an already-added source; prefer setting flags on "add")
Remove source	releases admin source delete <slug> [--ignore --reason <reason>]	manage_source action "remove" with identifier
Fetch releases	releases admin source fetch <slug> [--dry-run] [--max <n>]	manage_source action "fetch" with identifier
Get latest releases	releases tail [slug] --json [--org <org>]	get_latest_releases with source, organization, limit params
Search releases	releases search <query> --json	search with type: ["releases"]; search_releases is a deprecated alias
Evaluate URL	releases admin discovery evaluate <url> --json	evaluate_url with url param
Add org	releases admin org create <name> [--domain <d>] [--description <t>] [--category <c>] [--tags <t1,t2>]	manage_org action "add" with name, domain, description, category, tags
Edit org	releases admin org update <slug> [--category <c>]	manage_org action "edit" with identifier, category
Show org	releases admin org get <slug> --json	get_organization with identifier
Add tags to org	releases admin org tag add <slug> <tags...>	manage_org action "tag_add" with identifier, tags
Link account	releases admin org link <slug> --platform <p> --handle <h>	manage_org action "link_account" with identifier, platform, handle
Add product	releases admin product create <name> --org <org> [--category <c>] [--tags <t>]	manage_product action "add" with name, organization, category, tags
Ignore URL	releases admin policy ignore add --org <org> <url>	exclude_url action "ignore" with url, organization
Block URL	releases admin policy block add <url>	exclude_url action "block" with url
Get playbook	releases admin playbook <org>	manage_playbook action "get" with organization
Update playbook notes	releases admin playbook <org> --notes-file <path> (use - for stdin)	manage_playbook action "update_notes" with organization, notes

Valid categories (pass to manage_org/manage_product): see the enum in those tool descriptions or your system prompt. list_categories (now retired) has been folded into the two tool descriptions.

Listing Sources

Search for existing sources with optional filters:

• query — filter by name, slug, or URL
• organization — filter by org ID or slug
• product — filter by product ID or slug
• category — filter by category
• has_feed — only sources with a discovered feed URL

Use --json (CLI) for structured output. Typed tools always return JSON.

Adding Sources

Required: name and url. Optional: type (github, scrape, feed, agent — auto-detected from URL if omitted), organization (org ID or slug to associate with), feed_url (direct feed URL if known).

On slug collision the API auto-suffixes (changelog → changelog-2, -3, …) and the created row in the response tells you the resolved slug — no rename-and-retry needed.

Naming sources and products

Don't prefix names with the org name. The org is already shown as context on every page — repeating it in each child source produces noise like "Datadog › Datadog dd-trace-py". Pick the bare, recognizable name instead.

Rules, in priority order:

1. GitHub sources → use the repo name. DataDog/dd-trace-py → dd-trace-py, vercel/next.js → next.js. That's the name devs already recognize; the owner/repo byline underneath disambiguates.
2. Website/feed sources → strip the org name if present. Datadog Browser SDK → Browser SDK, Stripe API Changelog → API Changelog.
3. Keep the org prefix only when it's part of the canonical product name. Claude Code, GitHub Actions, Google Cloud Run, Amazon S3 — people say them that way. If you strip the prefix and what's left is the actual name people use, strip. If stripping produces something nobody would recognize on its own, keep the prefix.
4. Org-level content sources keep the prefix. Datadog Blog, Vercel Engineering Blog — "Blog" alone is meaningless, and org-prefix is the standard convention. Same for "Newsroom", "Announcements".
5. Products follow the same rules. A product under Vercel should be Next.js, not Vercel Next.js. A product under Datadog whose actual name is Agent stays Agent — the org context above it already says Datadog.

When in doubt: would a developer reading this name on its own (with the org already shown above) recognize what it is? If yes, strip. If no, keep the prefix.

Organization descriptions

When creating an org, include a brief one-sentence product description. This grounds AI summaries for lesser-known products, and it's also the primary signal for the entity vector index — the unified search tool (and the registry side of hybrid search) matches on description + category, not just name. A good description noticeably improves recall.

Embedding side effects

Adding or editing an org, product, or source triggers an entity embedding into the registry vector index in the background (fire-and-forget on the worker, never blocks the write). PATCHes are gated on the embed-relevant fields (name, description, category, domain, url) actually changing, so cosmetic edits and poll-driven metadata bumps don't re-embed. There's no manual step — if a write succeeds, treat the embedding as in-flight. If you ever need to verify or backfill, run releases admin embed status and then releases admin embed entities (remote mode only).

Removing Sources

When removing discovery results, also ignore the URL to prevent re-discovery. In CLI: releases admin source delete <slug> --ignore --reason "...". With typed tools: call manage_source action "remove" then exclude_url action "ignore".

Ignored URLs (org-scoped)

A URL ignored for one org can still be valid for another org. Always scope ignores to the relevant organization.

Blocked URLs (global)

For spam domains and known-bad URLs that should never be added for any org. Use block_type "domain" to block an entire domain.

Validation Workflow

After adding a source, validate it:

1. Add the source — provide name and URL
2. Fetch — trigger a fetch (CLI: --dry-run for preview, then real fetch; typed tools: manage_source action "fetch")
3. Check results — get latest releases and verify they have titles, dates, content
4. If bad: remove the source and ignore the URL
5. If good: the source is ready for production fetches

Primary Sources

An org can have one source marked as its primary changelog — the main, company-wide changelog.

is_primary is conditional, not default. Only set it when the source you are adding is clearly the org's primary changelog:

• Onboarding a new org with a single top-level changelog (e.g. example.com/changelog) — set is_primary=true on the add.
• Adding a supplementary or secondary source to an existing org (an engineering blog, a per-product changelog, an RSS feed alongside an already-primary page) — do not set is_primary. Leave the existing primary alone.
• The task prompt doesn't mention "primary" or similar — default to not setting it.

When it does apply, set it on the add call in one step, not via a follow-up edit:

manage_source(action="add", name="Changelog", url="https://example.com/changelog", organization="example-corp", is_primary=true)

The same applies on CLI: pass --primary to releases admin source create, not a follow-up source update.

Use manage_source(action="edit", is_primary=true) (or releases admin source update <slug> --primary on the CLI) only when promoting a source you added in a prior session — never in the same flow as the add.

Playbooks

A playbook is a per-org skill for fetching that org's releases. Same mental model as the global skills in this corpus, scoped to one organization. Agents load the playbook into context alongside global skills whenever they fetch from this org — the playbook overrides general rules with the org's specific behavior (naming conventions, what counts as a release, cross-source dedup, rollup cadence).

Each playbook has two layers:

• Header — auto-generated from source metadata. Shows source types, URLs, priorities, parseInstructions, and product groupings. Regenerates automatically on every source mutation. You never edit this directly.
• Agent notes — free-form markdown that you fully control. This is the most important part of the playbook. Write it like a skill an agent will follow — imperative, action-oriented, concise — not like human documentation.

Always read the playbook before fetching or working with an org's sources. Typed tool: manage_playbook action "get" with organization param. CLI: releases admin playbook <org>. If no playbook exists yet, one will be auto-generated on the next source mutation (create/update/delete).

Writing good agent notes

Write notes like a skill for the agent that will fetch from this org — imperative, action-oriented, concise. The reader is an agent about to do work; tell it what to do and what to watch for, not what things are.

Organize notes under these headings:

### Fetch instructions — One paragraph per source. Use imperative voice:

• What to do: "Set version=null", "Parse <h2> elements as version boundaries", "No filtering needed"
• What to expect: cadence, content quality, whether rendering is needed
• When to skip or deprioritize: "Only fetch when looking for launch announcements specifically"
• Cite version format examples where useful (e.g., "semver like 2.1.98")

### Traps — Concise warnings with bolded trigger labels:

• Each trap is a bullet with a bold label and a one-sentence explanation
• Example: **Doubled paths on Platform**: Relative doc links get prefixed with the source URL, producing doubled paths.
• Include disabled sources with "Don't re-discover" warnings so agents don't re-evaluate them
• Only include traps that would cause wasted work or bad data — skip informational notes

### Coverage — Two or three sentences max:

• Which sources are canonical vs supplementary
• Whether active sources cover the org's full release surface
• Any known gaps worth noting

### Release cadence — Call out rollup publishers explicitly. Some orgs don't ship incremental changelog entries at all — they publish seasonal, quarterly, or annual rollup pages that collect many features into one banner post or microsite (e.g. Shopify Editions, Brex Fall Release, Ramp quarterly blog). When this is the case, say so in the notes and tell the parser to classify matching pages as type: rollup. Example:

Ramp publishes quarterly rollups at /blog/new-on-ramp-q*-* and monthly editions at /blog/new-on-ramp-*-edition. Classify all entries from this source as type: rollup — individual features within a rollup are not separately indexed.

The parsing-changelogs skill ("Classifying Rollups" section) covers what rollups look like and when to set the type field. Your job in the playbook is to capture the org-specific signal so future fetches don't have to re-derive it from the page.

Levels of playbook quality

Compilation (fast, from metadata only): Write notes based on source metadata — URL, type, priority, parseInstructions. Good for bulk coverage but claims about page structure, cadence, and version format are inferred, not verified. Suitable for initial scaffolding or low-priority orgs.

Verified (thorough, from actual data): Before writing, query release data and fetch logs to ground every claim in observation:

1. releases list <slug> --json — Check actual version formats, titles, content length, publishedAt patterns
2. releases admin source fetch-log <slug> --json — Check for errors, success rates, stale data
3. Analyze: calculate real cadence from dates, identify empty content or null fields, spot date drift
4. Write notes citing specific data points, not general assumptions

Use the verified approach for high-value orgs, when onboarding new orgs with scrape sources, or when refreshing stale compilation-only playbooks. The difference: "this source likely needs JS rendering" (compilation) vs "all 50 releases have empty content — the RSS feed delivers summaries only, needs crawl mode on per-release pages" (verified).

Write notes during onboarding after you've fetched and validated sources. Update them when you discover new quirks or when source behavior changes. If notes are empty or stale, write them before doing fetch work — future agents (including yourself in later sessions) will benefit.

Updating notes: Use manage_playbook action "update_notes" with the complete notes content — it replaces the entire notes section. You can rewrite, reorganize, or clear notes at any time.

Changing source configuration: The header reflects current source metadata. To change things like parseInstructions, fetchPriority, or crawlEnabled, use manage_source action "edit" with metadata — the header updates automatically.

Product context: Playbooks group sources by product when products are configured. Some sources (like an org's engineering blog) aren't tied to a specific product but may contain content relevant to any product under that org — the playbook calls these out as "Organization-Level Sources" with a note about which products they may cover.

Rendering Control

The scrape adapter can fetch pages with or without a headless browser. Static-site providers (Docusaurus, VitePress, WordPress, Ghost, Mintlify) are fetched without rendering by default — this is ~10-30x faster.

To override the default for a specific source:

• releases admin source update <identifier> --no-render — force fast fetch (no headless browser)
• releases admin source update <identifier> --render — force headless browser rendering

Use --render when you know a source needs JavaScript execution. Use --no-render when you've verified the content is in the initial HTML for a provider not yet in the static list.

After adding a new scrape source with an unknown provider, check the first fetch results. If content is complete, consider setting --no-render and noting the provider behavior in the playbook.

Duplicate Detection

Before adding sources, search for overlapping URLs.

Common duplicates:

• Same repo via GitHub URL vs changelog page (the GitHub source is usually better)
• RSS feed URL vs the page it feeds from (keep the feed)
• With and without trailing slash or www. prefix