langsmith-skills

LangSmith Skills

⚠️ — This project is in early development. APIs and skill content may change.

Agent skills for observing and evaluating LLM applications with LangSmith. Query traces, build evaluation datasets, and create custom evaluators — all from your coding agent.

Looking for skills to build and improve agents with LangChain, LangGraph, or Deep Agents? See langchain-skills.

Supported Coding Agents

These skills can be installed for any agent supported by skills.sh, including Claude Code, Deep Agents CLI, Cursor, Windsurf, Goose, and many more.

Prerequisites

• A LangSmith API key

Installation

Quick Install

Using npx skills:

Local (current project):

npx skills add langchain-ai/langsmith-skills --skill '*' --yes

Global (all projects):

npx skills add langchain-ai/langsmith-skills --skill '*' --yes --global

To link skills to a specific agent (e.g. Claude Code):

npx skills add langchain-ai/langsmith-skills --agent claude-code --skill '*' --yes --global

---

Claude Code Plugin

Install directly as a Claude Code plugin:

/plugin marketplace add langchain-ai/langsmith-skills
/plugin install langsmith-skills@langsmith-skills

---

Install Script (Claude Code & Deep Agents CLI only)

Alternatively, clone the repo and use the install script:

# Install for Claude Code in current directory (default)
./install.sh

# Install for Claude Code in a specific project directory
./install.sh ~/my-project

# Install for Claude Code globally
./install.sh --global

# Install for DeepAgents CLI in a specific project directory
./install.sh --deepagents ~/my-project

# Install for DeepAgents CLI globally (includes agent persona)
./install.sh --deepagents --global

Flag / Argument	Description
DIRECTORY	Target project directory (default: current directory, ignored with --global)
--claude	Install for Claude Code (default)
--deepagents	Install for DeepAgents CLI
--global, -g	Install globally instead of current directory
--force, -f	Overwrite skills with same names as this package
--yes, -y	Skip confirmation prompts

Usage

After installation, set your API keys:

export LANGSMITH_API_KEY=<your-key>
export OPENAI_API_KEY=<your-key>      # For OpenAI models
export ANTHROPIC_API_KEY=<your-key>   # For Anthropic models

Then run your coding agent from the directory where you installed (for local installs) or from anywhere (for global installs).

Available Skills (3)

LangSmith

• langsmith-trace - Query and export traces (includes helper scripts)
• langsmith-dataset - Generate evaluation datasets from traces (includes helper scripts)
• langsmith-evaluator - Create custom evaluators (includes helper scripts)

Note: All skills include Python and TypeScript helper scripts for common operations.

Development

Agent configuration lives in config/. To update an existing installation:

./install.sh --force