Agent Skill
2/7/2026

langsmith-evaluator

Use this skill for ANY question about CREATING evaluators. Covers creating custom metrics, LLM as Judge evaluators, code-based evaluators, and uploading evaluation logic to LangSmith. Includes basic usage of evaluators to run evaluations.

L
langchain
20GitHub Stars
1Views
npx skills add langchain-ai/langchain-skills

SKILL.md

Namelangsmith-evaluator
DescriptionUse this skill for ANY question about CREATING evaluators. Covers creating custom metrics, LLM as Judge evaluators, code-based evaluators, and uploading evaluation logic to LangSmith. Includes basic usage of evaluators to run evaluations.

LangChain Skills

⚠️ — This project is in early development. APIs and skill content may change.

Agent skills for building agents with LangChain, LangGraph, and Deep Agents.

Want your agent to self-improve? Use langsmith-skills to observe, evaluate, and iterate on your LLM applications.

Supported Coding Agents

These skills can be installed via npx skills for any agent that supports the Agent Skills specification, including Claude Code, Cursor, Windsurf, and more.

Installation

Quick Install

Using npx skills:

Local (current project):

npx skills add langchain-ai/langchain-skills --skill '*' --yes

Global (all projects):

npx skills add langchain-ai/langchain-skills --skill '*' --yes --global

To link skills to a specific agent (e.g. Claude Code):

npx skills add langchain-ai/langchain-skills --agent claude-code --skill '*' --yes --global

Claude Code Plugin

Install directly as a Claude Code plugin:

/plugin marketplace add langchain-ai/langchain-skills
/plugin install langchain-skills@langchain-skills

Install Script (Claude Code & Deep Agents CLI only)

Alternatively, clone the repo and use the install script:

# Install for Claude Code in current directory (default)
./install.sh

# Install for Claude Code in a specific project directory
./install.sh ~/my-project

# Install for Claude Code globally
./install.sh --global

# Install for Deep Agents CLI in a specific project directory
./install.sh --deepagents ~/my-project

# Install for Deep Agents CLI globally (includes agent persona)
./install.sh --deepagents --global
Flag / ArgumentDescription
DIRECTORYTarget project directory (default: current directory, ignored with --global)
--claudeInstall for Claude Code (default)
--deepagentsInstall for Deep Agents CLI
--global, -gInstall globally instead of current directory
--force, -fOverwrite skills with same names as this package
--yes, -ySkip confirmation prompts

Usage

After installation, set your API keys:

export OPENAI_API_KEY=<your-key>      # For OpenAI models
export ANTHROPIC_API_KEY=<your-key>   # For Anthropic models

Then run your coding agent from the directory where you installed (for local installs) or from anywhere (for global installs).

Available Skills (13)

Getting Started

  • framework-selection - Framework comparison reference (LangChain vs LangGraph vs Deep Agents)
  • langchain-dependencies - Full package version and dependency management reference (Python + TypeScript)

Deep Agents

  • deep-agents-core - Agent architecture, harness setup, and SKILL.md format
  • deep-agents-memory - Memory, persistence, filesystem middleware
  • deep-agents-orchestration - Subagents, task planning, human-in-the-loop
  • managed-deep-agents - Managed Deep Agents private-preview REST API (/v1/deepagents): create agents, register MCP servers, create threads, stream runs

LangChain

  • langchain-fundamentals - Agents with create_agent, tools, structured output, middleware basics
  • langchain-middleware - Human-in-the-loop approval, custom middleware, Command resume patterns
  • langchain-rag - RAG pipeline (document loaders, embeddings, vector stores)

LangGraph

  • langgraph-fundamentals - StateGraph, nodes, edges, state reducers
  • langgraph-persistence - Checkpointers, thread_id, cross-thread memory
  • langgraph-cli - CLI lifecycle: scaffold, dev, build, deploy, langgraph.json config
  • langgraph-human-in-the-loop - Interrupts, human review, approval workflows
Skills Info
Original Name:langsmith-evaluatorAuthor:langchain