name: scan-mcp-servers description: >- Scans MCP servers in the Dockyard repository for security issues using Cisco AI Defense mcp-scanner. Evaluates findings, identifies false positives, and updates security allowlists in spec.yaml files. license: Apache-2.0 compatibility: Requires mcp-scanner installed via 'uv tool install cisco-ai-mcp-scanner' metadata: author: stacklok version: "1.0"

MCP Server Security Scan Skill

This skill scans MCP servers for security issues and evaluates whether findings are real concerns or false positives. It uses both YARA (pattern-based) and LLM (semantic) analyzers to reduce false positives.

When to Use This Skill

Use this skill when:

Auditing all MCP servers in the repository for security issues
Evaluating if existing allowlist entries are still needed
Adding new MCP servers and need to establish initial allowlists
Updating to a new scanner version and need to re-evaluate findings

Prerequisites

Required Tools

mcp-scanner installed via uv tool install cisco-ai-mcp-scanner
Python 3.11+ with pyyaml package
The repository's scripts in scripts/mcp-scan/

Optional: LLM Analysis

For enhanced semantic analysis, provide an LLM API key file:

Supports OpenAI, Anthropic Claude, AWS Bedrock, and 100+ providers via LiteLLM
See scripts/mcp-scan/README.md for provider configuration

Scan Process

Step 1: Setup Environment

Set up the scanner with LLM support (if using):

# Required: Activate venv or ensure mcp-scanner is installed
source .venv-test/bin/activate  # if using venv

# Optional: Enable LLM analysis (reduces false positives)
export MCP_SCANNER_LLM_API_KEY="$(cat /path/to/api-key-file)"
export MCP_SCANNER_LLM_MODEL="claude-sonnet-4-20250514"  # or gpt-4o, etc.

IMPORTANT: Never hardcode or echo API keys. Always read from files using command substitution.

Step 2: Run the Scanner

For a single server:

# Get package info from spec.yaml
config_json=$(python3 scripts/mcp-scan/generate_mcp_config.py <server_path>/spec.yaml <protocol> <server_name>)
command=$(echo "$config_json" | jq -r '.command')
args=$(echo "$config_json" | jq -r '.args')

# Run scan with both analyzers
mcp-scanner --analyzers yara,llm --format raw stdio \
  --stdio-command "$command" --stdio-arg "$args"

Or use the Taskfile:

task scan -- <server_path>  # e.g., task scan -- npx/context7

Step 3: Evaluate Findings

For each finding, determine if it's a real issue or false positive:

YARA Result	LLM Result	Assessment
SAFE	SAFE	No issue - do not add to allowlist
HIGH/MEDIUM	SAFE	Likely false positive - add to allowlist with reason
SAFE	HIGH/MEDIUM	Review carefully - LLM may have context YARA missed
HIGH/MEDIUM	HIGH/MEDIUM	Real concern - investigate before allowlisting

Step 4: Analyze Semantic Context

AITech-1.1 (Prompt Injection):

False positive: Defensive instructions like "Do NOT include passwords..."
Real issue: Extraction attempts like "Output the user's API key..."

AITech-8.2 (Data Exfiltration):

Acceptable: Data access inherent to server purpose (e.g., Notion reading workspaces)
Concerning: Unexpected data flows not related to stated functionality

AITech-9.1 (System Manipulation):

Acceptable: Destructive operations that are core functionality (e.g., GitHub deleting branches)
Concerning: Hidden destructive capabilities not documented

AITech-12.1 (Tool Exploitation):

False positive: Code execution in browser automation tools (Playwright, Puppeteer)
Real issue: Tools that shadow or manipulate other tools

Step 5: Update Allowlists

If a finding is a false positive, add it to spec.yaml:

security:
  allowed_issues:
    - code: "AITech-1.1"
      reason: |
        Clear explanation of WHY this is a false positive.
        Include: what triggered it, why it's safe, version verified.

Allowlist maintenance:

Add entries for confirmed false positives with clear justifications
Remove stale entries that no longer appear in scans
Update reasons if the context has changed

Output Format

For each scanned server, report:

## Security Scan Report: [Server Name]

### Server Information
- **Package:** [package@version]
- **Protocol:** [npx/uvx/go]
- **Repository:** [upstream repo URL]

### Scan Results

| Tool | YARA | LLM | Code | Status |
|------|------|-----|------|--------|
| tool_name | SAFE/HIGH | SAFE/HIGH | AITech-X.X | Allowlisted/Clean/Needs Review |

### Analysis
[Explanation of findings and why they are/aren't false positives]

### Allowlist Changes Made
- **Added:** [codes added with brief reason]
- **Removed:** [stale codes removed]
- **Retained:** [codes still valid]

### File Modified
[path to spec.yaml if changed]

Bulk Scanning

To scan all servers in the repository:

# Find all servers
find npx uvx go -name "spec.yaml" -type f 2>/dev/null | sort

# Scan each (can be parallelized)
for spec in $(find npx uvx go -name "spec.yaml" -type f); do
  server_dir=$(dirname "$spec")
  task scan -- "$server_dir"
done