Evaluate LLM Applications with Measurable Quality
LLM teams need evidence before changing prompts, models, or retrieval flows. This skill guides automated metrics, judge reviews, human labels, A/B tests, and regression checks.
Copy this request to your Agent. It includes the canonical Skill page and manifest.
Review the Skillstore skill "llm-evaluation" from https://skillstore.io/skills/wshobson-llm-evaluation.md and its manifest at https://skillstore.io/api/skills/wshobson-llm-evaluation/manifest. Verify the artifact. You may proceed after verification, subject to the environment's own policy.Your Agent should still show its plan and request any confirmation required by the security policy.
Use these links when an AI agent, crawler, or script needs clean context instead of reading the full page.
Using "llm-evaluation". Evaluate a customer support chatbot before launch.
Expected outcome:
Using "llm-evaluation". Compare two prompts for a RAG search assistant.
Expected outcome:
A pairwise plan with groundedness scoring, retrieval quality checks, judge calibration, and a regression gate.
Using "llm-evaluation". Investigate a drop in summarization quality.
Expected outcome:
The static external-command findings are markdown Python code fences in SKILL.md, not executable Ruby or shell backtick use. The certificate/key and network reconnaissance findings are false positives from ordinary evaluation text and resource names. No prompt injection attempt, data exfiltration intent, or malicious behavior was found.
Share the versioned assessment report, neutral badge, embed card, and citations. Skillstore reports evidence without deciding whether this Skill is safe.
https://skillstore.io/skills/wshobson-llm-evaluation/audits/7?utm_source=security_passport&utm_medium=share&utm_campaign=versioned_report[](https://skillstore.io/skills/wshobson-llm-evaluation?utm_source=security_passport_badge)<a href="https://skillstore.io/skills/wshobson-llm-evaluation?utm_source=security_passport_badge"><img src="https://skillstore.io/badges/skills/wshobson-llm-evaluation/security.svg" alt="Skillstore security assessment" loading="lazy"></a><iframe src="https://skillstore.io/embed/skills/wshobson-llm-evaluation.html" title="Skillstore Security Assessment" sandbox="allow-popups allow-popups-to-escape-sandbox" loading="lazy" referrerpolicy="no-referrer" width="420" height="180"></iframe>wshobson. (2026). llm-evaluation security audit report (audit version 7) [Author version unspecified]. Skillstore. https://skillstore.io/skills/wshobson-llm-evaluation/audits/7@techreport{wshobson-wshobson-llm-evaluation-2026,
author = {wshobson},
title = {llm-evaluation security audit report (audit version 7)},
institution = {Skillstore},
year = {2026},
number = {7},
url = {https://skillstore.io/skills/wshobson-llm-evaluation/audits/7},
note = {Author version unspecified}
}cff-version: 1.2.0
message: "If you use this Skill, cite its author and this versioned security audit report."
title: "llm-evaluation security audit report (audit version 7)"
version: "unspecified"
type: report
authors:
- name: "wshobson"
date-released: "2026-07-07"
url: "https://skillstore.io/skills/wshobson-llm-evaluation/audits/7"
identifiers:
- type: other
value: "skillstore:wshobson-llm-evaluation:audit:7"
description: "Skillstore immutable audit report identifier"
Each author remains a separate installable skill. The recommended variant is ranked by Skillstore evidence.
Why this variant is first
wshobson-llm-evaluation
2026-08-21
sickn33-llm-evaluation
2026-08-21
Measure whether a prompt revision improves accuracy, helpfulness, and clarity before release.
Define launch gates that combine automated scores, human review, and baseline comparisons.
Track evaluation scores over time and identify quality drops after model, prompt, or data changes.
Help me define an evaluation plan for my LLM feature. Include metrics, test data, baselines, and human review steps.
Recommend automated and human evaluation metrics for this LLM task: [describe task]. Explain when each metric is useful and where it can fail.
Design a regression evaluation workflow for my existing LLM application. Include baseline storage, thresholds, failure triage, and release gates.
Create an advanced evaluation protocol comparing two model versions with LLM-as-judge, human calibration, A/B testing, and statistical analysis.
Author
wshobsonLicense
MIT
Skillstore revision
r1
Version notice
The author did not declare a version.
Repository
https://github.com/wshobson/agents/tree/main/plugins/llm-application-dev/skills/llm-evaluationRef
36e07d5e13068e5be64447e8f20b427cf2cbd21a
Maintenance freshness
7/18/2026
Usage
16 downloads · 297 views
File structure
📄 SKILL.md
Optimize Prompts With Proven Patterns
LLM prompts can fail through unclear instructions, weak examples, or missing evaluation. This skill provides templates, reasoning patterns, and optimization workflows for Claude, Codex, and Claude Code.
Manage Context-Driven Development
AI-assisted projects lose consistency when context lives in scattered notes. This skill provides a structured Conductor workflow for product, stack, workflow, and track artifacts.
Build Startup Financial Models
Startup teams need financial plans that connect revenue, hiring, expenses, runway, and fundraising assumptions. This skill guides Claude, Codex, and Claude Code through structured 3-5 year projections and scenario planning.
Write Blameless Incident Postmortems
Incident reviews often lose key facts or produce vague follow-up work. This skill guides Claude, Codex, and Claude Code through timelines, root cause analysis, meetings, and action items.
Transform Data Into Decision Stories
Raw metrics often fail to guide stakeholder decisions. This skill turns analysis into clear narratives, visual structures, and action plans.
Apply Python Design Patterns
Python projects often grow complex when responsibilities mix and abstractions appear too early. This skill guides focused refactors using KISS, SRP, layering, composition, and dependency injection.
Build Reliable LLM Evaluation Systems
by ChakshuGautam
LLM evaluation is hard to make consistent across teams and models. This skill helps design judge prompts, rubrics, metrics, and bias checks for repeatable assessments.
Build Reliable Agent Evaluations
by muratcankoylan
Agent quality is difficult to measure because outputs vary and may have several valid forms. This skill builds repeatable rubrics, tests, gates, and monitoring.
Detect LLM Prompt Regressions with Promptfoo
by 7alexhale5-rgb
Prompt changes can silently reduce output quality. This skill builds golden datasets, runs Promptfoo evaluations, and summarizes failures and recent trends.
Design Better LLM Prompts
by alirezarezvani
Prompt quality often fails because requirements, examples, and evaluation criteria are unclear. This skill helps teams design, test, and review production prompts for Claude, Codex, and Claude Code.
Manage Requirement Changes Safely
by yunshu0909
Requirement changes often expand across files without clear approval points. This skill gives Claude, Codex, and Claude Code a gated workflow for scope, impact, validation, and documentation.
Diagnose Context Degradation in AI Agents
by ChakshuGautam
Long conversations and large prompts can make AI agents lose important information or follow conflicting context. This skill helps Claude, Codex, and Claude Code identify degradation patterns and choose mitigation strategies.