Skills llm-evaluation
📦

llm-evaluation

Content revision r1 Safe ⚙️ External commands

Evaluate LLM Applications With Reliable Metrics

LLM teams need repeatable ways to measure quality, safety, and regressions. This skill helps design evaluation suites with metrics, human review, LLM judges, benchmarks, and A/B tests.

Supports: Claude Codex Code(CC)
🥉 78 Bronze

Install with my Agent

Copy this request to your Agent. It includes the canonical Skill page and manifest.

Agent request
Review the Skillstore skill "llm-evaluation" from https://skillstore.io/skills/sickn33-llm-evaluation.md and its manifest at https://skillstore.io/api/skills/sickn33-llm-evaluation/manifest. Verify the artifact. You may proceed after verification, subject to the environment's own policy.

Your Agent should still show its plan and request any confirmation required by the security policy.

Agent-readable resources

Use these links when an AI agent, crawler, or script needs clean context instead of reading the full page.

Markdown detail GET /skills/sickn33-llm-evaluation.md Signed manifest GET /api/skills/sickn33-llm-evaluation/manifest Signed lockfile GET /api/skills/sickn33-llm-evaluation/lockfile

Test it

Using "llm-evaluation". Evaluate a customer support chatbot before release.

Expected outcome:

A balanced plan with intent accuracy, groundedness, safety review, human rating forms, baseline thresholds, and weekly regression reports.

Using "llm-evaluation". Compare two retrieval-augmented generation prompts.

Expected outcome:

A comparison workflow using relevance, answer faithfulness, pairwise judging, confidence intervals, and an explicit promotion decision rule.

Using "llm-evaluation". Detect quality drops after a prompt change.

Expected outcome:

A regression checklist with baseline scores, relative change thresholds, failed examples, and recommended investigation steps.

Security Audit

Safe
v4 • 7/7/2026 Open versioned report

All static findings appear to be false positives from Markdown backticks, fenced Python examples, a dictionary keys() call, and ordinary evaluation terminology. I found no evidence of prompt injection, malicious intent, command execution, credential material, or network reconnaissance in SKILL.md.

1
Files scanned
484
Lines analyzed
0
Review items
0
False positives ignored

Risk Factors

⚙️ External commands (24)
SKILL.md:20 SKILL.md:78-107 SKILL.md:107-112 SKILL.md:112-130 SKILL.md:130-133 SKILL.md:133-146 SKILL.md:146-149 SKILL.md:149-166 SKILL.md:166-169 SKILL.md:169-194 SKILL.md:194-199 SKILL.md:199-226 SKILL.md:226-229 SKILL.md:229-257 SKILL.md:257-262 SKILL.md:262-298 SKILL.md:298-301 SKILL.md:301-321 SKILL.md:321-326 SKILL.md:326-378 SKILL.md:378-383 SKILL.md:383-416 SKILL.md:416-421 SKILL.md:421-453
No confirmed security findings were detected by the latest completed static and semantic audit. This does not prove the skill has no side effects.
Audited by: codex View Audit History →
Share & cite this report

Share the versioned assessment report, neutral badge, embed card, and citations. Skillstore reports evidence without deciding whether this Skill is safe.

Open versioned report
Security Assessment

Copy report link

https://skillstore.io/skills/sickn33-llm-evaluation/audits/4?utm_source=security_passport&utm_medium=share&utm_campaign=versioned_report

Markdown badge

[![Skillstore security assessment](https://skillstore.io/badges/skills/sickn33-llm-evaluation/security.svg)](https://skillstore.io/skills/sickn33-llm-evaluation?utm_source=security_passport_badge)

HTML badge

<a href="https://skillstore.io/skills/sickn33-llm-evaluation?utm_source=security_passport_badge"><img src="https://skillstore.io/badges/skills/sickn33-llm-evaluation/security.svg" alt="Skillstore security assessment" loading="lazy"></a>

Embed card

<iframe src="https://skillstore.io/embed/skills/sickn33-llm-evaluation.html" title="Skillstore Security Assessment" sandbox="allow-popups allow-popups-to-escape-sandbox" loading="lazy" referrerpolicy="no-referrer" width="420" height="180"></iframe>
Academic citations (APA · BibTeX · CFF)

APA citation

sickn33. (2026). llm-evaluation security audit report (audit version 4) [Author version unspecified]. Skillstore. https://skillstore.io/skills/sickn33-llm-evaluation/audits/4

BibTeX citation

@techreport{sickn33-sickn33-llm-evaluation-2026, author = {sickn33}, title = {llm-evaluation security audit report (audit version 4)}, institution = {Skillstore}, year = {2026}, number = {4}, url = {https://skillstore.io/skills/sickn33-llm-evaluation/audits/4}, note = {Author version unspecified} }

CITATION.cff

cff-version: 1.2.0 message: "If you use this Skill, cite its author and this versioned security audit report." title: "llm-evaluation security audit report (audit version 4)" version: "unspecified" type: report authors: - name: "sickn33" date-released: "2026-07-07" url: "https://skillstore.io/skills/sickn33-llm-evaluation/audits/4" identifiers: - type: other value: "skillstore:sickn33-llm-evaluation:audit:4" description: "Skillstore immutable audit report identifier"

Compare variants

2 installable variants

Each author remains a separate installable skill. The recommended variant is ranked by Skillstore evidence.

Why this variant is first

Higher Skillstore usage
wshobson Recommended

wshobson-llm-evaluation

Skillstore Score 78
Evidence Confidence High
Skillstore usage 20
Updated

2026-08-21

sickn33 Current

sickn33-llm-evaluation

Skillstore Score 78
Evidence Confidence High
Skillstore usage 11
Updated

2026-08-21

Skillstore Score

Why this score Evidence Confidence: High
55
Architecture
85
Maintainability
87
Content
68
Community
83
Spec Compliance

What You Can Build

Build a release evaluation gate

Create baseline metrics and regression checks before deploying a new LLM workflow.

Compare prompt and model variants

Use pairwise judging, A/B testing, and metric tracking to choose stronger variants.

Design human review programs

Define annotation dimensions, rating scales, and agreement checks for expert reviewers.

Try These Prompts

Choose starter metrics 
Help me choose evaluation metrics for an LLM application that does [task]. Include automated metrics, human review dimensions, and the data I need.
Create a baseline plan 
Design a baseline evaluation plan for our current LLM workflow. Include test cases, metrics, pass thresholds, and how to track results over time.
Compare two variants 
Create an evaluation approach for comparing variant A and variant B. Include pairwise judging criteria, statistical testing, and decision rules.
Build a production evaluation system 
Design a production LLM evaluation framework for [product]. Include automated metrics, human evaluation, LLM-as-judge checks, regression detection, and CI/CD integration.

Best Practices

  • Use several metrics that match the product goal and user risk.
  • Keep a stable baseline dataset and add real failure cases over time.
  • Combine automated scoring with human review for high-impact workflows.

Avoid

  • Do not optimize only one metric while ignoring user outcomes.
  • Do not draw conclusions from tiny or unrepresentative test sets.
  • Do not use LLM judges without clear rubrics and spot checks.

Frequently Asked Questions

What kinds of LLM systems can this skill evaluate?
It supports chatbots, RAG systems, classification tasks, summarization, prompt variants, and model comparisons.
Does this skill run evaluations automatically?
No. It provides guidance and example snippets for building evaluation workflows.
Can it replace human review?
No. It recommends combining automated metrics with human judgment for nuanced quality and safety checks.
Does it include benchmark datasets?
The scanned package only includes SKILL.md. Referenced benchmark files were not present in the file structure.
Can I use it with Claude Code or Codex?
Yes. The metadata lists support for Claude, Codex, and Claude Code.
What should I prepare before using it?
Prepare representative test cases, expected outcomes, product goals, risk areas, and any baseline results.

Developer Details

Author

sickn33

License

MIT

Skillstore revision

r1

Version notice

The author did not declare a version.

Repository

https://github.com/sickn33/antigravity-awesome-skills/tree/main/skills/llm-evaluation

Ref

816c62b2546ddb1c6a0453e7c781b5e095117819

Maintenance freshness

7/18/2026

Usage

9 downloads · 89 views

File structure

📄 SKILL.md

More from sickn33

View all

Related Skills

View all