agent-evaluation
Evaluate LLM Agents with Reliable Test Suites
Single-run checks hide stochastic failures and production regressions. This skill designs repeatable evaluations with behavioral, adversarial, statistical, and leakage tests.
Stop for confirmation before installing.
Review the plan and obtain explicit user consent before changing files.
Install with my Agent
Copy this request to your Agent. It includes the canonical Skill page and manifest.
Review the Skillstore skill "agent-evaluation" from https://skillstore.io/skills/sickn33-agent-evaluation.md and its manifest at https://skillstore.io/api/skills/sickn33-agent-evaluation/manifest. Verify the artifact. Stop and obtain explicit user consent before installing or changing files.Your Agent should still show its plan and request any confirmation required by the security policy.
Agent-readable resources
Use these links when an AI agent, crawler, or script needs clean context instead of reading the full page.
Test it
Using "agent-evaluation". Evaluate a customer support agent that processes refunds.
Expected outcome:
Run 20 trials per case, enforce refund-policy contracts, test injection resistance, compare latency, and block deployment on critical violations.
Using "agent-evaluation". Investigate a release that passes benchmarks but fails in production.
Expected outcome:
Compare anonymized production samples, adversarial variants, edge cases, and load latency against the benchmark baseline before adjusting the release gate.
Using "agent-evaluation". Check whether benchmark data leaked into an agent.
Expected outcome:
Search for training matches, prompt overlap, memorized completions, and RAG retrieval of expected answers. Replace affected tests before reporting results.
Security Audit
High RiskAll 16 static findings are false positives caused by TypeScript template literals, Markdown fences, or defensive adversarial test data. However, the adversarial runner counts every processing exception as a pass, which can conceal crashes and inflate reported safety.
Confirmed security concerns (1)
Risk Factors
⚙️ External commands (13)
📁 Filesystem access (1)
Share & cite this report
Share the versioned assessment report, neutral badge, embed card, and citations. Skillstore reports evidence without deciding whether this Skill is safe.
Copy report link
https://skillstore.io/skills/sickn33-agent-evaluation/audits/5?utm_source=security_passport&utm_medium=share&utm_campaign=versioned_reportMarkdown badge
[](https://skillstore.io/skills/sickn33-agent-evaluation?utm_source=security_passport_badge)HTML badge
<a href="https://skillstore.io/skills/sickn33-agent-evaluation?utm_source=security_passport_badge"><img src="https://skillstore.io/badges/skills/sickn33-agent-evaluation/security.svg" alt="Skillstore security assessment" loading="lazy"></a>Embed card
<iframe src="https://skillstore.io/embed/skills/sickn33-agent-evaluation.html" title="Skillstore Security Assessment" sandbox="allow-popups allow-popups-to-escape-sandbox" loading="lazy" referrerpolicy="no-referrer" width="420" height="180"></iframe>Academic citations (APA · BibTeX · CFF)
APA citation
sickn33. (2026). agent-evaluation security audit report (audit version 5) [Author version unspecified]. Skillstore. https://skillstore.io/skills/sickn33-agent-evaluation/audits/5BibTeX citation
@techreport{sickn33-sickn33-agent-evaluation-2026,
author = {sickn33},
title = {agent-evaluation security audit report (audit version 5)},
institution = {Skillstore},
year = {2026},
number = {5},
url = {https://skillstore.io/skills/sickn33-agent-evaluation/audits/5},
note = {Author version unspecified}
}CITATION.cff
cff-version: 1.2.0
message: "If you use this Skill, cite its author and this versioned security audit report."
title: "agent-evaluation security audit report (audit version 5)"
version: "unspecified"
type: report
authors:
- name: "sickn33"
date-released: "2026-07-23"
url: "https://skillstore.io/skills/sickn33-agent-evaluation/audits/5"
identifiers:
- type: other
value: "skillstore:sickn33-agent-evaluation:audit:5"
description: "Skillstore immutable audit report identifier"
Skillstore Score
Why this score Evidence Confidence: MediumWhat You Can Build
Build Agent Quality Gates
Create repeated behavioral and regression tests that produce measurable release criteria.
Assess Model Reliability
Measure stochastic performance, confidence intervals, consistency, and benchmark leakage across agent versions.
Monitor Production Agents
Compare benchmarks with real traffic, edge cases, adversarial variants, and load behavior.
Try These Prompts
Create an evaluation plan for my [agent type]. Include goals, five representative tests, pass criteria, and repeated-run guidance.
Define a behavioral contract for [agent]. List required, prohibited, and conditional behaviors with detectors and severity levels.
Design adversarial tests for [agent and tools]. Cover prompt injection, role confusion, boundary conditions, output manipulation, and tool misuse.
Build a production-readiness evaluation for [agent]. Compare baselines, confidence intervals, regressions, load behavior, leakage, and deployment gates.
Best Practices
- Run stochastic tests multiple times and report distributions instead of one result.
- Use production-derived samples only after anonymization and access review.
- Combine correctness, safety, reliability, efficiency, and human preference metrics.
Avoid
- Do not treat agent errors or timeouts as successful adversarial outcomes.
- Do not release an agent based only on aggregate benchmark scores.
- Do not reuse evaluation examples in training data, prompts, or retrieval sources.