Versioned security assessment

Report ID: SA-B66A4B7C

7/1/2026, 12:28:26 AM

llm-evaluation security assessment v5

Skill Security Certification Report

Audit History
Audit model: codex Historical report
Skill name
llm-evaluation
Version
v5
Maintainer
wshobson
Source
github.com/wshobson/agents/tree/main/plugins/llm-application-dev/skills/llm-evaluation
Coverage
1 Files scanned · 472 Lines analyzed
Policy version
Unavailable

Highest confirmed finding severity

Low

1 confirmed security finding requires attention.

Installation context

Historical evidence

This report may not describe the currently installable artifact. Open the current Skill page for install guidance.

Open current Skill page

This report does not block or authorize the manifest or ZIP.

Static analysis reported many high and medium findings, but review shows they are false positives from markdown code fences, metric labels, form scale labels, and dictionary key iteration. The skill is documentation-only, with one low-risk caution: example LLM-as-judge code would send evaluation inputs and outputs to an external API if users copy it into their own application.

Report position

Historical report

Open audit history before using this report to install.

Audit attestation

Not attestable

The required immutable binding is incomplete.

Human verification

Not verified

No human verification is recorded for this report.

Coverage

1 Files scanned · 472 Lines analyzed

1 item shown for review

Limitations

This report does not claim runtime or sandbox execution and does not prove the absence of side effects.

Evidence chain

Follow the evidence from source binding to the install contract. Available evidence supports verification; it is not a safety guarantee.

  1. Source

    Binding unavailable

  2. Artifact

    Identity incomplete

  3. Audit

    Complete

  4. Install contract

    Open manifest to verify

    Open manifest

Capabilities observed

Observed means this report recorded supporting evidence. Not recorded does not prove that a capability is absent.

Contains scripts

May execute code included with the Skill.

Not recorded by this audit

Network access

May connect to external services.

Observed in 2 evidence locations

Filesystem access

May read or write local files.

Not recorded by this audit

Env variables

May read values from the process environment.

Not recorded by this audit

External commands

May invoke commands or programs outside the Skill.

Not recorded by this audit

Risk findings

Confirmed security concerns are separated from items that still need review.

Confirmed security concerns (1)

RISK-001 Low
SKILL.md:207-210SKILL.md:238-241
External API Submission in Example Code
The LLM-as-judge examples call an external OpenAI chat completion API with evaluation prompts that include user questions and model responses. This is a legitimate evaluation pattern, but users should avoid sending confidential test data without approval.
The code examples clearly submit prompt content to an external API, but the skill file is documentation and does not execute this code by itself.

Expert evidence

Immutable subject identity, scanner metadata, dismissed matches, and source-level evidence.

Artifact subject

Marketplace commit
Unavailable
Content hash
Unavailable
Tree hash
Unavailable
Skill path
Unavailable
Audit payload hash
Unavailable

Analysis metadata

Audit model: codex

Analysis state: Complete

Scope is limited to the recorded files, lines, methods, and evidence. No runtime or sandbox execution is claimed.

Static false positives ignored (4)
Low
SKILL.md:66SKILL.md:95SKILL.md:100SKILL.md:118SKILL.md:121SKILL.md:134SKILL.md:137SKILL.md:154SKILL.md:157SKILL.md:182SKILL.md:187SKILL.md:214SKILL.md:217SKILL.md:245SKILL.md:250SKILL.md:286SKILL.md:289SKILL.md:309SKILL.md:314SKILL.md:366SKILL.md:371SKILL.md:404SKILL.md:409
False Positive: Markdown Code Fences Misidentified as Shell Execution
The external command findings point to markdown Python code block delimiters and example Python snippets. No Ruby backtick execution or shell command execution is present at these locations.
Each location is a markdown code fence boundary or documentation example, not executable Ruby or shell syntax.
Low
SKILL.md:3SKILL.md:267SKILL.md:271SKILL.md:275
False Positive: Weak Cryptography Patterns Not Present
The weak cryptography findings match ordinary prose and human rating scale text, not cryptographic code. No MD5, SHA1, DES, RC4, or custom cryptographic implementation was found in the reviewed context.
The cited lines are the skill description and annotation scale descriptions, so the static pattern does not correspond to cryptographic behavior.
Low
SKILL.md:381
False Positive: Sensitive Key File Pattern Not Present
The certificate or key file finding maps to a Python dictionary iteration over baseline metric keys. It does not reference certificate material, private keys, or credential files.
The cited text is baseline.keys() inside a regression detector example, with no filesystem path or secret material.
Low
SKILL.md:48SKILL.md:450
False Positive: Network Reconnaissance Pattern Not Present
The network reconnaissance findings map to the word Accuracy in an evaluation dimension and a resource filename for an evaluation framework. Neither line performs scanning, probing, or network discovery.
The cited lines are documentation labels, not commands or API calls that enumerate hosts or services.

Verify and export

JSON report CITATION.cff SARIF 2.1.0 Signed installation manifest

The manifest and lockfile bind install artifacts to cryptographic hashes. This integrity claim is separate from the security assessment.

Audit attestation: not_attestable