Skills llm-evaluation Audit History
📦

Audit History

llm-evaluation - 7 audits

Version comparison

Capability and finding changes across audited versions, newest first.

VersionDateResultReview itemsChange vs previous
v7 LatestJul 7, 2026, 08:07 AM No confirmed findings0No capability change
v6 Jul 7, 2026, 08:07 AM No confirmed findings0External commands Network access
v5 Jul 1, 2026, 12:28 AM 1 confirmed0Network access External commands
v4 Jan 17, 2026, 08:09 AM No confirmed findings0No capability change
v3 Jan 17, 2026, 08:09 AM No confirmed findings0External commands
v2 Jan 4, 2026, 04:38 PM No confirmed findings0No capability change
v1 Jan 4, 2026, 04:38 PM No confirmed findings0Baseline

Jul 7, 2026, 08:07 AM

The static external-command findings are markdown Python code fences in SKILL.md, not executable Ruby or shell backtick use. The certificate/key and network reconnaissance findings are false positives from ordinary evaluation text and resource names. No prompt injection attempt, data exfiltration intent, or malicious behavior was found.

1
Files scanned
472
Lines analyzed
1
Review items
0
False positives ignored
Audited by: codex

Jul 7, 2026, 08:07 AM

The static external-command findings are markdown Python code fences in SKILL.md, not executable Ruby or shell backtick use. The certificate/key and network reconnaissance findings are false positives from ordinary evaluation text and resource names. No prompt injection attempt, data exfiltration intent, or malicious behavior was found.

1
Files scanned
472
Lines analyzed
1
Review items
0
False positives ignored
Audited by: codex

Jul 1, 2026, 12:28 AM

Static analysis reported many high and medium findings, but review shows they are false positives from markdown code fences, metric labels, form scale labels, and dictionary key iteration. The skill is documentation-only, with one low-risk caution: example LLM-as-judge code would send evaluation inputs and outputs to an external API if users copy it into their own application.

1
Files scanned
472
Lines analyzed
2
Review items
4
False positives ignored

Confirmed security concerns (1)

Low
External API Submission in Example Code
The LLM-as-judge examples call an external OpenAI chat completion API with evaluation prompts that include user questions and model responses. This is a legitimate evaluation pattern, but users should avoid sending confidential test data without approval.
The code examples clearly submit prompt content to an external API, but the skill file is documentation and does not execute this code by itself.
Static false positives ignored (4)

These static matches were dismissed by semantic review or matched schema-only tokens, so they are shown for transparency but do not drive the quality score.

Low
False Positive: Markdown Code Fences Misidentified as Shell Execution
The external command findings point to markdown Python code block delimiters and example Python snippets. No Ruby backtick execution or shell command execution is present at these locations.
Each location is a markdown code fence boundary or documentation example, not executable Ruby or shell syntax.
Low
False Positive: Weak Cryptography Patterns Not Present
The weak cryptography findings match ordinary prose and human rating scale text, not cryptographic code. No MD5, SHA1, DES, RC4, or custom cryptographic implementation was found in the reviewed context.
The cited lines are the skill description and annotation scale descriptions, so the static pattern does not correspond to cryptographic behavior.
Low
False Positive: Sensitive Key File Pattern Not Present
The certificate or key file finding maps to a Python dictionary iteration over baseline metric keys. It does not reference certificate material, private keys, or credential files.
The cited text is baseline.keys() inside a regression detector example, with no filesystem path or secret material.
Low
False Positive: Network Reconnaissance Pattern Not Present
The network reconnaissance findings map to the word Accuracy in an evaluation dimension and a resource filename for an evaluation framework. Neither line performs scanning, probing, or network discovery.
The cited lines are documentation labels, not commands or API calls that enumerate hosts or services.

Risk Factors

Audited by: codex

Jan 17, 2026, 08:09 AM

This skill contains only static documentation (SKILL.md) with no executable files. All static findings are false positives: markdown code block backticks were misidentified as Ruby/shell command execution, and JSON metadata fields were misclassified as cryptographic issues. The skill provides evaluation guidance only with no data access, network activity, or command execution capability.

2
Files scanned
649
Lines analyzed
1
Review items
0
False positives ignored
Audited by: claude

Jan 17, 2026, 08:09 AM

This skill contains only static documentation (SKILL.md) with no executable files. All static findings are false positives: markdown code block backticks were misidentified as Ruby/shell command execution, and JSON metadata fields were misclassified as cryptographic issues. The skill provides evaluation guidance only with no data access, network activity, or command execution capability.

2
Files scanned
649
Lines analyzed
1
Review items
0
False positives ignored
Audited by: claude

Jan 4, 2026, 04:38 PM

Pure documentation skill containing no executable files or tooling. All content is static markdown with illustrative code examples. No data access, network activity, file writes, or command execution is possible. The skill provides evaluation guidance only.

4
Files scanned
677
Lines analyzed
0
Review items
0
False positives ignored
No confirmed security findings were recorded for this completed audit.
Audited by: claude

Jan 4, 2026, 04:38 PM

Pure documentation skill containing no executable files or tooling. All content is static markdown with illustrative code examples. No data access, network activity, file writes, or command execution is possible. The skill provides evaluation guidance only.

4
Files scanned
677
Lines analyzed
0
Review items
0
False positives ignored
No confirmed security findings were recorded for this completed audit.
Audited by: claude