技能 pinchbench
📦
pinchbench
v1.0.0 内容修订 r1 严重 ⚙️ 外部命令🌐 网络访问📁 文件系统访问⚡ 包含脚本🔑 环境变量
使用 PinchBench 对 OpenClaw Agent 进行基准测试
模型团队需要可重复的证据来衡量 agent 性能。PinchBench 运行结构化的 OpenClaw 任务，并生成可比较的分数，用于分析和排行榜提交。
支持: Claude Codex Code(CC)
请勿自动安装此 Skill。
权威策略要求在执行任何安装操作前由操作人员审查。
使用我的智能体安装
将此请求复制给你的智能体，其中包含权威 Skill 页面和清单。
智能体请求
Review the Skillstore skill "pinchbench" from https://skillstore.io/skills/pinchbench-pinchbench.md and its manifest at https://skillstore.io/api/skills/pinchbench-pinchbench/manifest. Verify the artifact. Do not auto-install. Inspect the skill and report your findings, then wait for an operator or manual installation decision.你的智能体仍应展示其计划，并按安全策略请求必要确认。
Agent 可读资源
当 AI Agent、爬虫或脚本需要干净上下文，而不是读取完整页面时，请使用这些链接。
测试它
正在使用“pinchbench”。 为一个模型运行仅自动化套件，不上传。
预期结果:
运行完成后会生成每个任务的分数、总分、token 用量、成本指标和本地结果文件路径。
正在使用“pinchbench”。 比较两个已保存的基准测试运行。
预期结果:
比较结果会突出显示分数差异、失败任务、执行时间变化和成本效率权衡。
正在使用“pinchbench”。 为邮件分诊设计一个新的基准测试任务。
预期结果:
任务草案包含 prompt、预期行为、评分标准，以及用于可重复评分的自动化检查。
安全审计
严重v5 • 2026/7/9 打开版本化报告
PinchBench is a legitimate benchmark runner, but it is not low risk. Confirmed issues include pipe-to-shell dependency installation, unsandboxed task grading code, external agent execution, token handling, and host metadata upload. Many documentation and task-rubric alerts are false positives, while binary assets remain manual-review blind spots.
38
已扫描文件
7,808
分析行数
29
复核项
0
已忽略误报
已确认安全问题 (6)
严重
Pipe to shell pattern
RUN curl -LsSf https://astral.sh/uv/install.sh | sh
The Docker build pipes a downloaded installer directly into a shell. A compromised network path or upstream installer would execute during image build.
高
System Metadata Uploaded With Results
When uploads are enabled, the payload includes metadata.system and records OS, CPU, Python, memory, and a hostname hash. This goes beyond benchmark scores and may identify the host environment.
The upload payload explicitly includes collect_system_metadata output. The metadata function records platform, CPU, memory, Python, and hostname-derived data.
高
Registration Token Stored In Skill Directory
The registration helper writes the returned token to scripts/.pinchbench/config.json under the skill directory. Tokens in an installed skill tree can be accidentally shared or read by benchmarked agents.
The config path is inside the scripts directory and save_token_config writes the token there. _resolve_token later reads that stored token for uploads.
高
Installed Skills Copied Into Benchmark Workspace
prepare_task_workspace copies every directory under ~/.openclaw/workspace/skills into the benchmark workspace. This may expose private skill content or embedded credentials to the benchmarked model during tasks.
The code iterates over the user skill directory and copies each skill directory into the agent workspace. No allowlist or secret filtering is visible in that block.
中
System reconnaissance
"hostname_hash": hash(platform.node()) % 10000,
The code derives a hostname hash as part of system metadata. Even hashed, it contributes to host fingerprinting in uploaded results.
中
Agent Session Transcripts Deleted Without Confirmation
cleanup_agent_sessions removes stored session transcript files and sessions.json for the target agent. If the target agent ID collides with a real agent, benchmark runs can delete local conversation history.
The deletion code is explicit and is called before benchmark and judge runs. The risk is reduced because agent IDs are normally benchmark-prefixed, but collision is still possible.
能力复核项 (29)
这些是真实的本地能力，对此技能可能是预期行为，因此需要复核，但不会按已确认恶意行为计分。
高
Python subprocess.run
subprocess.run(
The code deletes an OpenClaw agent when its workspace appears stale. It uses argv lists, but it can remove local agent state.
高
Python subprocess.run
result = subprocess.run(
This launches the OpenClaw CLI with benchmark or judge prompts and workspace access. It avoids shell injection, but it triggers external agent execution with local side effects.
高
Python subprocess.run
result = subprocess.run(
This launches the OpenClaw CLI with benchmark or judge prompts and workspace access. It avoids shell injection, but it triggers external agent execution with local side effects.
高
Python subprocess.run
result = subprocess.run(
This launches the OpenClaw CLI with benchmark or judge prompts and workspace access. It avoids shell injection, but it triggers external agent execution with local side effects.
高
Python shutil operations
shutil.rmtree(workspace)
The runner recursively removes the agent workspace before each task. If workspace resolution is wrong, this can delete local files.
高
Python environment access
api_key = os.environ.get("OPENROUTER_API_KEY")
The code reads OPENROUTER_API_KEY from the environment. It is expected for validation, but it handles credential material.
高
Generic API/secret keys
api_key = os.environ.get("OPENROUTER_API_KEY")
The code reads OPENROUTER_API_KEY from the environment. It is expected for validation, but it handles credential material.
高
Generic API/secret keys
"Authorization": f"Bearer {api_key}",
The API key is placed in an Authorization bearer header for an external service. This is legitimate use, but it is sensitive credential transmission.
高
Python exec() function
exec(grading_code, namespace)
The grader executes Python code extracted from task markdown with exec(). Any malicious or modified task file could run arbitrary Python in the benchmark process.
高
Process exec
exec(grading_code, namespace)
The grader executes Python code extracted from task markdown with exec(). Any malicious or modified task file could run arbitrary Python in the benchmark process.
高
Python subprocess.run
result = subprocess.run(
The macOS metadata helper executes sysctl to collect hardware attributes. The arguments are fixed, but the command expands host fingerprinting data.
高
Linux /proc filesystem access
cpuinfo_path = Path("/proc/cpuinfo")
The upload path reads Linux /proc CPU and memory files for system metadata. This collects host fingerprinting information for submission payloads.
高
Linux /proc filesystem access
meminfo_path = Path("/proc/meminfo")
The upload path reads Linux /proc CPU and memory files for system metadata. This collects host fingerprinting information for submission payloads.
高
Python environment access
resolved_official_key = official_key or os.environ.get("PINCHBENCH_OFFICIAL_KEY")
The upload code reads authentication or official submission tokens from environment or local config. These values are sensitive credentials.
高
Python environment access
env_token = os.environ.get("PINCHBENCH_TOKEN")
The upload code reads authentication or official submission tokens from environment or local config. These values are sensitive credentials.
高
Configuration library
return config.get("token")
The upload code reads authentication or official submission tokens from environment or local config. These values are sensitive credentials.
高
Generic API/secret keys
token = data.get("token") or data.get("api_key")
The registration response is parsed for a token or api_key. This is credential material that is later stored and used for uploads.
中
Hardcoded URL
RUN curl -LsSf https://astral.sh/uv/install.sh | sh
The Docker build downloads an installer from an external URL. This is expected setup, but it is a real supply-chain dependency.
中
Temp directory access
run_root = Path("/tmp/pinchbench")
The runner uses a fixed directory under /tmp for benchmark workspaces. This is functional, but not as safe as a private temporary directory.
中
Temp directory access
agent_workspace = Path(f"/tmp/pinchbench/{run_id}/agent_workspace")
The runner uses a fixed directory under /tmp for benchmark workspaces. This is functional, but not as safe as a private temporary directory.
中
Hardcoded URL
specific_endpoint = f"https://openrouter.ai/api/v1/models/{encoded_model_id}"
The model validator contacts OpenRouter endpoints and can use an API key in the same request flow. This is legitimate but still external network activity.
中
Hardcoded URL
catalog_endpoint = "https://openrouter.ai/api/v1/models"
The model validator contacts OpenRouter endpoints and can use an API key in the same request flow. This is legitimate but still external network activity.
中
Temp directory access
workspace = Path(f"/tmp/pinchbench/{run_id}/{task.task_id}")
The fallback workspace uses a predictable path under /tmp. This can expose benchmark files to local path collision or permission risks.
中
Python shutil operations
shutil.rmtree(dest_skill_dir)
The runner deletes and copies skill directories inside the benchmark workspace. This can expose or overwrite copied skill content during task setup.
中
Python shutil operations
shutil.copytree(skill_dir_src, dest_skill_dir)
The runner deletes and copies skill directories inside the benchmark workspace. This can expose or overwrite copied skill content during task setup.
中
Temp directory access
judge_workspace = Path(f"/tmp/pinchbench/judge/{task.task_id}")
The judge uses predictable directories under /tmp for workspaces. This is expected, but not as isolated as a secure temporary directory.
中
Temp directory access
workspace = Path("/tmp/pinchbench/judge/workspace")
The judge uses predictable directories under /tmp for workspaces. This is expected, but not as isolated as a secure temporary directory.
中
Hardcoded URL
DEFAULT_SERVER_URL = "https://api.pinchbench.com"
The upload module posts benchmark results to the PinchBench API by default. This is intended, but it sends local run data off the machine.
中
Python environment access
resolved_server = server_url or os.environ.get("PINCHBENCH_SERVER_URL") or DEFAULT_SERVER_URL
The upload server can be overridden by environment variable. If the environment is compromised, results and auth headers can be sent away from the default API.
风险因素
⚙️ 外部命令 (50)
Dockerfile.benchmark:33 scripts/benchmark.py:277 scripts/lib_agent.py:136 scripts/lib_agent.py:179 scripts/lib_agent.py:221 scripts/lib_agent.py:230 scripts/lib_agent.py:578 scripts/lib_agent.py:611 scripts/lib_agent.py:753 scripts/lib_grading.py:107 scripts/lib_grading.py:216 scripts/lib_grading.py:318 scripts/lib_grading.py:319 scripts/lib_upload.py:317 scripts/lib_upload.py:396 scripts/run.sh:6 SKILL.md:23-37 SKILL.md:37-43 SKILL.md:43-44 SKILL.md:44-45 SKILL.md:45-46 SKILL.md:46-47 SKILL.md:47-48 SKILL.md:48-49 SKILL.md:49-50 SKILL.md:50-51 SKILL.md:51-52 SKILL.md:52-53 SKILL.md:53-54 SKILL.md:54-55 SKILL.md:55-56 SKILL.md:56-57 SKILL.md:57-58 SKILL.md:58-59 SKILL.md:59-60 SKILL.md:60-61 SKILL.md:61-62 SKILL.md:62-63 SKILL.md:63-64 SKILL.md:64-65 SKILL.md:65-71 SKILL.md:71 SKILL.md:72 SKILL.md:73 SKILL.md:74-75 SKILL.md:75-76 SKILL.md:76-77 SKILL.md:77-78 SKILL.md:78-84 SKILL.md:84-90
🌐 网络访问 (13)
📁 文件系统访问 (12)
Dockerfile.benchmark:21 Dockerfile.benchmark:30 scripts/benchmark.py:557 scripts/benchmark.py:562 scripts/lib_agent.py:293 scripts/lib_agent.py:298 scripts/lib_agent.py:340 scripts/lib_agent.py:341 scripts/lib_grading.py:157 scripts/lib_grading.py:298 scripts/lib_upload.py:354 scripts/lib_upload.py:355
🔑 环境变量 (13)
scripts/benchmark.py:270 scripts/lib_agent.py:27 scripts/lib_agent.py:59 scripts/lib_agent.py:59 scripts/lib_agent.py:60 scripts/lib_agent.py:61 scripts/lib_agent.py:67 scripts/lib_upload.py:64 scripts/lib_upload.py:66 scripts/lib_upload.py:135 scripts/lib_upload.py:289 scripts/lib_upload.py:293 scripts/lib_upload.py:159
检测到的模式
Pipe to shell patternSystem reconnaissance
审计者: codex 查看审计历史 →
分享与引用此报告
分享版本化评估报告、中性徽章、嵌入卡片和引用。Skillstore 只报告证据，不判断该 Skill 是否安全。
复制报告链接
https://skillstore.io/skills/pinchbench-pinchbench/audits/5?utm_source=security_passport&utm_medium=share&utm_campaign=versioned_reportMarkdown 徽章
[](https://skillstore.io/skills/pinchbench-pinchbench?utm_source=security_passport_badge)HTML 徽章
<a href="https://skillstore.io/skills/pinchbench-pinchbench?utm_source=security_passport_badge"><img src="https://skillstore.io/badges/skills/pinchbench-pinchbench/security.svg" alt="Skillstore security assessment" loading="lazy"></a>嵌入卡片
<iframe src="https://skillstore.io/embed/skills/pinchbench-pinchbench.html" title="Skillstore Security Assessment" sandbox="allow-popups allow-popups-to-escape-sandbox" loading="lazy" referrerpolicy="no-referrer" width="420" height="180"></iframe>学术引用 (APA · BibTeX · CFF)
APA 引用
pinchbench. (2026). pinchbench security audit report (audit version 5) [Author version 1.0.0]. Skillstore. https://skillstore.io/skills/pinchbench-pinchbench/audits/5BibTeX 引用
@techreport{pinchbench-pinchbench-pinchbench-2026,
author = {pinchbench},
title = {pinchbench security audit report (audit version 5)},
institution = {Skillstore},
year = {2026},
number = {5},
url = {https://skillstore.io/skills/pinchbench-pinchbench/audits/5},
note = {Author version 1.0.0}
}CITATION.cff
cff-version: 1.2.0
message: "If you use this Skill, cite its author and this versioned security audit report."
title: "pinchbench security audit report (audit version 5)"
version: "1.0.0"
type: report
authors:
- name: "pinchbench"
date-released: "2026-07-09"
url: "https://skillstore.io/skills/pinchbench-pinchbench/audits/5"
identifiers:
- type: other
value: "skillstore:pinchbench-pinchbench:audit:5"
description: "Skillstore immutable audit report identifier"
Skillstore 评分
评分依据 证据置信度: 中 73
架构
100
可维护性
87
内容
71
社区
83
规范符合性
你能构建什么
比较模型性能
在多个模型上运行同一套 OpenClaw 任务，并比较分数、成本和完成时间。
对 OpenClaw 工作流进行回归测试
检查最近的 agent、skill 或环境变更是否影响任务完成质量。
设计新的基准测试任务
使用任务模板和评分模式，为新的 agent 工作流添加可重复的评估。
试试这些提示
运行快速基准测试
使用 MODEL 在仅自动化套件上运行 PinchBench，并将结果保存在本地，不上传。
比较两个模型
在同一任务套件上分别为 MODEL_A 和 MODEL_B 运行 PinchBench，然后总结分数、成本和耗时差异。
调查薄弱任务
查看最新的 PinchBench 结果，找出分数低于 0.5 的任务，并说明可能原因和后续调试步骤。
准备自定义套件
为工作流自动化任务创建一个聚焦的 PinchBench 套件，包含评分标准，并说明如何安全运行。
最佳实践
- 在了解将提交哪些数据之前，请禁用上传运行。
- 使用专用的基准测试工作区和模型提供商的测试凭据。
- 在允许自定义任务评分代码在本地运行之前，先审查该代码。
避免
- 不要在进行基本检查之前，在主工作区运行完整套件。
- 不要在审查元数据和 token 设置之前上传基准测试结果。
- 不要添加在评分代码中具有宽泛文件系统或 shell 访问权限的自定义任务。
常见问题
PinchBench 评估什么？
它会评估 OpenClaw agent 在文件、邮件、研究、编码和工作流等实际任务中的性能。
这个 skill 支持哪些工具？
报告标明与 Claude、Codex 和 Claude Code marketplace surfaces 的兼容性。
它需要网络访问吗？
模型验证、provider 调用、注册和排行榜上传需要网络访问。本地运行可以跳过上传。
结果存储在哪里？
基准测试结果会作为本地结果文件写入配置的结果目录。
我可以只运行选定任务吗？
可以。runner 支持所有任务、仅自动化任务，或以逗号分隔的任务 ID 列表。
发布前我应该审查什么？
应审查命令执行、评分代码执行、数据上传行为、复制的 skills，以及打包的二进制资源。
开发者详情
作者
pinchbench许可证
MIT
作者版本
v1.0.0
Skillstore 修订
r1
引用
3e4b6c31a74a3bd1a291c98cf585d720cb9fbc88
维护新鲜度
2026/7/18
使用量
16 次下载 · 290 次浏览