技能 pinchbench
📦
pinchbench
v1.0.0 內容修訂 r1 嚴重 ⚙️ 外部命令🌐 網路存取📁 檔案系統存取⚡ 包含腳本🔑 環境變數
使用 PinchBench 對 OpenClaw Agents 進行基準測試
模型團隊需要可重複的證據來評估 agent 效能。PinchBench 會執行結構化的 OpenClaw 任務，並產生可比較的分數，用於分析與排行榜提交。
支援: Claude Codex Code(CC)
請勿自動安裝此 Skill。
權威政策要求在執行任何安裝操作前由操作人員審查。
使用我的智慧代理安裝
將此請求複製給你的智慧代理，其中包含權威 Skill 頁面和清單。
智能體請求
Review the Skillstore skill "pinchbench" from https://skillstore.io/skills/pinchbench-pinchbench.md and its manifest at https://skillstore.io/api/skills/pinchbench-pinchbench/manifest. Verify the artifact. Do not auto-install. Inspect the skill and report your findings, then wait for an operator or manual installation decision.你的智慧代理仍應顯示其計畫，並依安全政策請求必要確認。
Agent 可讀資源
當 AI Agent、爬蟲或腳本需要乾淨脈絡、而不是讀取完整頁面時，請使用這些連結。
測試它
正在使用「pinchbench」。 Run the automated-only suite for one model without uploading.
預期結果:
執行完成後會產生各任務分數、總分、token 用量、成本指標，以及本機結果檔案路徑。
正在使用「pinchbench」。 Compare two saved benchmark runs.
預期結果:
比較結果會凸顯分數差異、失敗任務、執行時間變化，以及成本效率取捨。
正在使用「pinchbench」。 Design a new benchmark task for email triage.
預期結果:
任務草稿包含 prompt、預期行為、評分標準，以及用於可重複評分的自動化檢查。
安全審計
嚴重v5 • 2026/7/9 開啟版本化報告
PinchBench is a legitimate benchmark runner, but it is not low risk. Confirmed issues include pipe-to-shell dependency installation, unsandboxed task grading code, external agent execution, token handling, and host metadata upload. Many documentation and task-rubric alerts are false positives, while binary assets remain manual-review blind spots.
38
已掃描檔案
7,808
分析行數
29
複核項目
0
已忽略的誤報
已確認的安全問題 (6)
嚴重
Pipe to shell pattern
RUN curl -LsSf https://astral.sh/uv/install.sh | sh
The Docker build pipes a downloaded installer directly into a shell. A compromised network path or upstream installer would execute during image build.
高
System Metadata Uploaded With Results
When uploads are enabled, the payload includes metadata.system and records OS, CPU, Python, memory, and a hostname hash. This goes beyond benchmark scores and may identify the host environment.
The upload payload explicitly includes collect_system_metadata output. The metadata function records platform, CPU, memory, Python, and hostname-derived data.
高
Registration Token Stored In Skill Directory
The registration helper writes the returned token to scripts/.pinchbench/config.json under the skill directory. Tokens in an installed skill tree can be accidentally shared or read by benchmarked agents.
The config path is inside the scripts directory and save_token_config writes the token there. _resolve_token later reads that stored token for uploads.
高
Installed Skills Copied Into Benchmark Workspace
prepare_task_workspace copies every directory under ~/.openclaw/workspace/skills into the benchmark workspace. This may expose private skill content or embedded credentials to the benchmarked model during tasks.
The code iterates over the user skill directory and copies each skill directory into the agent workspace. No allowlist or secret filtering is visible in that block.
中
System reconnaissance
"hostname_hash": hash(platform.node()) % 10000,
The code derives a hostname hash as part of system metadata. Even hashed, it contributes to host fingerprinting in uploaded results.
中
Agent Session Transcripts Deleted Without Confirmation
cleanup_agent_sessions removes stored session transcript files and sessions.json for the target agent. If the target agent ID collides with a real agent, benchmark runs can delete local conversation history.
The deletion code is explicit and is called before benchmark and judge runs. The risk is reduced because agent IDs are normally benchmark-prefixed, but collision is still possible.
能力複核項目 (29)
這些是真實的本機能力，對此技能而言可能是預期行為，因此需要複核，但不會按已確認的惡意行為計分。
高
Python subprocess.run
subprocess.run(
The code deletes an OpenClaw agent when its workspace appears stale. It uses argv lists, but it can remove local agent state.
高
Python subprocess.run
result = subprocess.run(
This launches the OpenClaw CLI with benchmark or judge prompts and workspace access. It avoids shell injection, but it triggers external agent execution with local side effects.
高
Python subprocess.run
result = subprocess.run(
This launches the OpenClaw CLI with benchmark or judge prompts and workspace access. It avoids shell injection, but it triggers external agent execution with local side effects.
高
Python subprocess.run
result = subprocess.run(
This launches the OpenClaw CLI with benchmark or judge prompts and workspace access. It avoids shell injection, but it triggers external agent execution with local side effects.
高
Python shutil operations
shutil.rmtree(workspace)
The runner recursively removes the agent workspace before each task. If workspace resolution is wrong, this can delete local files.
高
Python environment access
api_key = os.environ.get("OPENROUTER_API_KEY")
The code reads OPENROUTER_API_KEY from the environment. It is expected for validation, but it handles credential material.
高
Generic API/secret keys
api_key = os.environ.get("OPENROUTER_API_KEY")
The code reads OPENROUTER_API_KEY from the environment. It is expected for validation, but it handles credential material.
高
Generic API/secret keys
"Authorization": f"Bearer {api_key}",
The API key is placed in an Authorization bearer header for an external service. This is legitimate use, but it is sensitive credential transmission.
高
Python exec() function
exec(grading_code, namespace)
The grader executes Python code extracted from task markdown with exec(). Any malicious or modified task file could run arbitrary Python in the benchmark process.
高
Process exec
exec(grading_code, namespace)
The grader executes Python code extracted from task markdown with exec(). Any malicious or modified task file could run arbitrary Python in the benchmark process.
高
Python subprocess.run
result = subprocess.run(
The macOS metadata helper executes sysctl to collect hardware attributes. The arguments are fixed, but the command expands host fingerprinting data.
高
Linux /proc filesystem access
cpuinfo_path = Path("/proc/cpuinfo")
The upload path reads Linux /proc CPU and memory files for system metadata. This collects host fingerprinting information for submission payloads.
高
Linux /proc filesystem access
meminfo_path = Path("/proc/meminfo")
The upload path reads Linux /proc CPU and memory files for system metadata. This collects host fingerprinting information for submission payloads.
高
Python environment access
resolved_official_key = official_key or os.environ.get("PINCHBENCH_OFFICIAL_KEY")
The upload code reads authentication or official submission tokens from environment or local config. These values are sensitive credentials.
高
Python environment access
env_token = os.environ.get("PINCHBENCH_TOKEN")
The upload code reads authentication or official submission tokens from environment or local config. These values are sensitive credentials.
高
Configuration library
return config.get("token")
The upload code reads authentication or official submission tokens from environment or local config. These values are sensitive credentials.
高
Generic API/secret keys
token = data.get("token") or data.get("api_key")
The registration response is parsed for a token or api_key. This is credential material that is later stored and used for uploads.
中
Hardcoded URL
RUN curl -LsSf https://astral.sh/uv/install.sh | sh
The Docker build downloads an installer from an external URL. This is expected setup, but it is a real supply-chain dependency.
中
Temp directory access
run_root = Path("/tmp/pinchbench")
The runner uses a fixed directory under /tmp for benchmark workspaces. This is functional, but not as safe as a private temporary directory.
中
Temp directory access
agent_workspace = Path(f"/tmp/pinchbench/{run_id}/agent_workspace")
The runner uses a fixed directory under /tmp for benchmark workspaces. This is functional, but not as safe as a private temporary directory.
中
Hardcoded URL
specific_endpoint = f"https://openrouter.ai/api/v1/models/{encoded_model_id}"
The model validator contacts OpenRouter endpoints and can use an API key in the same request flow. This is legitimate but still external network activity.
中
Hardcoded URL
catalog_endpoint = "https://openrouter.ai/api/v1/models"
The model validator contacts OpenRouter endpoints and can use an API key in the same request flow. This is legitimate but still external network activity.
中
Temp directory access
workspace = Path(f"/tmp/pinchbench/{run_id}/{task.task_id}")
The fallback workspace uses a predictable path under /tmp. This can expose benchmark files to local path collision or permission risks.
中
Python shutil operations
shutil.rmtree(dest_skill_dir)
The runner deletes and copies skill directories inside the benchmark workspace. This can expose or overwrite copied skill content during task setup.
中
Python shutil operations
shutil.copytree(skill_dir_src, dest_skill_dir)
The runner deletes and copies skill directories inside the benchmark workspace. This can expose or overwrite copied skill content during task setup.
中
Temp directory access
judge_workspace = Path(f"/tmp/pinchbench/judge/{task.task_id}")
The judge uses predictable directories under /tmp for workspaces. This is expected, but not as isolated as a secure temporary directory.
中
Temp directory access
workspace = Path("/tmp/pinchbench/judge/workspace")
The judge uses predictable directories under /tmp for workspaces. This is expected, but not as isolated as a secure temporary directory.
中
Hardcoded URL
DEFAULT_SERVER_URL = "https://api.pinchbench.com"
The upload module posts benchmark results to the PinchBench API by default. This is intended, but it sends local run data off the machine.
中
Python environment access
resolved_server = server_url or os.environ.get("PINCHBENCH_SERVER_URL") or DEFAULT_SERVER_URL
The upload server can be overridden by environment variable. If the environment is compromised, results and auth headers can be sent away from the default API.
風險因素
⚙️ 外部命令 (50)
Dockerfile.benchmark:33 scripts/benchmark.py:277 scripts/lib_agent.py:136 scripts/lib_agent.py:179 scripts/lib_agent.py:221 scripts/lib_agent.py:230 scripts/lib_agent.py:578 scripts/lib_agent.py:611 scripts/lib_agent.py:753 scripts/lib_grading.py:107 scripts/lib_grading.py:216 scripts/lib_grading.py:318 scripts/lib_grading.py:319 scripts/lib_upload.py:317 scripts/lib_upload.py:396 scripts/run.sh:6 SKILL.md:23-37 SKILL.md:37-43 SKILL.md:43-44 SKILL.md:44-45 SKILL.md:45-46 SKILL.md:46-47 SKILL.md:47-48 SKILL.md:48-49 SKILL.md:49-50 SKILL.md:50-51 SKILL.md:51-52 SKILL.md:52-53 SKILL.md:53-54 SKILL.md:54-55 SKILL.md:55-56 SKILL.md:56-57 SKILL.md:57-58 SKILL.md:58-59 SKILL.md:59-60 SKILL.md:60-61 SKILL.md:61-62 SKILL.md:62-63 SKILL.md:63-64 SKILL.md:64-65 SKILL.md:65-71 SKILL.md:71 SKILL.md:72 SKILL.md:73 SKILL.md:74-75 SKILL.md:75-76 SKILL.md:76-77 SKILL.md:77-78 SKILL.md:78-84 SKILL.md:84-90
🌐 網路存取 (13)
📁 檔案系統存取 (12)
Dockerfile.benchmark:21 Dockerfile.benchmark:30 scripts/benchmark.py:557 scripts/benchmark.py:562 scripts/lib_agent.py:293 scripts/lib_agent.py:298 scripts/lib_agent.py:340 scripts/lib_agent.py:341 scripts/lib_grading.py:157 scripts/lib_grading.py:298 scripts/lib_upload.py:354 scripts/lib_upload.py:355
🔑 環境變數 (13)
scripts/benchmark.py:270 scripts/lib_agent.py:27 scripts/lib_agent.py:59 scripts/lib_agent.py:59 scripts/lib_agent.py:60 scripts/lib_agent.py:61 scripts/lib_agent.py:67 scripts/lib_upload.py:64 scripts/lib_upload.py:66 scripts/lib_upload.py:135 scripts/lib_upload.py:289 scripts/lib_upload.py:293 scripts/lib_upload.py:159
偵測到的模式
Pipe to shell patternSystem reconnaissance
審計單位: codex 檢視審計紀錄 →
分享與引用此報告
分享版本化評估報告、中性徽章、嵌入卡片與引用。Skillstore 只報告證據，不判斷該 Skill 是否安全。
複製報告連結
https://skillstore.io/skills/pinchbench-pinchbench/audits/5?utm_source=security_passport&utm_medium=share&utm_campaign=versioned_reportMarkdown 徽章
[](https://skillstore.io/skills/pinchbench-pinchbench?utm_source=security_passport_badge)HTML 徽章
<a href="https://skillstore.io/skills/pinchbench-pinchbench?utm_source=security_passport_badge"><img src="https://skillstore.io/badges/skills/pinchbench-pinchbench/security.svg" alt="Skillstore security assessment" loading="lazy"></a>嵌入卡片
<iframe src="https://skillstore.io/embed/skills/pinchbench-pinchbench.html" title="Skillstore Security Assessment" sandbox="allow-popups allow-popups-to-escape-sandbox" loading="lazy" referrerpolicy="no-referrer" width="420" height="180"></iframe>學術引用 (APA · BibTeX · CFF)
APA 引用
pinchbench. (2026). pinchbench security audit report (audit version 5) [Author version 1.0.0]. Skillstore. https://skillstore.io/skills/pinchbench-pinchbench/audits/5BibTeX 引用
@techreport{pinchbench-pinchbench-pinchbench-2026,
author = {pinchbench},
title = {pinchbench security audit report (audit version 5)},
institution = {Skillstore},
year = {2026},
number = {5},
url = {https://skillstore.io/skills/pinchbench-pinchbench/audits/5},
note = {Author version 1.0.0}
}CITATION.cff
cff-version: 1.2.0
message: "If you use this Skill, cite its author and this versioned security audit report."
title: "pinchbench security audit report (audit version 5)"
version: "1.0.0"
type: report
authors:
- name: "pinchbench"
date-released: "2026-07-09"
url: "https://skillstore.io/skills/pinchbench-pinchbench/audits/5"
identifiers:
- type: other
value: "skillstore:pinchbench-pinchbench:audit:5"
description: "Skillstore immutable audit report identifier"
Skillstore 評分
評分依據 證據置信度: 中 73
架構
100
可維護性
87
內容
71
社群
83
規範符合度
你可以打造什麼
比較模型效能
在多個模型上執行相同的 OpenClaw 任務套件，並比較分數、成本與完成時間。
對 OpenClaw 工作流程進行回歸測試
檢查近期的 agent、skill 或環境變更是否影響任務完成品質。
設計新的基準測試任務
使用任務範本與評分模式，為新的 agent 工作流程新增可重複的評估。
試試這些提示
執行快速基準測試
Run PinchBench with MODEL on the automated-only suite and save the results locally without uploading.
比較兩個模型
Run PinchBench for MODEL_A and MODEL_B on the same task suite, then summarize score, cost, and timing differences.
調查表現較弱的任務
Review the latest PinchBench results and identify tasks below 0.5 score with likely causes and next debugging steps.
準備自訂套件
Create a focused PinchBench suite for workflow automation tasks, include grading criteria, and explain how to run it safely.
最佳實務
- 在了解將提交哪些資料前，先以停用上傳的方式執行。
- 使用專用的基準測試工作區與模型供應商的測試憑證。
- 允許自訂任務評分程式碼在本機執行前，先加以審查。
避免
- 在進行基本檢查前，不要於主要工作區執行完整套件。
- 審查 metadata 與 token 設定前，不要上傳基準測試結果。
- 不要加入在評分程式碼中具有廣泛檔案系統或 shell 存取權限的自訂任務。
常見問題
PinchBench 評估什麼？
它會評估 OpenClaw agent 在檔案、電子郵件、研究、程式撰寫與工作流程等實務任務中的效能。
這個 skill 支援哪些工具？
報告標示與 Claude、Codex 和 Claude Code marketplace 介面的相容性。
它需要網路存取嗎？
模型驗證、供應商呼叫、註冊與排行榜上傳需要網路存取。本機執行可以略過上傳。
結果儲存在哪裡？
基準測試結果會作為本機結果檔案寫入已設定的結果目錄。
我可以只執行選定的任務嗎？
可以。runner 支援所有任務、僅自動化任務，或以逗號分隔的任務 ID 清單。
發布前我應該審查什麼？
請審查命令執行、評分程式碼執行、資料上傳行為、複製的 skills，以及隨附的二進位資產。
開發者詳細資訊
作者
pinchbench授權
MIT
作者版本
v1.0.0
Skillstore 修訂
r1
引用
3e4b6c31a74a3bd1a291c98cf585d720cb9fbc88
維護新鮮度
2026/7/18
使用量
16 次下載 · 290 次瀏覽