# Benchmark OpenClaw Agents With PinchBench

Agent teams need repeatable evidence before comparing models. PinchBench runs real-world OpenClaw tasks, grades outcomes, and can publish results to a leaderboard.

## Install

```bash
npx skillstore add pinchbench/pinchbench
```

## Metadata

- Status: approved
- Slug: pinchbench-pinchbench
- Version: 1.0.0
- Author version: 1.0.0
- Skillstore revision: r1
- Version status: valid
- Tree hash: 5bbbdb6709775a72147e8ca006236522641f0bff730a4dcca857a071dda91655
- Author: pinchbench
- GitHub username: pinchbench
- License: MIT
- Repository: https://github.com/pinchbench/skill/tree/main/
- Ref: 3e4b6c31a74a3bd1a291c98cf585d720cb9fbc88
- Supported tools: Claude, Codex, Claude Code
- Audit status: complete
- Agent install advisory: blocked
- Manual install advisory: allowed\_with\_warning
- Artifact signature: available
- Audit attestation: unavailable
- Human verification: not\_verified
- Risk factors: external\_commands, network, filesystem, scripts, env\_access
- Quality score: 38
- Quality tier: warning
- Public page: https://skillstore.pages.dev/skills/pinchbench-pinchbench
- Manifest: https://skillstore.pages.dev/api/skills/pinchbench-pinchbench/manifest

## Capabilities

- Runs 23 bundled benchmark tasks across productivity, research, writing, coding, email, and file workflows.
- Creates or reuses OpenClaw agents for selected model identifiers.
- Grades tasks with automated checks, LLM judge rubrics, or hybrid scoring.
- Records task transcripts, timing, token use, cost, and score summaries.
- Saves local result files and can upload authorized submissions to PinchBench.

## Use Cases

- Compare Agent Models: Run the same task suite across models and compare score, cost, time, and category performance.
- Validate OpenClaw Setup: Check that an OpenClaw installation can handle tools, memory, files, research, and multi-step workflows.
- Publish Benchmark Results: Generate reproducible local results and submit authorized runs to the PinchBench leaderboard.

## Prompt Templates

### Run A Quick Benchmark

```
Use PinchBench to run the automated-only suite for [model id] with uploads disabled. Summarize the final score and failed tasks.
```

### Compare Two Models

```
Run PinchBench for [model A] and [model B] using the same suite and run count. Compare category scores, cost, and time.
```

### Upload A Previous Run

```
Validate my saved PinchBench result at [file path], upload it to the leaderboard, and report the submission link.
```

### Audit A Custom Task

```
Review my new PinchBench task file for clear grading, safe workspace files, and reproducible behavior before I run it.
```

## Limitations

- Requires Python, uv, and a working OpenClaw installation.
- Some tasks call external services or model APIs, which can add cost.
- Leaderboard upload requires a PinchBench token and sends run metadata.
- Automated grading executes bundled task check code, so review tasks before running.

## Best Practices

- Start with automated-only and uploads disabled when testing a new model.
- Use the same suite, run count, and timeout multiplier when comparing models.
- Review task files and upload settings before running community benchmarks.

## Anti Patterns

- Running all tasks with uploads enabled before validating local behavior.
- Comparing models with different task suites or timeout settings.
- Sharing result files without checking transcripts and system metadata.

## Security Audit

- Audited at: 2026-07-09T07:39:04.466\+00:00
- Summary: PinchBench is a legitimate benchmark runner, but it is not low risk. Confirmed issues include pipe-to-shell dependency installation, unsandboxed task grading code, external agent execution, token handling, and host metadata upload. Many documentation and task-rubric alerts are false positives, while binary assets remain manual-review blind spots.

## Stats

- Views: 290
- Downloads: 22
- Favorites: 0
- Popularity score: 0