# Run and Audit Reproduction Commands

Short reproduction runs often produce inconsistent evidence and unclear results. This skill executes a selected command and organizes outcomes, metrics, logs, and repository changes.

## Install

```bash
npx skillstore add lllllllama/minimal-run-and-audit
```

## Metadata

- Status: approved
- Slug: lllllllama-minimal-run-and-audit
- Skillstore revision: r2
- Version status: missing
- Tree hash: 5db676cab5410d295fb0cb912c58874bf1aec086eadc88c924d96714b0512597
- Author: lllllllama
- GitHub username: lllllllama
- License: MIT
- Repository: https://github.com/lllllllama/ai-paper-reproduction-skill/tree/main/skills/minimal-run-and-audit/
- Ref: 656be3040aef5c047555a908cd5c695d22a4a548
- Supported tools: Claude, Codex, Claude Code
- Audit status: complete
- Agent install advisory: confirmation\_required
- Manual install advisory: allowed
- Artifact signature: available
- Audit attestation: unavailable
- Human verification: not\_verified
- Risk factors: external\_commands, filesystem
- Quality score: 38
- Quality tier: warning
- Public page: https://skillstore.pages.dev/skills/lllllllama-minimal-run-and-audit
- Manifest: https://skillstore.pages.dev/api/skills/lllllllama-minimal-run-and-audit/manifest

## Capabilities

- Executes a selected non-training command with a configurable timeout.
- Captures standard output, standard error, exit status, and timeout state.
- Extracts numeric metrics from command output using a defined text pattern.
- Compares Git status before and after execution to identify changed paths.
- Classifies runs as successful, partial, or blocked and prepares normalized evidence.

## Use Cases

- Verify a documented inference command: Run a repository inference example and capture its outcome, logs, metrics, and file changes.
- Audit a reproduction smoke test: Produce concise evidence showing whether a selected smoke test succeeded, failed, or timed out.
- Normalize evaluation evidence: Convert a short evaluation attempt into consistent artifacts for team review and comparison.

## Prompt Templates

### Run a smoke test

```
Run the selected smoke command in [repository path] with a [timeout] second limit. Report the outcome and captured evidence.
```

### Audit an inference example

```
Execute [inference command] in [repository path]. Capture logs, numeric metrics, timeout state, and repository changes without modifying scientific behavior.
```

### Compare an evaluation result

```
Run [evaluation command] and compare the observed metric with [expected baseline]. Mark unsupported comparisons and document any changed files.
```

### Audit a patched reproduction

```
Execute [command] after the approved patches. Separate execution facts from inferences and document patch effects, comparability limits, metrics, and blockers.
```

## Limitations

- It does not select the reproduction target or design the experiment.
- It does not install dependencies, obtain assets, or configure environments.
- It is intended for short verification commands, not training or long-running jobs.
- Automatic metric extraction may misidentify values in unstructured output.

## Best Practices

- Confirm the exact command, repository path, and timeout before execution.
- Use short documented inference, evaluation, smoke, or sanity commands.
- Review reported file changes and disclose any effect on scientific meaning.

## Anti Patterns

- Do not use this skill to start training or resume long-running jobs.
- Do not treat extracted metrics as validated scientific conclusions.
- Do not hide preprocessing, checkpoint, evaluation, or metric changes.

## Security Audit

- Audited at: 2026-08-08T08:54:36.809\+00:00
- Summary: The runner intentionally executes a user-supplied command, which is a confirmed high-risk capability despite avoiding shell interpretation. Two parent-directory references cross the audited package boundary, and the output wrapper dynamically executes a Python module; Markdown backticks and reporting text are false positives.

## Stats

- Views: 105
- Downloads: 11
- Favorites: 0
- Popularity score: 0