# Benchmark FlashInfer Kernels Accurately

GPU kernel comparisons are unreliable when timing methods and parameters vary. This skill guides reproducible FlashInfer benchmarks using CUPTI or CUDA events.

## Install

```bash
npx skillstore add flashinfer-ai/benchmark-kernel
```

## Metadata

- Status: approved
- Slug: flashinfer-ai-benchmark-kernel
- Skillstore revision: r2
- Version status: missing
- Tree hash: f7c9e4d41da011c2deb58a9bd3293fb74f37237751ad9250e9e2ef10497929d4
- Author: flashinfer-ai
- GitHub username: flashinfer-ai
- License: MIT
- Repository: https://github.com/flashinfer-ai/flashinfer/tree/main/.claude/skills/benchmark-kernel
- Ref: c43861a65bb95efcae259cd161c9d6f4dc7eec6f
- Supported tools: Claude, Codex, Claude Code
- Audit status: complete
- Agent install advisory: allowed
- Manual install advisory: allowed
- Artifact signature: available
- Audit attestation: unavailable
- Human verification: not\_verified
- Risk factors: external\_commands
- Quality score: 78
- Quality tier: bronze
- Public page: https://skillstore.pages.dev/skills/flashinfer-ai-benchmark-kernel
- Manifest: https://skillstore.pages.dev/api/skills/flashinfer-ai-benchmark-kernel/manifest

## Capabilities

- Guides single and batch benchmarks through flashinfer\_benchmark.py.
- Configures attention, FP8 GEMM, and MoE routines across supported backends.
- Explains CUPTI timing and automatic CUDA event fallback.
- Applies reference checks, warmups, measurement iterations, and reproducer generation.
- Shows custom Python timing with bench\_gpu\_time.
- Interprets median latency, variation, throughput, bandwidth, and CSV results.

## Use Cases

- Compare Kernel Backends: Measure identical attention or GEMM shapes across supported backends with correctness checks and consistent timing.
- Build Repeatable Benchmark Suites: Run parameterized test lists, preserve reproducer details, and save metrics for later analysis.
- Design Custom Timing Studies: Use bench\_gpu\_time for controlled experiments with warmups, iteration counts, cache behavior, and timing selection.

## Prompt Templates

### Start a FlashInfer Benchmark

```
Prepare a minimal FlashInfer decode-attention benchmark for my [GPU] using [backends]. Include prerequisites, reference checking, timing behavior, and result interpretation.
```

### Compare Kernel Backends

```
Build a fair comparison for [routine] across [backends] using [shape and data types]. Keep inputs, warmups, iterations, and validation identical.
```

### Plan a Batch Benchmark

```
Create a batch benchmark plan covering [workloads]. Define the test matrix, CSV fields, correctness checks, reproducer details, and comparison criteria.
```

### Design a Custom Timing Study

```
Design a bench_gpu_time experiment for [kernel]. Address CUPTI fallback, cache state, statistical stability, operation counts, throughput calculations, and likely measurement bias.
```

## Limitations

- Examples require a FlashInfer source checkout, a supported NVIDIA GPU, and a working CUDA environment.
- The preferred CUPTI path requires CUDA 13 or later and a compatible cupti-python package.
- Hardware and backend compatibility must be checked against the current FlashInfer support matrix.
- The skill provides instructions only; it does not execute benchmarks or validate measured results.

## Best Practices

- Enable reference checking before trusting performance comparisons.
- Record GPU model, software versions, parameters, warmups, and measurement iterations.
- Use identical inputs and timing methods when comparing backends.

## Anti Patterns

- Do not compare results collected with different shapes, data types, or timing methods.
- Do not treat one iteration as representative performance.
- Do not use output mismatches as valid performance evidence.

## Security Audit

- Audited at: 2026-07-23T17:43:36.644\+00:00
- Summary: Eighty-four Ruby or shell backtick findings and the network-reconnaissance finding are false positives caused by Markdown formatting and benchmark arguments. The sudo nvidia-smi example is a real privileged system change that can alter GPU clocks for all workloads, although it contains no injection path. No malicious intent, prompt injection, or data exfiltration was found.

## Stats

- Views: 291
- Downloads: 5
- Favorites: 0
- Popularity score: 0