# Find Duplicate Images in FiftyOne

Duplicate and near-duplicate images can distort dataset quality and model training. This skill guides Claude, Codex, or Claude Code through FiftyOne Brain workflows to find, review, and remove redundant samples.

## Install

```bash
npx skillstore add adonaivera/fiftyone-find-duplicates
```

## Metadata

- Status: approved
- Slug: adonaivera-fiftyone-find-duplicates
- Skillstore revision: r1
- Version status: missing
- Tree hash: a3fc2671a9a12b3201b800f1ca7c83bde9ea630fb1c770fd3f07fc7a925d3d83
- Author: AdonaiVera
- GitHub username: AdonaiVera
- License: MIT
- Repository: https://github.com/AdonaiVera/fiftyone-skills/tree/main/find-duplicates/skills/fiftyone-find-duplicates
- Ref: 6eec4a59f63ff83440a84c18d1aef79743bc714e
- Supported tools: Claude, Codex, Claude Code
- Audit status: complete
- Agent install advisory: allowed
- Manual install advisory: allowed
- Artifact signature: available
- Audit attestation: unavailable
- Human verification: not\_verified
- Risk factors: external\_commands, network
- Quality score: 70
- Public page: https://skillstore.pages.dev/skills/adonaivera-fiftyone-find-duplicates
- Manifest: https://skillstore.pages.dev/api/skills/adonaivera-fiftyone-find-duplicates/manifest

## Capabilities

- Set a FiftyOne dataset context and launch the app for delegated Brain operators.
- Discover installed Brain plugin operators and inspect their current schemas.
- Compute image similarity embeddings with configurable brain keys and models.
- Find exact or near-duplicate images with practical threshold guidance.
- Load duplicate views and filters for review in the FiftyOne App.
- Run deduplication operators after review to keep representative samples.

## Use Cases

- Clean Image Training Sets: Find redundant images before model training so validation and training data are less biased.
- Review Visual Redundancy: Group visually similar samples and inspect representatives before changing the dataset.
- Remove Exact Duplicate Media: Identify identical media files and use FiftyOne tools to keep one representative copy.

## Prompt Templates

### Find Near Duplicates

```
Find near-duplicate images in my FiftyOne dataset named DATASET_NAME. Guide me through setup, similarity computation, duplicate discovery, and review.
```

### Review Duplicate Groups

```
Show me how to view all samples with near-duplicate group IDs in the FiftyOne App and inspect one group at a time.
```

### Remove Exact Duplicates

```
Help me find exact duplicate media files in my active FiftyOne dataset, review the result, and remove duplicates only after confirmation.
```

### Tune Similarity Search

```
Compare threshold choices for near-duplicate detection, choose a fast embedding model, and reuse the similarity index for later searches.
```

## Limitations

- Requires a running FiftyOne MCP server and the @voxel51/brain plugin.
- Works with image datasets already loaded in FiftyOne, not standalone folders by itself.
- Similarity computation can be slow for large datasets.
- Deletion steps need careful review because they can remove dataset samples.

## Best Practices

- Confirm the Brain plugin and operator schemas before running expensive jobs.
- Start with the default near-duplicate threshold and adjust after visual review.
- Review duplicate views before running any deduplication operator.

## Anti Patterns

- Running deduplication before checking representatives in the app.
- Recomputing embeddings with new brain keys when an existing similarity index is reusable.
- Running dependency install commands from tool output without verification.

## Security Audit

- Audited at: 2026-07-05T01:39:53.935\+00:00
- Summary: All static external-command, network, and system-reconnaissance detections are false positives from Markdown examples, inline function names, local app URLs, and documentation links. No prompt injection or data-exfiltration intent was found. One semantic concern remains: install commands returned by tooling should be validated before any execution.

## Stats

- Views: 313
- Downloads: 10
- Favorites: 0
- Popularity score: 0