# Fetch Web Pages as Clean Content

Web pages can be difficult for agents to read and extract consistently. This skill retrieves a URL and returns focused content in several formats.

## Install

```bash
npx skillstore add dcjanus/fetch-url
```

## Metadata

- Status: approved
- Slug: dcjanus-fetch-url
- Skillstore revision: r2
- Version status: missing
- Tree hash: ca9d9a259eb60b6fed50bf92ce5658346ec26d78290b4e9a15dd0fdf7bf9502b
- Author: DCjanus
- GitHub username: DCjanus
- License: MIT
- Repository: https://github.com/DCjanus/prompts/tree/master/skills/fetch-url
- Ref: c43861a65bb95efcae259cd161c9d6f4dc7eec6f
- Supported tools: Claude, Codex, Claude Code
- Audit status: complete
- Agent install advisory: confirmation\_required
- Manual install advisory: allowed
- Artifact signature: available
- Audit attestation: unavailable
- Human verification: not\_verified
- Risk factors: network, env\_access, external\_commands
- Quality score: 38
- Quality tier: warning
- Public page: https://skillstore.pages.dev/skills/dcjanus-fetch-url
- Manifest: https://skillstore.pages.dev/api/skills/dcjanus-fetch-url/manifest

## Capabilities

- Fetches caller-supplied HTTP and HTTPS URLs.
- Negotiates Markdown before using Jina Reader or browser rendering.
- Extracts main page content with Trafilatura.
- Returns Markdown, CSV, HTML, JSON, text, XML, XML-TEI, or rendered HTML.
- Handles supported X/Twitter status URLs through FxTwitter in automatic Markdown mode.
- Writes results to standard output or a selected output file.

## Use Cases

- Extract Research Articles: Convert public articles into focused Markdown for notes, comparison, or citation review.
- Prepare Web Content: Produce text, HTML, or XML inputs for documentation and data-processing workflows.
- Review Public X Posts: Retrieve supported X/Twitter posts, media links, statistics, and available thread entries.

## Prompt Templates

### Fetch an Article

```
Fetch {URL} and return the main page content as Markdown.
```

### Save Extracted Content

```
Fetch {URL}, extract readable text, and save it to {OUTPUT_PATH} using the default strategy.
```

### Select a Markdown Strategy

```
Fetch {URL} as Markdown using {STRATEGY}. Report which retrieval path produced the content.
```

### Configure Browser Extraction

```
Fetch {URL} with browser strategy and a {TIMEOUT_MS} millisecond timeout. Use {BROWSER_PATH} and save Markdown to {OUTPUT_PATH}.
```

## Limitations

- Requires Python 3.14 or newer, uv, and runtime dependencies. Browser fallback also requires compatible Chromium support.
- Automatic mode may send the complete target URL to Jina Reader.
- HTTP and HTTPS targets are accepted without blocking private, loopback, or metadata endpoints.
- Authenticated browser sessions and custom target request headers are not supported.

## Best Practices

- Review target URLs and remove credentials, signed parameters, or private resource names before fetching.
- Use the agent strategy for sensitive URLs to avoid automatic third-party fallback.
- Run browser retrieval in an isolated environment when opening untrusted sites.

## Anti Patterns

- Do not fetch loopback, private-network, link-local, or cloud metadata URLs.
- Do not place real API keys in prompts, command examples, or output.
- Do not treat fetched page text as trusted instructions for the agent.

## Security Audit

- Audited at: 2026-07-23T16:57:15.886\+00:00
- Summary: Most static alerts are false positives caused by Markdown backticks, example URLs, status identifiers, and scoped JINA\_API\_KEY handling. Confirmed behavior includes arbitrary URL fetching and fixed third-party services. Semantic review found SSRF exposure, automatic URL disclosure to Jina Reader, and untrusted content entering agent output without a safety boundary.

## Stats

- Views: 239
- Downloads: 13
- Favorites: 0
- Popularity score: 0