Skills xcrawl-crawl
📦
xcrawl-crawl
v1.0.2 Content revision r1 Medium Risk ⚙️ External commands🌐 Network access📁 Filesystem access🔑 Env variables
Crawl Websites with Bounded XCrawl Jobs
Bulk website collection requires clear crawl boundaries and asynchronous result tracking. This skill guides XCrawl requests, filtering rules, and retrieval of original API responses.
Supports: Claude Codex Code(CC)
Install with my Agent
Copy this request to your Agent. It includes the canonical Skill page and manifest.
Agent request
Review the Skillstore skill "xcrawl-crawl" from https://skillstore.io/skills/xcrawl-api-xcrawl-crawl.md and its manifest at https://skillstore.io/api/skills/xcrawl-api-xcrawl-crawl/manifest. Verify the artifact. You may proceed after verification, subject to the environment's own policy.Your Agent should still show its plan and request any confirmation required by the security policy.
Agent-readable resources
Use these links when an AI agent, crawler, or script needs clean context instead of reading the full page.
Test it
Using "xcrawl-crawl". Describe an illustrative request for a documentation crawl limited to 20 pages and depth 2.
Expected outcome:
- Illustrative request: documentation paths only, 20-page limit, depth 2, Markdown and links, with TLS verification enabled.
- The create response supplies a task identifier and an initial pending status.
- The result response reports job status, available page data, and credit usage.
Using "xcrawl-crawl". Describe the expected response delivery when an approved crawl fails. Do not invent live results.
Expected outcome:
- The endpoint flow and submitted request payload accompany the original create and result response bodies.
- Any upstream error details are returned without presenting speculative page counts as completed coverage.
- No derived summary is added unless explicitly requested.
Security Audit
Medium Riskv1 • 10/17/2026 Open versioned report
All 132 static findings are false positives involving Markdown, documented configuration, or legitimate XCrawl authentication and API calls. A separate finding concerns the documented default that disables target TLS verification; both usage examples omit a secure override. No evidence found of malicious credential exfiltration, destructive operations, or embedded prompt injection.
1
Files scanned
239
Lines analyzed
0
Review items
0
False positives ignored
Confirmed security concerns (1)
Medium
Documented TLS Verification Bypass Default
The request table lists skip_tls_verification with default true. Both examples omit an override, potentially allowing intercepted or tampered target responses when following the documented default.
The insecure default and omitted overrides are explicit in the source. Confidence concerns the documented workflow; actual upstream enforcement was not tested.
Risk Factors
⚙️ External commands (50)
SKILL.md:17 SKILL.md:19 SKILL.md:21-25 SKILL.md:25-32 SKILL.md:32-33 SKILL.md:33-37 SKILL.md:37 SKILL.md:42-43 SKILL.md:43-44 SKILL.md:44-45 SKILL.md:45-51 SKILL.md:51-65 SKILL.md:65-69 SKILL.md:69-76 SKILL.md:76-80 SKILL.md:80-86 SKILL.md:86-88 SKILL.md:88-89 SKILL.md:89-95 SKILL.md:95-96 SKILL.md:96-97 SKILL.md:97-98 SKILL.md:98-99 SKILL.md:99-100 SKILL.md:100-101 SKILL.md:101-103 SKILL.md:103-107 SKILL.md:107 SKILL.md:108-109 SKILL.md:109-110 SKILL.md:110 SKILL.md:111 SKILL.md:112 SKILL.md:113 SKILL.md:114 SKILL.md:116-120 SKILL.md:120 SKILL.md:121-123 SKILL.md:123-127 SKILL.md:127 SKILL.md:128 SKILL.md:129-130 SKILL.md:130-131 SKILL.md:131 SKILL.md:132 SKILL.md:133 SKILL.md:135-139 SKILL.md:139 SKILL.md:140 SKILL.md:141
🌐 Network access (11)
📁 Filesystem access (10)
Audited by: codex
Share & cite this report
Share the versioned assessment report, neutral badge, embed card, and citations. Skillstore reports evidence without deciding whether this Skill is safe.
Copy report link
https://skillstore.io/skills/xcrawl-api-xcrawl-crawl/audits/1?utm_source=security_passport&utm_medium=share&utm_campaign=versioned_reportMarkdown badge
[](https://skillstore.io/skills/xcrawl-api-xcrawl-crawl?utm_source=security_passport_badge)HTML badge
<a href="https://skillstore.io/skills/xcrawl-api-xcrawl-crawl?utm_source=security_passport_badge"><img src="https://skillstore.io/badges/skills/xcrawl-api-xcrawl-crawl/security.svg" alt="Skillstore security assessment" loading="lazy"></a>Embed card
<iframe src="https://skillstore.io/embed/skills/xcrawl-api-xcrawl-crawl.html" title="Skillstore Security Assessment" sandbox="allow-popups allow-popups-to-escape-sandbox" loading="lazy" referrerpolicy="no-referrer" width="420" height="180"></iframe>Academic citations (APA · BibTeX · CFF)
APA citation
xcrawl-api. (2026). xcrawl-crawl security audit report (audit version 1) [Author version 1.0.2]. Skillstore. https://skillstore.io/skills/xcrawl-api-xcrawl-crawl/audits/1BibTeX citation
@techreport{xcrawl-api-xcrawl-api-xcrawl-crawl-2026,
author = {xcrawl-api},
title = {xcrawl-crawl security audit report (audit version 1)},
institution = {Skillstore},
year = {2026},
number = {1},
url = {https://skillstore.io/skills/xcrawl-api-xcrawl-crawl/audits/1},
note = {Author version 1.0.2}
}CITATION.cff
cff-version: 1.2.0
message: "If you use this Skill, cite its author and this versioned security audit report."
title: "xcrawl-crawl security audit report (audit version 1)"
version: "1.0.2"
type: report
authors:
- name: "xcrawl-api"
date-released: "2026-10-17"
url: "https://skillstore.io/skills/xcrawl-api-xcrawl-crawl/audits/1"
identifiers:
- type: other
value: "skillstore:xcrawl-api-xcrawl-crawl:audit:1"
description: "Skillstore immutable audit report identifier"
Skillstore Score
Why this score Evidence Confidence: Medium 55
Architecture
100
Maintainability
87
Content
65
Community
100
Spec Compliance
What You Can Build
Collect Documentation for Search
Crawl an approved documentation section with path filters and page limits, requesting Markdown and links for later indexing.
Extract Public Research Data
Configure structured extraction for authorized public pages and retrieve the completed job responses for later analysis.
Inspect Rendered Site Content
Request rendered pages and screenshots for a bounded website section, using desktop or mobile viewport settings.
Try These Prompts
Start a Small Crawl
Crawl [approved site URL] with a 10-page limit and depth 1. Request Markdown, keep TLS verification enabled, and return the original responses.
Collect a Documentation Section
Crawl [approved documentation URL], including only /docs/ paths and excluding /blog/. Limit to 100 pages and depth 2. Request Markdown and links. Keep TLS verification enabled and return original create and result responses.
Extract Structured Page Data
Crawl [approved site URL] with a 50-page limit and depth 2. Extract [fields] using [schema] and request links. Use [locale] and [device]. Keep TLS verification enabled. Poll until completion or failure and return original responses.
Configure an Asynchronous Workflow
Prepare a crawl for [approved site URL] with [page limit], [depth], and [path filters]. Configure [approved HTTPS webhook] for completed and failed events. Keep external links excluded and TLS verification enabled. Confirm scope before starting. Return original API responses.
Best Practices
- Set explicit page limits, depth, and path filters before starting any crawl.
- Keep TLS verification enabled and restrict crawl targets and webhook destinations to approved locations.
- Protect the local API key file and avoid sharing secrets in request payloads, logs, or conversation output.
Avoid
- Starting broad crawls without explicit scope, page limits, or authorization.
- Accepting the documented TLS verification bypass default for ordinary production targets.
- Treating an initial pending status or estimated page count as proof of completed coverage.
Frequently Asked Questions
What must I configure before using this skill?
Create ~/.xcrawl/config.json with your XCRAWL_API_KEY. An XCrawl account, available credits, network access, and curl or Node are required.
Does the skill run a local crawler?
No. It provides instructions for calling the external XCrawl service through curl or Node.
How can I restrict crawl scope?
Set page limits, maximum depth, and include or exclude filters. Keep subdomains and external links excluded unless explicitly needed.
Which output formats are documented?
HTML, raw HTML, Markdown, links, summaries, screenshots, and JSON are documented. Structured extraction supports a prompt and schema.
How are asynchronous results retrieved?
Start a crawl, record its identifier, and poll the result endpoint until completed or failed. Webhook configuration is also documented.
What security setting needs attention?
The guide documents skip_tls_verification as true by default. Set it to false to retain target certificate verification.
Developer Details
Author
xcrawl-apiLicense
MIT
Author version
v1.0.2
Skillstore revision
r1
Ref
e7474dc9298cb48291ba8e94ae55fd5e402b4b85
Maintenance freshness
10/17/2026
Usage
0 downloads · 0 views
File structure
📄 SKILL.md