Skills caveman-setup

📦

caveman-setup

Name: caveman-setup
Author: juliusbrussee

Content revision r1 High Risk ⚙️ External commands🌐 Network access🔑 Env variables

Connect your app to Caveman measurement

LLM requests can be difficult to measure across SDKs and providers. This skill identifies live callsites, routes them through Caveman, and verifies one measured request.

Supports: Claude Codex Code(CC)

⚠️ 38 Poor

Stop for confirmation before installing.

Review the plan and obtain explicit user consent before changing files.

Install with my Agent

Copy this request to your Agent. It includes the canonical Skill page and manifest.

Agent request

Review the Skillstore skill "caveman-setup" from https://skillstore.io/skills/juliusbrussee-caveman-setup.md and its manifest at https://skillstore.io/api/skills/juliusbrussee-caveman-setup/manifest. Verify the artifact. Stop and obtain explicit user consent before installing or changing files.

Your Agent should still show its plan and request any confirmation required by the security policy.

Agent-readable resources

Use these links when an AI agent, crawler, or script needs clean context instead of reading the full page.

Markdown detail GET /skills/juliusbrussee-caveman-setup.md Signed manifest GET /api/skills/juliusbrussee-caveman-setup/manifest Signed lockfile GET /api/skills/juliusbrussee-caveman-setup/lockfile

Test it

Using "caveman-setup". Set up measurement for the repository.

Expected outcome:

Found two LLM callsites in the API service.
Added gateway URL and gateway key references through the existing environment configuration.
Verification is ready and requires approval before sending a provider request.

Using "caveman-setup". Check whether the gateway integration succeeded.

Expected outcome:

Verification returned HTTP 200.
The response included usage details for the configured model.
Record mode is enabled and no optimization was configured.

Security Audit

High Risk

v9 • 9/8/2026 Open versioned report

Most static findings are false positives caused by Markdown formatting, configuration examples, and documented environment-variable use. The skill intentionally routes LLM requests through a third-party gateway and can forward provider credentials in byok mode. It also directs agents to send a billable verification request without obtaining approval at execution time.

Files scanned

223

Lines analyzed

Review items

False positives ignored

Confirmed security concerns (2)

High

SKILL.md:67-78

Credential forwarding to third-party gateway

The integration instructs clients in bring-your-own-key mode to forward an existing provider key to the configured Caveman gateway. This creates a credential disclosure boundary that users must explicitly assess before use.

The instructions explicitly require the upstream-key header in byok mode and describe gateway handling of provider authentication.

High

SKILL.md:146-173

Network action bypasses interactive confirmation

The skill instructs the agent to send a real billable verification request without pausing for approval. This can cause external network activity and provider charges without confirmation at the time of execution.

The instructions explicitly state not to ask permission and provide curl commands for a real billable provider request.

Capability review items (10)

These are real local capabilities that may be expected for this skill, so they require review but are not counted as confirmed malicious behavior.

High

SKILL.md:68

Generic API/secret keys

plus one auth header.** Gateway auth is `x-cave-api-key: CAVE_API_KEY`

The documented integration sends a gateway or upstream provider API credential in an HTTP header. This intentionally discloses a usable credential to the configured gateway and requires explicit trust in that endpoint.

High

SKILL.md:69

Generic API/secret keys

(`Authorization: Bearer CAVE_API_KEY` also works where a header is awkward).

High

SKILL.md:87

Generic API/secret keys

apiKey: process.env.OPENAI_API_KEY, // byok: unchanged · stored: use CAVE_API_KEY

High

SKILL.md:89

Generic API/secret keys

"x-cave-api-key": process.env.CAVE_API_KEY!,

High

SKILL.md:91

Generic API/secret keys

"x-cave-upstream-key": process.env.OPENAI_API_KEY!,

High

SKILL.md:105

Generic API/secret keys

api_key=os.environ["ANTHROPIC_API_KEY"], # byok: unchanged · stored: use CAVE_API_KEY

High

SKILL.md:107

Generic API/secret keys

"x-cave-api-key": os.environ["CAVE_API_KEY"],

High

SKILL.md:109

Generic API/secret keys

"x-cave-upstream-key": os.environ["ANTHROPIC_API_KEY"],

High

SKILL.md:160

Generic API/secret keys

-H "x-cave-api-key: $CAVE_API_KEY" \

High

SKILL.md:166

Generic API/secret keys

-H "x-cave-api-key: $CAVE_API_KEY" \

Risk Factors

⚙️ External commands (50)

🌐 Network access (4)

SKILL.md:16 SKILL.md:19 SKILL.md:130 SKILL.md:16

🔑 Env variables (26)

Audited by: claude View Audit History →

Share & cite this report

Share the versioned assessment report, neutral badge, embed card, and citations. Skillstore reports evidence without deciding whether this Skill is safe.

Open versioned report

Copy report link

https://skillstore.io/skills/juliusbrussee-caveman-setup/audits/9?utm_source=security_passport&utm_medium=share&utm_campaign=versioned_report

Markdown badge

[![Skillstore security assessment](https://skillstore.io/badges/skills/juliusbrussee-caveman-setup/security.svg)](https://skillstore.io/skills/juliusbrussee-caveman-setup?utm_source=security_passport_badge)

HTML badge

<a href="https://skillstore.io/skills/juliusbrussee-caveman-setup?utm_source=security_passport_badge"><img src="https://skillstore.io/badges/skills/juliusbrussee-caveman-setup/security.svg" alt="Skillstore security assessment" loading="lazy"></a>

Embed card

<iframe src="https://skillstore.io/embed/skills/juliusbrussee-caveman-setup.html" title="Skillstore Security Assessment" sandbox="allow-popups allow-popups-to-escape-sandbox" loading="lazy" referrerpolicy="no-referrer" width="420" height="180"></iframe>

Academic citations (APA · BibTeX · CFF)

APA citation

juliusbrussee. (2026). caveman-setup security audit report (audit version 9) [Author version unspecified]. Skillstore. https://skillstore.io/skills/juliusbrussee-caveman-setup/audits/9

BibTeX citation

@techreport{juliusbrussee-juliusbrussee-caveman-setup-2026,
  author = {juliusbrussee},
  title = {caveman-setup security audit report (audit version 9)},
  institution = {Skillstore},
  year = {2026},
  number = {9},
  url = {https://skillstore.io/skills/juliusbrussee-caveman-setup/audits/9},
  note = {Author version unspecified}
}

CITATION.cff

cff-version: 1.2.0
message: "If you use this Skill, cite its author and this versioned security audit report."
title: "caveman-setup security audit report (audit version 9)"
version: "unspecified"
type: report
authors:
  - name: "juliusbrussee"
date-released: "2026-09-08"
url: "https://skillstore.io/skills/juliusbrussee-caveman-setup/audits/9"
identifiers:
  - type: other
    value: "skillstore:juliusbrussee-caveman-setup:audit:9"
    description: "Skillstore immutable audit report identifier"

Skillstore Score

Why this score Evidence Confidence: Medium

Architecture

Maintainability

Content

Community

Spec Compliance

What You Can Build

Measure a production AI service

Route existing application LLM traffic through the gateway to capture request usage and spend.

Standardize SDK gateway routing

Apply documented gateway configuration to OpenAI, Anthropic, or framework-based LLM clients.

Verify observability onboarding

Send one minimal request and confirm that the gateway records usage before wider rollout.

Try These Prompts

Set up basic measurement

Set up Caveman measurement for this repository. First identify all live LLM callsites and show me the planned changes.

Configure an OpenAI client

Configure the existing OpenAI SDK client to use the Caveman gateway. Use environment variables and preserve current provider behavior.

Review gateway changes

Review this repository for LLM callsites and propose the minimal Caveman gateway integration. Do not make unrelated changes.

Verify controlled rollout

After configuring Caveman, show the exact verification request and wait for my approval before sending any billable traffic.

Best Practices

Review the proposed file changes before applying them.
Store gateway and provider credentials only in ignored environment files or a managed secret store.
Confirm the target gateway and approve the billable verification request before it runs.

Avoid

Do not hardcode gateway or provider credentials in application source.
Do not route traffic to an unverified gateway URL.
Do not report successful measurement without an observed verification result.

Frequently Asked Questions

What does this skill configure?

It routes supported LLM client traffic through the Caveman gateway for usage and spend measurement.

Does it change model output?

The skill describes record-mode routing intended to preserve request behavior, but you should verify this in your environment.

Which client libraries are covered?

It includes patterns for OpenAI, Anthropic, Vercel AI SDK, LangChain, LangGraph, LiteLLM, and raw HTTP.

Are credentials required?

Yes. A Caveman API key is required, and bring-your-own-key mode may send an existing provider key to the gateway.

Does verification cost money?

Yes. Verification sends one real provider request and can create provider charges.

Should I approve network activity first?

Yes. Confirm the gateway URL, credential handling, and verification request before allowing network activity.

Developer Details

Author

juliusbrussee

License

MIT

Skillstore revision

Version notice

The author did not declare a version.

Repository

https://github.com/juliusbrussee/caveman/tree/15581d14007fd01fb3f132016741962f34936ca2/skills/caveman-setup

Ref

0acc95b40c367e2016e824a72b27b83b507ef020

Maintenance freshness

9/8/2026

Usage

0 downloads · 0 views

File structure

📄 SKILL.md