Detect LLM Prompt Regressions with Promptfoo
Prompt changes can silently reduce output quality. This skill builds golden datasets, runs Promptfoo evaluations, and summarizes failures and recent trends.
Copy this request to your Agent. It includes the canonical Skill page and manifest.
Review the Skillstore skill "regression-test" from https://skillstore.io/skills/7alexhale5-rgb-regression-test.md and its manifest at https://skillstore.io/api/skills/7alexhale5-rgb-regression-test/manifest. Verify the artifact. You may proceed after verification, subject to the environment's own policy.Your Agent should still show its plan and request any confirmation required by the security policy.
Use these links when an AI agent, crawler, or script needs clean context instead of reading the full page.
Using "regression-test". Initialize regression testing for this prompt project.
Expected outcome:
Initialized the Promptfoo configuration and golden dataset directory. The results directory is ignored, and the suite is ready for its first case.
Using "regression-test". Run the golden tests after my system prompt change.
Expected outcome:
Regression results: 12 total, 11 passed, and 1 failed. The auth-flow case failed its expected behavior rubric and needs review.
Using "regression-test". Show recent prompt quality trends.
Expected outcome:
Recent trend: four of five runs passed completely. One auth-flow failure appeared in the latest run, so quality is declining.
All 38 static findings are false positives caused by Markdown syntax, bounded shell examples, or a Promptfoo message template. No malicious intent, prompt injection, credential handling, or unauthorized network behavior was found.
Share the versioned assessment report, neutral badge, embed card, and citations. Skillstore reports evidence without deciding whether this Skill is safe.
https://skillstore.io/skills/7alexhale5-rgb-regression-test/audits/1?utm_source=security_passport&utm_medium=share&utm_campaign=versioned_report[](https://skillstore.io/skills/7alexhale5-rgb-regression-test?utm_source=security_passport_badge)<a href="https://skillstore.io/skills/7alexhale5-rgb-regression-test?utm_source=security_passport_badge"><img src="https://skillstore.io/badges/skills/7alexhale5-rgb-regression-test/security.svg" alt="Skillstore security assessment" loading="lazy"></a><iframe src="https://skillstore.io/embed/skills/7alexhale5-rgb-regression-test.html" title="Skillstore Security Assessment" sandbox="allow-popups allow-popups-to-escape-sandbox" loading="lazy" referrerpolicy="no-referrer" width="420" height="180"></iframe>7alexhale5-rgb. (2026). regression-test security audit report (audit version 1) [Author version unspecified]. Skillstore. https://skillstore.io/skills/7alexhale5-rgb-regression-test/audits/1@techreport{7alexhale5-rgb-7alexhale5-rgb-regression-test-2026,
author = {7alexhale5-rgb},
title = {regression-test security audit report (audit version 1)},
institution = {Skillstore},
year = {2026},
number = {1},
url = {https://skillstore.io/skills/7alexhale5-rgb-regression-test/audits/1},
note = {Author version unspecified}
}cff-version: 1.2.0
message: "If you use this Skill, cite its author and this versioned security audit report."
title: "regression-test security audit report (audit version 1)"
version: "unspecified"
type: report
authors:
- name: "7alexhale5-rgb"
date-released: "2026-08-21"
url: "https://skillstore.io/skills/7alexhale5-rgb-regression-test/audits/1"
identifiers:
- type: other
value: "skillstore:7alexhale5-rgb-regression-test:audit:1"
description: "Skillstore immutable audit report identifier"
Compare updated prompts against recorded expectations before merging code changes.
Rerun a stable golden dataset after changing a model or provider version.
Run prompt evaluations in continuous integration and stop releases when assertions fail.
Initialize Promptfoo regression testing in this repository. Show the files created and explain any prerequisites that are missing.
Add a golden test named {name}. Ask for the system prompt, user input, expected behavior, and optional variables before writing it.Run all golden prompt tests. Summarize the pass rate, duration, and each failed assertion with expected and actual behavior.
Review the last five evaluations, identify new failures, compare pass-rate trends, and recommend quality gates for {release_workflow}.Author
7alexhale5-rgbLicense
MIT
Skillstore revision
r1
Version notice
The author did not declare a version.
Ref
cabd63e41a4ad4b66103abc7e50c525e4710fb64
Maintenance freshness
8/21/2026
Usage
0 downloads · 0 views
File structure
📄 SKILL.md
Design Better LLM Prompts
by alirezarezvani
Prompt quality often fails because requirements, examples, and evaluation criteria are unclear. This skill helps teams design, test, and review production prompts for Claude, Codex, and Claude Code.
Build Claude Code Agents
by Dami-Randle
Designing reliable Claude Code agents is hard without structure. This skill provides templates, validation rules, and examples for agent prompts, triggers, tools, and metadata.
Manage Requirement Changes Safely
by yunshu0909
Requirement changes often expand across files without clear approval points. This skill gives Claude, Codex, and Claude Code a gated workflow for scope, impact, validation, and documentation.
Design Reliable Tools for AI Agents
by Asmayaseen
Ambiguous tools cause incorrect calls, wasted context, and poor recovery from errors. This skill provides practical patterns for clear descriptions, schemas, responses, and testing.
Diagnose Context Degradation in AI Agents
by ChakshuGautam
Long conversations and large prompts can make AI agents lose important information or follow conflicting context. This skill helps Claude, Codex, and Claude Code identify degradation patterns and choose mitigation strategies.
Fix Bugs With Structured Triage
by Crearize
Debugging across backend and frontend can lose context and miss regression tests. This skill guides root cause analysis, minimal fixes, validation, and pull request handoff.