# Improve Existing AI Agents Systematically

Agent changes often lack reliable evidence, testing, and rollout controls. This skill provides a measured workflow for analysis, prompt refinement, evaluation, deployment, and monitoring.

## Install

```bash
npx skillstore add sickn33/agent-orchestration-improve-agent
```

## Metadata

- Status: approved
- Slug: sickn33-agent-orchestration-improve-agent
- Skillstore revision: r2
- Version status: missing
- Tree hash: cce5b8592581404cb4f52909a40a5202973786b34cba698fc9766af7245139cd
- Author: sickn33
- GitHub username: sickn33
- License: MIT
- Repository: https://github.com/sickn33/antigravity-awesome-skills/tree/main/skills/agent-orchestration-improve-agent
- Ref: ebdfe608f5de2b66ff37ab4af12af8ac4f5e8006
- Supported tools: Claude, Codex, Claude Code
- Audit status: complete
- Agent install advisory: allowed
- Manual install advisory: allowed
- Artifact signature: available
- Audit attestation: unavailable
- Human verification: not\_verified
- Risk factors: external\_commands
- Quality score: 69
- Public page: https://skillstore.pages.dev/skills/sickn33-agent-orchestration-improve-agent
- Manifest: https://skillstore.pages.dev/api/skills/sickn33-agent-orchestration-improve-agent/manifest

## Capabilities

- Defines baseline metrics for task success, accuracy, latency, token use, and user satisfaction.
- Classifies failures by instruction, context, format, tool, constraint, and edge-case causes.
- Guides targeted prompt revisions using examples, role definitions, self-checks, and output formats.
- Designs representative test suites and controlled A/B comparisons with acceptance metrics.
- Plans versioned rollouts with monitoring, rollback triggers, and post-deployment reviews.
- Establishes weekly, monthly, quarterly, and annual improvement cycles.

## Use Cases

- Improve Support Agent Reliability: Analyze recurring corrections, refine response rules, and test revisions before a controlled customer rollout.
- Reduce Agent Cost and Latency: Compare prompt variants against token use, response time, task success, and quality thresholds.
- Build an Agent Evaluation Program: Create test categories, review rubrics, version records, rollout gates, and recurring performance reviews.

## Prompt Templates

### Review an Agent Baseline

```
Analyze this agent using the supplied metrics and examples. Identify the three highest-impact failure modes and define measurable baseline indicators.
```

### Plan Prompt Improvements

```
Review the agent prompt, failure examples, and constraints. Propose targeted revisions, expected effects, and regression tests for each change.
```

### Design a Controlled Evaluation

```
Design an A/B evaluation for the original and revised agents using [test set]. Define metrics, sample allocation, acceptance thresholds, and review steps.
```

### Create a Production Optimization Cycle

```
Create an end-to-end optimization plan for [agent]. Include baseline analysis, prompt changes, adversarial tests, staged rollout, monitoring, rollback triggers, and a 30-day review.
```

## Limitations

- Requires representative metrics, feedback, or test cases from an existing agent.
- Does not implement evaluation tools, dashboards, deployment systems, or data pipelines.
- Suggested thresholds need adjustment for each environment, risk profile, and traffic level.
- Human review remains necessary for subjective quality, safety, and production decisions.

## Best Practices

- Change one major behavior at a time so evaluation results remain attributable.
- Use representative failures, edge cases, and adversarial inputs in every regression suite.
- Define acceptance and rollback thresholds before exposing users to a revised agent.

## Anti Patterns

- Do not deploy prompt changes based only on a few favorable examples.
- Do not compare variants with different tasks, rubrics, or evaluator instructions.
- Do not collect user interaction data without authorization, redaction, and retention controls.

## Security Audit

- Audited at: 2026-07-23T20:23:33.185\+00:00
- Summary: All 20 static findings are false positives caused by Markdown fences or benign metric text, with no executable Ruby or shell commands present. The skill does introduce moderate privacy and disclosure risks by requesting historical interaction analysis and visible reasoning traces without handling safeguards.

## Stats

- Views: 102
- Downloads: 10
- Favorites: 1
- Popularity score: 0