# Design reliable paid-ad experiments

Paid-ad tests can produce misleading results when variables, thresholds, and decision ownership are unclear. This skill creates structured test plans and interprets exported results with separate statistical and practical-effect checks.

## Install

```bash
npx skillstore add aaron-he-zhu/ad-test-designer
```

## Metadata

- Status: approved
- Slug: aaron-he-zhu-ad-test-designer
- Version: 19.0.0
- Author version: 19.0.0
- Skillstore revision: r2
- Version status: valid
- Tree hash: 30231805b68f14822e1aa888920de89a185434b27033ac7b128977accf71e1e4
- Author: aaron-he-zhu
- GitHub username: aaron-he-zhu
- License: Apache-2.0
- Repository: https://github.com/aaron-he-zhu/aaron-marketing-skills/tree/main/ad/orchestrate/ad-test-designer
- Ref: adcb3549b15782055d0beb6d29f113d20de08f92
- Supported tools: Claude, Codex, Claude Code
- Audit status: complete
- Agent install advisory: allowed
- Manual install advisory: allowed
- Artifact signature: available
- Audit attestation: unavailable
- Human verification: not\_verified
- Risk factors: filesystem, external\_commands, network
- Quality score: 77
- Quality tier: bronze
- Public page: https://skillstore.pages.dev/skills/aaron-he-zhu-ad-test-designer
- Manifest: https://skillstore.pages.dev/api/skills/aaron-he-zhu-ad-test-designer/manifest

## Capabilities

- Creates a falsifiable hypothesis and a one-variable-per-variant test matrix.
- Plans primary, secondary, and guardrail metrics for paid-ad and landing-page tests.
- Estimates sample size, duration, alpha, power, and minimum detectable effect requirements.
- Interprets user-provided exposure and conversion exports with effect and uncertainty reporting.
- Separates statistical detection from practical business relevance.
- Returns an undecided result when no owner-approved action rule exists.

## Use Cases

- Plan a landing-page test: Define a controlled hero or call-to-action test with metrics, sample targets, duration, and guardrails.
- Assess creative test results: Read a finished creative experiment export and report effect size, uncertainty, and practical-effect status.
- Design an incrementality study: Structure a geo or audience holdout test with a comparable control and a longer measurement window.

## Prompt Templates

### Start a simple A/B test

```
Design an A/B test for two landing-page hero variants. Baseline conversion rate is 3%, and I need to detect a 15% lift.
```

### Plan a creative experiment

```
Create a test matrix for four responsive search ad variants. Include one changed variable per variant, a primary metric, guardrails, sample size, and duration.
```

### Read an exported result

```
Review my finished experiment results with sessions and conversions by variant. Report effect size, confidence interval, statistical status, and practical-effect status.
```

### Design an incrementality test

```
Design a geo-holdout incrementality test for a paid campaign. Define the treatment, matched holdout, decision owner, precommitted rule, metrics, and read schedule.
```

## Limitations

- It uses only the brief and exported data that the user provides.
- It does not create ad copy, creative assets, or landing-page variants.
- It does not connect to ad-platform APIs by default.
- It does not make business decisions without a precommitted owner and rule.

## Best Practices

- Change one variable per variant and retain a control or holdout.
- Precommit metrics, sample size, alpha, power, and stopping rules before reviewing results.
- Name the decision owner and action rule before a test starts.

## Anti Patterns

- Do not declare a winner from a p-value alone.
- Do not stop early after repeated unplanned result checks.
- Do not combine several creative or landing-page changes in one variant.

## Security Audit

- Audited at: 2026-07-26T09:54:00.944\+00:00
- Summary: All 35 static alerts are false positives caused by Markdown backticks, relative documentation links, metadata, or the decimal alpha notation. The sole documented command runs a fixed local statistical helper and contains no evidence of network access, arbitrary code execution, data exfiltration, or prompt injection. The skill also explicitly treats exported CSV content as untrusted data and requires consent before saving results.

## Stats

- Views: 0
- Downloads: 3
- Favorites: 0
- Popularity score: 0