# Build Reliable Apache Airflow DAGs

Airflow pipelines often fail because dependencies, retries, sensors, and tests are designed inconsistently. This skill provides reusable patterns for production DAG design and validation.

## Install

```bash
npx skillstore add sickn33/airflow-dag-patterns
```

## Metadata

- Status: approved
- Slug: sickn33-airflow-dag-patterns
- Skillstore revision: r2
- Version status: missing
- Tree hash: 4b20de94d1a6ec73ad3bb5014ec985143032f0a9cf9e57a080479dc44397abee
- Author: sickn33
- GitHub username: sickn33
- License: MIT
- Repository: https://github.com/sickn33/antigravity-awesome-skills/tree/main/skills/airflow-dag-patterns
- Ref: ebdfe608f5de2b66ff37ab4af12af8ac4f5e8006
- Supported tools: Claude, Codex, Claude Code
- Audit status: complete
- Agent install advisory: allowed
- Manual install advisory: allowed
- Artifact signature: available
- Audit attestation: unavailable
- Human verification: not\_verified
- Risk factors: scripts, network, external\_commands
- Quality score: 78
- Quality tier: bronze
- Public page: https://skillstore.pages.dev/skills/sickn33-airflow-dag-patterns
- Manifest: https://skillstore.pages.dev/api/skills/sickn33-airflow-dag-patterns/manifest

## Capabilities

- Designs linear, branching, fan-out, and fan-in task dependencies.
- Creates TaskFlow API workflows with typed task inputs and outputs.
- Builds dynamic DAG factories from pipeline configuration.
- Configures file, S3, external DAG, and API sensors.
- Adds retry, failure callback, alerting, and trigger rule patterns.
- Provides DagBag tests for loading, structure, dependencies, and cycles.

## Use Cases

- Create a New Data Pipeline: Design an idempotent DAG with clear dependencies, retries, schedules, and observability.
- Standardize Airflow Operations: Define shared sensor, alerting, testing, and deployment patterns across DAG repositories.
- Validate Analytics Workflows: Review task order, data quality branches, catchup behavior, and safe retry handling.

## Prompt Templates

### Design a Basic DAG

```
Design a daily Airflow DAG for [source] to [target]. Include task dependencies, retries, ownership, start date, and catchup behavior.
```

### Add Sensors and Alerts

```
Extend my DAG with a [dependency type] sensor and failure alerts. Use efficient sensor settings, timeouts, retry backoff, and clear trigger rules.
```

### Build Dynamic DAGs

```
Create a dynamic DAG factory for these pipelines: [configurations]. Validate identifiers, schedules, source settings, shared defaults, and deterministic generation.
```

### Review Production Readiness

```
Review this Airflow design: [design summary]. Identify risks involving idempotency, XCom size, concurrency, backfills, sensors, testing, observability, and deployment.
```

## Limitations

- Does not deploy DAGs or modify an Airflow environment.
- Requires environment-specific connection identifiers, credentials, endpoints, and schedules.
- Examples need adaptation for installed Airflow and provider versions.
- Does not replace staging tests, backfill planning, or production approval.

## Best Practices

- Keep tasks idempotent, atomic, observable, and safe to retry.
- Use reschedule mode and explicit timeouts for long-running sensors.
- Test DAG loading, dependencies, branching, schedules, and cycles before deployment.

## Anti Patterns

- Do not pass large datasets through XCom or keep mutable global task state.
- Do not place expensive processing or network calls at DAG import time.
- Do not change production schedules or run backfills without impact review and approval.

## Security Audit

- Audited at: 2026-07-23T18:54:45.767\+00:00
- Summary: All ten static alerts are false positives caused by documentation examples, ordinary Python syntax, or Markdown backticks. No malicious intent, prompt injection, credential access, or data exfiltration was found.

## Stats

- Views: 132
- Downloads: 8
- Favorites: 0
- Popularity score: 0