# Design Production ML Pipelines

Production ML workflows often fail because data, training, validation, deployment, and monitoring are designed separately. This skill provides a structured lifecycle and orchestration guidance.

## Install

```bash
npx skillstore add sickn33/ml-pipeline-workflow
```

## Metadata

- Status: approved
- Slug: sickn33-ml-pipeline-workflow
- Skillstore revision: r2
- Version status: missing
- Tree hash: 5c22d2927dad58f331497a7b342593ae55e4eb8af8c8a35a5c17c23c6ef93de7
- Author: sickn33
- GitHub username: sickn33
- License: MIT
- Repository: https://github.com/sickn33/antigravity-awesome-skills/tree/main/skills/ml-pipeline-workflow
- Ref: 81e05e636292629114b76cbb3922fbe57672fc02
- Supported tools: Claude, Codex, Claude Code
- Audit status: complete
- Agent install advisory: allowed
- Manual install advisory: allowed
- Artifact signature: available
- Audit attestation: unavailable
- Human verification: not\_verified
- Risk factors: external\_commands
- Quality score: 78
- Quality tier: bronze
- Public page: https://skillstore.pages.dev/skills/sickn33-ml-pipeline-workflow
- Manifest: https://skillstore.pages.dev/api/skills/sickn33-ml-pipeline-workflow/manifest

## Capabilities

- Designs lifecycle stages from data ingestion through deployment and monitoring.
- Explains DAG patterns for Airflow, Dagster, Kubeflow, and Prefect.
- Structures data validation, feature engineering, versioning, and lineage practices.
- Defines model validation gates, baseline comparisons, and regression checks.
- Recommends canary, blue-green, shadow, and rollback deployment strategies.
- Provides troubleshooting steps for pipeline, training, deployment, and performance failures.

## Use Cases

- Plan a New ML Platform: Map lifecycle stages, dependencies, validation gates, and deployment controls before implementation.
- Operationalize a Research Model: Convert an experimental training process into a reproducible workflow with tracking, validation, and rollback.
- Review Pipeline Reliability: Assess an existing workflow for observability, idempotency, lineage, retries, and production monitoring.

## Prompt Templates

### Outline a Basic Pipeline

```
Design a simple ML pipeline for [use case]. Include ingestion, validation, training, evaluation, deployment, and monitoring stages.
```

### Choose an Orchestrator

```
Compare Airflow, Dagster, Kubeflow, and Prefect for [environment]. Recommend one based on scale, infrastructure, team skills, and operational needs.
```

### Add Production Controls

```
Review this pipeline design: [design]. Add data checks, lineage, experiment tracking, approval gates, retries, observability, and rollback controls.
```

### Design Continuous Training

```
Create a continuous training architecture for [model]. Define triggers, data validation, comparison gates, canary rollout, drift monitoring, and automated rollback.
```

## Limitations

- Provides guidance and examples but does not execute pipelines or deployments.
- Does not include the referenced resources, assets, or template directories.
- Requires environment-specific choices for cloud services, credentials, storage, and compute.
- Does not replace production testing, security review, or model governance.

## Best Practices

- Version data, code, configuration, and model artifacts together for reproducibility.
- Make every stage idempotent, independently testable, observable, and retry-aware.
- Use explicit validation and approval gates before production deployment.

## Anti Patterns

- Do not deploy a newly trained model without baseline comparison and rollback planning.
- Do not hide data transformations inside training jobs without lineage or versioning.
- Do not automate retraining from drift alerts without validating data and alert quality.

## Security Audit

- Audited at: 2026-08-04T14:32:44.895\+00:00
- Summary: All ten static findings are false positives caused by Markdown inline code markers and fenced code blocks. The reviewed content contains guidance and declarative examples, not shell execution.

## Stats

- Views: 112
- Downloads: 11
- Favorites: 0
- Popularity score: 0