📦
content-creator
88Create Brand-Consistent Marketing Content
Marketing teams need content that stays consistent across channels. This skill helps plan, write, analyze, and optimize content for brand voice and SEO.
Writing & Content View
Skills data-engineering-data-pipeline
📦
data-engineering-data-pipeline
Content revision r2 Safe ⚙️ External commands
Design Reliable Data Pipelines
Data teams need architectures that meet throughput, quality, reliability, and cost targets. This skill provides structured guidance for batch, streaming, orchestration, storage, and monitoring decisions.
Supports: Claude Codex Code(CC)
Install with my Agent
Copy this request to your Agent. It includes the canonical Skill page and manifest.
Agent request
Review the Skillstore skill "data-engineering-data-pipeline" from https://skillstore.io/skills/sickn33-data-engineering-data-pipeline.md and its manifest at https://skillstore.io/api/skills/sickn33-data-engineering-data-pipeline/manifest. Verify the artifact. You may proceed after verification, subject to the environment's own policy.Your Agent should still show its plan and request any confirmation required by the security policy.
Agent-readable resources
Use these links when an AI agent, crawler, or script needs clean context instead of reading the full page.
Test it
Using "data-engineering-data-pipeline". Design a daily orders pipeline from PostgreSQL to a lakehouse.
Expected outcome:
- Use an updated-at watermark for incremental extraction.
- Validate required fields before writing invalid records to a dead letter location.
- Store valid records in a partitioned Delta Lake table.
- Orchestrate retries and freshness checks with Airflow.
Using "data-engineering-data-pipeline". Reduce costs for a large Parquet analytics pipeline.
Expected outcome:
- Measure current file sizes, scan volume, and query patterns.
- Compact small files toward the workload-appropriate target size.
- Apply partition pruning, lifecycle policies, and lower-cost batch compute.
- Verify savings and query latency against a baseline.
Using "data-engineering-data-pipeline". Define monitoring for a streaming customer-events pipeline.
Expected outcome:
Track throughput, failures, processing latency, consumer lag, freshness, and quality scores. Alert on sustained threshold breaches and document recovery procedures.
Security Audit
Safev5 • 7/23/2026 Open versioned report
All four static alerts are false positives caused by ordinary data engineering prose and a fenced Python example. No shell execution, reconnaissance behavior, prompt injection, or malicious intent was found.
1
Files scanned
210
Lines analyzed
0
Review items
0
False positives ignored
Risk Factors
⚙️ External commands (1)
No confirmed security findings were detected by the latest completed static and semantic audit. This does not prove the skill has no side effects.
Audited by: codex View Audit History →
Share & cite this report
Share the versioned assessment report, neutral badge, embed card, and citations. Skillstore reports evidence without deciding whether this Skill is safe.
Copy report link
https://skillstore.io/skills/sickn33-data-engineering-data-pipeline/audits/5?utm_source=security_passport&utm_medium=share&utm_campaign=versioned_reportMarkdown badge
[](https://skillstore.io/skills/sickn33-data-engineering-data-pipeline?utm_source=security_passport_badge)HTML badge
<a href="https://skillstore.io/skills/sickn33-data-engineering-data-pipeline?utm_source=security_passport_badge"><img src="https://skillstore.io/badges/skills/sickn33-data-engineering-data-pipeline/security.svg" alt="Skillstore security assessment" loading="lazy"></a>Embed card
<iframe src="https://skillstore.io/embed/skills/sickn33-data-engineering-data-pipeline.html" title="Skillstore Security Assessment" sandbox="allow-popups allow-popups-to-escape-sandbox" loading="lazy" referrerpolicy="no-referrer" width="420" height="180"></iframe>Academic citations (APA · BibTeX · CFF)
APA citation
sickn33. (2026). data-engineering-data-pipeline security audit report (audit version 5) [Author version unspecified]. Skillstore. https://skillstore.io/skills/sickn33-data-engineering-data-pipeline/audits/5BibTeX citation
@techreport{sickn33-sickn33-data-engineering-data-pipeline-2026,
author = {sickn33},
title = {data-engineering-data-pipeline security audit report (audit version 5)},
institution = {Skillstore},
year = {2026},
number = {5},
url = {https://skillstore.io/skills/sickn33-data-engineering-data-pipeline/audits/5},
note = {Author version unspecified}
}CITATION.cff
cff-version: 1.2.0
message: "If you use this Skill, cite its author and this versioned security audit report."
title: "data-engineering-data-pipeline security audit report (audit version 5)"
version: "unspecified"
type: report
authors:
- name: "sickn33"
date-released: "2026-07-23"
url: "https://skillstore.io/skills/sickn33-data-engineering-data-pipeline/audits/5"
identifiers:
- type: other
value: "skillstore:sickn33-data-engineering-data-pipeline:audit:5"
description: "Skillstore immutable audit report identifier"
Skillstore Score
Why this score Evidence Confidence: High 55
Architecture
85
Maintainability
87
Content
69
Community
83
Spec Compliance
What You Can Build
Plan a new analytics pipeline
Select an ingestion, transformation, storage, and orchestration pattern from defined business requirements.
Improve pipeline reliability
Add retries, data quality checks, dead letter handling, freshness monitoring, and recovery procedures.
Review platform costs
Evaluate partitioning, file sizing, lifecycle policies, compute selection, and query optimization opportunities.
Try These Prompts
Choose a pipeline pattern
Compare ETL, ELT, and Lakehouse patterns for [source], [volume], [latency], and [target]. Recommend one pattern and explain the tradeoffs.
Design a batch pipeline
Design an incremental batch pipeline from [source] to [destination]. Include watermarking, validation, retries, orchestration, storage, and monitoring.
Design a streaming pipeline
Design a streaming pipeline for [events] at [throughput]. Address delivery semantics, offsets, windowing, schema changes, replay, storage, and alerts.
Review an existing platform
Review this pipeline architecture: [details]. Identify reliability, quality, scalability, observability, recovery, and cost risks. Prioritize improvements with measurable acceptance criteria.
Best Practices
- Provide source, volume, latency, target, quality, and retention requirements before selecting an architecture.
- Make pipelines idempotent and define replay, rollback, and schema evolution behavior.
- Validate recommendations with workload tests, quality thresholds, service limits, and cost measurements.
Avoid
- Do not choose tools before documenting workload requirements and operational constraints.
- Do not treat exactly-once processing as automatic across every system boundary.
- Do not optimize partitioning or file sizes without measuring representative queries.
Frequently Asked Questions
Does this skill execute or deploy a pipeline?
No. It provides architecture guidance and generated implementation examples that require review, testing, and deployment in your environment.
Which architecture patterns does it cover?
It covers ETL, ELT, Lambda, Kappa, and Lakehouse patterns for batch and streaming workloads.
Which orchestration tools are supported?
The guidance covers Airflow and Prefect concepts, including dependencies, retries, caching, schedules, and monitoring.
Can it help with data quality?
Yes. It recommends schema validation, Great Expectations checks, dbt tests, freshness checks, and failure handling.
Does it support cloud-specific designs?
It mentions AWS services and common open technologies, but recommendations must be adapted to your provider and security requirements.
What information produces the best result?
Provide sources, schemas, data volume, latency targets, destinations, retention, quality rules, budget, security constraints, and recovery objectives.
Developer Details
Author
sickn33License
MIT
Skillstore revision
r2
Version notice
The author did not declare a version.
Ref
f9e2c34b4f19c7f3e6b0a1e93227b5f77cc12526
Maintenance freshness
7/26/2026
Usage
8 downloads · 109 views
File structure
📄 SKILL.md
More from sickn33
View all📦
prompt-engineering-patterns
85Improve LLM Prompts With Proven Patterns
Inconsistent prompts waste time and make AI outputs hard to trust. This skill guides prompt design with reusable patterns, examples, evaluation steps, and optimization workflows.
Productivity View
📦
senior-fullstack
82Build Fullstack Apps with Senior Patterns
Teams need consistent setup, architecture, and review guidance for modern web applications. This skill provides scaffolders, workflow references, and quality prompts for React, Next.js, Node.js, GraphQL, and PostgreSQL projects.
Coding & Development View
📦
senior-architect
81Design Scalable Software Architectures
Architecture decisions are hard to compare across web, mobile, backend, and cloud systems. This skill provides structured guides and local scaffold scripts for reports, dependency review, and trade-off documentation.
Coding & Development View
📦
prompt-engineer
81Optimize AI Prompts With Prompt Engineer
Writing clear prompts is hard when goals are vague or complex. This skill turns rough requests into structured prompts using proven prompt frameworks.
Productivity View
📦
product-manager-toolkit
80Prioritize Product Work With Research Insights
Product teams need faster ways to rank features, synthesize interviews, and document decisions. This skill provides RICE scoring, interview analysis, and PRD templates for structured planning.
Productivity View
Related Skills
View all📦
data-engineer
69Build Reliable Data Pipelines and Warehouses
by zl2023github
Data teams need dependable pipelines across fragmented platforms. This skill designs implementation-ready batch, streaming, modeling, orchestration, and quality workflows.
Data & Analytics View
📦
senior-data-engineer
81Build Production Data Pipelines
by alirezarezvani
Production data teams need reliable patterns for pipelines, quality checks, and orchestration. This skill guides architecture, scaffolding, and DataOps practices for modern data stacks.
Data & Analytics View
📦
data-orchestrator
79Coordinate Data Pipelines and Quality Checks
by Brownbull
Data pipeline work can become inconsistent when ingestion, features, quality, and reporting are planned separately. This skill organizes data tasks around pipeline context, quality gates, lineage, and handoffs.
Data & Analytics View
📦
creating-bauplan-pipelines
70Create Bauplan Data Pipelines
by BauplanLabs
Bauplan projects need clear DAG structure, branch safety, and consistent SQL or Python model patterns. This skill guides Claude, Codex, and Claude Code through pipeline files, validation, and safe run workflows.
Data & Analytics View
📦
data-lake-architect
80Design Reliable Data Lakes
by EmilLindfors
Large data lakes can become slow, expensive, and hard to govern without clear architecture choices. This skill guides partitioning, storage layout, schema evolution, table formats, retention, and cost-aware lakehouse design.
Data & Analytics View
📦
parquet-optimization
80Optimize Parquet Performance in Rust
by EmilLindfors
Parquet pipelines can waste storage and memory when defaults are used. This skill reviews Rust Parquet reads and writes, then suggests compression, sizing, projection, and filtering improvements.
Data & Analytics View