Skills data-lake-architect

📦

data-lake-architect

Name: data-lake-architect
Author: EmilLindfors

v1.0.0 Content revision r1 Safe ⚙️ External commands

Design Reliable Data Lakes

Large data lakes can become slow, expensive, and hard to govern without clear architecture choices. This skill guides partitioning, storage layout, schema evolution, table formats, retention, and cost-aware lakehouse design.

Supports: Claude Codex Code(CC)

🥈 80 Silver

Install with my Agent

Copy this request to your Agent. It includes the canonical Skill page and manifest.

Agent request

Review the Skillstore skill "data-lake-architect" from https://skillstore.io/skills/emillindfors-data-lake-architect.md and its manifest at https://skillstore.io/api/skills/emillindfors-data-lake-architect/manifest. Verify the artifact. You may proceed after verification, subject to the environment's own policy.

Your Agent should still show its plan and request any confirmation required by the security policy.

Agent-readable resources

Use these links when an AI agent, crawler, or script needs clean context instead of reading the full page.

Markdown detail GET /skills/emillindfors-data-lake-architect.md Signed manifest GET /api/skills/emillindfors-data-lake-architect/manifest Signed lockfile GET /api/skills/emillindfors-data-lake-architect/lockfile

Test it

Using "data-lake-architect". We ingest 500 GB of events daily and query by date and event type.

Expected outcome:

Use daily time partitions first, then consider event type only if cardinality is controlled and filters are consistent. Target 100 MB to 1 GB files and monitor small file growth.

Using "data-lake-architect". Our schema changes often and several services write to the same table.

Expected outcome:

Prefer Iceberg because it supports schema evolution, concurrent writes, snapshot isolation, and partition evolution. Budget for metadata operations and operational learning.

Using "data-lake-architect". Our data lake has many tiny files and slow queries.

Expected outcome:

Prioritize compaction, larger writer batches, and partition simplification. Check whether hourly partitions can become daily partitions without hurting common queries.

Security Audit

Safe

v4 • 7/6/2026 Open versioned report

Static analysis flagged Markdown fences and data modeling terms as command execution or reconnaissance. Review of SKILL.md found read-only architecture guidance, no executable scripts, no prompt injection, and no data exfiltration intent.

Files scanned

551

Lines analyzed

Review items

False positives ignored

Risk Factors

No confirmed security findings were detected by the latest completed static and semantic audit. This does not prove the skill has no side effects.

Audited by: codex View Audit History →

Share & cite this report

Share the versioned assessment report, neutral badge, embed card, and citations. Skillstore reports evidence without deciding whether this Skill is safe.

Open versioned report

Copy report link

https://skillstore.io/skills/emillindfors-data-lake-architect/audits/4?utm_source=security_passport&utm_medium=share&utm_campaign=versioned_report

Markdown badge

[![Skillstore security assessment](https://skillstore.io/badges/skills/emillindfors-data-lake-architect/security.svg)](https://skillstore.io/skills/emillindfors-data-lake-architect?utm_source=security_passport_badge)

HTML badge

<a href="https://skillstore.io/skills/emillindfors-data-lake-architect?utm_source=security_passport_badge"><img src="https://skillstore.io/badges/skills/emillindfors-data-lake-architect/security.svg" alt="Skillstore security assessment" loading="lazy"></a>

Embed card

<iframe src="https://skillstore.io/embed/skills/emillindfors-data-lake-architect.html" title="Skillstore Security Assessment" sandbox="allow-popups allow-popups-to-escape-sandbox" loading="lazy" referrerpolicy="no-referrer" width="420" height="180"></iframe>

Academic citations (APA · BibTeX · CFF)

APA citation

EmilLindfors. (2026). data-lake-architect security audit report (audit version 4) [Author version 1.0.0]. Skillstore. https://skillstore.io/skills/emillindfors-data-lake-architect/audits/4

BibTeX citation

@techreport{emillindfors-emillindfors-data-lake-architect-2026,
  author = {EmilLindfors},
  title = {data-lake-architect security audit report (audit version 4)},
  institution = {Skillstore},
  year = {2026},
  number = {4},
  url = {https://skillstore.io/skills/emillindfors-data-lake-architect/audits/4},
  note = {Author version 1.0.0}
}

CITATION.cff

cff-version: 1.2.0
message: "If you use this Skill, cite its author and this versioned security audit report."
title: "data-lake-architect security audit report (audit version 4)"
version: "1.0.0"
type: report
authors:
  - name: "EmilLindfors"
date-released: "2026-07-06"
url: "https://skillstore.io/skills/emillindfors-data-lake-architect/audits/4"
identifiers:
  - type: other
    value: "skillstore:emillindfors-data-lake-architect:audit:4"
    description: "Skillstore immutable audit report identifier"

Skillstore Score

Why this score Evidence Confidence: High

Architecture

100

Maintainability

Content

Community

Spec Compliance

What You Can Build

Plan a New Lakehouse

Design a layered storage model, table format choice, and partitioning approach before ingestion starts.

Fix Slow Analytical Queries

Review file sizes, partition granularity, compression, and metadata design to reduce query latency.

Control Data Lake Costs

Create retention tiers, compression choices, and compaction practices for lower storage and scan costs.

Try These Prompts

Review My Layout

Review this data lake layout and suggest improvements for raw, processed, and curated zones. Ask for any missing details first.

Choose Partitions

Help me choose partition columns for a table with these query patterns, daily volume, and cardinality details. Explain the trade-offs.

Select a Table Format

Compare raw Parquet and Iceberg for my workload. Consider schema evolution, concurrent writers, update needs, metadata cost, and team operations.

Audit a Lakehouse Architecture

Audit this lakehouse architecture for scalability, cost, retention, compaction, schema evolution, and query performance. Return prioritized recommendations.

Best Practices

Start with query patterns, data volume, and retention needs before choosing partitions.
Keep raw data immutable so pipelines can reprocess history when logic changes.
Use file size, compression, and compaction targets as operational requirements.

Avoid

Do not partition by high-cardinality fields unless a clear bucketing strategy exists.
Do not mix raw source data and curated business tables in the same storage zone.
Do not choose Iceberg only for trend alignment without operational ownership.

Frequently Asked Questions

Can this skill design a complete lakehouse architecture?

Yes. It can propose storage zones, table formats, partitioning, retention tiers, and performance practices from your requirements.

Does it support both Parquet and Iceberg guidance?

Yes. It explains when raw Parquet is enough and when Iceberg provides better operational behavior.

Can it optimize an existing data lake?

Yes. Share the current layout, table sizes, query patterns, and pain points for a focused review.

Does it execute commands or change data?

No. The skill provides architecture guidance and uses read-only analysis tools.

What information should I provide?

Provide daily data volume, query filters, latency goals, schema change frequency, retention rules, and storage platform.

Is this only for Rust data engineering?

No. It references Rust examples, but the architecture guidance applies to general cloud data lakes.

Developer Details

Author

EmilLindfors

License

MIT

Author version

v1.0.0

Skillstore revision

Repository

https://github.com/EmilLindfors/claude-marketplace/tree/main/plugins/rust-data-engineering/skills/data-lake-architect

Ref

a06681402992ceae98ba04d54cfd4ab004862696

Maintenance freshness

7/18/2026

Usage

4 downloads · 146 views

File structure

📄 SKILL.md