implementing-data-cicd skill
Set up CI/CD for data pipelines — SQL/dbt linting (SQLFluff), compilation and test gates, dbt Slim CI with state:modified and deferral, environment promotion (dev/staging/prod), and running only changed models on pull requests. Use when adding CI checks to a dbt or SQL project, automating pipeline tests on PRs, speeding up CI, or promoting changes across environments.
- GitHub stars
- 27
- Files
- 1
- Last commit
- Aug 30, 2026
- License
- —
- Allowed tools
- —
What does the implementing-data-cicd skill do?
implementing-data-cicd is an agent skill. Set up CI/CD for data pipelines — SQL/dbt linting (SQLFluff), compilation and test gates, dbt Slim CI with state:modified and deferral, environment promotion (dev/staging/prod), and running only changed models on pull requests. Use when adding CI checks to a dbt or SQL project, automating pipeline tests on PRs, speeding up CI, or promoting changes across environments.
How to install the implementing-data-cicd skill
Each command copies the skill folder into the client's skills directory. Restart the client or start a new session to load it.Install the implementing-data-cicd skill in Plugin
/plugin marketplace add Unknown-333/awesome-data-engineering-skills /plugin install data-engineering-skills@awesome-data-engineering-skills
Install the implementing-data-cicd skill in Claude Code
~/.claude/skills/implementing-data-cicd/SKILL.md
$ npx degit Unknown-333/awesome-data-engineering-skills/skills/implementing-data-cicd ~/.claude/skills/implementing-data-cicdInstall the implementing-data-cicd skill in Claude Code (project)
.claude/skills/implementing-data-cicd/SKILL.md
$ npx degit Unknown-333/awesome-data-engineering-skills/skills/implementing-data-cicd .claude/skills/implementing-data-cicdInstall the implementing-data-cicd skill in Codex
~/.agents/skills/implementing-data-cicd/SKILL.md
$ npx degit Unknown-333/awesome-data-engineering-skills/skills/implementing-data-cicd ~/.agents/skills/implementing-data-cicdInstall the implementing-data-cicd skill in Gemini CLI
~/.gemini/skills/implementing-data-cicd/SKILL.md
$ npx degit Unknown-333/awesome-data-engineering-skills/skills/implementing-data-cicd ~/.gemini/skills/implementing-data-cicdInstall the implementing-data-cicd skill in Claude.ai
npx degit Unknown-333/awesome-data-engineering-skills/skills/implementing-data-cicd implementing-data-cicd && zip -r implementing-data-cicd.zip implementing-data-cicd # claude.ai → Customize → Skills → + → Create skill → Upload a skill → implementing-data-cicd.zip
SKILL.md
View on GitHubImplementing Data CI/CD
When to use
- Adding CI checks to a dbt/SQL/pipeline repo.
- Automating lint, compile, and tests on pull requests.
- Speeding up CI by building only changed models (Slim CI).
- Promoting changes across dev → staging → prod.
- Do NOT use for reviewing an individual PR's logic (use
reviewing-data-pipeline-code).
Workflow
- [ ] Lint SQL (SQLFluff) and check formatting on every PR
- [ ] Compile the project to catch ref/Jinja errors early
- [ ] Build + test only changed models against prod parents (Slim CI)
- [ ] Run in an isolated CI schema; tear it down after
- [ ] Promote across environments with the same code, different targets
- Lint first — SQLFluff catches style and some correctness issues fast and cheap, before spinning up a warehouse.
- Compile —
dbt compile(orparse) fails on badref()/Jinja without running SQL. - Slim CI — build and test only
state:modified+using a productionmanifest.jsonas state, and defer unchanged parents to prod so CI doesn't rebuild the whole project. - Isolate — run into a unique CI schema (e.g. per PR) and drop it afterward so runs don't collide.
- Promote — the same code runs against dev/staging/prod via
--target; never fork logic per environment.
Patterns
dbt Slim CI (GitHub Actions sketch):
- run: dbt deps
- run: dbt compile
- run: |
dbt build --select state:modified+ \
--defer --state ./prod-artifacts \
--target ci
--select state:modified+ builds changed models and their children; --defer --state points unchanged parents at prod, so CI stays fast. A fresh/stale
prod-artifacts/manifest.json is required for correct selection.
SQLFluff config (.sqlfluff) — pin the dialect and templater so lint matches
your warehouse and dbt:
[sqlfluff]
dialect = snowflake
templater = dbt
Environment promotion — dev (developer schemas) → staging (full build on
merge) → prod (scheduled). Same repo, different profiles.yml targets and
credentials.
Common pitfalls
- Rebuilding the whole project on every PR — slow and expensive; use Slim CI
with
state:modified+ deferral. - Stale/missing state manifest —
state:modifiedselects the wrong nodes; refresh prod artifacts in CI. - Shared CI schema across PRs — concurrent runs clobber each other; use a unique schema per PR and drop it.
- Skipping
dbt depsin CI — macro/package-not-found failures. - Different logic per environment — drift and "works in dev" bugs; vary only the target/config, not the code.
- No teardown — orphaned CI schemas accumulate cost.
Files in this skill
1implementing-data-cicd skill questions
How do I install the implementing-data-cicd skill in Claude Code?
Run npx degit Unknown-333/awesome-data-engineering-skills/skills/implementing-data-cicd ~/.claude/skills/implementing-data-cicd. Claude Code loads skills from ~/.claude/skills (all projects) or .claude/skills (one project) and uses a skill when its description matches the task.
Which clients can use the implementing-data-cicd skill?
Any client that reads the Agent Skills format: Claude Code, Codex, Gemini CLI and claude.ai (upload as a ZIP). Scripts in the skill run only where the client allows code execution.