capture-postgres-create skill
✓ VerifiedCreate and configure PostgreSQL CDC captures to stream database changes into Estuary collections using flowctl.
- GitHub stars
- 7
- Files
- 1
- Last commit
- Nov 4, 2026
- License
- Apache-2.0
- Allowed tools
- —
What does the capture-postgres-create skill do?
capture-postgres-create creates and configure PostgreSQL CDC captures to stream database changes into Estuary collections using flowctl.
Summary generated from the skill's SKILL.md.
About the capture-postgres-create skill
Setting up continuous data streams from PostgreSQL databases into Estuary collections is what this skill enables. It assists developers and data engineers in configuring Change Data Capture (CDC) for various PostgreSQL environments, including vanilla installations, Amazon RDS, Aurora, Google Cloud SQL, Supabase, and Neon. The skill guides the user through the entire lifecycle of a capture, from verifying logical replication settings and WAL levels to generating the necessary configuration files.
An agent should load this skill when a user needs to connect a database to Estuary or set up replication slots. It utilizes the flowctl command-line tool to manage the workflow, including querying the connector registry for correct image tags, discovering table schemas, and publishing the final catalog. The process produces a structured set of files, including a flow.yaml specification and SOPS-encrypted credential files, ensuring that sensitive information remains secure while automating the creation of write and read schemas for each captured table.
What an agent can do with it
- 1Configure PostgreSQL CDCSet up a continuous stream of database changes from a specific Postgres variant into Estuary.
- 2Identify Correct Connector VersionsFind the appropriate container image tag for a specific database type using the connector registry.
- 3Troubleshoot Replication IssuesDiagnose common errors such as permission denials, missing publications, or connection timeouts.
Written from the skill's SKILL.md. Every use case is checked against a quote from it.
How to install the capture-postgres-create skill
Each command copies the skill folder into the client's skills directory. Restart the client or start a new session to load it.Install the capture-postgres-create skill in Plugin
/plugin marketplace add estuary/agent-skills /plugin install estuary-captures@estuary
Install the capture-postgres-create skill in Claude Code
~/.claude/skills/capture-postgres-create/SKILL.md
$ npx degit estuary/agent-skills/skills/capture-postgres-create ~/.claude/skills/capture-postgres-createInstall the capture-postgres-create skill in Claude Code (project)
.claude/skills/capture-postgres-create/SKILL.md
$ npx degit estuary/agent-skills/skills/capture-postgres-create .claude/skills/capture-postgres-createInstall the capture-postgres-create skill in Codex
~/.agents/skills/capture-postgres-create/SKILL.md
$ npx degit estuary/agent-skills/skills/capture-postgres-create ~/.agents/skills/capture-postgres-createInstall the capture-postgres-create skill in Gemini CLI
~/.gemini/skills/capture-postgres-create/SKILL.md
$ npx degit estuary/agent-skills/skills/capture-postgres-create ~/.gemini/skills/capture-postgres-createInstall the capture-postgres-create skill in Claude.ai
npx degit estuary/agent-skills/skills/capture-postgres-create capture-postgres-create && zip -r capture-postgres-create.zip capture-postgres-create # claude.ai → Customize → Skills → + → Create skill → Upload a skill → capture-postgres-create.zip
Automated checks
Ran Nov 16, 2026Full report →Passed our automated checks. That is not a security audit: read the scripts before you let an agent run them. How checks work
- Source repository linked and reachableestuary/agent-skills/skills/capture-postgres-create is public and reachablegithub.com/estuary/agent-skills/tree/HEAD/skills/capture-pos ↗Pass
- Open source license (OSI)Apache-2.0spdx.org/licenses/Apache-2.0.html ↗Pass
- Commit or release in the last yearLast commit 2026-11-04github.com/estuary/agent-skills/commits ↗Pass
- No download-and-run or credential theft in scriptsNo scripts in the skill folderPass
- No hidden instructions in SKILL.mdNo override phrases, safeguard switches, credential requests or hidden charactersgithub.com/estuary/agent-skills/blob/HEAD/skills/capture-pos ↗Pass
- Pre-approved tools are scopedNo pre-approved tools; the agent asks before using onegithub.com/estuary/agent-skills/blob/HEAD/skills/capture-pos ↗Pass
- SKILL.md follows the Agent Skills formatValid name and description; says when to use it; 286 linesgithub.com/estuary/agent-skills/blob/HEAD/skills/capture-pos ↗Pass
SKILL.md
View on GitHubCreate PostgreSQL Capture
Create a PostgreSQL CDC capture that continuously streams changes from Postgres tables into Estuary collections.
Applies to: source-postgres, source-amazon-aurora-postgres, source-amazon-rds-postgres, source-google-cloud-sql-postgres, source-supabase-postgres, source-neon-postgres
Step 0: Load Connector Documentation
Before proceeding, fetch the official connector docs for prerequisites, config reference, and cloud-specific setup.
Always load the main page: https://docs.estuary.dev/reference/Connectors/capture-connectors/PostgreSQL/
Then load the variant subpage based on the user's Postgres type:
| Variant | Docs URL |
|---|---|
| Self-hosted / Vanilla | Main page covers this |
| Amazon Aurora | Main page covers this |
| Azure Database | Main page covers this |
| Amazon RDS | https://docs.estuary.dev/reference/Connectors/capture-connectors/PostgreSQL/amazon-rds-postgres/ |
| Google Cloud SQL | https://docs.estuary.dev/reference/Connectors/capture-connectors/PostgreSQL/google-cloud-sql-postgres/ |
| Supabase | https://docs.estuary.dev/reference/Connectors/capture-connectors/PostgreSQL/Supabase/ |
| Neon | https://docs.estuary.dev/reference/Connectors/capture-connectors/PostgreSQL/neon-postgres/ |
Use WebFetch to load these pages. Together they cover:
- Prerequisites (WAL level, user creation, publication setup, watermarks table)
- Full config property reference
- Cloud-specific setup instructions
- SSH tunnel configuration
- Network access / IP allowlisting
This skill provides the flowctl workflow and troubleshooting that docs don't cover.
Step 1: Gather Requirements
Before writing any YAML, ask the user:
- Postgres variant? — Vanilla, Amazon RDS, Aurora, Google Cloud SQL, Supabase, Neon
- Network path? — Direct connection (cloud-hosted with IP allowlist), SSH tunnel (private network/on-prem), Private Link (AWS/Azure/GCP), or ngrok (local dev)
- Non-default data plane? — Most users use the default. Ask if they need a non-default data plane.
- Tables to capture? — All standard tables in the publication is the default. A specific subset, partitioned tables (must capture child partitions, not the parent), views (require batch connector instead of CDC), or tables not yet in the publication all require additional configuration.
- History mode? — Standard CDC (
false) or full event history (true)
Step 2: Find the Correct Connector Version
Always use the latest numbered version tag. Query the connector registry to find it:
# For standard Postgres
flowctl raw get --table connector_tags \
--query 'documentation_url=eq.https://go.estuary.dev/source-postgres' \
--query 'select=image_tag,documentation_url' \
--output yaml
To find any Postgres variant version:
flowctl raw get --table connector_tags \
--query 'documentation_url=ilike.*<connector-name>*' \
--query 'select=image_tag,documentation_url' \
--output yaml
Choose the connector image based on the user's Postgres variant:
| Variant | Connector Image |
|---|---|
| Vanilla / self-hosted | ghcr.io/estuary/source-postgres |
| Amazon RDS | ghcr.io/estuary/source-amazon-rds-postgres |
| Amazon Aurora | ghcr.io/estuary/source-amazon-aurora-postgres |
| Google Cloud SQL | ghcr.io/estuary/source-google-cloud-sql-postgres |
| Supabase | ghcr.io/estuary/source-supabase-postgres |
| Neon | ghcr.io/estuary/source-neon-postgres |
Step 3: Help User Complete Prerequisites
Walk the user through the prerequisites from the docs page loaded in Step 0. Key items:
- Logical replication —
SHOW wal_level;must returnlogical - WAL retention safety — Recommend setting
max_slot_wal_keep_size(50GB is a good starting point, more for high-change-rate databases). Without this, Postgres retains unbounded WAL if the capture ever stalls, which can fill the disk and crash the database. The connector does not check this — it's a safety net for the source database, not a replication prerequisite. - Capture user — needs REPLICATION permission and SELECT access
- Publication —
CREATE PUBLICATION flow_publication FOR ALL TABLES; - Watermarks table — recommended for backfill accuracy
If the user is on a managed service (RDS, Cloud SQL, Supabase), refer to the cloud-specific section in the docs.
Step 4: Create the Capture Spec File
Build flow.yaml using the config reference from the docs. Minimal required config:
captures:
<tenant>/<path>/source-postgres:
endpoint:
connector:
image: ghcr.io/estuary/source-postgres:<version>
config:
address: "<host>:<port>"
database: "<database_name>"
user: "<username>"
credentials:
auth_type: "UserPassword"
password: "<password>"
historyMode: false
bindings: []
For SSH tunnel, add networkTunnel.sshForwarding block — see docs for full config.
For AWS IAM or GCP IAM auth, set credentials.auth_type accordingly — see docs for required fields.
Step 5: Discover Tables
flowctl discover --source flow.yaml
Note: The first discover attempt occasionally fails with a generic error. If this happens, simply retry — it typically succeeds on the second attempt.
What this does:
- Connects to the database
- Discovers all tables in the publication
- Generates JSON schemas from table structures
- Updates
flow.yamlwith bindings - Creates SOPS-encrypted config file
Generated file structure:
flow.yaml # Updated with bindings
source-postgres.config.yaml # SOPS-encrypted credentials
<tenant>/
flow.yaml # Import file
<path>/
flow.yaml # Collection definitions
public/
<table>.write.schema.yaml # Write schema per table
<table>.read.schema.yaml # Read schema per table
Step 6: Review and Customize Bindings
After discovery, review the generated bindings in flow.yaml:
- Remove unwanted tables — Delete the binding entry
- Rename target collections — Change the
targetfield
Step 7: Publish
flowctl catalog publish --source flow.yaml --auto-approve
The --auto-approve flag is required for non-interactive use.
Step 8: Verify
# Check status (expect PENDING → Backfilling → OK: Streaming CDC Events)
flowctl catalog status <tenant>/<path>/source-postgres
# View recent logs
flowctl logs --task <tenant>/<path>/source-postgres --since 5m | jq -c '{ts, message}'
# Read captured data
flowctl collections read --collection <tenant>/<path>/public/<table> | head -20
Status progression:
WARNING: waiting for task shards to be ready(PENDING) — normal for ~30 secondsBackfilling Tables (N tables backfilling)— initial data syncOK: Streaming CDC Events— capture is running normally
Step 9: Test Real-Time CDC
Insert a row in the source database and verify it appears in the collection within seconds:
flowctl collections read --collection <tenant>/<path>/public/<table> | \
jq 'select(.<key_field> == "<test_value>")'
Troubleshooting
"no connector tag found for image"
Cause: Wrong image tag (e.g., :dev)
Fix: Query connector_tags for the correct version (see Step 2).
"Config failed schema validation"
Cause: Config structure is wrong. Common mistakes:
- Password at root level instead of
credentials.password - Missing
credentials.auth_type: "UserPassword" - Missing
historyMode: false
"connection refused" or timeout
Cause: Database not reachable from Estuary cloud
Fix:
- Verify firewall allows Estuary IPs (see docs for current list)
- Use SSH tunnel for private networks
- For local dev:
ngrok tcp 5432
"permission denied for replication"
Cause: User lacks REPLICATION permission
Fix:
ALTER USER flow_capture WITH REPLICATION;
GRANT pg_read_all_data TO flow_capture; -- Postgres 14+
"publication does not exist"
Cause: Publication not created or wrong name in advanced config
Fix:
CREATE PUBLICATION flow_publication FOR ALL TABLES;
No tables discovered
Cause: Publication exists but has no tables, or the capture user lacks permissions to see them. This is a very common support issue.
Debug:
-- Check publication has tables (run as admin)
SELECT p.pubname, pt.schemaname, pt.tablename
FROM pg_publication p
JOIN pg_publication_tables pt ON p.oid = pt.pubid
ORDER BY p.pubname, pt.schemaname, pt.tablename;
-- Check capture user can see tables (run as capture user)
SELECT table_schema, table_name, table_type
FROM information_schema.tables
WHERE table_schema NOT IN ('pg_catalog', 'pg_internal', 'information_schema');
If the publication is empty, add tables or recreate with FOR ALL TABLES. If the capture user sees fewer tables than admin, it's a permissions/schema visibility issue.
Capture stuck in PENDING
Wait 30-60 seconds — this is normal during shard assignment. If still stuck:
flowctl logs --task <tenant>/<path>/source-postgres --since 5m | jq 'select(.level == "error" or .level == "warn")'
Watermarks table warning
Warning: relation "public.flow_watermarks" does not exist
Impact: Capture still works, but backfill accuracy may be affected.
Fix: Create the watermarks table per docs prerequisites.
Discover fails on first attempt
This is a known intermittent issue. Simply retry flowctl discover --source flow.yaml — it typically succeeds on the second attempt.
Related Skills
connector-disable-enable— Pause/restart existing capturesconnector-delete-recreate— Nuclear option for stuck capturesestuary-logs— Deep log analysisestuary-catalog-status— Status checking
Files in this skill
1capture-postgres-create skill questions
How do I install the capture-postgres-create skill in Claude Code?
Run npx degit estuary/agent-skills/skills/capture-postgres-create ~/.claude/skills/capture-postgres-create. Claude Code loads skills from ~/.claude/skills (all projects) or .claude/skills (one project) and uses a skill when its description matches the task.
Which clients can use the capture-postgres-create skill?
Any client that reads the Agent Skills format: Claude Code, Codex, Gemini CLI and claude.ai (upload as a ZIP). Scripts in the skill run only where the client allows code execution.
Which PostgreSQL variants are supported?
The skill supports vanilla/self-hosted, Amazon Aurora, Amazon RDS, Google Cloud SQL, Supabase, and Neon.
What are the main prerequisites for setting up a Postgres capture?
Key requirements include setting the WAL level to logical, creating a capture user with REPLICATION permissions, setting up a publication, and optionally creating a watermarks table.
What should I do if the 'flowctl discover' command fails on the first attempt?
This is a known intermittent issue; you should simply retry the command, as it typically succeeds on the second attempt.
How can I verify if my CDC capture is running correctly?
You can check the status using 'flowctl catalog status' and look for the 'OK: Streaming CDC Events' message, or read recent logs using 'flowctl logs'.