materialize-redshift-create skill
✓ VerifiedCreate a Redshift materialization to stream Estuary collections into Amazon Redshift tables using flowctl.
- GitHub stars
- 7
- Files
- 1
- Last commit
- Nov 4, 2026
- License
- Apache-2.0
- Allowed tools
- —
What does the materialize-redshift-create skill do?
materialize-redshift-create creates a Redshift materialization to stream Estuary collections into Amazon Redshift tables using flowctl.
Summary generated from the skill's SKILL.md.
About the materialize-redshift-create skill
Setting up Redshift as a destination for captured data is what this skill enables. It assists developers in creating Redshift materializations to stream Estuary collections into Amazon Redshift tables using the flowctl command-line tool. An agent should load this skill when a user requests to send, materialize, or load data into Redshift.
The process involves gathering specific requirements such as Redshift endpoints, S3 staging buckets, AWS credentials, and network paths. It guides the user through fetching official connector documentation via WebFetch, querying the connector registry for the latest image tags, and building a flow.yaml specification file. The skill manages the entire lifecycle from requirement gathering and configuration to publishing the catalog and verifying the task status.
This tool handles complex configurations including SSH tunnels, delta updates for event logs, and hard delete settings. It also provides troubleshooting guidance for common issues like column mismatches, memory errors on Redshift Serverless, and S3 timeout errors caused by region mismatches.
What an agent can do with it
- 1Configure Redshift data streamingSet up a materialization to move data from Estuary collections into an Amazon Redshift cluster.
- 2Generate flow configuration filesBuild a flow.yaml file with the necessary endpoint, credentials, and binding details for Redshift.
- 3Troubleshoot materialization errorsIdentify and fix issues such as S3 timeouts or memory errors on Redshift Serverless.
- 4Verify data sync statusCheck if a materialization is pending, backfilling, or running normally.
Written from the skill's SKILL.md. Every use case is checked against a quote from it.
How to install the materialize-redshift-create skill
Each command copies the skill folder into the client's skills directory. Restart the client or start a new session to load it.Install the materialize-redshift-create skill in Plugin
/plugin marketplace add estuary/agent-skills /plugin install estuary-materializations@estuary
Install the materialize-redshift-create skill in Claude Code
~/.claude/skills/materialize-redshift-create/SKILL.md
$ npx degit estuary/agent-skills/skills/materialize-redshift-create ~/.claude/skills/materialize-redshift-createInstall the materialize-redshift-create skill in Claude Code (project)
.claude/skills/materialize-redshift-create/SKILL.md
$ npx degit estuary/agent-skills/skills/materialize-redshift-create .claude/skills/materialize-redshift-createInstall the materialize-redshift-create skill in Codex
~/.agents/skills/materialize-redshift-create/SKILL.md
$ npx degit estuary/agent-skills/skills/materialize-redshift-create ~/.agents/skills/materialize-redshift-createInstall the materialize-redshift-create skill in Gemini CLI
~/.gemini/skills/materialize-redshift-create/SKILL.md
$ npx degit estuary/agent-skills/skills/materialize-redshift-create ~/.gemini/skills/materialize-redshift-createInstall the materialize-redshift-create skill in Claude.ai
npx degit estuary/agent-skills/skills/materialize-redshift-create materialize-redshift-create && zip -r materialize-redshift-create.zip materialize-redshift-create # claude.ai → Customize → Skills → + → Create skill → Upload a skill → materialize-redshift-create.zip
Automated checks
Ran Nov 16, 2026Full report →Passed our automated checks. That is not a security audit: read the scripts before you let an agent run them. How checks work
- Source repository linked and reachableestuary/agent-skills/skills/materialize-redshift-create is public and reachablegithub.com/estuary/agent-skills/tree/HEAD/skills/materialize ↗Pass
- Open source license (OSI)Apache-2.0spdx.org/licenses/Apache-2.0.html ↗Pass
- Commit or release in the last yearLast commit 2026-11-04github.com/estuary/agent-skills/commits ↗Pass
- No download-and-run or credential theft in scriptsNo scripts in the skill folderPass
- No hidden instructions in SKILL.mdNo override phrases, safeguard switches, credential requests or hidden charactersgithub.com/estuary/agent-skills/blob/HEAD/skills/materialize ↗Pass
- Pre-approved tools are scopedNo pre-approved tools; the agent asks before using onegithub.com/estuary/agent-skills/blob/HEAD/skills/materialize ↗Pass
- SKILL.md follows the Agent Skills formatValid name and description; says when to use it; 188 linesgithub.com/estuary/agent-skills/blob/HEAD/skills/materialize ↗Pass
SKILL.md
View on GitHubCreate Redshift Materialization
Create a Redshift materialization using flowctl to stream data from Estuary collections into Amazon Redshift tables.
Applies to: materialize-redshift (Redshift Provisioned and Serverless)
Step 0: Load Connector Documentation
Before proceeding, fetch the official connector docs for prerequisites, config reference, and setup instructions.
Load the docs page: https://docs.estuary.dev/reference/Connectors/materialization-connectors/amazon-redshift/
Use WebFetch to load this page. It covers:
- Prerequisites (Redshift cluster, S3 bucket, IAM credentials)
- Full config property reference
- SSH tunnel configuration
- Network access and IP allowlisting
Search Kapa for tribal knowledge (if the Estuary MCP is configured):
Search kapa ai knowledge sources for "materialize redshift common issues"
If Kapa MCP is not configured, the user can set it up: https://docs.estuary.dev/features/mcp-integration/
This skill provides the flowctl workflow and troubleshooting that docs don't cover.
Step 1: Gather Requirements
Before writing any YAML, ask the user:
- Redshift endpoint? — Cluster hostname and port (default: 5439)
- Database and schema? — Target database name, schema (default:
public) - S3 staging bucket? — Bucket name and region (same region as Redshift cluster is recommended for best performance)
- AWS credentials? — Access key ID and secret access key with S3 read/write
- Network path? — Direct connection (with IP allowlist) or SSH tunnel
- Non-default data plane? — Most users use the default. Ask if they need a non-default data plane.
- Source collections? — Which Estuary collections to materialize
- Hard deletes? — Off by default. Without it, deleted rows stay in Redshift marked with
_meta/op: 'd'. Enable to physically remove them. - Delta updates? — Off by default. Switches from one-row-per-key (standard merge) to append-only. Use for event logs or history tables.
- Sync schedule? — Controls how often batches are written to Redshift (default: 30 minutes,
0sfor real-time). Affects latency and compute cost.
Key constraints to communicate upfront:
- Best practice: one materialization per schema (multiple bindings, not multiple materializations)
- All identifiers are lowercase unless
enable_case_sensitive_identifieris enabled on the cluster - Maximum document size is 4 MB per document
Step 2: Find the Correct Connector Version
Always use the latest numbered version tag. Query the connector registry:
flowctl raw get --table connector_tags \
--query 'documentation_url=eq.https://go.estuary.dev/materialize-redshift' \
--query 'select=image_tag,documentation_url' \
--output yaml
Use the returned image_tag — never hardcode a version.
Step 3: Help User Complete Prerequisites
Walk the user through prerequisites from the docs loaded in Step 0:
- Redshift accessible — Cluster publicly accessible or SSH tunnel configured
- S3 bucket — Same region as Redshift cluster, for staging data
- IAM user — With read/write access to the S3 bucket
- Redshift user — With CREATE TABLE permissions on the target schema
Refer to the docs page for exact setup steps.
Step 4: Create the Spec File
Build flow.yaml using the config reference from the docs. Minimal required config:
materializations:
<TENANT>/<PATH>/materialize-redshift:
endpoint:
connector:
image: ghcr.io/estuary/materialize-redshift:<VERSION>
config:
address: "<CLUSTER-ENDPOINT>:5439"
user: "<USERNAME>"
password: "<PASSWORD>"
database: "<DATABASE>"
schema: "public"
awsAccessKeyId: "<AWS-ACCESS-KEY-ID>"
awsSecretAccessKey: "<AWS-SECRET-ACCESS-KEY>"
bucket: "<S3-BUCKET-NAME>"
region: "<AWS-REGION>"
bucketPath: "estuary-staging"
bindings:
- source: <TENANT>/<collection-path>
resource:
table: "<table_name>"
Important: Table names should be lowercase. The bucket field is just the bucket name — no s3:// prefix.
For SSH tunnel, add networkTunnel.sshForwarding block — see docs for full config.
Step 5: Publish
flowctl catalog publish --source flow.yaml --auto-approve
Step 6: Verify
# Check status
flowctl catalog status <TENANT>/<PATH>/materialize-redshift
# View logs
flowctl logs --task <TENANT>/<PATH>/materialize-redshift --since 5m | jq -c '{ts, message}'
Status progression:
PENDING— Normal for ~30 seconds during shard assignmentBACKFILLING— Initial data sync from collectionsOK— Running normally with real-time updates
Troubleshooting
"column does not exist" (SQLSTATE 42703)
Cause: Column was dropped or renamed directly in Redshift outside of Estuary
Fix: Avoid altering connector-created tables directly. If the schema drifted, you may need to drop the table and let the connector recreate it (trigger a backfill by incrementing backfill counter on the binding).
"cannot drop sort key column"
Cause: The connector tried to alter a table but can't drop a sort key column
Fix: If using always_drop_tables_on_backfill, disable it. Otherwise, manually drop and recreate the table.
OOM on Serverless
Cause: Query ran out of memory on Redshift Serverless with low RPU
Fix: Increase RPU (Redshift Processing Units) on the Serverless workgroup, or reduce the number of fields being materialized using field selection.
S3 timeout errors
Cause: Network or region mismatch between S3 bucket and Redshift cluster
Fix:
- Verify S3 bucket is in the same AWS region as Redshift
- Check S3 bucket permissions (IAM user needs read/write)
- Verify Redshift cluster can reach S3 (VPC endpoints or internet access)
Slow type migrations on large tables
Cause: Redshift performs full table rewrite for type changes
Fix: This is expected behavior. Do not interrupt — let it complete. For very large tables, consider scheduling during low-traffic periods.
Materialization stuck in PENDING
Wait 30-60 seconds — this is normal during shard assignment. If still stuck:
flowctl logs --task <TENANT>/<PATH>/materialize-redshift --since 5m | jq 'select(.level == "error")'
Documents exceeding 4 MB limit
Cause: Individual documents are larger than Redshift's 4 MB per-document limit
Fix: Use field selection to exclude large fields (e.g., binary blobs, nested arrays), or create a derivation to truncate/filter before materializing.
Related Skills
schema-field-selection— Reduce materialized fields to avoid size limitsconnector-disable-enable— Pause/restart existing materializationsconnector-delete-recreate— Nuclear option for stuck materializationsestuary-logs— Deep log analysisestuary-catalog-status— Status checking
Files in this skill
1materialize-redshift-create skill questions
How do I install the materialize-redshift-create skill in Claude Code?
Run npx degit estuary/agent-skills/skills/materialize-redshift-create ~/.claude/skills/materialize-redshift-create. Claude Code loads skills from ~/.claude/skills (all projects) or .claude/skills (one project) and uses a skill when its description matches the task.
Which clients can use the materialize-redshift-create skill?
Any client that reads the Agent Skills format: Claude Code, Codex, Gemini CLI and claude.ai (upload as a ZIP). Scripts in the skill run only where the client allows code execution.
How do I handle documents that exceed the 4 MB limit?
You should use field selection to exclude large fields like binary blobs or nested arrays, or create a derivation to truncate or filter the data before materialization.
How can I physically remove deleted rows from Redshift?
You must enable the 'Hard deletes' option, which is off by default.