Manages LLM provider credentials and integrations within Arize to enable evaluators and other features to invoke AI models.
- GitHub stars
- 41,038
- Files
- 3
- Last commit
- Nov 11, 2026
- License
- MIT
- Allowed tools
- —
What does the arize-ai-provider-integration skill do?
arize-ai-provider-integration manages LLM provider credentials and integrations within Arize to enable evaluators and other features to invoke AI models.
Summary generated from the skill's SKILL.md.
About the arize-ai-provider-integration skill
Managing LLM provider credentials within Arize is what this skill enables for developers building AI evaluation workflows. It allows for the creation, retrieval, updating, and deletion of AI integrations that store the necessary authentication details for various models. By registering these credentials, users can enable Arize evaluators and other platform features to invoke models on their behalf.
An agent should load this skill when a user needs to connect a provider like OpenAI, Anthropic, Azure OpenAI, AWS Bedrock, Vertex AI, Gemini, or NVIDIA NIM to their Arize account. The skill relies on the ax CLI and a configured Arize profile to execute commands. It produces integration IDs, which are base64-encoded identifiers required for downstream operations like creating evaluators. The tool supports different authentication methods including default API keys, bearer tokens, and proxy configurations with custom headers.
What an agent can do with it
- 1Configure cloud provider metadataSet up AWS Bedrock or Vertex AI integrations using role ARNs or GCP project details.
Written from the skill's SKILL.md. Every use case is checked against a quote from it.
How to install the arize-ai-provider-integration skill
Each command copies the skill folder into the client's skills directory. Restart the client or start a new session to load it.Install the arize-ai-provider-integration skill in Claude Code
~/.claude/skills/arize-ai-provider-integration/SKILL.md
$ npx degit github/awesome-copilot/skills/arize-ai-provider-integration ~/.claude/skills/arize-ai-provider-integrationInstall the arize-ai-provider-integration skill in Claude Code (project)
.claude/skills/arize-ai-provider-integration/SKILL.md
$ npx degit github/awesome-copilot/skills/arize-ai-provider-integration .claude/skills/arize-ai-provider-integrationInstall the arize-ai-provider-integration skill in Codex
~/.agents/skills/arize-ai-provider-integration/SKILL.md
$ npx degit github/awesome-copilot/skills/arize-ai-provider-integration ~/.agents/skills/arize-ai-provider-integrationInstall the arize-ai-provider-integration skill in Gemini CLI
~/.gemini/skills/arize-ai-provider-integration/SKILL.md
$ npx degit github/awesome-copilot/skills/arize-ai-provider-integration ~/.gemini/skills/arize-ai-provider-integrationInstall the arize-ai-provider-integration skill in Claude.ai
npx degit github/awesome-copilot/skills/arize-ai-provider-integration arize-ai-provider-integration && zip -r arize-ai-provider-integration.zip arize-ai-provider-integration # claude.ai → Customize → Skills → + → Create skill → Upload a skill → arize-ai-provider-integration.zip
Automated checks
Ran Nov 15, 2026Full report →Passed our automated checks. That is not a security audit: read the scripts before you let an agent run them. How checks work
- Source repository linked and reachablegithub/awesome-copilot/skills/arize-ai-provider-integration is public and reachablegithub.com/github/awesome-copilot/tree/HEAD/skills/arize-ai- ↗Pass
- Open source license (OSI)MITspdx.org/licenses/MIT.html ↗Pass
- Commit or release in the last yearLast commit 2026-11-11github.com/github/awesome-copilot/commits ↗Pass
- No download-and-run or credential theft in scriptsNo scripts in the skill folderPass
- No hidden instructions in SKILL.mdNo override phrases, safeguard switches, credential requests or hidden charactersgithub.com/github/awesome-copilot/blob/HEAD/skills/arize-ai- ↗Pass
- Pre-approved tools are scopedNo pre-approved tools; the agent asks before using onegithub.com/github/awesome-copilot/blob/HEAD/skills/arize-ai- ↗Pass
- SKILL.md follows the Agent Skills formatValid name and description; says when to use it; 281 linesgithub.com/github/awesome-copilot/blob/HEAD/skills/arize-ai- ↗Pass
SKILL.md
View on GitHubArize AI Integration Skill
SPACE— Most--spaceflags and theARIZE_SPACEenv var accept a space name (e.g.,my-workspace) or a base64 space ID (e.g.,U3BhY2U6...). Find yours withax spaces list. Note:ai-integrations createdoes not accept--space— AI integrations are account-scoped. Use--spaceonly withlist,get,update, anddelete.
Concepts
- AI Integration = stored LLM provider credentials registered in Arize; used by evaluators to call a judge model and by other Arize features that need to invoke an LLM on your behalf
- Provider = the LLM service backing the integration (e.g.,
openAI,anthropic,awsBedrock) - Integration ID = a base64-encoded global identifier for an integration (e.g.,
TGxtSW50ZWdyYXRpb246MTI6YUJjRA==); required for evaluator creation and other downstream operations - Scoping = visibility rules controlling which spaces or users can use an integration
- Auth type = how Arize authenticates with the provider:
default(provider API key),proxy_with_headers(proxy via custom headers), orbearer_token(bearer token auth)
Prerequisites
Proceed directly with the task — run the ax command you need. Do NOT check versions, env vars, or profiles upfront.
If an ax command fails, troubleshoot based on the error:
command not foundor version error → see references/ax-setup.md401 Unauthorized/ missing API key → runax profiles showto inspect the current profile. If the profile is missing or the API key is wrong, follow references/ax-profiles.md to create/update it. If the user doesn't have their key, direct them to https://app.arize.com/admin > API Keys- Space unknown → run
ax spaces listto pick by name, or ask the user - LLM provider call fails (missing OPENAI_API_KEY / ANTHROPIC_API_KEY) → run
ax ai-integrations list --space SPACEto check for platform-managed credentials. If none exist, ask the user to provide the key or create an integration via the arize-ai-provider-integration skill - Security: Never read
.envfiles or search the filesystem for credentials. Useax profilesfor Arize credentials andax ai-integrationsfor LLM provider keys. If credentials are not available through these channels, ask the user.
List AI Integrations
List all integrations accessible in a space:
ax ai-integrations list --space SPACE
Filter by name (case-insensitive substring match):
ax ai-integrations list --space SPACE --name "openai"
Paginate large result sets:
# Get first page
ax ai-integrations list --space SPACE --limit 20 -o json
# Get next page using cursor from previous response
ax ai-integrations list --space SPACE --limit 20 --cursor CURSOR_TOKEN -o json
Key flags:
| Flag | Description |
|---|---|
--space |
Space name or ID to filter integrations |
--name |
Case-insensitive substring filter on integration name |
--limit |
Max results (1–100, default 15) |
--cursor |
Pagination token from a previous response |
-o, --output |
Output format: table (default) or json |
Response fields:
| Field | Description |
|---|---|
id |
Base64 integration ID — copy this for downstream commands |
name |
Human-readable name |
provider |
LLM provider enum (see Supported Providers below) |
has_api_key |
true if credentials are stored |
model_names |
Allowed model list, or null if all models are enabled |
enable_default_models |
Whether default models for this provider are allowed |
function_calling_enabled |
Whether tool/function calling is enabled |
auth_type |
Authentication method: default, proxy_with_headers, or bearer_token |
Get a Specific Integration
ax ai-integrations get NAME_OR_ID
ax ai-integrations get NAME_OR_ID -o json
ax ai-integrations get NAME_OR_ID --space SPACE # required when using name instead of ID
Use this to inspect an integration's full configuration or to confirm its ID after creation.
Create an AI Integration
Before creating, always list integrations first — the user may already have a suitable one:
ax ai-integrations list --space SPACE
If no suitable integration exists, create one. The required flags depend on the provider.
OpenAI
ax ai-integrations create \
--name "My OpenAI Integration" \
--provider openAI \
--api-key $OPENAI_API_KEY
Anthropic
ax ai-integrations create \
--name "My Anthropic Integration" \
--provider anthropic \
--api-key $ANTHROPIC_API_KEY
Azure OpenAI
ax ai-integrations create \
--name "My Azure OpenAI Integration" \
--provider azureOpenAI \
--api-key $AZURE_OPENAI_API_KEY \
--base-url "https://my-resource.openai.azure.com/"
AWS Bedrock
AWS Bedrock uses IAM role-based auth. Provide the ARN of the role Arize should assume via --provider-metadata:
ax ai-integrations create \
--name "My Bedrock Integration" \
--provider awsBedrock \
--provider-metadata '{"role_arn": "arn:aws:iam::123456789012:role/ArizeBedrockRole"}'
Vertex AI
Vertex AI uses GCP service account credentials. Provide the GCP project and region via --provider-metadata:
ax ai-integrations create \
--name "My Vertex AI Integration" \
--provider vertexAI \
--provider-metadata '{"project_id": "my-gcp-project", "location": "us-central1"}'
Gemini
ax ai-integrations create \
--name "My Gemini Integration" \
--provider gemini \
--api-key $GEMINI_API_KEY
NVIDIA NIM
ax ai-integrations create \
--name "My NVIDIA NIM Integration" \
--provider nvidiaNim \
--api-key $NVIDIA_API_KEY \
--base-url "https://integrate.api.nvidia.com/v1"
Custom (OpenAI-compatible endpoint)
ax ai-integrations create \
--name "My Custom Integration" \
--provider custom \
--base-url "https://my-llm-proxy.example.com/v1" \
--api-key $CUSTOM_LLM_API_KEY
Supported Providers
| Provider | Required extra flags |
|---|---|
openAI |
--api-key <key> |
anthropic |
--api-key <key> |
azureOpenAI |
--api-key <key>, --base-url <azure-endpoint> |
awsBedrock |
--provider-metadata '{"role_arn": "<arn>"}' |
vertexAI |
--provider-metadata '{"project_id": "<gcp-project>", "location": "<region>"}' |
gemini |
--api-key <key> |
nvidiaNim |
--api-key <key>, --base-url <nim-endpoint> |
custom |
--base-url <endpoint> |
Optional flags for any provider
| Flag | Description |
|---|---|
--model-name |
Allowed model name (repeat for multiple, e.g. --model-name gpt-4o --model-name gpt-4o-mini); omit to allow all models |
--enable-default-models |
Enable the provider's default model list |
--function-calling-enabled |
Enable tool/function calling support |
--auth-type |
Authentication type: default, proxy_with_headers, or bearer_token |
--headers |
Custom headers as JSON object or file path (for proxy auth) |
--provider-metadata |
Provider-specific metadata as JSON object or file path |
After creation
Capture the returned integration ID (e.g., TGxtSW50ZWdyYXRpb246MTI6YUJjRA==) — it is needed for evaluator creation and other downstream commands. If you missed it, retrieve it:
ax ai-integrations list --space SPACE -o json
# or by name/ID directly:
ax ai-integrations get NAME_OR_ID
Update an AI Integration
update is a partial update — only the flags you provide are changed. Omitted fields stay as-is.
# Rename
ax ai-integrations update NAME_OR_ID --name "New Name"
# Rotate the API key
ax ai-integrations update NAME_OR_ID --api-key $OPENAI_API_KEY
# Change the model list (replaces all existing model names)
ax ai-integrations update NAME_OR_ID --model-name gpt-4o --model-name gpt-4o-mini
# Update base URL (for Azure, custom, or NIM)
ax ai-integrations update NAME_OR_ID --base-url "https://new-endpoint.example.com/v1"
Add --space SPACE when using a name instead of ID. Any flag accepted by create can be passed to update.
Delete an AI Integration
Warning: Deletion is permanent. Evaluators that reference this integration will no longer be able to run.
ax ai-integrations delete NAME_OR_ID --force
ax ai-integrations delete NAME_OR_ID --space SPACE --force # required when using name instead of ID
Omit --force to get a confirmation prompt instead of deleting immediately.
Troubleshooting
| Problem | Solution |
|---|---|
ax: command not found |
See references/ax-setup.md |
401 Unauthorized |
API key may not have access to this space. Verify key and space ID at https://app.arize.com/admin > API Keys |
No profile found |
Run ax profiles show --expand; set ARIZE_API_KEY env var or write ~/.arize/config.toml |
Integration not found |
Verify with ax ai-integrations list --space SPACE |
has_api_key: false after create |
Credentials were not saved — re-run update with the correct --api-key or --provider-metadata |
| Evaluator runs fail with LLM errors | Check integration credentials with ax ai-integrations get INT_ID; rotate the API key if needed |
provider mismatch |
Cannot change provider after creation — delete and recreate with the correct provider |
Related Skills
- arize-evaluator: Create LLM-as-judge evaluators that use an AI integration → use
arize-evaluator - arize-experiment: Run experiments that use evaluators backed by an AI integration → use
arize-experiment
Save Credentials for Future Use
See references/ax-profiles.md § Save Credentials for Future Use.
Files in this skill
3arize-ai-provider-integration skill questions
How do I install the arize-ai-provider-integration skill in Claude Code?
Run npx degit github/awesome-copilot/skills/arize-ai-provider-integration ~/.claude/skills/arize-ai-provider-integration. Claude Code loads skills from ~/.claude/skills (all projects) or .claude/skills (one project) and uses a skill when its description matches the task.
Which clients can use the arize-ai-provider-integration skill?
Any client that reads the Agent Skills format: Claude Code, Codex, Gemini CLI and claude.ai (upload as a ZIP). Scripts in the skill run only where the client allows code execution.
What is an AI Integration in Arize?
An AI Integration consists of stored LLM provider credentials registered in Arize, which are used by evaluators to call judge models or by other features to invoke an LLM.
Can I change the provider of an existing integration?
No, you cannot change the provider once an integration has been created; you must delete the existing integration and create a new one with the correct provider.
How do I handle AWS Bedrock authentication?
AWS Bedrock uses IAM role-based authentication. You must provide the ARN of the role Arize should assume using the --provider-metadata flag.
What happens if I delete an AI integration?
Deletion is permanent, and any evaluators that reference that specific integration will no longer be able to run.