Selects and evaluates base models by querying SageMaker Hub to resolve model IDs, licenses, and benchmarks.
- GitHub stars
- 938
- Files
- 4
- Last commit
- Nov 10, 2026
- License
- Apache-2.0
- Allowed tools
- —
What does the model-selection skill do?
model-selection selects and evaluates base models by querying SageMaker Hub to resolve model IDs, licenses, and benchmarks.
Summary generated from the skill's SKILL.md.
About the model-selection skill
Finding and evaluating the right base model for machine learning projects is what this skill enables. Developers use it to resolve specific model IDs, check licenses, and review benchmarks by querying SageMaker Hub. It is particularly useful when a user mentions a model family like Llama, Mistral, or Nova, as the tool ensures the exact Hub model ID is correctly identified.
An agent should load this skill when a user needs to select or change a base model or wants to evaluate options before fine-tuning. The process begins by verifying the AWS region and using the aws___call_aws tool to list available SageMaker Hubs. It filters out hubs containing AI Registry in their description to focus on JumpStart models. The skill relies on a use_case_spec.md file to guide the selection process and uses python scripts to retrieve model names from the chosen hub.
Once models are discovered, the tool presents them alongside license information from references/model-licenses.md. It helps users navigate through available options in public or private hubs, ensuring that any recommended model is actually available in the user's specific environment.
What an agent can do with it
- 1Identify specific model IDsResolve a general model family name into a precise SageMaker Hub model ID.
- 2Check model licensingVerify the legal license associated with a specific model name before selection.
- 3Discover available model hubsFind and select between different public or private SageMaker Hubs in a specific region.
Written from the skill's SKILL.md. Every use case is checked against a quote from it.
How to install the model-selection skill
Each command copies the skill folder into the client's skills directory. Restart the client or start a new session to load it.Install the model-selection skill in Plugin
/plugin marketplace add awslabs/agent-plugins /plugin install sagemaker-ai@agent-plugins-for-aws
Install the model-selection skill in Claude Code
~/.claude/skills/model-selection/SKILL.md
$ npx degit awslabs/agent-plugins/plugins/sagemaker-ai/skills/model-selection ~/.claude/skills/model-selectionInstall the model-selection skill in Claude Code (project)
.claude/skills/model-selection/SKILL.md
$ npx degit awslabs/agent-plugins/plugins/sagemaker-ai/skills/model-selection .claude/skills/model-selectionInstall the model-selection skill in Codex
~/.agents/skills/model-selection/SKILL.md
$ npx degit awslabs/agent-plugins/plugins/sagemaker-ai/skills/model-selection ~/.agents/skills/model-selectionInstall the model-selection skill in Gemini CLI
~/.gemini/skills/model-selection/SKILL.md
$ npx degit awslabs/agent-plugins/plugins/sagemaker-ai/skills/model-selection ~/.gemini/skills/model-selectionInstall the model-selection skill in Claude.ai
npx degit awslabs/agent-plugins/plugins/sagemaker-ai/skills/model-selection model-selection && zip -r model-selection.zip model-selection # claude.ai → Customize → Skills → + → Create skill → Upload a skill → model-selection.zip
Automated checks
Ran Nov 15, 2026Full report →Passed our automated checks. That is not a security audit: read the scripts before you let an agent run them. How checks work
- Source repository linked and reachableawslabs/agent-plugins/plugins/sagemaker-ai/skills/model-selection is public and reachablegithub.com/awslabs/agent-plugins/tree/HEAD/plugins/sagemaker ↗Pass
- Open source license (OSI)Apache-2.0spdx.org/licenses/Apache-2.0.html ↗Pass
- Commit or release in the last yearLast commit 2026-11-10github.com/awslabs/agent-plugins/commits ↗Pass
- No download-and-run or credential theft in scripts1 script read; no download-and-run, remote eval, base64 execution or credential uploadPass
- No hidden instructions in SKILL.mdNo override phrases, safeguard switches, credential requests or hidden charactersgithub.com/awslabs/agent-plugins/blob/HEAD/plugins/sagemaker ↗Pass
- Pre-approved tools are scopedNo pre-approved tools; the agent asks before using onegithub.com/awslabs/agent-plugins/blob/HEAD/plugins/sagemaker ↗Pass
- SKILL.md follows the Agent Skills formatValid name and description; says when to use it; 77 linesgithub.com/awslabs/agent-plugins/blob/HEAD/plugins/sagemaker ↗Pass
SKILL.md
View on GitHubModel Selection
Guides the user through selecting a base model based on their use case.
When to Use
- User asks which model to use
- User wants to select or change their base model
- User mentions a model name or family (e.g., "Llama", "Mistral", "Nova") — the exact Hub model ID still needs to be resolved
- User wants to evaluate a base model before deciding whether to finetune
Prerequisites
- A
use_case_spec.mdfile exists. If not, activate the use-case-specification skill to generate it first.
Workflow
Step 1: Check Region
Run:
python -c "import boto3; print(boto3.session.Session().region_name)"
None→ STOP. Tell user: "Set your region viaexport AWS_DEFAULT_REGION=us-west-2oraws configure."- Set → store REGION in context, continue.
Step 2: Discover Hub
-
List all available SageMaker Hubs in the user's region by calling the SageMaker
ListHubsAPI using theaws___call_awstool. -
From the results, filter out any hub whose
HubDescriptioncontains "AI Registry" — these do not contain JumpStart models. -
The remaining hubs are eligible (e.g.,
SageMakerPublicHuband any private hubs). -
If exactly one eligible hub exists, use it automatically — do not ask the user.
-
If multiple eligible hubs exist, present them to the user and ask which one to use. Example:
I found the following model hubs: - SageMakerPublicHub — SageMaker Public Hub - Private-Hub-XYZ — Private Hub models Which hub would you like to use? -
Store the selected hub name for use in subsequent steps.
Step 3: Select Base Model
First, retrieve all available SageMaker Hub model names by running: python model-selection/scripts/get_model_names.py <hub-name>.
Present all available models to the user with their licenses before making any recommendations. Cross-reference the model list with references/model-licenses.md and display each as <model name> - [<license>](<url>). For example: "Qwen3-4B - Apache 2.0"
If you already know the model the user wants to use (from conversation context or planning files), confirm that it's in the list, display its license, and move on. Otherwise, help the user pick a model following the instructions in references/model-selection.md.
Important: Make sure to remember this list of available models when helping with model selection. Don't recommend a model that's not available to the user.
Step 4: Confirm Selection
Present a summary to the user:
Here's what we've selected:
- Base model: [model name]
Ask if they'd like to proceed with this model.
References
references/model-selection.md— Model selection instructions and benchmark descriptionsreferences/model-licenses.md— Model license information for display during model selection
Files in this skill
4model-selection skill questions
How do I install the model-selection skill in Claude Code?
Run npx degit awslabs/agent-plugins/plugins/sagemaker-ai/skills/model-selection ~/.claude/skills/model-selection. Claude Code loads skills from ~/.claude/skills (all projects) or .claude/skills (one project) and uses a skill when its description matches the task.
Which clients can use the model-selection skill?
Any client that reads the Agent Skills format: Claude Code, Codex, Gemini CLI and claude.ai (upload as a ZIP). Scripts in the skill run only where the client allows code execution.
What should I do if my AWS region is not set?
You must set your region by using 'export AWS_DEFAULT_REGION=us-west-2' or by running 'aws configure'.
What is required before I can select a model?
A use_case_spec.md file must exist; if it does not, you should use the use-case-specification skill to create one first.
How does the tool handle multiple available model hubs?
If more than one eligible hub is found, the tool will present the options to the user and ask them to choose which one to use.