Tool Selection for AI Agents: A Reusable Prompt Pattern

Give an agent five tools and it picks well. Give it twenty-five and it starts reaching for the wrong one, calling a write tool when a read would do, or chaining three calls where one was enough. Tool selection for AI agents is the quiet reliability problem behind a lot of agent misbehavior, and most advice solves it with retrieval code rather than the thing you can reuse across projects: a prompt that picks the right small set of tools for a job.

The research is clear that accuracy falls as the tool count climbs. What's missing is a prompt-layer policy you can paste, with a shortlist step and a least-privilege check built in.

Why too many tools breaks the agent

Every tool you expose ships its description into the prompt, and those descriptions compete for the model's attention. Show too many and the model can't tell which one fits; show too few and the correct tool isn't on the menu at all. There's a real ceiling on how many tools a model chooses well from, and it's lower than the number most agents are handed.

The naive fix is to dump every tool into the system prompt and hope. It works at five tools and falls apart at twenty. The better move is to narrow the menu before the agent reasons, so it's always choosing from a short, relevant set.

The shortlist-then-choose pattern

The pattern that holds up has two stages, and a prompt can run both.

Shortlist. Before the agent reasons about the task, narrow the full tool catalog to the few that plausibly apply. At small scale a prompt does this directly from tool names and one-line descriptions. At large scale you back it with semantic retrieval over a tool index, but the decision the agent sees is still a short menu.

Choose with a contract. The agent picks from the shortlist and states which tool, why, and at what privilege. That last part matters: a read-only lookup shouldn't reach for a tool that can write. Encoding "prefer the least-privileged tool that does the job" in the contract catches a whole class of overreach.

Namespacing helps the shortlist stay legible. Grouping tools by service and resource, like jira_search and jira_create, gives the model boundaries it can reason about instead of a flat pile of names. The shortlist plus a naming convention does more for accuracy than swapping models.

What you can do with a tool-selection prompt

• Shortlist the 3-5 tools that fit a job before the agent commits to one
• Force a stated reason for each tool choice, so logs explain themselves
• Apply a least-privilege check that prefers read over write when both work
• Catch redundant tool chains where one call would have done
• Keep the choice auditable with a fixed selection contract
• Swap the underlying tools without rewriting the selection logic

Anatomy of the selection contract

The reusable piece reads the task and the catalog and emits a scoped, justified selection.

Variables → {{task}}, {{tool_catalog}}, {{privilege_rules}}

Prompt → Shortlist tools that could plausibly serve {{task}}.
         From the shortlist, choose the least-privileged tool
         that fully does the job.

Output contract (locked):
  shortlist:  [ 3-5 tools, each with a one-line reason ]
  chosen:     { tool, why, privilege_level }
  rejected:   [ tool + why not, for the close calls ]

Because chosen always carries a why and a privilege_level, every tool call leaves a trace you can audit. The rejected block is the part that saves you later: when an agent does something odd, the close-call it didn't pick usually explains the reasoning.

Where models differ

Models weigh tool descriptions differently, so the same catalog selects differently across them. Claude reads structured tool descriptions carefully and honors a least-privilege instruction stated as a rule. GPT-4o is faster but more eager to chain tools, so the "one call if one call works" constraint needs to be explicit and restated near the end of the prompt. Gemini is sensitive to description length and tends to favor whichever tool is described first, which makes ordering and namespacing matter more. Test the contract against your model rather than assuming the shortlist transfers.

An opinionated take: most agents are overloaded with tools because adding one feels free. It isn't. Every tool you expose taxes the selection accuracy of every other tool. Audit your agent's tool list and cut anything it hasn't needed in a week; a lean catalog beats a clever router. The MCP Tool Wiring Playbook builds on this: a {{task}} variable drives a shortlist-then-choose pass with a least-privilege check, so the agent's tool menu stays scoped to the job.

Step-by-step: scoping an agent's tools

1. Inventory the catalog

List every tool with a one-line description. If you can't describe a tool in one line, the model can't choose it well either.

2. Namespace by service and resource

Rename tools so the prefix tells the model what domain they belong to. db_read, db_write, jira_search. Boundaries help selection.

3. Run the shortlist pass

For a given {{task}}, narrow to three to five candidates before the agent reasons about which to call.

4. Apply the privilege rule

Encode "prefer the least-privileged tool that fully does the job" in {{privilege_rules}}. This is what stops a read job from grabbing a write tool.

5. Log the rejected set

Keep the close calls. They're the cheapest debugging artifact you'll ever collect.

Variables you'll set

What goes wrong, and the guard

The selection prompt has its own failure mode: it confidently shortlists a tool that almost fits, then the agent uses it for a job it can't quite do. The guard is the success_check on the chosen tool, the same discipline that keeps a planner honest. If the chosen tool can't satisfy the task, the contract should return "no suitable tool" rather than force a bad fit. There's also a security angle worth naming: tool descriptions are an injection surface, and a poisoned description can steer selection. Treat the catalog as data you control, and review prompt injection defense for coding agents before you expose tools to untrusted content.

For wiring the chosen tools to the agent's actual instructions, see MCP tool wiring for AI agents, and for picking which MCP servers a job needs in the first place, the free picker below covers it.

Getting started

1. List every tool in one line each.
2. Namespace them by service and resource.
3. Run a shortlist pass for a real task.
4. Add the least-privilege rule to {{privilege_rules}}.
5. Log the rejected set and review it.
6. Add retrieval only when the catalog outgrows the prompt.
7. Start from the free MCP Server Picker to scope servers before tools.

When one agent's tool set isn't enough and you split work across agents, see multi-agent orchestration prompt patterns, and browse the MCP packs for the rest of the wiring story.