Tool Use

This talk proposes legally registered agent organizations, scoped JWT authority, and blockchain audit trails for open-internet commerce; it is an architecture proposal, not validation.

Generative UI needs more than a component catalog: agents require explicit layout rules, schemas, and design-system constraints to produce stable interfaces across repeated requests.

Paid agent tools turn a skill file into spending authority. Builders need budgets, transaction logs, and counterparty screening before letting agents purchase data or services unattended.

Agent authorization should scale with stakes and reversibility: logs may cover coding changes, while open-ended payments need scoped, time-bound, independently verifiable mandates.

SAGE uses an expensive VLM only when a learner is uncertain, tests its advice against environment rewards, and distills useful guidance into a policy that needs no VLM at runtime.

A wallet-equipped agent crossed paywalls and completed paid email and phone actions under a spending cap. The engineering lesson is to enforce budgets in the wallet, not in prompts.

For fuzzy requests, an agent should identify the missing constraint with the most decision value, elicit it in the right modality, then choose a response format suited to the task.

Agent checkout needs more than payment tools: enforce user-set limits outside the agent, expose structured merchant data, honor cancellation, and log every decision.

An AWS demo treats a robot as an agent tool: an LLM selects existing hardware policies while separate agents handle thinking, communication, and voice. The live failures expose the safety gap.

Maersk’s production agents depend less on a clever loop than on executable SOPs, bounded tools, replayable traces, and a correction system shared by experts and engineers.

Once agents mutate external state, timeouts mean unknown outcomes. Builders need idempotent tools, bounded retries, scoped credentials, durable traces, and explicit recovery paths.

Vercel CLI 59.6.2 adds agent-friendly control over DNS, domains, projects, observability, and membership, with JSON output and confirmation gates for risky actions.

Ling 3.0 Flash Fin adds a finance-focused reasoning and tool-calling option to AI Gateway, with separate model IDs for automatic billing or a hard stop after the free period.

Exa’s internal agent clone combines writing examples, decision evals, and permission-scoped tools—a concrete pattern for encoding judgment without sharing full access.

Notion treats GTM automation as a shared context system for humans and agents, not an AI layer over disconnected tools. The key design choice is keeping risky customer actions human-approved.

Web agents should validate fetched content before spending model tokens on it and reserve browser automation for interactive steps. Explicit failures beat silently passing CAPTCHAs or blocked HTML downstream.

GLM 5.3 Flash joins Vercel AI Gateway with text and vision input, a 1M-token context window, function calling, structured output, and streaming.

Qwen 3.8 Flash is now selectable in Vercel AI Gateway and coding agents, with text-and-image input, a 1M-token context window, and responses up to 65k tokens.

Vercel Connect gives agents runtime-minted, task-scoped credentials instead of stored provider tokens, adding per-user identity, revocation, audit logs, and usage visibility.

Run SDK executes agent-generated JavaScript or TypeScript behind narrow host functions, with resumable approval and auth boundaries instead of application-level eval access.

Production agents need bounded write authority, infrastructure-stamped identity, and human-only overrides; a broad token and tool list cannot control blast radius.

A tool-exposed ontology steered models closer to human therapy patterns without fine-tuning, showing how explicit action vocabularies can improve agent behavior.

ConceptTS turns LLM-proposed concepts into executable labels and interpretable forecasting bottlenecks, enabling concept-level inspection and intervention.

Vercel agents can now manage DNS, domains, project state, observability, and members through structured CLI commands, while purchases and destructive actions retain confirmation gates.

DeepSeek V4 Flash Vision adds screenshot, image, and chart input to Vercel AI Gateway while retaining tool use, reasoning, and caching, but its experimental ID signals production risk.

MidTool trains general tool use before post-training, using API, MCP, document, web, PDF, and code data. Qwen3 4B and 8B variants improved across three downstream tool-use benchmarks.

Docker argues that agent safety belongs in a model-agnostic runtime: contain each task, grant narrowly scoped capabilities, and judge new access requests outside the agent's trust boundary.

Hugging Face automated research-artifact outreach with a CLI, one skill, and a sandbox. The case shows when an agent can replace custom workflow code, but undisclosed automated outreach raises trust questions.

Agent instructions are not security boundaries. Put identity, scoped capabilities, policy gates, audit receipts, and revocation outside the model before granting production access.

Healthcare agents can use X12 transactions as a typed contract across calls, portals, and claims. The standard constrains actions and validation, but payer data can still be stale or wrong.

Chat SDK now supports Instagram bots for DMs, media, reactions, story replies, and interactive cards, with buffered streaming and Meta’s 24-hour reply window.

Real-time video models are becoming cheap and responsive enough for agent interfaces, but builders still need global GPU routing, streaming infrastructure, and multi-model orchestration.

A talking guitar is a useful agent-interface case study: connect speech input, a local model, synthesis, pitch detection, and physical control, then expose where latency and segmentation break.

Nereu argues that coding agents need an engine-native vocabulary, not more generated glue code: tags, reusable systems, and selective scene context turn game intent into reliable actions.

Twin has a coding agent build and continuously repair an executable simulator before acting. The harness lifts the same base model from 7.8 to 93.3 on ARC-AGI-3 by validating every observed transition.

Reliable browser agents pair a programmable CLI with Chrome DevTools Protocol input, then sense, act, and verify one move at a time. Save solved paths as deterministic code instead of re-reasoning.

The long-tail web is unlikely to expose clean agent APIs. Browser agents need pixels as ground truth, with code and network access used opportunistically for speed rather than as universal substitutes.

AI Gateway now exposes Exa search as a keyless tool for any gateway model, and eve uses it by default. The free period ends August 31, so keep cost assumptions temporary.

Cross-region RL rollouts become practical if workers receive exact sparse weight deltas instead of full checkpoints. The proposal turns scattered inference GPUs into an elastic rollout fleet.

Model ML uses GPT-5.6 Sol to turn finance research and analysis into editable, traceable decks and workbooks, showing a concrete agent workflow beyond chat output.