Routing / Scoring / Evaluation
Decide if an agent needs a tool
Screen out conversational turns before selecting from a tool catalog.
01 The question
Does answering this turn require calling a tool at all?
- true
- Requires fetching, changing, or acting on external state
- false
- Can be answered from conversation or general knowledge alone
Copy includes the complete instructions, criteria, usage notes, and attribution.
02 Input
The current user turn and enough conversation context to resolve references.
{
"user_turn": "Which production services are failing right now?"
}03 Answer & policy
A Noul probability that answering the turn requires a tool.
Your code decides what happens next.
The source defaults to 0.5 for this gate. Passing it should start tool selection, not execute a tool. Your application must still validate arguments and permissions.
04 Use it in your code
Node.js 24 · TypeSafe SDK 0.6.0 · Set TYPESAFE_API_KEY in your environment. Run on your server; API calls incur provider charges.
import { noul, TypeSafeClient, type JsonValue } from "@typesafe-ai/sdk";
// Illustrative input, not a recorded model test.
const state: JsonValue = {
"user_turn": "Which production services are failing right now?"
};
const client = new TypeSafeClient();
try {
const response = await client.systemOne({
model: "jev-latest",
state,
questions: {
needs_tool: noul(
"Does answering this turn require calling a tool at all?",
{
"true": "Requires fetching, changing, or acting on external state",
"false": "Can be answered from conversation or general knowledge alone"
}
)
},
});
const probability = response.answers.needs_tool.noul;
console.log({ probability, next: probability >= 0.5 ? "select_tool" : "answer_directly" });
} catch (error) {
console.error("Decision unavailable; use your fallback or human review.", error);
process.exitCode = 1;
}The wrapper and example input are provided by Jev Directory. Checked against SDK types; no live model call was made. Pin a model version before evaluating production behavior.
Before you adapt it
- A need for fresh information is different from permission to change external state.
- This small gate omits the source’s ranking and full-schema second stage.
More about the original project or pattern
What it does
Awesome Jev Apps collects runnable examples across document processing, AI infrastructure, business operations, data engineering, finance, research, moderation, and personal workflows. Each application is designed to stand alone, includes sample data, and follows a repository-wide application specification.
What you can reuse
The most useful layer is the application contract: a self-contained folder, explicit dependencies, provider configuration, copied shared client, sample data, offline tests, and a small README. The repository also keeps a machine-readable catalog.json, making the collection browsable without scraping its Markdown index.
How it fits
Each app turns a domain-specific state into one or more typed Jev questions. Ordinary Python code handles thresholds, ranking, escalation, formatting, and any exact computation. The collection repeatedly separates semantic judgment from deterministic policy.
Setup and compatibility
The documented quick start uses Python 3.11 or newer and uv. Apps can be opened with their included sample data before a user supplies personal files, although live decisions still require a configured provider key.
Limitations
This directory reviewed the collection structure, not every application. Offline tests can validate deterministic logic without establishing live inference quality. Provider compatibility, model context limits, and individual app behavior may change; inspect the selected app’s own README before use.
Sources
Primary sources: Awesome Jev Apps and its application specification.