Jev Directory

Evaluation / Routing

Detect an urgent support ticket

Turn an urgency question into a measurable signal for your response queue.

NoulSource extractionSource reviewedNot live-tested

01 The question

is_urgent noul

Does this ticket need a response today?

true
The customer states a deadline, an outage, or ongoing business impact.
false
No time pressure is expressed, or the customer says it can wait.

Copy includes the complete instructions, criteria, usage notes, and attribution.

02 Input

One support ticket with its timing, outage, or business-impact context.

Illustrative input · JSON
{
  "ticket": "Our checkout integration is down and customers cannot pay. We need help restoring it today."
}

03 Answer & policy

A Noul probability of needing a response today.

Your code decides what happens next.

Collect probabilities against labeled tickets before selecting a cutoff. This example reports the signal and keeps every ticket in the review queue.

04 Use it in your code

Node.js 24 · TypeSafe SDK 0.6.0 · Set TYPESAFE_API_KEY in your environment. Run on your server; API calls incur provider charges.

decision.ts · TypeScript
import { noul, TypeSafeClient, type JsonValue } from "@typesafe-ai/sdk";

// Illustrative input, not a recorded model test.
const state: JsonValue = {
  "ticket": "Our checkout integration is down and customers cannot pay. We need help restoring it today."
};
const client = new TypeSafeClient();
try {
  const response = await client.systemOne({
    model: "jev-latest",
    state,
    questions: {
      is_urgent: noul(
        "Does this ticket need a response today?",
        {
          "true": "The customer states a deadline, an outage, or ongoing business impact.",
          "false": "No time pressure is expressed, or the customer says it can wait."
        }
      )
    },
  });

  console.log({
    urgencyProbability: response.answers.is_urgent.noul,
    action: "review_ticket",
  });
} catch (error) {
  console.error("Decision unavailable; use your fallback or human review.", error);
  process.exitCode = 1;
}
Download .ts ↓SDK setup ↗Copy and download include license notices.

The wrapper and example input are provided by Jev Directory. Checked against SDK types; no live model call was made. Pin a model version before evaluating production behavior.

Before you adapt it

  • Do not substitute an accuracy target for a probability threshold.
  • Evaluate missed urgent tickets separately; mistakes have different costs.
More about the original project or pattern

What it does

jevcal runs typed decision questions against labeled examples, measures how confidence relates to correctness, and selects thresholds intended to meet a target on held-out data. It writes a decision lock file and an HTML report, and can remeasure the locked configuration in CI to flag accuracy, coverage, or model-version drift.

What you can reuse

The portable artifact is the bundle around a question: questions.yaml, labeled JSONL, saved predictions, thresholds, and evidence. The split between measurement and compilation means previously collected predictions can be inspected without spending on another model call. A cascade runtime can pass low-confidence items to a fallback and fail closed when none exists.

How it fits

Representative states and labels become an evaluation set. Jev produces typed answers and probabilities. Deterministic evaluation code selects thresholds, reports coverage and errors, and decides when a slower fallback is needed.

Setup and compatibility

The documented workflow uses Python 3.10 or newer and installs from GitHub. A no-key demo uses a deliberately imperfect simulator so a user can inspect the report format before connecting Jev.

Limitations

The sample metrics in the README are simulator output, not a Jev benchmark. The author recommends roughly one hundred or more labeled rows per question and warns that labels, ambiguous questions, model changes, and rounded probabilities can all affect a threshold. Jev Directory has not reproduced the author-reported live verification.

Sources

Primary source: jevcal repository.