Scoring

Score a bug report’s usefulness

Combine specific quality judgments with weights you control.

ScoreAdapted exampleSource reviewedNot live-tested

01 The questions

reproduction score

How usable are this report’s reproduction steps?

0
No steps or triggering context.
1
Some context, but important steps are missing.
2
Clear steps and environment sufficient for an initial attempt.
behavior score

How clearly does the report distinguish observed and expected behavior?

0
Neither is clear.
1
One is clear, but the other is missing or vague.
2
Both are specific and distinguishable.

Copy includes the complete instructions, criteria, usage notes, and attribution.

02 Input

A bug report, including reproduction steps, environment, and observed behavior.

Illustrative input · JSON
{
  "report": "On Android 15, open Settings and tap Export. The app closes before showing the share sheet. Expected: a CSV export. Reproduced three times on version 2.4."
}

03 Answer & policy

Two expected scores on a 0–2 rubric, each with confidence. Scores can be fractional.

Your code decides what happens next.

The example normalizes each score to 0–1 and weights reproducibility 60%, behavior 40%. Low-confidence judgments go to review before ranking.

04 Use it in your code

Node.js 24 · TypeSafe SDK 0.6.0 · Set TYPESAFE_API_KEY in your environment. Run on your server; API calls incur provider charges.

decision.ts · TypeScript
import { score, TypeSafeClient, type JsonValue } from "@typesafe-ai/sdk";

// Illustrative input, not a recorded model test.
const state: JsonValue = {
  "report": "On Android 15, open Settings and tap Export. The app closes before showing the share sheet. Expected: a CSV export. Reproduced three times on version 2.4."
};
const client = new TypeSafeClient();
try {
  const response = await client.systemOne({
    model: "jev-latest",
    state,
    questions: {
      reproduction: score(
        "How usable are this report’s reproduction steps?",
        [
          "No steps or triggering context.",
          "Some context, but important steps are missing.",
          "Clear steps and environment sufficient for an initial attempt."
        ]
      ),
      behavior: score(
        "How clearly does the report distinguish observed and expected behavior?",
        [
          "Neither is clear.",
          "One is clear, but the other is missing or vague.",
          "Both are specific and distinguishable."
        ]
      )
    },
  });

  const { reproduction, behavior } = response.answers;
  const review = Math.min(reproduction.confidence, behavior.confidence) < 0.7;
  const usefulness = 0.6 * (reproduction.score / 2) + 0.4 * (behavior.score / 2);
  console.log({ usefulness, action: review ? "review_scores" : "rank_report" });
} catch (error) {
  console.error("Decision unavailable; use your fallback or human review.", error);
  process.exitCode = 1;
}
Download .ts ↓SDK setup ↗Copy and download include license notices.

The wrapper and example input are provided by Jev Directory. Checked against SDK types; no live model call was made. Pin a model version before evaluating production behavior.

Before you adapt it

  • Weights are editorial choices, not model measurements.
  • A well-written report can still describe an incorrect or low-impact issue.
More about the original project or pattern

What it does

Composite scoring breaks a broad evaluation into separate dimensions, asks a narrow Score question for each one, and combines the resulting values with a formula in application code. This avoids hiding business priorities inside one compound question.

What you can reuse

Reuse the separation between judgment and weighting. Jev evaluates well-defined rubrics; code normalizes and combines the outputs. When priorities change, developers can alter a coefficient or decision boundary without rewriting every question or rerunning unrelated dimensions.

How it fits

One shared state is evaluated against several independent Score questions. Their typed results enter a deterministic formula, followed by sorting, routing, or a policy threshold. The formula remains inspectable and testable without a model call.

Setup and compatibility

The pattern is documented for the TypeSafe SDK. Current syntax and model selection should come from the official quick start and the specific composite-scoring page.

Limitations

Weights express product policy; they do not correct a weak rubric or unrepresentative evaluation data. Normalization must respect each scale, and a weighted total can hide a disqualifying condition. Use an explicit veto question when one dimension must never be averaged away.

Sources

Primary source: TypeSafe composite scoring.