How to write a Jev question that your code can use

A useful Jev question starts with a decision your application already understands. Write the question so a teammate could label examples consistently before asking a model to do it.

The following workflow is an editorial starting point, not a tested promise of higher accuracy. It follows the separation between context and questions described in TypeSafe’s state documentation.

Name one decision

Start with a concrete action: send a ticket to a team, flag a patch for inspection, or decide whether a tool result should remain available.

Avoid using one question to cover every dimension of quality. “Is this change good?” can mean correct, secure, readable, tested, or useful. Each interpretation leads to a different decision.

For a code-review screen, a narrower draft is: “Does the changed code introduce a path to an incorrect runtime result?” The source extraction from Jev Review adds scope and criteria around that concern.

Give the question the facts it needs

Use named fields for the relevant message, patch, policy, or conversation. Include enough context to interpret them, but leave out records that cannot affect the answer.

A patch question may require surrounding lines. A policy question may require the actual policy text. A classification question may need the earlier message that explains “that plan.”

Missing context and excess context create different problems. Test both with deliberate examples rather than assuming a longer input is always better.

Define the boundary cases

Write what counts as a yes answer or as each option. Then try a near miss.

For an outage check, “the service stopped working” may qualify, while “will this work next month?” should not. If you intended a deadline or ongoing business impact to matter, say so explicitly.

Keep the instructions and criteria aligned. Do not ask whether a condition is present and then define the positive answer as its absence. Avoid making readers infer the intended condition from the question’s identifier.

Separate instructions from untrusted input

Customer messages and retrieved documents can contain text that attempts to steer classification. Treat them as material to evaluate, not as authority for changing your application rules.

TypeSafe documents adversarial-content sensitivity among the Jev 1.13 limitations. A strongly worded prompt is not a substitute for testing or for enforcing permissions in code.

Add examples that contradict the criteria, contain irrelevant instructions, or argue for their own classification. Record the failures instead of silently removing difficult cases.

Batch only independent questions

If two questions read the same input independently, they can be submitted together. If the second needs the first answer to choose what evidence to inspect, make that dependency explicit in your application.

Our release-note example illustrates independent checks over shared state. It does not claim a particular speedup.

Version the whole contract

Save the question, criteria, input shape, model identifier, and downstream policy together. Keep a small labeled evaluation set with clear expected behavior.

When you revise wording, compare the same examples again. Look for improvements and regressions by failure category, not just a single headline score. A reusable prompt becomes more useful when another developer can understand what changed and why.