Jev API

Jev evaluates structured decisions from a state object and a set of questions. It uses a dedicated API path separate from the OpenAI-compatible and Anthropic-compatible APIs, making it suitable for classification, scoring, and confidence-based evaluation.

Endpoint and authentication

POST https://<LLMPOOL_HOST>/jev/v1/systemone
Authorization: Bearer YOUR_API_KEY
Content-Type: application/json

Create the API key in Board under API Keys. The model field must use the Jev model ID shown on the Models page. Do not send the upstream provider model name.

Request format

Requests must include model, state, and questions. state and questions must be non-empty JSON objects. Their inner fields are passed through for the selected Jev model to interpret.

This example evaluates whether an assistant answer is correct:

curl -X POST "https://<LLMPOOL_HOST>/jev/v1/systemone" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "JEV_MODEL_FROM_MODELS_PAGE",
    "state": {
      "user_message": "How do I request a refund?",
      "assistant_reply": "Submit a refund request from the order details page."
    },
    "questions": {
      "correctness": {
        "type": "choice",
        "instructions": "Decide whether the answer correctly solves the user request.",
        "criteria": {
          "pass": "The answer is correct, complete, and directly solves the request.",
          "fail": "The answer contains an error, misses important information, or does not solve the request."
        }
      }
    }
  }'

You can include multiple evaluation dimensions in questions. A dimension commonly contains type, instructions, and optional criteria; the exact format depends on the Questions API supported by the selected Jev model.

Response format

LLMPool preserves the dynamic answers returned by Jev and rewrites the response model to the client-facing model ID from the Models page. Successful responses contain at least answers and usage.input_tokens:

{
  "id": "jev_request_xxx",
  "model": "JEV_MODEL_FROM_MODELS_PAGE",
  "answers": {
    "correctness": {
      "type": "choice",
      "choice": "pass",
      "confidence": 0.98
    }
  },
  "usage": {
    "input_tokens": 128,
    "output_tokens": 0,
    "total_tokens": 128
  }
}

If the upstream omits usage.output_tokens, LLMPool treats it as 0. Fields such as id, provider, and usage.cost are preserved as upstream metadata. Jev billing is based on input tokens; the account page is the source of truth for current pricing and charges.

Use the Playground

After signing in to Board, open Playground → Jev, choose an API key and model, enter the user message, assistant reply, and evaluation dimensions, and view the structured result. The Playground uses the same request format documented here.