jevper

https://github.com/zhulinchng/jevper
Admin
jevper implements the Jev System One interface over any OpenAI-compatible model, providing typed...
Tokens:119,558
Snippets:701
Trust Score:7.8
License:Apache-2.0
Update:2 weeks ago
Show doc for...
Tokens:
Raw
### Install, pull models, and serve ollaya

Source: https://github.com/zhulinchng/jevper/blob/main/docs/local-servers.md

Install ollaya 0.7.0, add its binary to PATH, pull the laya:en and laya:typed-decisions models, then start the server on 127.0.0.1:11435. The installer does not require root and installs to ~/.local.

```bash
curl -fsSL https://ollaya.dev/install.sh | sh      # 0.7.0; no root installs to ~/.local
export PATH="$HOME/.local/bin:$PATH"
ollaya pull laya:en
ollaya pull laya:typed-decisions
ollaya serve                                       # 127.0.0.1:11435
```

--------------------------------

### Example

Source: https://github.com/zhulinchng/jevper/blob/main/docs/api.md

Documentation for the Example constructor.

```APIDOC
## Example

### Description
Constructs an example demonstration for a question. The example is replayed into the prompt as the answer the model is asked to imitate.

### Constructor
`Example(state=..., answer=..., probabilities=None)`

### Parameters
- **state** (any JSON-serializable value) - Required - The input state for the example. Must be JSON-serializable; non-serializable values are refused.
- **answer** (str, int, or bool) - Required - The answer to the question. Can be a label (e.g., `"B"`), a `Choice` criteria key, a `Score` level index, or a bool for `Noul`.
- **probabilities** (dict) - Optional - A dictionary of probabilities. Only read by `method="structured"`. Defaults to a one-hot distribution over `answer`.

### Validation
- `probabilities` must carry exactly the option keys of a `Choice`, the level indexes of a `Score`, or a `True`/`False` key for a `Noul`. Wrong key sets raise `InvalidQuestionError`.
- Values must be finite and `>= 0`. For `Noul`, values must be in `[0, 1]`.
- Non-finite numbers or non-JSON-serializable `state` are refused.
```

--------------------------------

### Development setup with uv, including pytest

Source: https://github.com/zhulinchng/jevper/blob/main/README.md

Set up the development environment for jevper. Clones the repository, creates a virtual environment with uv, installs with test extras, and runs the test suite.

```sh
git clone https://github.com/zhulinchng/jevper && cd jevper
uv venv && uv pip install -e '.[test]'
pytest -q
```

--------------------------------

### Install and run MLflow integration tests

Source: https://github.com/zhulinchng/jevper/blob/main/docs/mlflow.md

Install the test and MLflow extras, then run the MLflow integration suite. The suite skips unless MLflow is installed.

```sh
uv pip install -e '.[test,mlflow]'    # mlflow[gateway,langchain] + openai + anthropic + pytest
pytest tests/test_mlflow.py -q
```

--------------------------------

### Example source resolution flowchart (Mermaid)

Source: https://github.com/zhulinchng/jevper/blob/main/docs/few-shot.md

Flowchart showing the precedence of example sources: question.examples, then system_one, then SystemOneClient. The first non-empty level wins.

```mermaid
flowchart TD
    A["question.examples"] -->|"non-empty"| U["use them"]
    A -->|"empty"| B["system_one(examples=...)"]
    B -->|"mapping entry for this question id, or the bare sequence"| U
    B -->|"empty"| C["SystemOneClient(examples=...)"]
    C -->|"mapping entry, or the bare sequence"| U
    C -->|"empty"| D["no examples for this question"]
```

--------------------------------

### Run duck client example

Source: https://github.com/zhulinchng/jevper/blob/main/docs/complete-example.md

Runs the duck client example with no server, using a constant model.

```sh
python examples/duck_client.py
```

--------------------------------

### Install and run CLM with vLLM encoder

Source: https://github.com/zhulinchng/jevper/blob/main/docs/local-servers.md

Install contrastive-lm and start the CLM server with a vLLM-served Qwen3-8B encoder. The encoder runs on port 8090 with pooling runner and last-token embeddings; clm-serve serves the API and playground on :8700 and fetches the 75 MB head into ~/.cache/clm/ on first run.

```bash
pip install contrastive-lm

# the encoder: Qwen3-8B, pooling runner, last-token embeddings
vllm serve Qwen/Qwen3-8B --served-model-name qwen3-8b --runner pooling \
  --max-model-len 2048 --port 8090 &

# the API and playground on :8700; fetches the 75 MB head into ~/.cache/clm/ on first run
clm-serve
```

--------------------------------

### Install and run kev from source

Source: https://github.com/zhulinchng/jevper/blob/main/docs/local-servers.md

Install kev from source because the PyPI package named 'kev' is a different project (K.E.V. ORM). The serve command runs the Kev-4B model on port 8009; the first run downloads the adapter and base model from HuggingFace.

```bash
git clone https://github.com/jaredpalmer/kev.git && cd kev
pip install -e ".[serve]"
python -m kev.serve --run jaredpalmer/kev-4b --port 8009
```

--------------------------------

### Install incident-triage with uv

Source: https://github.com/zhulinchng/jevper/blob/main/examples/incident-triage/README.md

Creates a virtual environment and installs the project in editable mode from the wheel.

```sh
uv venv && uv pip install -e .
```

--------------------------------

### Define few-shot examples with Example(state, answer, probabilities)

Source: https://github.com/zhulinchng/jevper/blob/main/docs/few-shot.md

Constructs two few-shot examples for the System One wire format. The first uses a label answer; the second includes a probabilities distribution, which is only rendered by method="structured" but is validated for every method before any provider call.

```python
Example(state="Charged twice for one order", answer="billing")
Example(state="Login fails after reset", answer="technical", probabilities={"billing": 0.05, "technical": 0.9})
```

--------------------------------

### Attach few-shot examples at question, call, and constructor levels

Source: https://github.com/zhulinchng/jevper/blob/main/README.md

Attach few-shot examples as chat turns at three levels: question, per call, and constructor. Precedence is question → per call → constructor, and the first non-empty level wins. `examples` is excluded from `model_dump()`, so question dumps keep exactly the Jev wire keys.

```python
from jevper import Choice, Example, SystemOneClient

question = Choice(
    criteria={"billing": "...", "technical": "..."},
    examples=[Example(state="Charged twice for one order", answer="billing")],
)

client = SystemOneClient(OpenAI(), model="gpt-5.6-terra",
                         examples=[Example(state="Login fails", answer="technical")])  # fallback for every question

client.system_one(state=..., questions={"intent": question},
                  examples={"intent": [...]})  # or a bare sequence for all questions
```

--------------------------------

### Install jevper in editable mode with uv

Source: https://github.com/zhulinchng/jevper/blob/main/docs/getting-started.md

Installs the package in editable mode for a local checkout, using uv.

```sh
uv pip install -e .
```

--------------------------------

### Create a decision with SystemOneClient

Source: https://github.com/zhulinchng/jevper/blob/main/docs/decisions.md

Use client.decisions.create with a choice question. The response contains answers in question order and a usage object. The example prints the answer name, choice, confidence, probabilities, and total tokens.

```python
from openai import OpenAI
from jevper import SystemOneClient

client = SystemOneClient(OpenAI(), model="gpt-5.6-terra")

decision = client.decisions.create(
    input="I was charged twice for the same subscription this month.",
    questions=[
        {
            "type": "choice",
            "name": "department",
            "instructions": "Which department should handle this complaint?",
            "choices": [
                {"value": "billing", "description": "Payments, invoices, and refunds."},
                {"value": "technical", "description": "Problems using the product."},
                {"value": "sales", "description": "Pricing, plans and upgrades."},
            ],
        }
    ],
)

answer = decision.answers[0]
print(answer.name, answer.choice, answer.confidence)
print([(entry.value, entry.probability) for entry in answer.probabilities])
print(decision.usage.total_tokens)
```

--------------------------------

### SystemOneClient with OpenAI base_url and endpoint name as model

Source: https://github.com/zhulinchng/jevper/blob/main/docs/mlflow.md

Client setup for the standalone gateway. The base_url points to the gateway's /v1 root and the model must be the endpoint name (local-chat), not the backend model name; a wrong name is refused by the gateway. With method="auto", the client pays one 404 for the missing /v1/responses route and answers on chat.

```python
client = SystemOneClient(
    OpenAI(base_url="http://127.0.0.1:5000/v1", api_key="dummy"),   # the route's /v1 root
    model="local-chat",                                             # the endpoint name is the model
    method="auto",
)
```

--------------------------------

### Build and preview documentation with MkDocs

Source: https://github.com/zhulinchng/jevper/blob/main/docs/contributing.md

Run the strict build from the repository root to validate the documentation, then preview the rendered site locally. The build must pass before a documentation change is submitted.

```sh
uv run --extra docs mkdocs build --strict
```

```sh
uv run --extra docs mkdocs serve
```

--------------------------------

### Install dependencies for complete_call.py

Source: https://github.com/zhulinchng/jevper/blob/main/docs/complete-example.md

Install the required packages. The comment notes that the complete_call.py program uses the OpenAI client and Chat Completions only; to use Anthropic's Messages surface, install anthropic and pin api="messages".

```sh
pip install jevper openai
# The complete_call.py program uses the OpenAI client and Chat Completions only.
# To use Anthropic's Messages surface, install anthropic and pin api="messages".
```

--------------------------------

### Install jevper with pip

Source: https://github.com/zhulinchng/jevper/blob/main/README.md

Install the jevper package with pip. Requires Python 3.10 or newer and the pydantic>=2.7 runtime dependency.

```sh
pip install jevper
```

--------------------------------

### SystemOneClient

Source: https://github.com/zhulinchng/jevper/blob/main/docs/api.md

Constructor documentation for the SystemOneClient class, including parameters and usage notes.

```APIDOC
## SystemOneClient

### Description
Creates a client for the Jev service, wrapping an underlying OpenAI or Anthropic SDK client. It provides a unified interface for making decisions via the Decisions API.

### Constructor
```python
SystemOneClient(client, *, model, method="auto", api="auto", reasoning=None, examples=(),
                structured_outputs=True, normalize_probabilities=True, top_logprobs=20,
                max_concurrency=8, n_retry_malformed=1, retry=None, temperature=None,
                extra_body=None, extra_headers=None, prompt_cache_key=None,
                native=False, noul_requires_question=True)
```

### Parameters
- **client** (object) - Required - Any object exposing `responses.create`, `chat.completions.create` and/or `messages.create` (or `post` when `api="systemone"`). The client stays owned by the caller.
- **model** (str) - Required - Model id sent with every call; overridable per `system_one` call.
- **method** (str) - Optional - Default `"auto"`. One of `"auto"`, `"logprobs"`, `"grammar"`, `"structured"` or `"discrete"`.
- **api** (str) - Optional - Default `"auto"`. One of `"auto"`, `"chat_completions"`, `"responses"` or `"messages"`.
- **reasoning** (ReasoningConfig) - Optional - Default `None`. A `ReasoningConfig`; `None` disables reasoning entirely.
- **examples** (tuple or dict) - Optional - Default `()`. Default few-shot examples: a sequence for all questions, or a mapping keyed by question id.
- **structured_outputs** (bool) - Optional - Default `True`.
- **normalize_probabilities** (bool) - Optional - Default `True`.
- **top_logprobs** (int) - Optional - Default `20`.
- **max_concurrency** (int) - Optional - Default `8`.
- **n_retry_malformed** (int) - Optional - Default `1`.
- **retry** (RetryPolicy) - Optional - Default `None`.
- **temperature** (float) - Optional - Default `None`.
- **extra_body** (dict) - Optional - Default `None`.
- **extra_headers** (dict) - Optional - Default `None`.
- **prompt_cache_key** (str) - Optional - Default `None`.
- **native** (bool) - Optional - Default `False`.
- **noul_requires_question** (bool) - Optional - Default `True`.

### Notes
- Only `client` is positional; every other option is keyword-only.
- Unknown keyword arguments raise `TypeError` from the constructor.
- The client object must not be asynchronous when using `SystemOneClient`; use `AsyncSystemOneClient` for async clients.
- The client's own retry loop is disabled when used through this client; `RetryPolicy` is the only retry loop.
- The `responses` surface speaks both OpenAI's Responses API and the OpenResponses specification.

### Example
```python
from jevper import SystemOneClient
from openai import OpenAI

client = OpenAI(base_url="https://api.typesafe.ai/v1", api_key="...")
system_one = SystemOneClient(client, model="gpt-4o")
```
```

--------------------------------

### InvalidQuestionError for example answer not matching any option

Source: https://github.com/zhulinchng/jevper/blob/main/docs/few-shot.md

Error raised when an example answer does not match any option of the question. Shown as a validation failure before any provider call.

```text
InvalidQuestionError: question 'intent': example 1: answer 'nope' does not match any option of this question
```

--------------------------------

### Python client for opencode Zen with SystemOneClient

Source: https://github.com/zhulinchng/jevper/blob/main/docs/jev-comparison.md

Configures SystemOneClient to point at the opencode Zen host. The base URL must be the host, not the endpoint, because jevper appends /systemone itself. Using the full endpoint causes a 404 with an HTML page reported as ProviderError.

```python
client = SystemOneClient(
    OpenAI(base_url="https://opencode.ai/zen/v1", api_key=key),
    model="jev-1.13-free",
    api="systemone",
)
```

--------------------------------

### model_dump() excludes examples

Source: https://github.com/zhulinchng/jevper/blob/main/docs/few-shot.md

Shows that examples are excluded from the dump because the field is declared with Field(exclude=True). The output confirms the dump contains only type, instructions, and criteria.

```python
Choice(criteria={"billing": "...", "technical": "..."}, examples=[...]).model_dump()
# {"type": "choice", "instructions": None, "criteria": {"billing": "...", "technical": "..."}}
```

--------------------------------

### Connect to kev with SystemOneClient

Source: https://github.com/zhulinchng/jevper/blob/main/docs/local-servers.md

Configure a SystemOneClient to talk to a local kev server. The base_url points at kev's /v1 endpoint and the model is set to kev-latest; the api parameter is "systemone" to match TypeSafe's System One API.

```python
from openai import OpenAI
from jevper import SystemOneClient

client = SystemOneClient(
    OpenAI(base_url="http://127.0.0.1:8009/v1", api_key="local"),
    model="kev-latest",
    api="systemone",
)
```

--------------------------------

### Install jevper agent skill

Source: https://github.com/zhulinchng/jevper/blob/main/README.md

Add the official Agent skill for jevper using npx. This is an optional integration for agent environments.

```sh
npx skills add zhulinchng/jevper-skill
```

--------------------------------

### Connect jevper to a local server with SystemOneClient

Source: https://github.com/zhulinchng/jevper/blob/main/docs/local-servers.md

Connects jevper to a local OpenAI-compatible server (Ollama at 127.0.0.1:11434) by passing a custom base_url and api_key. Uses extra_body to disable thinking and reads a real probability distribution from the server's logprobs.

```python
from openai import OpenAI
from jevper import Choice, SystemOneClient

client = SystemOneClient(
    OpenAI(base_url="http://127.0.0.1:11434/v1", api_key="local"),
    model="qwen3.5:9b",
    extra_body={"reasoning": {"effort": "none"}},  # Responses thinking off; see below
)

response = client.system_one(
    state="I was charged twice for the same subscription this month.",
    questions={
        "intent": Choice(
            criteria={
                "billing": "money, invoices, refunds, charges",
                "technical": "errors, crashes, login or performance problems",
                "sales": "pricing, plans, purchasing, upgrades",
            }
        )
    },
)

response.answers["intent"].probabilities  # a real distribution, read from the server's logprobs
```

--------------------------------

### SystemOneClient with OpenAI gpt-5.6 for prompt caching

Source: https://github.com/zhulinchng/jevper/blob/main/README.md

Create a SystemOneClient for OpenAI's gpt-5.6 model. Call system_one twice with different states but the same rubric to demonstrate prompt caching: the shared prefix is reused. The client is shaped so the state comes last, enabling prefix caching.

```python
client = SystemOneClient(OpenAI(), model="gpt-5.6")

client.system_one(state=record_a, questions=rubric)
client.system_one(state=record_b, questions=rubric)   # the shared prefix is reused
```

--------------------------------

### list_models() / alist_models()

Source: https://github.com/zhulinchng/jevper/blob/main/docs/api.md

Documentation for the list_models() and alist_models() methods, which retrieve the models offered by the deployment from the System One endpoint's GET /v1/models.

```APIDOC
## list_models() / alist_models()

### Description
Lists the models offered by the deployment, read from the System One endpoint's `GET /v1/models` and parsed as the service's own OpenAPI declares it. The synchronous version is `list_models()` and the async twin is `alist_models()`.

### Method
GET

### Endpoint
/v1/models

### Parameters
None

### Request Example
None

### Response
#### Success Response (200)
- **models** (array) - List of model metadata objects.
  - **name** (string) - The model name.
  - **description** (string) - Model description; defaults to `""` if omitted.
  - **release_date** (string) - Release date; defaults to `None` if omitted.

#### Response Example
{
  "models": [
    {
      "name": "model-name",
      "description": "model description",
      "release_date": "2024-01-01"
    }
  ]
}

### Error Handling
- If the gateway answers with a different shape (e.g., OpenAI-style `{"object": "list", "data": [...]}`), the call raises `MalformedAnswerError`.
- Failures follow the `retry` policy and arrive as a `ProviderError` carrying the provider's status.
- A client whose `get` is a coroutine (or is not) is refused before the request, naming the class to use instead.
- The SDK's own retry loop is switched off for this request, so one call is one policy rather than two nested ones.
```

--------------------------------

### Python client for System One with openai and jevper

Source: https://github.com/zhulinchng/jevper/blob/main/docs/jev-comparison.md

Creates a SystemOneClient using an OpenAI client with a custom base URL and the systemone API. The client object is duck-typed, so any object with post(path, body=…, cast_to=…) works. The model is set to "jev-latest" and the api to "systemone".

```python
from openai import OpenAI
from jevper import Choice, Noul, SystemOneClient

client = SystemOneClient(
    OpenAI(base_url="https://api.typesafe.ai/v1", api_key=key), model="jev-latest", api="systemone"
)

response = client.system_one(
    state="I was charged twice for the same order. Can someone look into this?",
    questions={
        "refund": Noul(instructions="Is the customer asking for a refund?"),
        "team": Choice(instructions="Which team should handle this?",
                       criteria={"billing": "Payments", "support": "Customer problems"}),
    },
)
```

--------------------------------

### Use system_one for the underlying Jev interface

Source: https://github.com/zhulinchng/jevper/blob/main/docs/index.md

Call client.system_one directly to get answers keyed by your own question IDs. The response can be converted to the Decisions schema with response.to_decision().

```python
from jevper import Choice, SystemOneClient

client = SystemOneClient(OpenAI(), model="gpt-5.6-terra")

response = client.system_one(
    state="I was charged twice for the same subscription this month.",
    questions={
        "intent": Choice(
            instructions="Pick the intent of the message.",
            criteria={
                "billing": "money, invoices, refunds, charges",
                "technical": "errors, crashes, login or performance problems",
                "sales": "pricing, plans, purchasing, upgrades",
            },
        )
    },
)

answer = response.answers["intent"]
print(answer.choice, answer.probabilities, answer.confidence)
```
Tokens:
Raw