Laya

https://nandhakishorm.github.io/laya/
Admin
Laya is an on-device, non-autoregressive System 1 decision engine that computes calibrated discrete...
Tokens:66,847
Snippets:422
Trust Score:9.7
Update:1 week ago
Show doc for...
Tokens:
Raw
### Install and launch the MCP server

Source: https://nandhakishorm.github.io/laya/cli-mcp

Install the optional MCP dependency and start the server. The server speaks MCP over stdio, not HTTP. The module form `python -m laya.mcp.server` is equivalent to the console script.

```bash
python -m pip install "laya[mcp]"
laya-mcp-server
# equivalent module form:
python -m laya.mcp.server
```

--------------------------------

### Example optimizer budget output

Source: https://nandhakishorm.github.io/laya/finetune

Example output printed by --dry-run and at the start of a run, showing the post-calibration training count and effective batch.

```text
optimizer budget: 1100 train item(s), effective batch up to 64 (micro_batch 8 x grad_accum 8), 72 optimizer updates over 4 epoch(s)

```

--------------------------------

### Install laya and view help

Source: https://nandhakishorm.github.io/laya/cli-mcp

Installs the package and shows the complete option list for the `laya` entry point.

```bash
python -m pip install laya
laya --help
```

--------------------------------

### Run Laya with example compose override

Source: https://nandhakishorm.github.io/laya/docker

Runs Laya using the example compose file as an override, which provides commented configuration with request, checkpoint, and secret-file mounts.

```bash
docker compose -f compose.yaml -f compose.example.yml run --build --rm laya

```

--------------------------------

### Start HTTP serving with laya-serve using compose.http.yaml

Source: https://nandhakishorm.github.io/laya/docker

Starts the laya-serve service alongside the laya service using the HTTP override file. The service publishes on 127.0.0.1 only and has no authentication until LAYA_API_KEY is set. The health endpoint does not require authentication.

```bash
docker compose -f compose.yaml -f compose.http.yaml up --build laya-serve
curl -s localhost:8000/health
curl -s localhost:8000/v1/systemone -H 'content-type: application/json' \
  --data @examples/docker/request.json

```

--------------------------------

### Install documentation requirements with uv pip

Source: https://nandhakishorm.github.io/laya/fast-backends

Installs documentation requirements using uv pip. Expected to check 3 packages (already installed) and note that the virtualenv has no pip.

```bash
uv pip install --python /home/ckl/projects/S/laya/.venv/bin/python -r requirements-docs.txt
```

--------------------------------

### Run CPU quickstart with docker compose

Source: https://nandhakishorm.github.io/laya/docker

Builds the checkout and runs the sample request on CPU. Requires Docker Engine or Docker Desktop with Compose v2 or newer, 8 GB RAM, and 10 GB free disk. The first request downloads a public Hugging Face checkpoint (no account needed) and prints JSON with choice, score, and noul. Weights are stored in a named volume; subsequent runs use the same command without --build.

```bash
docker compose run --build --rm laya

```

--------------------------------

### Run NVIDIA GPU/CUDA quickstart with docker compose

Source: https://nandhakishorm.github.io/laya/docker

Runs the sample on GPU using the CUDA override file. Requires a compatible NVIDIA driver and the NVIDIA Container Toolkit. The override selects GPU 0 and defaults to LAYA_DEVICE=cuda. Set LAYA_GPU_ID to use another host index or UUID. Rebuild when switching between CPU and CUDA configurations.

```bash
docker compose -f compose.yaml -f compose.cuda.yaml run --build --rm laya

```

--------------------------------

### Start the shared HTTP server with environment variables

Source: https://nandhakishorm.github.io/laya/cli-mcp

Launches the laya-serve HTTP server with environment variables LAYA_HOST, LAYA_PRELOAD, and LAYA_IDLE_UNLOAD_SECONDS. The server is used by MCP clients via HTTP, while MCP still uses stdio with the editor.

```bash
LAYA_HOST=127.0.0.1 LAYA_PRELOAD=0 LAYA_IDLE_UNLOAD_SECONDS=300 laya-serve

```

--------------------------------

### laya-evals CLI quickstart: validate, run, compare

Source: https://nandhakishorm.github.io/laya/evals

Shows the three main CLI commands: validate a dataset format, run a labelled set against a checkpoint with thresholds and a baseline, and compare a saved report to a baseline. Exit codes: 0 on success, 1 on threshold/baseline failure, 2 on usage error. The 'run' command prints metrics to stdout and writes JSON/Markdown reports when flags are given.

```bash
# check the format without a model
laya-evals validate research/evals/fixture.jsonl

# score a labelled set on one checkpoint, with thresholds and a baseline
laya-evals run data.jsonl --model english --device cpu \
    --min-accuracy 0.8 --max-ece 0.05 --score-within 0.25 --slice language \
    --json report.json --markdown report.md

# compare a saved report to a baseline
laya-evals compare report.json --baseline baseline.json --tolerance choice_accuracy=0.02

```

--------------------------------

### Start laya-serve with CUDA override

Source: https://nandhakishorm.github.io/laya/docker

Adds the CUDA override for NVIDIA support. It repeats the build args and device reservation for laya-serve because it is a separate service and overrides for laya never reach it.

```bash
docker compose -f compose.yaml -f compose.http.yaml -f compose.cuda.yaml up --build laya-serve

```

--------------------------------

### Serve HTTP API on DGX Spark

Source: https://nandhakishorm.github.io/laya/docker-platforms

Starts the HTTP API service on the DGX Spark by adding compose.http.yaml to the Spark override. The HTTP serving settings apply unchanged.

```bash
docker compose -f compose.yaml -f compose.http.yaml -f compose.spark.yaml up --build laya-serve

```

--------------------------------

### Build and run laya-serve with ModelScope via Compose

Source: https://nandhakishorm.github.io/laya/docker

Starts the laya-serve service using the ModelScope override. The MODELSCOPE_MODEL argument defaults to 'multilingual'. For NVIDIA, add '-f compose.cuda.yaml' before 'up'; ordering between overrides does not matter.

```bash
docker compose -f compose.yaml -f compose.http.yaml -f compose.modelscope.yaml up --build laya-serve

```

--------------------------------

### Build with specific ModelScope model types via Compose

Source: https://nandhakishorm.github.io/laya/docker

Shows how to select the 'english' or 'all' checkpoint by setting MODELSCOPE_MODEL. Each command builds and starts laya-serve with the corresponding baked model.

```bash
MODELSCOPE_MODEL=english docker compose -f compose.yaml -f compose.http.yaml -f compose.modelscope.yaml up --build laya-serve

```

```bash
MODELSCOPE_MODEL=all docker compose -f compose.yaml -f compose.http.yaml -f compose.modelscope.yaml up --build laya-serve

```

--------------------------------

### Decide with a JSON schema

Source: https://nandhakishorm.github.io/laya/structured

Pass a JSON schema to agent.decide() to get typed values. The schema must be an object with properties; each property becomes a question. The example shows enum, integer, and boolean properties.

```python
import laya

schema = {
    "type": "object",
    "properties": {
        "department": {"type": "string", "enum": ["billing", "support", "sales"],
                       "description": "Which team should handle this?"},
        "urgency": {"type": "integer", "minimum": 0, "maximum": 2},
        "needs_human": {"type": "boolean"},
    },
}

agent = laya.load("convaiinnovations/laya")
values = agent.decide("I was charged twice, refund me.", schema=schema)
# {"department": "billing", "urgency": 2, "needs_human": True}
```

--------------------------------

### Install laya with CrewAI extra

Source: https://nandhakishorm.github.io/laya/crewai

Install the laya package with the crewai extra using pip.

```bash
pip install "laya[crewai]"
```

--------------------------------

### Run laya-serve with LAYA_API_KEY_FILE from a mounted file

Source: https://nandhakishorm.github.io/laya/docker

Runs the laya-serve service with a read-only mounted API key file, setting LAYA_API_KEY_FILE so the key is read at startup and moved into LAYA_API_KEY. The _FILE variable is removed before the server execs.

```bash
docker compose -f compose.yaml -f compose.http.yaml run --rm \
  --volume "$PWD/laya_api_key:/run/secrets/laya_api_key:ro" \
  -e LAYA_API_KEY_FILE=/run/secrets/laya_api_key \
  --service-ports laya-serve

```

--------------------------------

### agent.backend_object

Source: https://nandhakishorm.github.io/laya/compile-and-fast-path

Exposes the installed backend object.

```APIDOC
## agent.backend_object

### Description
Property that returns the backend object currently installed on the agent.

### Method
Property access

### Returns
- **backend_object** (object) - The installed backend object.

### Example
```python
obj = agent.backend_object
```
```

--------------------------------

### View training CLI help

Source: https://nandhakishorm.github.io/laya/cli-mcp

Shows the help for the training CLI, also reachable as `laya train`.

```bash
laya-train --help
```

--------------------------------

### agent.deaccelerate()

Source: https://nandhakishorm.github.io/laya/compile-and-fast-path

Removes a backend installed through the class layer.

```APIDOC
## agent.deaccelerate()

### Description
Removes the backend that was installed via the class layer, reverting to the default behavior.

### Method
Method call

### Parameters
None

### Example
```python
agent.deaccelerate()
```
```

--------------------------------

### Reproduce the fine-tuned browser agent

Source: https://nandhakishorm.github.io/laya/finetune_browser_agent

Commands to download the model and run the verification script. The first command downloads the checkpoint, the second syncs dependencies, and the third runs the verification on a recorded browser step.

```bash
huggingface-cli download cklxx/laya-browser --local-dir laya-browser
cd laya-browser/code && uv sync --extra fast
uv run python verify.py v10s            # downloads the checkpoint, answers one recorded browser step

```

--------------------------------

### Baseline comparability failure output

Source: https://nandhakishorm.github.io/laya/evals

Example output when the baseline is not comparable, showing the key and both values.

```text
FAIL: baseline is not comparable: dataset_sha256 (dataset bytes): baseline is <sha>, this run is <sha>

```

--------------------------------

### Run Laya with Docker Compose and Docker build/run

Source: https://nandhakishorm.github.io/laya/docker

Shows how to run Laya with Docker Compose using environment variables, and how to build and run a Docker image directly with runtime environment variables and a cache volume mount.

```bash
LAYA_MODEL=english OMP_NUM_THREADS=2 docker compose run --build --rm laya

docker build -t laya:local .
docker run --rm -e LAYA_MODEL=english -e OMP_NUM_THREADS=2 \
  -v laya-model-cache:/home/laya/.cache/huggingface laya:local

```

--------------------------------

### Decide with a pydantic model

Source: https://nandhakishorm.github.io/laya/structured

Use a pydantic BaseModel as the schema. Requires installing laya[structured]. The model fields map to the same question types as the JSON schema rows.

```python
from typing import Literal
from pydantic import BaseModel

class Ticket(BaseModel):
    department: Literal["billing", "support", "sales"]
    urgency: Literal[0, 1, 2]
    needs_human: bool

ticket = agent.decide("I was charged twice, refund me.", schema=Ticket)
```

--------------------------------

### Build and run laya-serve with ModelScope using plain Docker

Source: https://nandhakishorm.github.io/laya/docker

Builds a local image with the multilingual checkpoint and runs it with HF_HUB_OFFLINE=1 to serve the baked weights. The container listens on 127.0.0.1:8000.

```bash
docker build --build-arg MODELSCOPE_MODEL=multilingual \
  -t laya:local .
docker run --rm -e HF_HUB_OFFLINE=1 -p 127.0.0.1:8000:8000 laya:local laya-serve

```

--------------------------------

### Set LAYA_PORT with docker compose up

Source: https://nandhakishorm.github.io/laya/docker

Sets LAYA_PORT to 9000 and starts the laya-serve service with the HTTP compose file. The port is both the published host port and the container bind port.

```bash
LAYA_PORT=9000 docker compose -f compose.yaml -f compose.http.yaml up --build laya-serve

```

--------------------------------

### Get detailed DecisionResult with return_details=True

Source: https://nandhakishorm.github.io/laya/structured

Pass `return_details=True` to `agent.decide` to get a `DecisionResult` with per-field confidence, probabilities, raw answers, usage, and routing. The `usage` dict includes `input_tokens`, `output_tokens` (always 0), `state_tokens`, `state_tokens_dropped`, `truncated`, and `truncated_questions`; an optional `options` key appears only when a head budget leaves some question's options sharing a token span. `answer_confidence` is `max(p)` and is what `min_confidence` gates on, while `confidence` is normalized entropy for `choice`/`score` fields and `max(p_true, 1 - p_true)` for `noul` fields.

```python
result = agent.decide(state, schema=Ticket, return_details=True)
result.values["department"]            # "billing"
result.answer_confidence["department"] # 0.94  max(p): the quantity min_confidence gates on
result.confidence["department"]        # 0.79  1 - H(p)/log(k) on this 3-option `choice` field
result.probabilities["department"]     # {"billing": 0.94, "support": 0.06, "sales": 0.0}
result.usage                           # {"input_tokens": ..., "output_tokens": 0,
                                       #  "state_tokens": ..., "state_tokens_dropped": ...,
                                       #  "truncated": ..., "truncated_questions": [...]}
result.routing                         # the Router decision, when a Router answered
```

--------------------------------

### Build documentation with zensical

Source: https://nandhakishorm.github.io/laya/fast-backends

Builds documentation with zensical in strict mode. Expected output: 'No issues found' and no 'griffe:' lines.

```bash
zensical build --strict --clean
```

--------------------------------

### Run laya-train from a CSV file

Source: https://nandhakishorm.github.io/laya/finetune

Builds training items from a CSV file, fine-tunes a model, and saves a checkpoint. The label column becomes a choice question over its distinct values in order of first appearance, with id from --question-id (default 'label') and instructions from --instructions. The question is saved as questions.json beside the checkpoint.

```bash
laya-train --data tickets.csv --text-column body --label-column department \
           --base english --out ./tickets-checkpoint

```

--------------------------------

### Fine-tune Laya from Python using laya.train.finetune

Source: https://nandhakishorm.github.io/laya/finetune

Shows how to call `laya.train.finetune` with a training file, language, checkpoint directory, and a `TrainConfig` that sets `loss` to `soft-ce` and enables `shuffle_options` for the 'choice' field. The returned report includes training item count, skipped items, and calibration data. After fine-tuning, the model is loaded with `laya.load`.

```python
import laya
from laya.train import TrainConfig, finetune

report = finetune("train.jsonl", "english", "./my-checkpoint",
                  TrainConfig(loss="soft-ce", shuffle_options=("choice",)))
print(report["train_items"], report["skipped"], report["calibration"])

agent = laya.load("./my-checkpoint")

```

--------------------------------

### Shadow logger and router setup

Source: https://nandhakishorm.github.io/laya/staged-adoption

Sets up a Laya Router with a shadow logger hook that records prediction results. The handle function catches exceptions around router.predict and always runs the incumbent action, ensuring Laya's decision is not executed.

```python
from laya import Router

class ShadowLog:
    def on_predict_end(self, ctx):
        result = ctx.results[0] if ctx.results else None
        write_shadow_record({
            "run_id": ctx.run_id,
            "model": ctx.model,
            "decision": ctx.decision,
            "result": result,
            "elapsed_ms": ctx.elapsed_ms,
            "error": None if ctx.error is None else repr(ctx.error),
        })

router = Router(hooks=[ShadowLog()])

def handle(request):
    try:
        laya_result = router.predict(request.state, request.questions)
    except Exception as exc:
        record_laya_failure(request, exc)
        return run_incumbent_action(request)
    # The shadow result is recorded by the hook; do not execute it here.
    return run_incumbent_action(request)
```

--------------------------------

### JSONL training data example

Source: https://nandhakishorm.github.io/laya/finetune

Each line is one case and may ask several questions of any type. The 'expected' field holds one answer per question in the format laya-evals reads; 'gold' holds a teacher's probabilities per question. A file may mix both label shapes.

```json
{"state": "We were billed twice for March.",
 "questions": {"department": {"type": "choice", "instructions": "Which team should handle this?",
                              "criteria": {"billing": "invoices, refunds", "technical": "bugs, outages"}},
               "urgent": {"type": "noul", "instructions": "Does this need an answer today?"}},
 "expected": {"department": "billing", "urgent": true}}

```

--------------------------------

### Dry-run check before a long run

Source: https://nandhakishorm.github.io/laya/finetune

Builds training items with the tokenizer and config of --base without loading weights or training. Prints how many rows were read, how many items they produced, how many questions were skipped and why, and the optimizer-update budget.

```bash
laya-train --data tickets.csv --text-column body --label-column department --dry-run

```
Tokens:
Raw