NandhaKishorM /
项目仓库laya
Non-autoregressive System 1 decision engine. Typed choice, score and yes/no decisions over any text in a single forward pass, in 100+ languages, with a router that picks the right checkpoint per request.
Star 增减趋势
仅展示实际采集历史；历史不足或采集过期时不计算净增。
| 周期 | Star 净增 |
|---|---|
| 最近 24 小时 | 历史积累中 |
| 最近 7 天 | 历史积累中 |
| 最近 30 天 | 历史积累中 |
| 最近 365 天 | 历史积累中 |
仓库档案
创建时间：2026-09-18T04:46:33Z · 开始观察：2026-10-20T08:03:48Z
项目文档
<p align="center">
<picture>
<source media="(prefers-color-scheme: dark)" srcset="https://raw.githubusercontent.com/NandhaKishorM/laya/main/assets/logo-lockup-dark.png" />
<img src="https://raw.githubusercontent.com/NandhaKishorM/laya/main/assets/logo-lockup.png" alt="Laya" width="330" />
</picture>
</p>
**Multilingual, non-autoregressive System 1 decision engine.** Typed decisions over 100+ languages in a single forward pass — 33 ms — trained with reinforcement learning against strictly proper scoring rules (RLCD), with a router that picks the right checkpoint per request.
<div align="center">
[](https://colab.research.google.com/drive/15d4Yv__KHeHjshVb-6PRTfqVllxih2S3?usp=sharing)
[](https://pypi.org/project/laya/)
[](https://nandhakishorm.github.io/laya/)
[](https://huggingface.co/convaiinnovations/laya)
[](https://huggingface.co/convaiinnovations/laya-multilingual)
[](https://huggingface.co/spaces/convaiinnovations/laya-demo)
[](https://dev.to/nandakishor_m_6cc0adfde9f/i-built-non-autoregressive-decision-models-a-year-ago-then-a-frontier-lab-called-it-a-18me)
[](https://www.buymeacoffee.com/nandakishorm)
[](https://opensource.org/licenses/Apache-2.0)
</div>
## Installation
```bash
python -m pip install laya
```
With [uv](https://docs.astral.sh/uv/), run `uv add laya` in a uv project or `uv pip install laya` in a virtual environment.
Python 3.10 or newer. Optional extras: `laya[serve]` (HTTP server), `laya[mcp]` (MCP server), `laya[langchain]` (LangChain and LangGraph), `laya[llamaindex]` (LlamaIndex selectors), `laya[crewai]` (CrewAI routing), `laya[onnx]` (ONNX Runtime), `laya[fast]` (TileLang GPU fast path), `laya[structured]` (pydantic models in `decide`). Step-by-step setup for each platform, CPU-only or GPU PyTorch builds, and troubleshooting are in [Installation details](#installation-details).
For TypeScript / Node.js / browser, see [`laya-ts/`](https://github.com/NandhaKishorM/laya/tree/main/laya-ts/). npm releases (`npm install laya-ts`) are published from this repository's `laya-ts-v*` release tags.
**Long documents.** `laya-multilingual` reads up to 8,192 tokens with `max_len=8192`. Measured accuracy and time by document length, reproducible with [`research/scripts/bench_long_context.py`](https://github.com/NandhaKishorM/laya/blob/main/research/scripts/bench_long_context.py):
<p align="center">
<img src="https://raw.githubusercontent.com/NandhaKishorM/laya/main/assets/long_context_8192.png" alt="laya-multilingual with max_len=8192: 16 to 18 of 20 requests correct with up to about 4,000 tokens of text before them, more variable beyond" width="100%" />
</p>
## Quickstart
> **Long documents: `laya-multilingual` reads up to 8,192 tokens.** It ships with a 1,024-token limit that cuts long documents off, so pass `max_len=8192` for them:
>
> ```python
> result = router.predict(long_document, questions, model="multilingual", max_len=8192)
> ```
>
> In the table above, 16 to 18 of 20 requests were answered correctly with up to about 4,000 tokens of text before them; beyond that results vary (8 to 17 of 20), so check long-document accuracy on your own data. Short inputs give identical answers with `max_len=8192`, and speed follows the input's real length, not the limit: short inputs are unchanged, and a 4,000-token input takes about 1.7 s on an Apple GPU. Name the checkpoint with `model="multilingual"`, since long mostly-English text would otherwise route to the English checkpoint.
```python
from laya import Router
router = Router() # downloads a checkpoint on first use; Router(preload=True) loads all three up front
state = "Hi, we were billed twice for March. Please refund the duplicate today or we will cancel our plan."
questions = {
"department": {"type": "choice", "instructions": "Which department should handle this?",
"criteria": {"billing": "invoices, payments, refunds",
"technical": "bugs, outages, system errors",
"other": "everything else"}},
"urgency": {"type": "score", "instructions": "How urgent is this?",
"criteria": ["not urgent", "soon", "blocking"]},
"churn_risk": {"type": "noul", "instructions": "Does the user threaten to cancel or leave?"},
}
result = router.predict(state, questions)
print(result["answers"]["department"]["choice"]) # billing
print(result["answers"]["churn_risk"]["noul"]) # probability the answer is yes
print(result["routing"]["model"]) # english
```
The same call works in any of 100+ languages. The `Router` detects the script and language and sends non-English text to `laya-multilingual`:
```python
for text in ["मुझसे मार्च में दो बार शुल्क लिया गया, कृपया डुप्लिकेट राशि वापस करें।",
"La aplicación se cierra cada vez que abro la configuración."]:
r = router.predict(text, {"department": questions["department"]})
print(r["routing"]["model"], r["answers"]["department"]["choice"])
# multilingual billing
# multilingual technical
```
From the command line, `laya "My payment failed twice" --preset triage` answers a ready-made question set. More in the [full quickstart](#quickstart-route-mode-recommended) and the [docs](https://nandhakishorm.github.io/laya/).
## Fine-tune for better accuracy
The shipped checkpoints work zero-shot, but fine-tuning on decisions from your own domain is where accuracy jumps. On the typed-decisions benchmark (2,000 decisions across four workflows), the fine-tuned `laya-typed-decisions` checkpoint scores **0.766** accuracy, against **0.362** for the base English checkpoint on the same decisions.
**[Fine-tuning notebook](https://github.com/NandhaKishorM/laya/blob/main/notebooks/laya_finetune_typed_decisions_2xT4_kaggle.ipynb)** (Kaggle 2xT4) and **[Apple Silicon script](https://github.com/NandhaKishorM/laya/blob/main/notebooks/laya_finetune_typed_decisions_mps.py)** (MPS / CPU): run the whole loop (build dataset, train with RLCD, calibrate temperatures, evaluate, export). Details in [Fine-Tuning](#fine-tuning).
## Documentation
**[nandhakishorm.github.io/laya](https://nandhakishorm.github.io/laya/)**: guides for [prediction hooks](https://nandhakishorm.github.io/laya/hooks/), [schema-driven decisions](https://nandhakishorm.github.io/laya/structured/), [Docker](https://nandhakishorm.github.io/laya/docker/) and [LangChain and LangGraph](https://nandhakishorm.github.io/laya/langchain/), plus a full [API reference](https://nandhakishorm.github.io/laya/reference/).
## What's new in 0.4.12
* **An opt-in reliability signal for `choice` answers.** `OptionStabilityHook` re-asks a question
with its options reordered and relabelled, scores every variant in the same forward pass, and
reports how often the answer survives. A clear request held at `stability=1.0` here; a vague one
answered `legal` at confidence 0.33 held at `0.17`, which is the case confidence alone does not
catch. `accept` and `escalate` are required arguments, not defaults, because the right cutoffs
depend on your option counts and on the calibration defect in #1245 (#908, issue #635). It costs
what the extra rows cost: on CPU, 1.98x for `variants="fast"` and 3.69x for `"full"`.
* **`collate_items` builds its tensors once.** The shared batch collator filled torch tensors row
by row; it now fills NumPy arrays and wraps them, which is 4.8x faster on a 128-row batch
(1.207 ms to 0.250 ms) and produces identical decisions (#1279). The ONNX agent reuses question
tokenizations across states in `predict_batch` the way the PyTorch agent already did (#1278).
* **`PredictLong` in the .NET SDK**, ported from `ONNXAgent.predict_long`, with golden fixtures
re-recorded from the Python package (#1283).
* **The shortlist docs now say which embedder to use.** Measured on 500 rows per dataset,
`embed_fn_from_agent` scores *below* asking one plain question (0.224 against 0.416 on
BANKING77), while a dedicated bi-encoder reaches 0.630, and embedding the user's message alone
rather than the instructions plus the JSON state is better again on CLINC150 and MASSIVE (#1281,
issue #102).
* **The logo was drawn semi-transparent** and is now solid ink, which is most visible on dark
backgrounds where the ring had been desaturating toward grey.
Earlier releases are in the [GitHub releases](https://github.com/NandhaKishorM/laya/releases).
---
<p align="center">
<img src="https://raw.githubusercontent.com/NandhaKishorM/laya/main/assets/laya_vs_jev_full.png" alt="Laya versus TypeSafe Jev: accuracy on shared public datasets, every application workflow, all 51 languages, speed, calibration, and the cost of not preloading" width="100%" />
</p>
Laya evaluates typed questions (`choice`, `score`, `noul`) over any state (text, email, ticket or JSON document) in **a single forward pass** — 33 ms for one question, 7.2 ms/question batched, measured on a T4. No text generation, so nothing to parse and nothing to hallucinate.
Three checkpoints, and a `Router` that picks between them per request:
| | encoder | params | context | use it for |
|---|---|---|---|---|
| [`laya`](https://huggingface.co/convaiinnovations/laya) | ModernBERT-large | 421M | 512 | English |
| [`laya-multilingual`](https://huggingface.co/convaiinnovations/laya-multilingual) | mmBERT-base | 322M | 1024 (up to 8,192) | 100+ languages, 2x faster |
| [`laya-typed-decisions`](https://huggingface.co/convaiinnovations/laya-typed-decisions) | ModernBERT-large | 421M | 1024 | the typed-decisions workflows |
## Where to go next
| document | what is in it |
|---|---|
| Quickstart below | the shortest path to a working call |
| [`examples/README.md`](https://github.com/NandhaKishorM/laya/blob/main/examples/README.md) | 41 runnable examples as an eight-stage learning path: first call → schema design → production service |
| [`LOCAL_SETUP.md`](https://github.com/NandhaKishorM/laya/blob/main/LOCAL_SETUP.md) | running this checkout on macOS/Intel: pinned stack, checkpoint manifest, the two portability fixes, every verification run and its result |
| [`BENCHMARKS.md`](https://github.com/NandhaKishorM/laya/blob/main/BENCHMARKS.md) | every benchmark run consolidated, all 51 languages, with the limits stated plainly |
| [`research/README.md`](https://github.com/NandhaKishorM/laya/blob/main/research/README.md) | the harnesses that produce those numbers, and how to re-run them |
## Installation details
Python 3.10 or newer. The dependencies set that floor: `huggingface_hub` 1.x, `transformers` 5.x and `torch` 2.14 all require 3.10.
**Optional PyTorch build selection:** If you need a CPU-only or GPU-specific PyTorch build, follow [PyTorch's installation guide](https://pytorch.org/get-started/locally/) after creating your virtual environment and before installing Laya. Replace `pip` or `pip3` in the selected command with the environment's Python executable followed by `-m pip`.
If you already use a virtual environment, install the PyPI release with:
```bash
python -m pip install laya
```
For a new environment, choose the commands for your platform below. Run them from your project directory; the explicit Python paths keep installation and verification in the same environment.
**macOS / Linux** (with Python 3.10 or newer):
On Debian/Ubuntu, the system Python may require `sudo apt install python3-venv` before creating a virtual environment. If `venv` reports that `ensurepip` is unavailable, install that package and retry.
```bash
python3 -m venv .venv
.venv/bin/python -m pip install laya
.venv/bin/python -I -c "import laya; print(laya.__version__)"
```
**Windows PowerShell** (this example uses an installed Python 3.11):
```powershell
py -3.11 -m venv .venv
.\.venv\Scripts\python.exe -m pip install laya
.\.venv\Scripts\python.exe -I -c "import laya; print(laya.__version__)"
```
Both checks print the installed Laya version without loading a checkpoint. `-I` excludes the current directory from the import search path, so a local source copy cannot mask a missing installation. Keep using the same virtual environment's Python when running your application.
**Intel GPU (XPU)**
Install a supported Intel GPU driver first. For an XPU-enabled PyTorch build, install its wheel before Laya; the default PyPI wheel may be CPU-only. PyTorch's validated hardware and OS list is in the [Intel GPU guide](https://docs.pytorch.org/docs/2.14/notes/get_start_xpu.html).
```powershell
.\.venv\Scripts\python.exe -m pip install torch --index-url https://download.pytorch.org/whl/xpu
.\.venv\Scripts\python.exe -m pip install laya
.\.venv\Scripts\python.exe -c "import torch; print(torch.xpu.is_available())"
```
For a source checkout, replace `pip install laya` with `pip install -e .`. Laya automatically selects an available XPU when no device is specified; you can also request one explicitly with `device="xpu"` in `laya.load()` or `Router(device="xpu")`.
**Install from GitHub**
To use the development version instead of the PyPI release, create the virtual environment above and replace its installation command with the appropriate command below. Git must be installed.
```bash
# macOS / Linux
.venv/bin/python -m pip install "git+https://github.com/NandhaKishorM/laya.git"
```
```powershell
# Windows PowerShell
.\.venv\Scripts\python.exe -m pip install "git+https://github.com/NandhaKishorM/laya.git"
```
Run the same version check afterward. The GitHub version follows the repository's default branch and may differ from the published release.
**Install with uv**
[uv](https://docs.astral.sh/uv/) creates the virtual environment, downloads a matching Python if none is installed, and installs into it. The commands are the same on macOS, Linux and Windows PowerShell:
```bash
uv venv --python 3.12
uv pip install laya
```
`uv pip install` targets the `.venv` in the current directory without activating it, so run the version check for your platform above afterward. Extras and the GitHub version install the same way: `uv pip install "laya[serve]"`, `uv pip install "git+https://github.com/NandhaKishorM/laya.git"`. For a CPU-only or GPU-specific PyTorch build, add `--torch-backend=auto` to pick the build that matches the machine's GPU driver, or name one such as `--torch-backend=cpu`; [uv's PyTorch guide](https://docs.astral.sh/uv/guides/integration/pytorch/) has the list.
If your application is a uv project, add Laya as a dependency instead:
```bash
uv add laya
uv run python -I -c "import laya; print(laya.__version__)"
```
`--torch-backend` applies to `uv pip` only; in a uv project, uv's PyTorch guide shows how to set the PyTorch index in `pyproject.toml`.
**Model setup and troubleshooting**
Continue with the [Router quickstart](#quickstart-route-mode-recommended) to run inference. Loading a Hub checkpoint requires access to Hugging Face on its first download; the quickstart's `Router(preload=True)` loads all three configured checkpoints at construction.
- **`ModuleNotFoundError: No module named 'laya'`:** run both installation and your script with the same virtual environment's Python executable shown above. In an editor, select that interpreter as well.
- **Missing `rl_agent_config.json`:** this file ships with a Laya checkpoint alongside `model.safetensors`; it is not a configuration file you need to create in the source repository. For a local model, pass the directory containing those checkpoint files.
- **Checkpoint downloads and cache:** `Router()` is lazy: the first prediction downloads only the
selected checkpoint, then reuses the Hugging Face cache. Set `HF_HUB_CACHE` to move that cache.
The English and typed-decisions checkpoints have about 421M parameters each; multilingual has
about 322M (see the table above). Download sizes also depend on weight precision and files.
- **Memory limits:** `Router()` keeps up to two checkpoints resident by default (`max_loaded=2`),
evicting the least recently used when needed. Use `Router(max_loaded=1)` to reduce resident
memory, at the cost of reloads when switching checkpoints. `Router(preload=True)` loads all
three up front and raises the resident limit to fit them; it is not an out-of-memory remedy.
- **Windows + Python 3.14:** the model-construction crash reported in
[#123](https://github.com/NandhaKishorM/laya/issues/123) was fixed in Laya 0.3.7 by
[#195](https://github.com/NandhaKishorM/laya/pull/195) and verified on the reporter's Windows 11
setup without a manual patch. Upgrade older Laya installations with your environment's
Python executable followed by `-m pip install -U laya`.
---
### Command line
Installing the package also installs a `laya` command for quick local testing, no script needed:
```bash
laya "I was charged twice, please refund" # routing decision only; works offline, no download
laya "Refactor this service" --predict # full answers (downloads the checkpoint on first use)
laya "Mein Konto wurde zweimal belastet" --lang de # force a language instead of detecting it
laya "My payment failed twice" --lang-guess en # a soft hint: nudge routing, still fall through to detection
laya "My payment failed twice" --model ml # pin a checkpoint: names, aliases and casing all resolve as the SDK resolves them
laya "My payment failed twice" --preset triage # answer a ready-made preset (triage, email, guard, moderation, router)
laya --batch tickets.txt --predict # score a file of requests, one per line, in one batch
cat tickets.txt | laya --batch - --predict --json # stdin; one JSON line of answers per request
laya "Where is my card" --questions intents.json # answer your own questions, written in a JSON file
laya "Refund my card" --predict --min-confidence 0.9 # mark an answer the model is unsure of
laya # interactive mode
```
`--lang` is decisive: a real code picks the checkpoint and skips detection. `--lang-guess` is the
soft sibling core checks in between, so a probable-but-uncertain language can nudge routing without
forcing it — and a hint that resolves to nothing falls through to the built-in detector, exactly the
`Router(lang_guess=...)` behaviour the library offers. Neither flag changes the routing decision's
precedence rules; `--lang-guess` just adds a hint those rules read before detection.
Routing alone never downloads a checkpoint, so it returns in milliseconds. `--predict` loads the routed checkpoint, which needs network access to the Hugging Face hub the first time; if a checkpoint cannot be downloaded, the CLI says so instead of crashing. `--batch` (with or without `--predict`) sends the whole file through `Router.predict_batch` in one process, so the requests share checkpoint loads and forward passes — measured 2.6x on 20 tickets vs looping `predict` one by one, with `--batch-size N` to bound the forward pass, `--sort-by-length` to group similarly sized requests inside it, and `--json` for JSONL output. Batch routing (`laya --batch FILE`, no `--predict`) likewise answers with `route_batch` in one pass, still without loading anything.
`--sort-by-length` reaches the length grouping `Agent.predict_batch` has done si