### Start reference server and generate reference outputs

Source: https://github.com/zhongkaifu/tensorsharp/blob/main/eng/validation/bonsai2.md

Starts the publisher's server with the exact model under test, F16 K/V caches, and a bounded context. The example uses four 2,048-token slots. Then runs the reference script to generate golden outputs.

```sh
artifacts/bonsai2/llama.cpp-prism/build/bin/llama-server \
  -m /path/to/Ternary-Bonsai-2-27B-PQ2_0.gguf -ngl 99 \
  -c 8192 -np 4 -fa on -ctk f16 -ctv f16 -b 512 -ub 512 \
  --host 127.0.0.1 --port 8083 --jinja
```

```sh
python3 eng/validation/bonsai2-bench.py reference \
  --url http://127.0.0.1:8083 --tokens 32 \
  --golden docs/validation/bonsai2/pq2-golden.json \
  --output docs/validation/bonsai2/prism-pq2-reference.json
```

--------------------------------

### Install TensorSharp and run with tensor parallelism

Source: https://github.com/zhongkaifu/tensorsharp/blob/main/docs/getting_started.md

Install the CUDA toolkit, then set up the environment, clone the repo, build, and run the CLI with --tp 2 on the cuda or ggml_cuda backend. The example includes a comment about setting LD_LIBRARY_PATH for RunPod Ubuntu 24.04 and adding the dotnet backports PPA for older Ubuntu.

```bash
# On RunPod's Ubuntu 24.04 images, point the loader at the CUDA compat libraries first:
export LD_LIBRARY_PATH=/usr/local/cuda-12.6/compat:$LD_LIBRARY_PATH
# On older Ubuntu releases the .NET 10 SDK comes from the backports PPA:
add-apt-repository ppa:dotnet/backports

apt update && apt install dotnet-sdk-10.0
git clone https://github.com/zhongkaifu/TensorSharp.git
cd TensorSharp
mkdir models
wget "https://huggingface.co/ggml-org/gemma-4-E4B-it-GGUF/resolve/main/gemma-4-E4B-it-Q8_0.gguf?download=true" -O models/gemma-4-E4B-it-Q8_0.gguf
bash TensorSharp.GGML.Native/build-linux.sh
dotnet build -c Release

# 2 GPUs in one process
TensorSharp.Cli/bin/TensorSharp.Cli --model models/gemma-4-E4B-it-Q8_0.gguf \
    --backend cuda --interactive --max-tokens 20000 --tp 2

# Same thing on the GGML CUDA backend (add TENSORSHARP_TP_DEVICES=0,2 to pick GPUs)
TensorSharp.Cli/bin/TensorSharp.Cli --model models/gemma-4-E4B-it-Q8_0.gguf \
    --backend ggml_cuda --interactive --max-tokens 20000 --tp 2
```

--------------------------------

### Print full option reference with --help

Source: https://github.com/zhongkaifu/tensorsharp/blob/main/docs/getting_started.md

Both the CLI and server print their full option reference when started with no arguments or with --help. This shows the description, default, range, and an example per flag.

```bash
dotnet run --project TensorSharp.Cli -c Release -- --help
dotnet run --project TensorSharp.Server.Host -c Release -- --help
```

--------------------------------

### Enable knowledge library and start chat server

Source: https://github.com/zhongkaifu/tensorsharp/blob/main/TensorSharp.Server.Host/API_EXAMPLES.md

Set the TS_KNOWLEDGE_DIRECTORY environment variable and start the chat server with the specified model, backend, host, and port. This must be done before starting the chat server.

```bash
export TS_KNOWLEDGE_DIRECTORY=/absolute/path/to/library
dotnet TensorSharp.Server.Host/bin/TensorSharp.Server.Host.dll \
  --model /absolute/path/to/chat-model.gguf --backend ggml_metal \
  --host 127.0.0.1 --port 5001
```

--------------------------------

### Verify .NET SDK installation with dotnet --list-sdks

Source: https://github.com/zhongkaifu/tensorsharp/blob/main/DEVELOPMENT.md

Run this command in a new terminal after installing the .NET 10 SDK to verify that a 10.0.x SDK appears. A runtime-only installation is not sufficient; the full SDK is required.

```bash
dotnet --list-sdks
```

--------------------------------

### macOS/MLX host setup and benchmark pipeline

Source: https://github.com/zhongkaifu/tensorsharp/blob/main/benchmarks/engine_comparison/README.md

One-time host setup for native libs, server, and a Metal llama-server build, followed by the same pipeline as CI: download models, run the matrix with ggml_metal and mlx backends, and generate the report.

```bash
# One-time host setup: native libs + server + a Metal llama-server build
bash TensorSharp.GGML.Native/build-macos.sh
bash TensorSharp.Backends.MLX/build-native-macos.sh
dotnet build TensorSharp.Server.Host/TensorSharp.Server.Host.csproj -c Release
git clone https://github.com/ggml-org/llama.cpp ~/tensorsharp-bench/llama.cpp
cmake -S ~/tensorsharp-bench/llama.cpp -B ~/tensorsharp-bench/llama.cpp/build \
    -DCMAKE_BUILD_TYPE=Release -DLLAMA_CURL=OFF -DLLAMA_BUILD_SERVER=ON
cmake --build ~/tensorsharp-bench/llama.cpp/build --config Release --target llama-server -j

# Models, benchmark, report — same pipeline as CI
cd benchmarks/engine_comparison
export BENCH_MODEL_ROOT=~/tensorsharp-bench/models
export BENCH_LLAMA_SERVER=~/tensorsharp-bench/llama.cpp/build/bin/llama-server
python3 download_models.py --config benchmark_config_ci.json --models gemma4-12b
python3 run_matrix.py --config benchmark_config_ci.json \
    --backends ggml_metal,mlx --models gemma4-12b
python3 report.py --config benchmark_config_ci.json
```

--------------------------------

### Start server for MiniMax-H3 video generation with audio

Source: https://github.com/zhongkaifu/tensorsharp/blob/main/USAGE.md

Generates video with 32 kHz stereo audio. Size, steps, and frame count are startup flags; 640x384 is the documented starting point. Each run writes an .mp4 and a sidecar .wav.

```bash
dotnet TensorSharp.Server.Host/bin/TensorSharp.Server.Host.dll \
    --model ./models/minimax_h3_fl2va_pruned-Q4_K.gguf --backend ggml_cuda \
    --video-width 640 --video-height 384 --video-steps 20 --video-frames 22
```

--------------------------------

### Start TensorSharp server with Web UI and APIs

Source: https://github.com/zhongkaifu/tensorsharp/blob/main/docs/models/mistral3.md

Starts the TensorSharp server, providing a Web UI and OpenAI/Ollama-compatible APIs on http://localhost:5000. Uses the specified model, projector, and backend.

```bash
dotnet run --project TensorSharp.Server.Host -c Release -- --model models/mistralai_Mistral-Small-3.1-24B-Instruct-2503-Q4_K_M.gguf \
  --mmproj models/mmproj-mistralai_Mistral-Small-3.1-24B-Instruct-2503-f16.gguf \
  --backend ggml_cuda --max-tokens 4096
```

--------------------------------

### Dockerfile: install dependencies, runtime, user, and copy build output

Source: https://github.com/zhongkaifu/tensorsharp/blob/main/TensorSharp.Server.Host/Dockers/Dockerfile_GPU.txt

Dockerfile steps for the TensorSharp GPU host: installs required packages (libgomp1, libicu70, curl, wget), installs the ASP.NET Core 10 runtime, creates a non-root user with UID 1000 (matching Hugging Face Spaces), and copies the build output. The first line comments note that libgomp1 is the OpenMP runtime for native GGML CPU kernels)Skip rest?

```Dockerfile
RUN apt-get update \
    && apt-get install -y --no-install-recommends \
        libgomp1 libicu70 curl wget ca-certificates \
    && rm -rf /var/lib/apt/lists/*
```

```Dockerfile
RUN curl -fsSL https://dot.net/v1/dotnet-install.sh -o /tmp/dotnet-install.sh \
    && bash /tmp/dotnet-install.sh --channel 10.0 --runtime aspnetcore --install-dir /usr/share/dotnet \
    && ln -sf /usr/share/dotnet/dotnet /usr/local/bin/dotnet \
    && rm /tmp/dotnet-install.sh
```

```Dockerfile
RUN useradd -m -u 1000 -s /bin/bash user \
    && mkdir -p /home/user/app/models \
    && chown -R 1000:1000 /home/user
USER 1000
ENV HOME=/home/user \
    PATH=/home/user/.local/bin:$PATH
WORKDIR /home/user/app
```

```Dockerfile
COPY --chown=1000:1000 --from=build /src/TensorSharp.Server.Host/bin/ ./
```

--------------------------------

### curl command to post traffic-light example

Source: https://github.com/zhongkaifu/tensorsharp/blob/main/docs/models/jev.md

Posts the pre-made traffic-light example JSON file to the SystemOne endpoint. The file contains an embedded image, so it can be sent as-is.

```bash
curl http://127.0.0.1:5000/v1/systemone \
  -H 'Content-Type: application/json' \
  --data-binary @docs/examples/jev-traffic-light.json
```

--------------------------------

### Download model and projector with huggingface_hub

Source: https://github.com/zhongkaifu/tensorsharp/blob/main/docs/models/mistral3.md

Installs the huggingface_hub package and downloads the Q4_K_M model and f16 mmproj files into the models directory. Requires pip install -U huggingface_hub.

```bash
python -m pip install -U huggingface_hub
hf download bartowski/mistralai_Mistral-Small-3.1-24B-Instruct-2503-GGUF mistralai_Mistral-Small-3.1-24B-Instruct-2503-Q4_K_M.gguf --local-dir models
hf download bartowski/mistralai_Mistral-Small-3.1-24B-Instruct-2503-GGUF mmproj-mistralai_Mistral-Small-3.1-24B-Instruct-2503-f16.gguf --local-dir models
```

--------------------------------

### Start llama-server and generate golden reference outputs

Source: https://github.com/zhongkaifu/tensorsharp/blob/main/docs/models/muse-glimmer.md

Starts a llama-server on port 8899 with the Muse-Glimmer GGUF, then runs the two reference-generation scripts against it. The server must be running before the Python scripts are invoked.

```bash
llama-server -m Muse-Glimmer-30B-UD-IQ2_XXS.gguf -ngl 99 -c 8192 --port 8899
```

```bash
python .parity/gen_ref.py      http://127.0.0.1:8899 .parity/ref_text.json
```

```bash
python .parity/gen_ref_long.py http://127.0.0.1:8899 .parity/ref_text_long.json
```

--------------------------------

### Skills field in GET /api/models

Source: https://github.com/zhongkaifu/tensorsharp/blob/main/TensorSharp.Server.Host/API_EXAMPLES.md

Shows the skills field returned by GET /api/models, indicating whether skills are enabled, installable, and script execution is allowed.

```json
"skills": { "enabled": true, "installable": true, "allowScripts": false, "count": 7 }
```

--------------------------------

### Skill object JSON structure

Source: https://github.com/zhongkaifu/tensorsharp/blob/main/docs/agent_skills.md

Example of a skill object returned by the API. origin is discovered for scanned skills and installed for uploaded ones; only installed skills can be deleted. /api/skills wraps the list with enabled, installable, skills, and errors.

```json
{
  "id": "pdf",
  "object": "skill",
  "name": "pdf",
  "description": "Extract text and tables from PDF files...",
  "license": "Apache-2.0",
  "compatibility": "Requires python3 with pypdf installed.",
  "files": [
    {"path": "scripts/extract_tables.py", "bytes": 4021, "kind": "script", "text": true},
    {"path": "references/forms.md", "bytes": 18233, "kind": "reference", "text": true}
  ],
  "bytes": 41288,
  "origin": "installed",
  "warnings": [],
  "modified": "2026-08-29T12:00:00Z"
}
```

--------------------------------

### Start server with configuration file (auto-downloads model)

Source: https://github.com/zhongkaifu/tensorsharp/blob/main/USAGE.md

Reads all settings from a reusable JSON file. The model is auto-downloaded on first run. The second command overrides the backend to CPU.

```bash
dotnet TensorSharp.Server.Host/bin/TensorSharp.Server.Host.dll --config config/server-basic.json
```

```bash
dotnet TensorSharp.Server.Host/bin/TensorSharp.Server.Host.dll --config config/server-basic.json --backend ggml_cpu
```

--------------------------------

### Quick start with wrapper script

Source: https://github.com/zhongkaifu/tensorsharp/blob/main/TensorAgent/skills/playwright/SKILL.md

Opens a page, clicks an element, types text, presses Enter, and takes a screenshot. Read the linked snapshot to obtain current element refs.

```bash
node "$PWCLI" open https://playwright.dev --headed
# Read the linked snapshot to obtain current element refs.
node "$PWCLI" click e15
node "$PWCLI" type "Playwright"
node "$PWCLI" press Enter
node "$PWCLI" screenshot
```

--------------------------------

### Start MiniMax-H3 video generation with Ref2VA checkpoint

Source: https://github.com/zhongkaifu/tensorsharp/blob/main/website/server.html

Command-line invocation to start the TensorSharp.Server host with the MiniMax-H3 Ref2VA checkpoint, using the same geometry as the FL2VA example. This checkpoint uses identity/appearance references into a new scene.

```bash
dotnet TensorSharp.Server.Host/bin/TensorSharp.Server.Host.dll \
    --model ./models/minimax_h3_ref2va_pruned-Q4_K.gguf --backend ggml_cuda \
    --video-width 640 --video-height 384 --video-steps 20 --video-frames 22
```

--------------------------------

### Launch CLI: set TENSORSHARP_MODELS, build, generate and edit with qwen-image-2.1

Source: https://github.com/zhongkaifu/tensorsharp/blob/main/docs/models/qwenimage21.md

Setup commands for launching the CLI. Set TENSORSHARP_MODELS to choose where models are stored (default resolves to ../models relative to config/). Build the CLI and server host projects, then run text-to-image or image editing with the qwen-image-2.1 config. Use --backend ggml_cuda on NVIDIA, --backend cpu for pure C#.

```bash
export TENSORSHARP_MODELS="$PWD/../models"
```

```bash
dotnet build TensorSharp.Cli/TensorSharp.Cli.csproj -c Release

```

```bash
dotnet build TensorSharp.Server.Host/TensorSharp.Server.Host.csproj -c Release

```

```bash
dotnet run --project TensorSharp.Cli -c Release --no-build -- \
  --config config/qwen-image-2.1.json \
  --prompt 'A small orange cat beside a blue ceramic vase, soft daylight, detailed photograph' \
  --width 2048 --height 2048 --diffusion-steps 40 --cfg 1 \
  --diffusion-seed 42 --output generated.png
```

```bash
dotnet run --project TensorSharp.Cli -c Release --no-build -- \
  --config config/qwen-image-2.1.json \
  --image generated.png \
  --prompt 'Change the blue vase to a red vase. Preserve the cat, lighting and composition.' \
  --width 2048 --height 2048 --diffusion-steps 40 --cfg 1 \
  --diffusion-seed 42 --output edited.png
```

--------------------------------

### Build and run Qwen-Image-2.1 with config/qwen-image-2.1.json

Source: https://github.com/zhongkaifu/tensorsharp/blob/main/website/models-image.html

Builds the CLI and server host, then runs text-to-image, image editing, and the server with the ready-made config. The config selects ggml_metal; append --backend ggml_cuda on NVIDIA, or use --backend cpu for pure C#.

```bash
dotnet build TensorSharp.Cli/TensorSharp.Cli.csproj -c Release
dotnet build TensorSharp.Server.Host/TensorSharp.Server.Host.csproj -c Release

# Text-to-image
dotnet run --project TensorSharp.Cli -c Release --no-build -- \
  --config config/qwen-image-2.1.json \
  --prompt 'A small orange cat beside a blue ceramic vase, soft daylight, detailed photograph' \
  --width 2048 --height 2048 --diffusion-steps 40 --cfg 1 \
  --diffusion-seed 42 --output generated.png

# Image editing
dotnet run --project TensorSharp.Cli -c Release --no-build -- \
  --config config/qwen-image-2.1.json \
  --image generated.png \
  --prompt 'Change the blue vase to a red vase. Preserve the cat, lighting and composition.' \
  --width 2048 --height 2048 --diffusion-steps 40 --cfg 1 \
  --diffusion-seed 42 --output edited.png

# Server + Web UI
dotnet run --project TensorSharp.Server.Host -c Release --no-build -- \
  --config config/qwen-image-2.1.json --host 127.0.0.1 --port 5000
```

--------------------------------

### Start agentic session with code execution tools

Source: https://github.com/zhongkaifu/tensorsharp/blob/main/website/agentic.html

Run the TensorSharp CLI with the pdf skill and code execution enabled, disabling command networking and package installation.

```bash
dotnet run --project TensorSharp.Cli -c Release -- \
  --model models/model.gguf --backend ggml_metal \
  --skills-dir ./skills --skill pdf --code-exec -i
```

--------------------------------

### C# example: Create KnowledgeStore, import, search, and read

Source: https://github.com/zhongkaifu/tensorsharp/blob/main/docs/knowledge-system.md

Creates a KnowledgeStore with built-in extractor, imports a markdown file, searches for 'launch requirements', and reads the top hit with context. Requires an absolute path to the library and a markdown file. The core project has no dependency on TensorAgent or a running inference model.

```csharp
using TensorSharp.Knowledge;

var options = new KnowledgeOptions();
using var store = new KnowledgeStore(
    "/absolute/path/to/library", new BuiltInKnowledgeExtractor(options), options: options);
var collection = await store.CreateCollectionAsync("Project research");
await using var input = File.OpenRead("notes.md");
var document = await store.ImportAsync(collection.Id, "notes.md", input, "text/markdown");
var hits = await store.SearchAsync("launch requirements", new KnowledgeSearchOptions
{
    CollectionIds = new[] { collection.Id }, Limit = 6, MaxCharacters = 8000
});
if (hits.Count > 0)
{
    var context = store.Read(hits[0].Reference, maxCharacters: 8000,
        adjacentChunks: 1, collectionIds: new[] { collection.Id });
}
```

--------------------------------

### Start server with Qwen-Image-2.1 LoRA plug-in and request image generation

Source: https://github.com/zhongkaifu/tensorsharp/blob/main/TensorSharp.Server.Host/API_EXAMPLES.md

Starts the server with a step-distillation LoRA plug-in, making its recipe (8 steps, CFG 1) the default for every image request. Weights download and are hash-checked on first use. The curl request omits steps and cfg to use the plug-in's recipe.

```bash
dotnet TensorSharp.Server.Host/bin/TensorSharp.Server.Host.dll --config config/qwen-image-2.1.json \
  --lora config/lora/qwen-image-2.1-pruna-8step.json
```

```bash
curl --fail-with-body http://localhost:5000/api/image-generate \
  -H 'Content-Type: application/json' \
  -d '{"prompt":"A cat beside a blue vase, soft daylight","width":1024,"height":1024,"seed":42}'
```

--------------------------------

### Run TensorSharp.Cli with ggml_cuda backend

Source: https://github.com/zhongkaifu/tensorsharp/blob/main/docs/models/deepseek4.md

Starts a chat session using the ggml CUDA backend, pointing --model at the first shard of an IQ4_XS quantized model. Requires 2x80 GB GPUs for this example.

```bash
# point --model at the FIRST shard; this IQ4_XS example needs 2x80 GB GPUs
TensorSharp.Cli --model DeepSeek-V4-Flash-UD-IQ4_XS-00001-of-00004.gguf \
    --backend ggml_cuda --layer-split 2 --chat
```

--------------------------------

### Prepare iOS build prerequisites

Source: https://github.com/zhongkaifu/tensorsharp/blob/main/TensorAgent/README.md

Run these one-time setup scripts to build native components and stage Python dependencies.

```bash
TensorSharp.GGML.Native/build-ios.sh        # GgmlOps.xcframework (device + simulator)
eng/fetch-python-ios.sh                     # CPython 3.13 for iOS
TensorAgent/scripts/prepare-python.sh       # stage the interpreter and its packages

```

--------------------------------

### Send a chat completion request with curl

Source: https://github.com/zhongkaifu/tensorsharp/blob/main/TensorSharp.Server.Host/API_EXAMPLES.md

Example curl command to send a chat completion request to the running server at localhost:5000. The server must be started first. The response will be a JSON object with the model's reply.

```bash
curl -s http://localhost:5000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model":"gemma-4-E4B-it-Q8_0.gguf","messages":[{"role":"user","content":"Reply with one short hello."}],"max_tokens":32}'
```

--------------------------------

### Launch TensorSharp.Server.Host for Qwen-Image 2.1

Source: https://github.com/zhongkaifu/tensorsharp/blob/main/docs/models/qwenimage21.md

Launch the TensorSharp.Server.Host with the Qwen-Image 2.1 configuration. Open http://127.0.0.1:5000 afterwards.

```bash
dotnet run --project TensorSharp.Server.Host -c Release --no-build -- \
  --config config/qwen-image-2.1.json --host 127.0.0.1 --port 5000
```

--------------------------------

### Run TensorSharp.Cli with direct-CUDA backend

Source: https://github.com/zhongkaifu/tensorsharp/blob/main/docs/models/deepseek4.md

Starts a chat session using the direct-CUDA engine (no ggml), streaming weights into per-GPU arenas with layer-split across 2 local GPUs. Requires 2x80 GB GPUs for this IQ4_XS example.

```bash
# direct-CUDA engine (no ggml): weights stream into per-GPU arenas, layer-split
# across 2 local GPUs (2x80 GB for this IQ4_XS example)
TensorSharp.Cli --model DeepSeek-V4-Flash-UD-IQ4_XS-00001-of-00004.gguf \
    --backend cuda --layer-split 2 --chat
```

--------------------------------

### Run Wan video generation with config file

Source: https://github.com/zhongkaifu/tensorsharp/blob/main/docs/models/wan.md

Use the ready-made config to start the server and run text-to-video and image-to-video generation. The config downloads missing models on first run (~12.9 GB).

```bash
TensorSharp.Server.Host --config config/wan-video-ti2v-5b-turbo.json

TensorSharp.Cli --config config/wan-video-ti2v-5b-turbo.json \
  --prompt "a red fox trotting through falling snow" --output fox.mp4

# Image-to-video: the image becomes frame 0, the prompt drives the motion.
TensorSharp.Cli --config config/wan-video-ti2v-5b-turbo.json \
  --image first_frame.png --prompt "she turns and smiles" --output clip.mp4
```

--------------------------------

### Run image input smoke test

Source: https://github.com/zhongkaifu/tensorsharp/blob/main/docs/models/jev.md

Runs the extended smoke test with image input against a server started with the vision tower. The --image flag posts the traffic-light example and requires two unambiguous answers; refusal of non-inline image input is checked with or without the flag.

```powershell
# Image input, against a server started WITH the vision tower. --image posts the
# traffic-light example and requires its two unambiguous answers; the refusal of
# non-inline image input is checked with or without the flag.
python eng/jev-extended-smoke.py --endpoint http://127.0.0.1:5000 --image --output artifacts/jev/image
```

--------------------------------

### Embedding API curl examples

Source: https://github.com/zhongkaifu/tensorsharp/blob/main/TensorSharp.Server.Host/API_EXAMPLES.md

Start an embedding service with --model encoder.gguf --embeddings and use these curl commands to call /v1/embeddings and /api/embed. While --embeddings is set, generation routes return HTTP 400 with the message 'This server hosts an embedding model. Use /v1/embeddings or /api/embed.'

```bash
curl http://127.0.0.1:5000/v1/embeddings -H 'Content-Type: application/json' \
  -d '{"model":"all-MiniLM-L6-v2-Q8_0","input":["read a file","open a document"],"encoding_format":"float"}'
curl http://127.0.0.1:5000/api/embed -H 'Content-Type: application/json' \
  -d '{"model":"all-MiniLM-L6-v2-Q8_0","input":["read a file"],"truncate":false}'
```

--------------------------------

### Start standalone server with TS_KNOWLEDGE_DIRECTORY

Source: https://github.com/zhongkaifu/tensorsharp/blob/main/docs/getting_started.md

Sets the knowledge directory and launches the standalone server with a model and backend. Use a backend available on your host. The server exposes /api/knowledge for managing sources and accepts knowledgeCollections on chat requests.

```bash
export TS_KNOWLEDGE_DIRECTORY="$HOME/TensorSharpKnowledge"
dotnet TensorSharp.Server.Host/bin/TensorSharp.Server.Host.dll \
  --model models/gemma-4-E4B-it-Q8_0.gguf --backend ggml_metal \
  --host 127.0.0.1 --port 5001
```

--------------------------------

### Run TensorSharp server with skill execution enabled

Source: https://github.com/zhongkaifu/tensorsharp/blob/main/docs/playwright_agent.md

Starts a local model server with skill scripts, shell commands, package installation, and network access enabled. Use the appropriate model backend for the machine. Without an explicit skills directory, conventional repository `.agents/skills` roots are searched from the current directory to the nearest Git boundary, nearest first.

```sh
dotnet run --project TensorSharp.Server.Host -c Release -- \
  --model /path/to/model.gguf --backend ggml_metal \
  --host 127.0.0.1 --port 5037 \
  --skills-dir "$PWD/TensorAgent/skills" \
  --skills-allow-exec --skills-allow-network --skills-max-rounds 32 \
  --code-exec --code-exec-allow-install --code-exec-allow-network \
  --code-exec-timeout 180
```

--------------------------------

### Gemma 4 E4B Q8_0 quick start (Linux + NVIDIA)

Source: https://github.com/zhongkaifu/tensorsharp/blob/main/website/models-text.html

Verified Gemma 4 E4B quick start for Linux + NVIDIA. Downloads the Q8_0 GGUF, builds with CUDA enabled, writes a prompt file, then runs the CLI and server with `--backend ggml_cuda`. Requires the .NET 10 SDK and Hugging Face CLI. For Apple Silicon omit the CUDA env assignment and use `ggml_metal`; for Vulkan use `TENSORSHARP_GGML_NATIVE_ENABLE_VULKAN=ON` and `ggml_vulkan`. For image/video/audio input also download the mmproj file and pass `--mmproj`.

```bash
hf download ggml-org/gemma-4-E4B-it-GGUF gemma-4-E4B-it-Q8_0.gguf --local-dir models
TENSORSHARP_GGML_NATIVE_ENABLE_CUDA=ON dotnet build TensorSharp.slnx -c Release -p:TensorSharpSkipMlxNative=true
echo "Explain why local inference is useful." > prompt.txt

# Text needs no mmproj.
dotnet run --project TensorSharp.Cli -c Release --no-build -- --model models/gemma-4-E4B-it-Q8_0.gguf \
    --input prompt.txt --max-tokens 300 --backend ggml_cuda
dotnet run --project TensorSharp.Server.Host -c Release --no-build -- --model models/gemma-4-E4B-it-Q8_0.gguf \
    --backend ggml_cuda
```

--------------------------------

### GET /api/version

Source: https://github.com/zhongkaifu/tensorsharp/blob/main/website/http-api.html

Get the server version.

```APIDOC
## GET /api/version

### Description
Get the server version.

### Method
GET

### Endpoint
/api/version

### Response
#### Success Response (200)
- **version** (string) - The server version.

#### Response Example
{
  "version": "1.0.0"
}
```

--------------------------------

### Start server with MiniMax-H3 config file

Source: https://github.com/zhongkaifu/tensorsharp/blob/main/USAGE.md

Uses a shipped config file to run the same host with the reference checkpoint; only the denoiser differs, so nothing re-downloads.

```bash
dotnet TensorSharp.Server.Host/bin/TensorSharp.Server.Host.dll --config config/minimax-h3-fl2va.json
```

--------------------------------

### Launch TensorSharp CLI with TP on GGML CUDA

Source: https://github.com/zhongkaifu/tensorsharp/blob/main/TENSOR_PARALLELISM_PLAN.md

Command-line examples for launching TensorSharp with tensor parallelism on the GGML CUDA backend. The first runs on 2 GPUs, the second selects specific devices via TENSORSHARP_TP_DEVICES, and the third configures multi-node execution with a node ID and peer addresses. These commands demonstrate the basic TP invocation, device selection, and multi-node setup.

```bash
TensorSharp.Cli --model qwen3.5-9b.gguf --backend ggml_cuda --tp 2

```

```bash
TENSORSHARP_TP_DEVICES=0,2 TensorSharp.Cli --model m.gguf --backend ggml_cuda --tp 2

```

```bash
TensorSharp.Cli --model m.gguf --backend ggml_cuda --tp 2 \
  --tp-node-id 0 --tp-peers "10.0.0.1:9500,10.0.0.2:9500"

```

--------------------------------

### Download and run Qwen3.5-9B with mmproj

Source: https://github.com/zhongkaifu/tensorsharp/blob/main/MODEL_DOWNLOADS.md

Downloads Qwen3.5-9B model and vision projector, then runs CLI and server with multimodal support. Use for text and image tasks.

```bash
hf download unsloth/Qwen3.5-9B-GGUF Qwen3.5-9B-UD-Q4_K_XL.gguf --local-dir models
hf download unsloth/Qwen3.5-9B-GGUF mmproj-F16.gguf --local-dir models

dotnet TensorSharp.Cli/bin/TensorSharp.Cli.dll --model models/Qwen3.5-9B-UD-Q4_K_XL.gguf --mmproj models/mmproj-F16.gguf --input prompt.txt --max-tokens 300 --backend ggml_cuda
dotnet TensorSharp.Server.Host/bin/TensorSharp.Server.Host.dll --model models/Qwen3.5-9B-UD-Q4_K_XL.gguf --mmproj models/mmproj-F16.gguf --backend ggml_cuda
```

--------------------------------

### Start server with multimodal projector

Source: https://github.com/zhongkaifu/tensorsharp/blob/main/USAGE.md

Hosts a multimodal model with an explicit projector file for vision capabilities.

```bash
dotnet TensorSharp.Server.Host/bin/TensorSharp.Server.Host.dll --model ./models/model.gguf --mmproj ./models/mmproj.gguf --backend ggml_cuda
```

--------------------------------

### Canvas Setup - Standard p5.js Structure

Source: https://github.com/zhongkaifu/tensorsharp/blob/main/TensorAgent/skills/algorithmic-art/SKILL.md

Standard p5.js canvas setup: createCanvas in setup, and draw contains the generative algorithm. The draw can be static with noLoop or animated.

```javascript
function setup() {
  createCanvas(1200, 1200);
  // Initialize your system
}

function draw() {
  // Your generative algorithm
  // Can be static (noLoop) or animated
}
```

--------------------------------

### Build and launch TensorSharp server with CUDA (recommended)

Source: https://github.com/zhongkaifu/tensorsharp/blob/main/docs/models/deepseek41.md

Builds the native library and server, then launches the server with explicit microbatch and scheduler settings. Requires .NET 10 SDK, CMake, C++ compiler, and CUDA toolkit with nvcc on PATH. Adjust CUDA architectures for other GPUs. The launch uses A40 architecture 8.6 and references the repaired download.

```bash
TENSORSHARP_GGML_NATIVE_ENABLE_CUDA=ON \
  TENSORSHARP_GGML_NATIVE_CUDA_ARCHITECTURES=86 \
  bash TensorSharp.GGML.Native/build-linux.sh
dotnet build TensorSharp.Server.Host/TensorSharp.Server.Host.csproj -c Release \
  -p:CudaArch=compute_86 -p:TensorSharpSkipGgmlNative=true

CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 MAX_CONTEXT=65536 \
  TS_CPU_MOE_THREADS=32 TS_DSV4_UBATCH=256 \
  TS_DSV41_ENGRAM_WARM=0 \
  TS_DSV41_COMPACT_RAW_GATHER=0 KV_CACHE_DTYPE=f16 \
  TS_SCHED_MAX_RUNNING_SEQS=4 TS_SCHED_MAX_BATCHED_TOKENS=4096 \
  TS_SCHED_PREFILL_CHUNK=256 TS_SCHED_SOLO_PREFILL_CHUNK=8192 \
  dotnet TensorSharp.Server.Host/bin/TensorSharp.Server.Host.dll \
  --model /workspace/models/deepseek41-q2-q5/Q2_K-Q5/DeepSeek-V4.1-Flash-EngramQ5-Q2_K-00001-of-00010.gguf \
  --backend ggml_cuda --layer-split 8 --port 5000
```

--------------------------------

### GET /api/skills/{skill_id}

Source: https://github.com/zhongkaifu/tensorsharp/blob/main/docs/agent_skills.md

Get a specific skill in Web UI shape.

```APIDOC
## GET /api/skills/{skill_id}

### Description
Get a specific skill in Web UI shape.

### Method
GET

### Endpoint
/api/skills/{skill_id}

### Parameters
#### Path Parameters
- **skill_id** (string) - Required - The skill identifier.

### Response
#### Success Response (200)
- Skill object.

#### Response Example
{
  "id": "pdf",
  "object": "skill",
  "name": "pdf",
  "description": "Extract text and tables from PDF files...",
  "license": "Apache-2.0",
  "compatibility": "Requires python3 with pypdf installed.",
  "files": [],
  "bytes": 41288,
  "origin": "installed",
  "warnings": [],
  "modified": "2026-08-29T12:00:00Z"
}
```

--------------------------------

### Manually download Qwen-Image-2.1 files and run server

Source: https://github.com/zhongkaifu/tensorsharp/blob/main/website/models-image.html

Downloads the four required files (diffusion transformer, VAE, text encoder, vision encoder) via Hugging Face CLI, then runs the server with explicit model paths. The companion scan matches a qwen_image_2.1_vae safetensors and names containing qwen3vl-8b or qwen3-vl-8b; a projector saved as mmproj-BF16.gguf needs --qwen-image-mmproj.

```bash
hf download Abiray/Qwen-Image-2.1-GGUF qwen_image_2.1_Q4_K_M.gguf --local-dir models
hf download Comfy-Org/Qwen-Image-2.1 vae/qwen_image_2.1_vae_bf16.safetensors --local-dir models
hf download Qwen/Qwen3-VL-8B-Instruct-GGUF Qwen3VL-8B-Instruct-Q4_K_M.gguf mmproj-Qwen3VL-8B-Instruct-F16.gguf --local-dir models

dotnet TensorSharp.Server.Host/bin/TensorSharp.Server.Host.dll --model models/qwen_image_2.1_Q4_K_M.gguf \
    --qwen-image-vae models/vae/qwen_image_2.1_vae_bf16.safetensors \
    --qwen-image-vl models/Qwen3VL-8B-Instruct-Q4_K_M.gguf \
    --qwen-image-mmproj models/mmproj-Qwen3VL-8B-Instruct-F16.gguf \
    --backend ggml_cuda
```

--------------------------------

### GET /v1/skills/{skill_id}

Source: https://github.com/zhongkaifu/tensorsharp/blob/main/docs/agent_skills.md

Get a specific skill, including its instructions from the SKILL.md body.

```APIDOC
## GET /v1/skills/{skill_id}

### Description
Get a specific skill, including its instructions from the SKILL.md body.

### Method
GET

### Endpoint
/v1/skills/{skill_id}

### Parameters
#### Path Parameters
- **skill_id** (string) - Required - The skill identifier.

### Response
#### Success Response (200)
- Skill object with instructions field.

#### Response Example
{
  "id": "pdf",
  "object": "skill",
  "name": "pdf",
  "description": "Extract text and tables from PDF files...",
  "instructions": "# PDF Skill\n...",
  "license": "Apache-2.0",
  "compatibility": "Requires python3 with pypdf installed.",
  "files": [],
  "bytes": 41288,
  "origin": "installed",
  "warnings": [],
  "modified": "2026-08-29T12:00:00Z"
}
```

--------------------------------

### DELETE /api/skills/{name}

Source: https://github.com/zhongkaifu/tensorsharp/blob/main/website/http-api.html

Removes an installed skill. Only skills with origin 'installed' can be deleted.

```APIDOC
## DELETE /api/skills/{name}

### Description
Removes an installed skill. Only a skill uploaded via the API (origin `installed`) can be deleted; skills discovered under an operator-configured directory are refused.

### Method
DELETE

### Endpoint
/api/skills/{name}

### Parameters
#### Path Parameters
- **name** (string) - Required - The name of the skill to delete.

### Response
#### Success Response (200)
- **removed** (boolean) - Always `true` on success.

#### Response Example
{
  "removed": true
}
```

--------------------------------

### Linux + NVIDIA: Build and run TensorSharp.Cli with Gemma 4 E4B

Source: https://github.com/zhongkaifu/tensorsharp/blob/main/website/cli.html

Quick start commands for Linux with NVIDIA. Builds the native GGML backend with CUDA enabled, downloads the benchmark-verified gemma-4-E4B-it-Q8_0.gguf model, creates a prompt file, and runs the CLI with the model, input file, max tokens, and ggml_cuda backend. Requires .NET 10 SDK, Git, network access, CMake, and a C++ toolchain.

```bash
TENSORSHARP_GGML_NATIVE_ENABLE_CUDA=ON dotnet build TensorSharp.slnx -c Release -p:TensorSharpSkipMlxNative=true
curl --create-dirs --fail -L "https://huggingface.co/ggml-org/gemma-4-E4B-it-GGUF/resolve/main/gemma-4-E4B-it-Q8_0.gguf?download=true" -o models/gemma-4-E4B-it-Q8_0.gguf
echo "Explain local inference in one paragraph." > prompt.txt
dotnet run --project TensorSharp.Cli -c Release --no-build -- --model models/gemma-4-E4B-it-Q8_0.gguf --input prompt.txt --max-tokens 128 --backend ggml_cuda
```

--------------------------------

### Install dependencies with pip

Source: https://github.com/zhongkaifu/tensorsharp/blob/main/TensorAgent/skills/slack-gif-creator/SKILL.md

Installs the required Python packages: pillow, imageio, and numpy.

```bash
pip install pillow imageio numpy
```

--------------------------------

### Run server and CLI with --config

Source: https://github.com/zhongkaifu/tensorsharp/blob/main/USAGE.md

Shows how to launch the server and CLI with a JSON config file. Command-line options always override file values, and each override prints a stderr line. Repeat --config to layer files.

```bash
dotnet TensorSharp.Server.Host/bin/TensorSharp.Server.Host.dll --config config/server-basic.json
dotnet TensorSharp.Cli/bin/TensorSharp.Cli.dll       --config config/cli-basic.json
```

--------------------------------

### Run server from already-built source tree (all backends)

Source: https://github.com/zhongkaifu/tensorsharp/blob/main/website/server.html

Run the server from an already-built source tree using the TensorSharp.Server.Host.dll. These commands cover Apple Silicon (Metal), NVIDIA (CUDA), Vulkan (with device selection), and multimodal (with explicit projector). The --model argument is required for inference.

```bash
# Apple Silicon / Metal
dotnet TensorSharp.Server.Host/bin/TensorSharp.Server.Host.dll --model ./models/model.gguf --backend ggml_metal

# NVIDIA / GGML CUDA
dotnet TensorSharp.Server.Host/bin/TensorSharp.Server.Host.dll --model ./models/model.gguf --backend ggml_cuda

# AMD, Intel, or NVIDIA / Vulkan; inspect device indices first
dotnet TensorSharp.Server.Host/bin/TensorSharp.Server.Host.dll --list-gpus
dotnet TensorSharp.Server.Host/bin/TensorSharp.Server.Host.dll --model ./models/model.gguf --backend ggml_vulkan --gpu-device 1

# Multimodal: the projector is always explicit
dotnet TensorSharp.Server.Host/bin/TensorSharp.Server.Host.dll --model ./models/model.gguf --mmproj ./models/mmproj.gguf --backend ggml_cuda
```

--------------------------------

### Run Qwen-Image-2.1 CLI generation, edit, and server

Source: https://github.com/zhongkaifu/tensorsharp/blob/main/MODEL_DOWNLOADS.md

Run the Qwen-Image-2.1 CLI for generation, editing, and server hosting using the ready-made config. The config downloads missing files into $TENSORSHARP_MODELS/qwen-image-2.1/ (or models/qwen-image-2.1/) and selects ggml_metal; append --backend ggml_cuda on NVIDIA or --backend cpu for pure-C#. For local editing, add --mask selection.png at the first input image's dimensions, optionally --mask-feather 8 --mask-crop.

```bash
dotnet run --project TensorSharp.Cli -c Release --no-build -- \
  --config config/qwen-image-2.1.json \
  --prompt 'A small orange cat beside a blue ceramic vase, soft daylight, detailed photograph' \
  --width 2048 --height 2048 --diffusion-steps 40 --cfg 1 \
  --diffusion-seed 42 --output generated.png
dotnet run --project TensorSharp.Cli -c Release --no-build -- \
  --config config/qwen-image-2.1.json \
  --image generated.png \
  --prompt 'Change the blue vase to a red vase. Preserve the cat, lighting and composition.' \
  --width 2048 --height 2048 --diffusion-steps 40 --cfg 1 \
  --diffusion-seed 42 --output edited.png
dotnet run --project TensorSharp.Server.Host -c Release --no-build -- \
  --config config/qwen-image-2.1.json --host 127.0.0.1 --port 5000
```

--------------------------------

### Download and run Mistral Small 3.1 with mmproj

Source: https://github.com/zhongkaifu/tensorsharp/blob/main/MODEL_DOWNLOADS.md

Download the Mistral Small 3.1 24B Instruct GGUF and its mmproj, then run with the CLI and server host for text and image input.

```bash
hf download bartowski/mistralai_Mistral-Small-3.1-24B-Instruct-2503-GGUF mistralai_Mistral-Small-3.1-24B-Instruct-2503-Q4_K_M.gguf --local-dir models
hf download bartowski/mistralai_Mistral-Small-3.1-24B-Instruct-2503-GGUF mmproj-mistralai_Mistral-Small-3.1-24B-Instruct-2503-f16.gguf --local-dir models

dotnet TensorSharp.Cli/bin/TensorSharp.Cli.dll --model models/mistralai_Mistral-Small-3.1-24B-Instruct-2503-Q4_K_M.gguf --mmproj models/mmproj-mistralai_Mistral-Small-3.1-24B-Instruct-2503-f16.gguf --input prompt.txt --max-tokens 300 --backend ggml_cuda
dotnet TensorSharp.Server.Host/bin/TensorSharp.Server.Host.dll --model models/mistralai_Mistral-Small-3.1-24B-Instruct-2503-Q4_K_M.gguf --mmproj models/mmproj-mistralai_Mistral-Small-3.1-24B-Instruct-2503-f16.gguf --backend ggml_cuda
```

--------------------------------

### GET /api/skills/{skill_id}/files/{*path}

Source: https://github.com/zhongkaifu/tensorsharp/blob/main/docs/agent_skills.md

Get one bundled file as plain text. Nested paths bind whole.

```APIDOC
## GET /api/skills/{skill_id}/files/{*path}

### Description
Get one bundled file as plain text. Nested paths bind whole.

### Method
GET

### Endpoint
/api/skills/{skill_id}/files/{*path}

### Parameters
#### Path Parameters
- **skill_id** (string) - Required - The skill identifier.
- **path** (string) - Required - The file path within the skill.

### Response
#### Success Response (200)
- Plain text file content.

#### Response Example
```
# Forms Reference
...
```
```

--------------------------------

### Global install of Playwright CLI

Source: https://github.com/zhongkaifu/tensorsharp/blob/main/TensorAgent/skills/playwright/SKILL.md

Installs the CLI globally via npm and shows help. This is an alternative to using the wrapper script.

```bash
npm install -g @playwright/cli@0.1.21
playwright-cli --help
```

--------------------------------

### Python multiturn test script examples

Source: https://github.com/zhongkaifu/tensorsharp/blob/main/TensorSharp.Server.Host/testdata/README.md

Shows three example invocations of test_multiturn.py: with no arguments, with a model name, and with a max-tokens limit.

```bash
python3 test_multiturn.py
```

```bash
python3 test_multiturn.py --model gemma-4-E4B-it-Q8_0.gguf
```

```bash
python3 test_multiturn.py --max-tokens 120
```

--------------------------------

### Install huggingface_hub and download GGUF

Source: https://github.com/zhongkaifu/tensorsharp/blob/main/docs/models/diffusiongemma.md

Installs the huggingface_hub Python package and downloads the Q4_K_M GGUF file from the unsloth repo into the models directory.

```bash
python -m pip install -U huggingface_hub
hf download unsloth/diffusiongemma-26B-A4B-it-GGUF diffusiongemma-26B-A4B-it-Q4_K_M.gguf --local-dir models
```

--------------------------------

### Start TensorSharp server with GPT OSS model

Source: https://github.com/zhongkaifu/tensorsharp/blob/main/docs/models/gptoss.md

Start the TensorSharp server with the GPT OSS model, providing a Web UI and OpenAI/Ollama-compatible APIs on http://localhost:5000.

```bash
dotnet run --project TensorSharp.Server.Host -c Release -- --model models/gpt-oss-20b-MXFP4.gguf \
  --backend ggml_cuda --max-tokens 4096
```