Spark-X2.5

https://github.com/xhtoken/spark-x2.5
Admin
Spark-X2.5 is an open series of compact, general-purpose language models featuring native context...
Tokens:10,618
Snippets:72
Trust Score:6.6
License:Apache-2.0
Update:4 days ago
Show doc for...
Tokens:
Raw
### Install vLLM Spark2_5 plugin from source

Source: https://github.com/xhtoken/spark-x2.5/wiki/FAQ

Alternative installation from the GitHub repository. Clone the repo and run pip install . in the plugin directory.

```bash
git clone https://github.com/XHToken/Spark-plugin.git
cd ./Spark-plugin && pip install .
```

--------------------------------

### Install Spark Plugin

Source: https://github.com/xhtoken/spark-x2.5/blob/main/README.md

Install the Spark plugin inside the container using uv.

```bash
pip install uv
uv venv ~/spark2_5
source ~/spark2_5/bin/activate
git clone https://github.com/XHToken/Spark-plugin.git
cd ./Spark-plugin
uv pip install .
```

--------------------------------

### Start vLLM Server

Source: https://github.com/xhtoken/spark-x2.5/blob/main/README.md

Launch the vLLM server with the specified model configuration.

```bash
vllm serve "/models/Spark-X2.5-4B" \
 --port "30000" \
 --trust-remote-code \
 --served-model-name spark25 \
 --tensor-parallel-size 1 \
 --gpu-memory-utilization 0.7 \
 --enable-prefix-caching \
 --chat-template /models/Spark-X2.5-4B/chat_template.jinja
```

--------------------------------

### Install Spark-MLX-LLM

Source: https://github.com/xhtoken/spark-x2.5/blob/main/README.md

Clone and install the Spark-MLX-LLM repository with environment-specific dependencies.

```bash
git clone https://github.com/XHToken/Spark-MLX-LLM.git
cd Spark-MLX-LLM

python3 -m venv .venv
source .venv/bin/activate

# Apple silicon
python -m pip install -e .
# Linux CPU
python -m pip install -e '.[cpu]'
# Linux with CUDA 12
python -m pip install -e '.[cuda12]' 
# Linux with CUDA 13
python -m pip install -e '.[cuda13]'
```

--------------------------------

### Deploy vLLM on Ascend A3

Source: https://github.com/xhtoken/spark-x2.5/blob/main/README.md

Setup and run the vLLM container for Ascend A3 hardware.

```bash
export IMAGE=quay.io/ascend/vllm-ascend:nightly-main-a3
docker pull "$IMAGE"

export DEVICE0=/dev/davinci0
export DEVICE1=/dev/davinci1
export MODEL_CACHE="${HOME}/.cache"

mkdir -p "$MODEL_CACHE"

docker run --rm \
    --name vllm-ascend \
    --shm-size=1g \
    --device "$DEVICE0" \
    --device "$DEVICE1" \
    --device /dev/davinci_manager \
    --device /dev/devmm_svm \
    --device /dev/hisi_hdc \
    -v /usr/local/dcmi:/usr/local/dcmi \
    -v /usr/local/bin/npu-smi:/usr/local/bin/npu-smi \
    -v /usr/local/Ascend/driver/lib64/:/usr/local/Ascend/driver/lib64/ \
    -v /usr/local/Ascend/driver/version.info:/usr/local/Ascend/driver/version.info \
    -v /etc/ascend_install.info:/etc/ascend_install.info \
    -v "$MODEL_CACHE:/root/.cache" \
    -p 8000:8000 \
    -it "$IMAGE" bash
```

--------------------------------

### Launch Inference Server on Ascend NPU

Source: https://github.com/xhtoken/spark-x2.5/blob/main/README.md

Start the API server on Ascend NPU hardware with the required device and volume mappings.

```bash
docker run -it --rm -e ASCEND_USE_FIA=1 --network=host --ipc=host --shm-size=16g \
    --device=/dev/davinci0 --device=/dev/davinci1 --device=/dev/davinci2 --device=/dev/davinci3 \
    --device=/dev/davinci4 --device=/dev/davinci5 --device=/dev/davinci6 --device=/dev/davinci7 \
    --device=/dev/davinci8 --device=/dev/davinci9 --device=/dev/davinci10 --device=/dev/davinci11 \
    --device=/dev/davinci12 --device=/dev/davinci13 --device=/dev/davinci14 --device=/dev/davinci15 \
    --device=/dev/davinci_manager \
    --device=/dev/devmm_svm \
    --device=/dev/hisi_hdc \
    --volume /usr/local/sbin:/usr/local/sbin \
    --volume /usr/local/Ascend/driver:/usr/local/Ascend/driver \
    --volume /usr/local/Ascend/firmware:/usr/local/Ascend/firmware \
    --volume /etc/ascend_install.info:/etc/ascend_install.info \
    --volume /var/queue_schedule:/var/queue_schedule \
    --volume ~/.cache/:/root/.cache/ \
    --volume "$MODEL_PATH:/root/Spark-X2.5-4B:ro" \
    --entrypoint=python \
    "$SGLANG_IMAGE" \
    -m sglang.launch_server \
      --model-path /root/Spark-X2.5-4B \
      --served-model-name spark2.5 \
      --tool-call-parser spark25 \
      --reasoning-parser qwen3 \
      --tp-size 1 \
      --mem-fraction-static 0.8 \
      --context-length 1048576 \
      --chat-template /root/Spark-X2.5-4B/chat_template.jinja \
      --host 0.0.0.0 \
      --port 30000
```

--------------------------------

### Deploy vLLM on Ascend 950DT

Source: https://github.com/xhtoken/spark-x2.5/blob/main/README.md

Setup and run the vLLM container for Ascend 950DT hardware.

```bash
export IMAGE=quay.io/ascend/vllm-ascend:nightly-main-a5
docker pull "$IMAGE"

export MODEL_CACHE="${HOME}/.cache"

mkdir -p "$MODEL_CACHE"

docker run --rm \
    --name vllm-ascend \
    --net=host \
    --shm-size=1g \
    --device /dev/davinci0 \
    --device /dev/davinci_manager \
    --device /dev/devmm_svm \
    --device /dev/hisi_hdc \
    -v /usr/local/dcmi:/usr/local/dcmi \
    -v /usr/local/Ascend/driver/tools/hccn_tool:/usr/local/Ascend/driver/tools/hccn_tool \
    -v /usr/local/bin/npu-smi:/usr/local/bin/npu-smi \
    -v /usr/local/Ascend/driver/lib64/:/usr/local/Ascend/driver/lib64/ \
    -v /usr/local/Ascend/driver/version.info:/usr/local/Ascend/driver/version.info \
    -v /etc/ascend_install.info:/etc/ascend_install.info \
    -v "$MODEL_CACHE:/root/.cache" \
    -it "$IMAGE" bash
```

--------------------------------

### Install vLLM Spark2_5 plugin via pip

Source: https://github.com/xhtoken/spark-x2.5/wiki/FAQ

Install the vllm-spark2_5-plugin into the same Python environment that runs vLLM. On Ascend, use --no-deps to avoid overwriting the vLLM-Ascend build. The official vllm/vllm-openai Docker image also requires this plugin.

```bash
uv pip install vllm-spark2_5-plugin
```

--------------------------------

### Deploy vLLM on Ascend A2

Source: https://github.com/xhtoken/spark-x2.5/blob/main/README.md

Setup and run the vLLM container for Ascend A2 hardware.

```bash
export IMAGE=quay.io/ascend/vllm-ascend:nightly-main
docker pull "$IMAGE"

export DEVICE=/dev/davinci0
export MODEL_CACHE="${HOME}/.cache"

mkdir -p "$MODEL_CACHE"

docker run --rm \
    --name vllm-ascend \
    --shm-size=1g \
    --device "$DEVICE" \
    --device /dev/davinci_manager \
    --device /dev/devmm_svm \
    --device /dev/hisi_hdc \
    -v /usr/local/dcmi:/usr/local/dcmi \
    -v /usr/local/bin/npu-smi:/usr/local/bin/npu-smi \
    -v /usr/local/Ascend/driver/lib64/:/usr/local/Ascend/driver/lib64/ \
    -v /usr/local/Ascend/driver/version.info:/usr/local/Ascend/driver/version.info \
    -v /etc/ascend_install.info:/etc/ascend_install.info \
    -v "$MODEL_CACHE:/root/.cache" \
    -p 8000:8000 \
    -it "$IMAGE" bash
```

--------------------------------

### Launch Inference Server on NVIDIA GPU

Source: https://github.com/xhtoken/spark-x2.5/blob/main/README.md

Start the OpenAI-compatible API server on an NVIDIA GPU with a context length of 1,048,576 tokens.

```bash
docker run --rm -it \
  --gpus '"device=0"' \
  --ipc=host \
  -p 30000:30000 \
  -v "$MODEL_PATH":/root/Spark-X2.5-4B:ro \
  lmsysorg/sglang:nightly-dev-cu13-20260827-20621aa1 \
  python -m sglang.launch_server \
    --model-path /root/Spark-X2.5-4B \
    --served-model-name spark2.5 \
    --tool-call-parser spark25 \
    --reasoning-parser qwen3 \
    --tp-size 1 \
    --mem-fraction-static 0.8 \
    --context-length 1048576 \
    --chat-template /root/Spark-X2.5-4B/chat_template.jinja \
    --host 0.0.0.0 \
    --port 30000
```

--------------------------------

### Set Model Path

Source: https://github.com/xhtoken/spark-x2.5/blob/main/README.md

Define the absolute path to the Spark-X2.5-4B checkpoint before starting the container.

```bash
export MODEL_PATH=/absolute/path/to/Spark-X2.5-4B
```

--------------------------------

### Run Spark-X2.5 with lms CLI

Source: https://github.com/xhtoken/spark-x2.5/blob/main/README.md

Commands to list available models and initiate a chat session with Spark-X2.5 using the lms CLI.

```bash
# List available models
lms ls

# Run chat with Spark-X2.5
lms chat spark-x2.5
```

--------------------------------

### Run Spark-X2.5 with Ollama

Source: https://github.com/xhtoken/spark-x2.5/wiki/FAQ

Requires Ollama v0.34.1 or later. Both models have a single 'latest' tag with unquantized weights; for limited VRAM, pull an official GGUF quantization from Hugging Face instead.

```bash
ollama run SparkLLM/Spark-X2.5-1.7B
# or
ollama run SparkLLM/Spark-X2.5-4B
```

--------------------------------

### Deploy vLLM on NVIDIA GPU

Source: https://github.com/xhtoken/spark-x2.5/blob/main/README.md

Run the official vLLM Docker image for NVIDIA GPU deployment.

```bash
docker run --rm --gpus all \
  --ipc=host \
  -p 30000:30000 \
  -v "$MODEL_PATH:/models/Spark-X2.5-4B:ro" \
  vllm/vllm-openai:latest \
  --model /models/Spark-X2.5-4B \
  --port 30000 \
  --trust-remote-code \
  --served-model-name spark25 \
  --tensor-parallel-size 1 \
  --gpu-memory-utilization 0.7 \
  --enable-prefix-caching \
  --chat-template /models/Spark-X2.5-4B/chat_template.jinja
```

--------------------------------

### Pull SGLang Image for Ascend NPUs

Source: https://github.com/xhtoken/spark-x2.5/blob/main/README.md

Select the appropriate image based on your hardware (A3 or A2) and pull it.

```bash
# A3 daily build
export SGLANG_IMAGE=quay.io/ascend/sglang:main-cann9.0.0-a3

# A2 daily build (use this instead on A2 hardware)
export SGLANG_IMAGE=quay.io/ascend/sglang:main-cann9.0.0-910b

docker pull "$SGLANG_IMAGE"
```

--------------------------------

### Run Inference via Client

Source: https://github.com/xhtoken/spark-x2.5/blob/main/README.md

Send a chat completion request to the vLLM server using curl.

```bash
curl -s http://127.0.0.1:30000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "spark25",
    "messages": [{"role": "user", "content": "安徽的省会在哪里？"}],
    "temperature": 1.0, 
    "top_k": -1,
    "top_p": 0.95
  }'
```

--------------------------------

### Run Ollama Model

Source: https://github.com/xhtoken/spark-x2.5/blob/main/README.md

Execute the Spark-X2.5 model using Ollama.

```bash
ollama run SparkLLM/Spark-X2.5-1.7B
```

--------------------------------

### Pull SGLang Image for NVIDIA GPUs

Source: https://github.com/xhtoken/spark-x2.5/blob/main/README.md

Use the pre-built image compatible with the Spark-X2.5 runtime.

```bash
docker pull lmsysorg/sglang:nightly-dev-cu13-20260827-20621aa1
```

--------------------------------

### Run MLX Inference

Source: https://github.com/xhtoken/spark-x2.5/blob/main/README.md

Execute inference using the spark-mlx-generate command.

```bash
spark-mlx-generate \
  --device gpu \
  --dtype bfloat16 \
  --model XHToken/Spark-X2.5-1.7B \
  --prompt "安徽的省会在哪里？" \
  --max-tokens 512 \
  --temp 0
```

--------------------------------

### Query Inference API

Source: https://github.com/xhtoken/spark-x2.5/blob/main/README.md

Send a chat completion request to the running server using curl.

```bash
curl -s http://localhost:30000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "spark2.5",
    "messages": [
      {
        "role": "user",
        "content": "安徽的省会在哪里？"
      }
    ],
    "max_tokens": 131072,
    "temperature": 1,
    "top_k": -1,
    "top_p": 0.95,
    "repetition_penalty": 1,
    "presence_penalty": 0,
    "frequency_penalty": 0
  }'
```

--------------------------------

### Cite Spark-X2.5

Source: https://github.com/xhtoken/spark-x2.5/blob/main/README.md

BibTeX citation for the Spark-X2.5 model series.

```bibtex
@misc{sparkx2.5,
    title  = {Spark-X2.5 4B&1.7B: Pushing the Limits of Agentic Capabilities in On-Device Models},
    author = {SparkLLM Team},
    year   = {2026}
}
```

--------------------------------

### Disable thinking mode per request with chat_template_kwargs

Source: https://github.com/xhtoken/spark-x2.5/wiki/FAQ

Use this JSON snippet to disable thinking mode for a single request, which can prevent looping and improve stability in everyday chat. Set enable_thinking to false in chat_template_kwargs.

```json
"chat_template_kwargs": {"enable_thinking": false}
```
Tokens:
Raw