Qwen3.5-35B-A3B

https://huggingface.co/Qwen/Qwen3.5-35B-A3B
Admin
Qwen3.5-35B-A3B is a multimodal AI model with 35B total parameters and 3B activated parameters,...
Tokens:5,179,053
Snippets:17,201
Trust Score:10
Update:1 month ago
Show doc for...
Tokens:
Raw
### vLLM Setup and Usage

Source: https://huggingface.co/Qwen/Qwen3.5-35B-A3B-Base/blame/main/config.json

Installing vLLM, starting the server, and calling it with curl.

```bash
# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Qwen/Qwen3.5-35B-A3B-Base"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Qwen/Qwen3.5-35B-A3B-Base",
		"messages": [
			{"role": "user", "content": "Hello!"}
		]
	}'
```

--------------------------------

### SGLang Setup and Usage

Source: https://huggingface.co/Qwen/Qwen3.5-35B-A3B-Base/blame/main/config.json

Installing SGLang, starting the server, and calling it with curl.

```bash
# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Qwen/Qwen3.5-35B-A3B-Base" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Qwen/Qwen3.5-35B-A3B-Base",
		"messages": [
			{"role": "user", "content": "Hello!"}
		]
	}'
```

--------------------------------

### vLLM Server Setup and Usage

Source: https://huggingface.co/Qwen/Qwen3.5-35B-A3B-GPTQ-Int4/blob/bafc49ee05bd5ebe0a9d928775f08be8affe5334/model.safetensors-00002-of-00014.safetensors

Install vLLM, start the server, and call it using the OpenAI-compatible API.

```APIDOC
## vLLM Server Setup and Usage

### Description
Install vLLM, start the server, and call it using the OpenAI-compatible API.

### Install and Serve
```bash
pip install vllm
vllm serve "Qwen/Qwen3.5-35B-A3B-GPTQ-Int4"
```

### Call the Server
```bash
curl -X POST "http://localhost:8000/v1/chat/completions" \
    -H "Content-Type: application/json" \
    --data '{
        "model": "Qwen/Qwen3.5-35B-A3B-GPTQ-Int4",
        "messages": [
            {
                "role": "user",
                "content": [
                    {
                        "type": "text",
                        "text": "Describe this image in one sentence."
                    },
                    {
                        "type": "image_url",
                        "image_url": {
                            "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
                        }
                    }
                ]
            }
        ]
    }'
```

### Docker Usage
```bash
docker model run hf.co/Qwen/Qwen3.5-35B-A3B-GPTQ-Int4
```
```

--------------------------------

### Use Qwen3.5-35B-A3B with Transformers pipeline

Source: https://huggingface.co/Qwen/Qwen3.5-35B-A3B/tree/b4be5dbefc8633c4d73fb6d79ff52040fdf6ae9a

High-level helper to load the model and generate a response for an image-text prompt. The example sends an image URL and a question, then prints the response.

```python
# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="Qwen/Qwen3.5-35B-A3B")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

```
Tokens:
Raw