GLM-5

https://github.com/zai-org/glm-5
Admin
GLM-5 is a large-scale language model with 744B parameters designed for complex systems engineering...
Tokens:43,358
Snippets:441
Trust Score:7.8
License:Apache-2.0
Update:1 month ago
Show doc for...
Tokens:
Raw
### Install TokenSpeed

Source: https://github.com/zai-org/glm-5/blob/main/_autodocs/api-reference/deployment-frameworks.md

Install the TokenSpeed framework for fast token generation.

```bash
pip install tokenspeed
```

--------------------------------

### Install Unsloth

Source: https://github.com/zai-org/glm-5/blob/main/_autodocs/api-reference/deployment-frameworks.md

Install the Unsloth library for optimized inference and fine-tuning.

```bash
pip install unsloth
```

--------------------------------

### Install Skill from GitHub

Source: https://github.com/zai-org/glm-5/blob/main/_autodocs/api-reference/ecosystem-skills.md

Manual installation method for bypassing rate limits by cloning directly from the repository.

```bash
git clone https://github.com/zai-org/GLM-OCR.git
cd GLM-OCR/skills/glmocr
# Follow that skill's SKILL.md for installation
```

--------------------------------

### Design-to-Code Pipeline Installation

Source: https://github.com/zai-org/glm-5/blob/main/_autodocs/api-reference/ecosystem-skills.md

Installs skills for converting PRDs to applications and replicating existing websites.

```bash
# Convert designs to code
npx clawhub@latest install glmv-prd-to-app glmv-web-replication
```

--------------------------------

### Install Slime Infrastructure

Source: https://github.com/zai-org/glm-5/blob/main/_autodocs/api-reference/fine-tuning.md

Install the required Slime package for asynchronous RL fine-tuning.

```bash
pip install 'slime>=0.3.0'
```

--------------------------------

### Install all OCR skills

Source: https://github.com/zai-org/glm-5/blob/main/_autodocs/api-reference/ecosystem-skills.md

Install the complete suite of OCR skills using the Clawhub CLI.

```bash
npx clawhub@latest install glmocr glmocr-table glmocr-formula glmocr-handwriting
```

--------------------------------

### Install Dependencies and Verify Model

Source: https://github.com/zai-org/glm-5/blob/main/_autodocs/quick-start.md

Install required Python packages and verify model loading via a one-liner script.

```bash
# Install required packages
pip install transformers>=5.15.0 torch accelerate

# Download and cache model (first run, ~30s)
python -c "
from transformers import AutoTokenizer, AutoModelForCausalLM

model_id = 'zai-org/GLM-5.3-Flash'
tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(model_id, trust_remote_code=True)
print('Model ready!')
"
```

--------------------------------

### Install KTransformers

Source: https://github.com/zai-org/glm-5/blob/main/_autodocs/api-reference/deployment-frameworks.md

Install the KTransformers package via pip.

```bash
pip install ktransformers
```

--------------------------------

### Install a Single Skill using Clawhub

Source: https://github.com/zai-org/glm-5/blob/main/skills/glm-master-skill/SKILL.md

Use this command to install a specific GLM skill from Clawhub. Ensure you have npx installed.

```bash
npx clawhub@latest install <skill-name>
```

--------------------------------

### Install ms-swift and dependencies

Source: https://github.com/zai-org/glm-5/blob/main/_autodocs/api-reference/fine-tuning.md

Required packages for running fine-tuning tasks with ms-swift.

```bash
pip install 'ms-swift>=4.4.0'
pip install 'torch>=2.0' 'transformers>=5.15.0'
```

--------------------------------

### Install SGLang

Source: https://github.com/zai-org/glm-5/blob/main/_autodocs/api-reference/deployment-frameworks.md

Install the SGLang package via pip.

```bash
pip install sglang
```

--------------------------------

### Install PEFT for LoRA

Source: https://github.com/zai-org/glm-5/blob/main/_autodocs/api-reference/fine-tuning.md

Install the PEFT library required for LoRA fine-tuning.

```bash
pip install 'peft>=0.4.0'
```

--------------------------------

### Install GLM-V Skills

Source: https://github.com/zai-org/glm-5/blob/main/_autodocs/api-reference/ecosystem-skills.md

Install all available GLM-V skills using the clawhub CLI.

```bash
# All GLM-V skills
npx clawhub@latest install glmv-caption glmv-prompt-gen glmv-resume-screen glmv-grounding glmv-doc-based-writing glmv-pdf-to-ppt glmv-pdf-to-web glmv-prd-to-app glmv-web-replication
```

--------------------------------

### Complete Inference Pipeline

Source: https://github.com/zai-org/glm-5/blob/main/_autodocs/api-reference/model-loading.md

A full example demonstrating model loading with device mapping, chat template application, and generation parameters.

```python
from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

# Load model and tokenizer
print("Loading model...")
model_id = "zai-org/GLM-5.3"
tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    trust_remote_code=True,
    torch_dtype=torch.bfloat16,
    device_map="auto"
)
model.eval()

# Prepare input
messages = [
    {"role": "user", "content": "Write a Python function to calculate Fibonacci numbers."}
]

# Apply chat template
text = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True
)

# Tokenize
model_inputs = tokenizer(text, return_tensors="pt").to(model.device)

# Generate
with torch.no_grad():
    generated_ids = model.generate(
        **model_inputs,
        max_new_tokens=1024,
        temperature=0.7,
        top_p=0.9,
        do_sample=True
    )

# Decode
output = tokenizer.decode(
    generated_ids[0][model_inputs["input_ids"].shape[-1]:],
    skip_special_tokens=True
)
print(output)
```

--------------------------------

### Content Generation Pipeline Installation

Source: https://github.com/zai-org/glm-5/blob/main/_autodocs/api-reference/ecosystem-skills.md

Installs skills required for document-based writing, image generation, and PDF-to-web conversion.

```bash
# Generate from documents and create web assets
npx clawhub@latest install glmv-doc-based-writing glm-image-gen glmv-pdf-to-web
```

--------------------------------

### Fine-tune with Slime

Source: https://github.com/zai-org/glm-5/blob/main/_autodocs/configuration.md

Installation and training command for RL-based fine-tuning using the Slime framework.

```bash
# Install slime
pip install 'slime>=0.3.0'

# RL-based fine-tuning
slime train \
    --model zai-org/GLM-5.3 \
    --data your-data.jsonl \
    --output-dir ./output \
    --learning-rate 1e-5 \
    --num-epochs 3
```

--------------------------------

### Deploy vLLM on Ascend NPU

Source: https://github.com/zai-org/glm-5/blob/main/_autodocs/api-reference/deployment-frameworks.md

Install the vLLM-Ascend package and launch the OpenAI-compatible API server on Ascend hardware.

```bash
# Install vLLM-Ascend
pip install vllm-ascend

# Launch server
python -m vllm.entrypoints.openai.api_server \
    --model zai-org/GLM-5.3 \
    --trust-remote-code \
    --device ascend
```

--------------------------------

### Install GLM-5 Dependencies

Source: https://github.com/zai-org/glm-5/blob/main/_autodocs/README.md

Commands to install the required packages for different deployment scenarios.

```bash
pip install transformers>=5.15.0 torch accelerate
```

```bash
pip install vllm>=0.6.0
```

```bash
pip install 'ms-swift>=4.4.0'
```

```bash
pip install transformers torch accelerate vllm 'ms-swift>=4.4.0'
```

--------------------------------

### Install All Listed GLM Skills via Clawhub

Source: https://github.com/zai-org/glm-5/blob/main/skills/glm-master-skill/SKILL.md

Installs all the GLM skills listed in the catalog by passing them as arguments to the npx clawhub install command. This is useful for setting up the entire suite of skills.

```bash
npx clawhub@latest install glmocr glmocr-table glmocr-formula glmocr-handwriting glm-image-generation glmv-caption glmv-prompt-gen glmv-resume-screen glmv-grounding glmv-doc-based-writing glmv-pdf-to-ppt glmv-pdf-to-web glmv-prd-to-app glmv-web-replication
```

--------------------------------

### Fine-tune with ms-swift

Source: https://github.com/zai-org/glm-5/blob/main/_autodocs/configuration.md

Installation and training command for fine-tuning GLM-5.3 using the ms-swift framework.

```bash
# Install ms-swift
pip install 'ms-swift>=4.4.0'

# Fine-tune GLM-5.3
swift sft \
    --model_type glm5 \
    --model_id_or_path zai-org/GLM-5.3 \
    --dataset alpaca-en \
    --output_dir ./output \
    --batch_size 4 \
    --num_train_epochs 3 \
    --learning_rate 5e-5 \
    --warmup_ratio 0.1
```

--------------------------------

### Initialize Message List

Source: https://github.com/zai-org/glm-5/blob/main/_autodocs/types.md

Example of a message list structure compatible with tokenizer chat templates.

```python
messages = [
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": "What is AI?"},
    {"role": "assistant", "content": "AI is..."},
]

# Apply to model via tokenizer.apply_chat_template(messages)
```

--------------------------------

### Install Multiple GLM Skills using Clawhub

Source: https://github.com/zai-org/glm-5/blob/main/skills/glm-master-skill/SKILL.md

This command allows for the installation of multiple GLM skills simultaneously using Clawhub. It's a convenient way to set up several skills at once.

```bash
npx clawhub@latest install glmocr
npx clawhub@latest install glmv-caption
npx clawhub@latest install glm-image-generation
```

--------------------------------

### Configure 8-bit Quantization

Source: https://github.com/zai-org/glm-5/blob/main/_autodocs/configuration.md

Programmatic setup for 8-bit quantization using BitsAndBytes with Hugging Face Transformers.

```python
from transformers import BitsAndBytesConfig, AutoModelForCausalLM

quantization_config = BitsAndBytesConfig(
    load_in_8bit=True,
    bnb_8bit_compute_dtype="float16",
    bnb_8bit_use_double_quant=True,
    bnb_8bit_quant_type="nf8"
)

model = AutoModelForCausalLM.from_pretrained(
    "zai-org/GLM-5.3",
    quantization_config=quantization_config,
    device_map="auto",
    trust_remote_code=True
)
```

--------------------------------

### Launch SGLang Server

Source: https://github.com/zai-org/glm-5/blob/main/_autodocs/configuration.md

Command to start the SGLang server with specific model path, tensor parallelism, and memory settings.

```bash
python -m sglang.launch_server \
    --model-path zai-org/GLM-5.3-Flash \
    --trust-remote-code \
    --tp-size 1 \
    --port 8000 \
    --mem-fraction 0.9 \
    --context-length 128000 \
    --dtype float16
```

--------------------------------

### Launch SGLang server for GLM-5

Source: https://github.com/zai-org/glm-5/blob/main/_autodocs/project-overview.md

Starts the SGLang server instance for the specified model path.

```bash
python -m sglang.launch_server --model-path zai-org/GLM-5.3-Flash
```

--------------------------------

### Minimal Setup for GLM-5

Source: https://github.com/zai-org/glm-5/blob/main/_autodocs/README.md

Initializes the tokenizer and model for basic text generation using the Hugging Face Transformers library.

```python
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("zai-org/GLM-5.3-Flash", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained("zai-org/GLM-5.3-Flash", trust_remote_code=True)

input_ids = tokenizer("Hello", return_tensors="pt").input_ids
output_ids = model.generate(input_ids, max_new_tokens=50)
print(tokenizer.decode(output_ids[0], skip_special_tokens=True))
```

--------------------------------

### Install glm-image-gen via CLI

Source: https://github.com/zai-org/glm-5/blob/main/_autodocs/api-reference/ecosystem-skills.md

Use the npx command to install the image generation skill package.

```bash
npx clawhub@latest install glm-image-gen
```

--------------------------------

### Document Processing Pipeline Installation

Source: https://github.com/zai-org/glm-5/blob/main/_autodocs/api-reference/ecosystem-skills.md

Installs the necessary skills for text, table, and formula extraction.

```bash
# Extract text and structure from documents
npx clawhub@latest install glmocr glmocr-table glmocr-formula
```

--------------------------------

### Install and use glmocr-formula

Source: https://github.com/zai-org/glm-5/blob/main/_autodocs/api-reference/ecosystem-skills.md

Install the formula extraction skill to retrieve LaTeX/MathML from documents.

```bash
npx clawhub@latest install glmocr-formula
```

```python
from glmocr_formula import extract_formulas

# Extract formulas from image/PDF
formulas = extract_formulas("math_document.pdf")

# Returns list of formulas with positions
# [
#     {"formula": "E=mc^2", "latex": "E=mc^{2}"},
#     {"formula": "∫f(x)dx", "latex": "\\int f(x) dx"}
# ]
```

--------------------------------

### Install Transformers dependencies

Source: https://github.com/zai-org/glm-5/blob/main/_autodocs/api-reference/deployment-frameworks.md

Install the required packages for local inference using the Transformers library.

```bash
pip install transformers>=5.15.0 accelerate>=1.14.0
```

--------------------------------

### Example Request with Clear Thinking and Reasoning Effort

Source: https://github.com/zai-org/glm-5/blob/main/_autodocs/api-reference/generation-inference.md

A sample JSON request body demonstrating both clear_thinking and reasoning_effort parameters.

```json
{
    "messages": [
        {
            "role": "user",
            "content": "What is the capital of France?"
        }
    ],
    "clear_thinking": true,
    "reasoning_effort": "max"
}
```

--------------------------------

### Install and use glmocr-table

Source: https://github.com/zai-org/glm-5/blob/main/_autodocs/api-reference/ecosystem-skills.md

Install the table extraction skill to parse structured data from images.

```bash
npx clawhub@latest install glmocr-table
```

```python
from glmocr_table import extract_table

# Extract table structure
table_data = extract_table("table_image.png")

# Returns structured format
# {
#     "headers": ["Col1", "Col2", "Col3"],
#     "rows": [
#         ["value1", "value2", "value3"],
#         ["value4", "value5", "value6"]
#     ]
# }
```

--------------------------------

### Standard Skill Integration Pattern

Source: https://github.com/zai-org/glm-5/blob/main/_autodocs/api-reference/ecosystem-skills.md

General workflow for installing, importing, configuring, and executing a skill.

```python
# 1. Install
# npx clawhub@latest install <skill-name>

# 2. Import
from skill_name import function_name

# 3. Configure (usually via environment)
import os
os.environ["ZHIPU_API_KEY"] = "..."

# 4. Use
result = function_name(input_data, **options)
```

--------------------------------

### Launch SGLang Server

Source: https://github.com/zai-org/glm-5/blob/main/_autodocs/api-reference/deployment-frameworks.md

Commands to start the SGLang server for GLM-5.3-Flash, including support for multi-GPU configurations.

```bash
# Launch SGLang server with GLM-5.3-Flash
python -m sglang.launch_server \
    --model-path zai-org/GLM-5.3-Flash \
    --trust-remote-code \
    --port 8000

# With multiple GPUs
python -m sglang.launch_server \
    --model-path zai-org/GLM-5.3-Flash \
    --trust-remote-code \
    --tp-size 2
```

--------------------------------

### Python Client Integration

Source: https://github.com/zai-org/glm-5/blob/main/_autodocs/integration-examples.md

Standard and streaming request examples using the OpenAI Python client to interact with the vLLM server.

```python
import openai
import json

# Configure client
openai.api_key = "token-abc123"  # vLLM requires a token (can be dummy)
openai.api_base = "http://localhost:8000/v1"

def generate_with_vllm(prompt, max_tokens=256):
    """Generate text using vLLM server."""
    response = openai.ChatCompletion.create(
        model="zai-org/GLM-5.3-Flash",
        messages=[
            {"role": "system", "content": "You are helpful."},
            {"role": "user", "content": prompt}
        ],
        temperature=0.7,
        max_tokens=max_tokens,
        stream=False
    )
    return response.choices[0].message.content

# Use
result = generate_with_vllm("Explain quantum computing in simple terms")
print(result)
```

```python
import openai

openai.api_key = "token"
openai.api_base = "http://localhost:8000/v1"

def stream_generation(prompt):
    """Stream text generation."""
    response = openai.ChatCompletion.create(
        model="zai-org/GLM-5.3-Flash",
        messages=[{"role": "user", "content": prompt}],
        stream=True,
        max_tokens=512
    )
    
    for chunk in response:
        if "choices" in chunk:
            delta = chunk["choices"][0].get("delta", {})
            content = delta.get("content", "")
            if content:
                print(content, end="", flush=True)

# Use
stream_generation("Write a short story about a robot")
print()  # Newline after streaming
```

--------------------------------

### API Usage with cURL

Source: https://github.com/zai-org/glm-5/blob/main/_autodocs/integration-examples.md

Commands to start the server and interact with the generation and chat endpoints.

```bash
# Start server
python api_server.py

# Generate
curl -X POST http://localhost:5000/generate \
  -H "Content-Type: application/json" \
  -d '{"prompt": "Hello", "max_tokens": 100}'

# Chat
curl -X POST http://localhost:5000/chat \
  -H "Content-Type: application/json" \
  -d '{"messages": [{"role": "user", "content": "Hi"}]}'
```

--------------------------------

### Launch vLLM Server for GLM-5.3-Flash

Source: https://github.com/zai-org/glm-5/blob/main/_autodocs/configuration.md

Command to start the vLLM API server with the required remote code trust.

```bash
python -m vllm.entrypoints.openai.api_server \
    --model zai-org/GLM-5.3-Flash \
    --trust-remote-code
```

--------------------------------

### Launch vLLM Server

Source: https://github.com/zai-org/glm-5/blob/main/_autodocs/integration-examples.md

Shell script to terminate existing vLLM processes and start the GLM-5.3-Flash model server.

```bash
#!/bin/bash
# launch_vllm_server.sh

# Kill existing server
pkill -f "vllm.entrypoints"

# Launch vLLM with GLM-5.3-Flash
python -m vllm.entrypoints.openai.api_server \
    --model zai-org/GLM-5.3-Flash \
    --trust-remote-code \
    --dtype auto \
    --gpu-memory-utilization 0.9 \
    --max-num-seqs 256 \
    --port 8000 \
    --host 0.0.0.0
```

--------------------------------

### Prepare Instruction-Response Dataset in Python

Source: https://github.com/zai-org/glm-5/blob/main/_autodocs/api-reference/fine-tuning.md

Creates a JSONL file containing instruction, input, and output fields for fine-tuning.

```python
import json

data = [
    {
        "instruction": "Translate to Spanish",
        "input": "Hello world",
        "output": "Hola mundo"
    },
    {
        "instruction": "Write Python code to",
        "input": "calculate factorial",
        "output": "def factorial(n):\n    return 1 if n <= 1 else n * factorial(n-1)"
    }
]

with open("dataset.jsonl", "w") as f:
    for item in data:
        f.write(json.dumps(item) + "\n")
```

--------------------------------

### Run RL Fine-tuning with Slime

Source: https://github.com/zai-org/glm-5/blob/main/_autodocs/api-reference/fine-tuning.md

Execute basic RL fine-tuning using the slime CLI.

```bash
# Basic RL fine-tuning
slime train \
    --model zai-org/GLM-5.3 \
    --data ./train-data.jsonl \
    --output-dir ./output/glm5-rl \
    --num-epochs 3 \
    --learning-rate 1e-5 \
    --batch-size 32
```

--------------------------------

### Load and Use Tokenizer

Source: https://github.com/zai-org/glm-5/blob/main/_autodocs/api-reference/model-loading.md

Initialize the tokenizer and apply chat templates for instruction-following models.

```python
from transformers import AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("zai-org/GLM-5.3", trust_remote_code=True)

# Encode text
text = "Hello, world!"
tokens = tokenizer.encode(text)
decoded = tokenizer.decode(tokens)

# With chat template (for instruction-following models)
messages = [
    {"role": "user", "content": "What is 2+2?"}
]
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
token_ids = tokenizer.apply_chat_template(messages, tokenize=True, add_generation_prompt=True)
```

--------------------------------

### Run PPO fine-tuning

Source: https://github.com/zai-org/glm-5/blob/main/_autodocs/api-reference/fine-tuning.md

Execute Policy Optimization using preference data.

```bash
swift ppo \
    --model_type glm5 \
    --model_id_or_path zai-org/GLM-5.3 \
    --dataset dpo-en \
    --output_dir ./output/glm5-ppo \
    --batch_size 4 \
    --num_train_epochs 2 \
    --learning_rate 2e-5 \
    --beta 0.1
```

--------------------------------

### Install and use glmocr-handwriting

Source: https://github.com/zai-org/glm-5/blob/main/_autodocs/api-reference/ecosystem-skills.md

Install the handwriting recognition skill to process handwritten notes or forms.

```bash
npx clawhub@latest install glmocr-handwriting
```

```python
from glmocr_handwriting import recognize_handwriting

# Recognize handwritten text
text = recognize_handwriting("handwritten_note.jpg")

# Can also extract structure
result = recognize_handwriting("form_scan.pdf")
# Returns: recognized text + layout structure
```

--------------------------------

### Deploy API Server with vLLM

Source: https://github.com/zai-org/glm-5/blob/main/_autodocs/quick-start.md

Run an OpenAI-compatible API server using vLLM.

```bash
# Install vLLM
pip install vllm

# Start server (runs on localhost:8000)
python -m vllm.entrypoints.openai.api_server \
    --model zai-org/GLM-5.3-Flash \
    --trust-remote-code

# In another terminal, use it:
# curl -X POST http://localhost:8000/v1/chat/completions \
#   -H "Content-Type: application/json" \
#   -d '{
#     "model": "zai-org/GLM-5.3-Flash",
#     "messages": [{"role": "user", "content": "Hi"}]
#   }'
```

--------------------------------

### Install and use glmocr

Source: https://github.com/zai-org/glm-5/blob/main/_autodocs/api-reference/ecosystem-skills.md

Install the general OCR skill and extract text from files, URLs, or batches.

```bash
npx clawhub@latest install glmocr
```

```python
# Pseudocode - see official SKILL.md for exact API
from glmocr import extract_text

# From file
text = extract_text("document.png")

# From URL
text = extract_text("https://example.com/image.jpg")

# Batch processing
texts = extract_text(["image1.png", "image2.png", "image3.png"])
```

--------------------------------

### Run Supervised Fine-tuning (SFT)

Source: https://github.com/zai-org/glm-5/blob/main/_autodocs/api-reference/fine-tuning.md

Execute SFT on a GLM-5 model using a standard dataset.

```bash
swift sft \
    --model_type glm5 \
    --model_id_or_path zai-org/GLM-5.3 \
    --dataset alpaca-en \
    --output_dir ./output/glm5-sft \
    --batch_size 4 \
    --num_train_epochs 3 \
    --learning_rate 5e-5 \
    --warmup_ratio 0.1 \
    --grad_accumulation_steps 2 \
    --save_steps 100 \
    --eval_steps 100 \
    --logging_steps 10
```

--------------------------------

### Reference SGLang Ascend Documentation

Source: https://github.com/zai-org/glm-5/blob/main/_autodocs/api-reference/deployment-frameworks.md

Placeholder for SGLang Ascend documentation link.

```bash
# See SGLang Ascend documentation
# https://docs.sglang.io/docs/hardware-platforms/ascend-npus/
```

--------------------------------

### Run Distributed RL Fine-tuning

Source: https://github.com/zai-org/glm-5/blob/main/_autodocs/api-reference/fine-tuning.md

Perform multi-GPU training using torchrun and slime.

```bash
# Multi-GPU with torch.distributed
torchrun --nproc_per_node=4 -m slime train \
    --model zai-org/GLM-5.3 \
    --data ./train-data.jsonl \
    --output-dir ./output/glm5-rl-dist \
    --num-epochs 5 \
    --batch-size 128
```

--------------------------------

### Validating Inputs

Source: https://github.com/zai-org/glm-5/blob/main/_autodocs/errors.md

Example of validating sequence length against model constraints before initiating generation.

```python
if input_ids.shape[-1] + max_new_tokens > max_length:
    raise ValueError(f"Sequence too long: {input_ids.shape[-1] + max_new_tokens} > {max_length}")
```

--------------------------------

### Configure and Load API Key

Source: https://github.com/zai-org/glm-5/blob/main/_autodocs/api-reference/ecosystem-skills.md

Store the API key in a .env file and load it into the application environment using python-dotenv.

```bash
# ~/.env
ZHIPU_API_KEY=sk_xxx...

# Load in code
from dotenv import load_dotenv
import os
load_dotenv()
api_key = os.getenv("ZHIPU_API_KEY")
```

--------------------------------

### Identify KeyError triggers

Source: https://github.com/zai-org/glm-5/blob/main/_autodocs/errors.md

Examples of KeyError exceptions raised when message fields are missing.

```text
KeyError: 'role'  # Missing role in message
KeyError: 'content'  # Missing content in message
```

--------------------------------

### Handle ImportError for missing dependencies

Source: https://github.com/zai-org/glm-5/blob/main/_autodocs/errors.md

Catch missing library errors during import to prompt for installation.

```text
ImportError: transformers is not installed
ImportError: torch is not installed
```

```python
try:
    from transformers import AutoTokenizer, AutoModelForCausalLM
except ImportError as e:
    print(f"Install required packages: pip install transformers torch")
    raise
```

--------------------------------

### Configure Inference Pipeline

Source: https://github.com/zai-org/glm-5/blob/main/_autodocs/api-reference/model-loading.md

Sets up a text-generation pipeline with specific sampling parameters like temperature and top_p.

```python
from transformers import pipeline

generator = pipeline(
    "text-generation",
    model="zai-org/GLM-5.3",
    tokenizer="zai-org/GLM-5.3",
    trust_remote_code=True
)

output = generator(
    "Translate to French: Hello world",
    max_new_tokens=512,
    temperature=0.7,
    top_p=0.9,
    do_sample=True
)
```

--------------------------------

### Reference xLLM Ascend Documentation

Source: https://github.com/zai-org/glm-5/blob/main/_autodocs/api-reference/deployment-frameworks.md

Placeholder for xLLM Ascend documentation link.

```bash
# See xLLM documentation for Ascend support
# https://github.com/jd-opensource/xllm
```

--------------------------------

### Using PreTrainedTokenizer and BatchEncoding

Source: https://github.com/zai-org/glm-5/blob/main/_autodocs/types.md

Demonstrates loading a tokenizer and accessing the resulting BatchEncoding dictionary fields.

```python
from transformers import AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("zai-org/GLM-5.3", trust_remote_code=True)

# Tokenization returns BatchEncoding
encoding = tokenizer("Hello world", return_tensors="pt")

class BatchEncoding(dict):
    input_ids: torch.Tensor           # Token IDs
    attention_mask: torch.Tensor      # Attention mask
    token_type_ids: Optional[torch.Tensor]  # Segment IDs (if applicable)

# Access
token_ids = encoding['input_ids']       # shape: (1, seq_len)
attention = encoding['attention_mask']  # shape: (1, seq_len)
```

--------------------------------

### Chat Completion Request

Source: https://github.com/zai-org/glm-5/blob/main/_autodocs/api-reference/generation-inference.md

Example of a request structure for GLM-5 models, demonstrating the use of reasoning_effort and clear_thinking parameters.

```APIDOC
## POST /v1/chat/completions

### Description
Sends a chat completion request to the model with specific reasoning and output control parameters.

### Request Body
- **messages** (array) - Required - List of message objects with role and content.
- **reasoning_effort** (string) - Optional - Controls the reasoning intensity (e.g., "low", "high", "max").
- **clear_thinking** (boolean) - Optional - If true, hides internal reasoning tokens from the output.

### Request Example
{
    "messages": [
        {
            "role": "user",
            "content": "What is the capital of France?"
        }
    ],
    "clear_thinking": true,
    "reasoning_effort": "max"
}
```

--------------------------------

### Perform First Text Generation

Source: https://github.com/zai-org/glm-5/blob/main/_autodocs/quick-start.md

Load the model and tokenizer to generate text from a simple input string.

```python
from transformers import AutoTokenizer, AutoModelForCausalLM

# Load
tokenizer = AutoTokenizer.from_pretrained("zai-org/GLM-5.3-Flash", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained("zai-org/GLM-5.3-Flash", trust_remote_code=True)

# Generate
input_ids = tokenizer("Hello, world!", return_tensors="pt").input_ids
output_ids = model.generate(input_ids, max_new_tokens=50)
print(tokenizer.decode(output_ids[0], skip_special_tokens=True))
```

--------------------------------

### Use glmv-web-replication for Frontend Design

Source: https://github.com/zai-org/glm-5/blob/main/_autodocs/api-reference/ecosystem-skills.md

Replicate website designs by analyzing public URLs.

```bash
npx clawhub@latest install glmv-web-replication
```

```python
from glmv_web_replication import replicate_website

# Replicate website design
output = replicate_website(
    url="https://example.com",
    target_framework="react",  # or "vue", "vanilla"
    output_dir="./replicated-site"
)
# Returns: React/Vue code matching visual design
```

--------------------------------

### Request Reasoning Effort via OpenAI Client

Source: https://github.com/zai-org/glm-5/blob/main/_autodocs/configuration.md

Example of passing the reasoning_effort parameter in the extra_body field of an OpenAI ChatCompletion request.

```python
import openai

response = openai.ChatCompletion.create(
    model="zai-org/GLM-5.3-Flash",
    messages=[{"role": "user", "content": "Solve this problem..."}],
    extra_body={
        "reasoning_effort": "max"  # or "high", "low"
    }
)
```

--------------------------------

### Content Generation Pipeline Implementation

Source: https://github.com/zai-org/glm-5/blob/main/_autodocs/api-reference/ecosystem-skills.md

Automates content creation by summarizing documents, generating illustrations, and building websites.

```python
from glmv_doc_based_writing import generate_from_document
from glm_image_gen import generate_image
from glmv_pdf_to_web import pdf_to_website

# Generate content from source
summary = generate_from_document("whitepaper.pdf", task="summarize")

# Generate supporting image
image = generate_image(f"Illustration of: {summary[:100]}")

# Create website
website = pdf_to_website("whitepaper.pdf")
```

--------------------------------

### Batch Generation with Multiple Prompts

Source: https://github.com/zai-org/glm-5/blob/main/_autodocs/api-reference/generation-inference.md

Demonstrates how to tokenize and generate text for a list of multiple prompts simultaneously using the transformers library.

```python
from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

tokenizer = AutoTokenizer.from_pretrained("zai-org/GLM-5.3", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained("zai-org/GLM-5.3", trust_remote_code=True)

prompts = [
    "The meaning of life is",
    "The future of technology is",
    "The best programming language is"
]

# Tokenize all prompts
inputs = tokenizer(prompts, return_tensors="pt", padding=True)

with torch.no_grad():
    outputs = model.generate(
        **inputs,
        max_new_tokens=50,
        do_sample=True,
        temperature=0.7
    )

# Decode all outputs
generated_texts = tokenizer.batch_decode(outputs, skip_special_tokens=True)
for text in generated_texts:
    print(text)
    print("---")
```

--------------------------------

### Batch Processing Prompts with GLM-5

Source: https://github.com/zai-org/glm-5/blob/main/_autodocs/integration-examples.md

Uses the transformers library to tokenize multiple prompts and generate responses in a single batch. Requires torch and transformers installed with trust_remote_code enabled.

```python
from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

tokenizer = AutoTokenizer.from_pretrained("zai-org/GLM-5.3-Flash", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    "zai-org/GLM-5.3-Flash",
    trust_remote_code=True,
    torch_dtype=torch.bfloat16,
    device_map="auto"
)

prompts = [
    "What is machine learning?",
    "Explain neural networks",
    "What are transformers?"
]

# Tokenize all at once
inputs = tokenizer(prompts, return_tensors="pt", padding=True)

# Generate in batch
with torch.no_grad():
    output_ids = model.generate(
        inputs["input_ids"].to(model.device),
        attention_mask=inputs["attention_mask"].to(model.device),
        max_new_tokens=100,
        temperature=0.7,
        do_sample=True
    )

# Decode results
results = tokenizer.batch_decode(output_ids, skip_special_tokens=True)
for prompt, result in zip(prompts, results):
    print(f"Prompt: {prompt}")
    print(f"Response: {result}\n")
```

--------------------------------

### Clone GLM Skill Repository from GitHub

Source: https://github.com/zai-org/glm-5/blob/main/skills/glm-master-skill/SKILL.md

This command clones a GLM skill's repository from GitHub. After cloning, refer to the skill's specific SKILL.md for detailed setup instructions.

```bash
git clone https://github.com/zai-org/<REPO>.git
```

--------------------------------

### Launch vLLM Server for GLM-5

Source: https://github.com/zai-org/glm-5/blob/main/_autodocs/api-reference/model-loading.md

Use the vLLM entrypoint to serve the GLM-5.3-Flash model with remote code execution enabled.

```bash
# Launch vLLM server with GLM-5.3-Flash
python -m vllm.entrypoints.openai.api_server \
    --model zai-org/GLM-5.3-Flash \
    --trust-remote-code \
    --dtype auto
```

--------------------------------

### Tokenizer Loading and Usage

Source: https://github.com/zai-org/glm-5/blob/main/_autodocs/api-reference/model-loading.md

How to initialize a tokenizer and process text for GLM-5 models.

```APIDOC
## AutoTokenizer.from_pretrained

### Description
Loads the tokenizer associated with a GLM-5 model and demonstrates text encoding/decoding and chat template application.

### Example
```python
from transformers import AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained("zai-org/GLM-5.3", trust_remote_code=True)

# Encode/Decode
tokens = tokenizer.encode("Hello, world!")
decoded = tokenizer.decode(tokens)

# Chat Template
messages = [{"role": "user", "content": "What is 2+2?"}]
token_ids = tokenizer.apply_chat_template(messages, tokenize=True, add_generation_prompt=True)
```
```

--------------------------------

### Load and Generate with GLM-5.3-Flash

Source: https://github.com/zai-org/glm-5/blob/main/_autodocs/INDEX.txt

Initializes the tokenizer and model using Hugging Face Transformers and performs basic text generation.

```python
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("zai-org/GLM-5.3-Flash", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained("zai-org/GLM-5.3-Flash", trust_remote_code=True)

input_ids = tokenizer("Hello, world!", return_tensors="pt").input_ids
output_ids = model.generate(input_ids, max_new_tokens=50)
print(tokenizer.decode(output_ids[0], skip_special_tokens=True))
```

--------------------------------

### Catching Exceptions

Source: https://github.com/zai-org/glm-5/blob/main/_autodocs/errors.md

Demonstrates the recommended pattern of catching specific exceptions before generic ones.

```python
try:
    ...
except OutOfMemoryError:
    # Handle specific case
except Exception as e:
    # Handle generic case
```

--------------------------------

### Evaluation During Training

Source: https://github.com/zai-org/glm-5/blob/main/_autodocs/api-reference/fine-tuning.md

Configure evaluation parameters during the fine-tuning process using the swift sft command.

```bash
swift sft \
    --model_type glm5 \
    --model_id_or_path zai-org/GLM-5.3 \
    --dataset alpaca-en \
    --output_dir ./output \
    --eval_dataset_size 200 \
    --eval_steps 100 \
    --save_steps 100
```

--------------------------------

### Configure vLLM Environment Variables

Source: https://github.com/zai-org/glm-5/blob/main/_autodocs/configuration.md

Set performance and backend environment variables for vLLM execution.

```bash
# GPU memory fraction
export VLLM_GPU_MEMORY_UTILIZATION=0.9

# Attention backend
export VLLM_ATTENTION_BACKEND=flash_attn

# Seed
export VLLM_SEED=42
```

--------------------------------

### Perform basic text generation with GLM-5

Source: https://github.com/zai-org/glm-5/blob/main/_autodocs/api-reference/generation-inference.md

Initializes the tokenizer and model to generate text from a prompt using PyTorch.

```python
from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

tokenizer = AutoTokenizer.from_pretrained("zai-org/GLM-5.3", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained("zai-org/GLM-5.3", trust_remote_code=True)

prompt = "The future of artificial intelligence is"
input_ids = tokenizer.encode(prompt, return_tensors="pt")

with torch.no_grad():
    output_ids = model.generate(input_ids, max_new_tokens=100)

generated_text = tokenizer.decode(output_ids[0], skip_special_tokens=True)
print(generated_text)
```

--------------------------------

### Load model with 8-bit quantization

Source: https://github.com/zai-org/glm-5/blob/main/_autodocs/quick-start.md

Use this configuration when hardware VRAM is below 24GB to reduce memory footprint.

```python
# Enable 8-bit quantization
from transformers import BitsAndBytesConfig

config = BitsAndBytesConfig(load_in_8bit=True)
model = AutoModelForCausalLM.from_pretrained(
    "zai-org/GLM-5.3-Flash",
    quantization_config=config,
    device_map="auto",
    trust_remote_code=True
)
```

--------------------------------

### Stream Generation with vLLM

Source: https://github.com/zai-org/glm-5/blob/main/_autodocs/api-reference/generation-inference.md

Deploy a vLLM server and consume the stream using the OpenAI Python client.

```bash
# Start vLLM server
python -m vllm.entrypoints.openai.api_server \
    --model zai-org/GLM-5.3-Flash \
    --trust-remote-code
```

```python
import openai

openai.api_base = "http://localhost:8000/v1"
openai.api_key = "token-abc123"

# Stream response
response = openai.ChatCompletion.create(
    model="zai-org/GLM-5.3-Flash",
    messages=[
        {"role": "user", "content": "Write a short story about a robot."}
    ],
    stream=True,
    max_tokens=512
)

for chunk in response:
    if "choices" in chunk:
        delta = chunk["choices"][0].get("delta", {})
        content = delta.get("content", "")
        if content:
            print(content, end="", flush=True)
```

--------------------------------

### Perform simple text completion with GLM-5

Source: https://github.com/zai-org/glm-5/blob/main/_autodocs/integration-examples.md

Loads the model and tokenizer to generate text from a single prompt string.

```python
from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

# Load model and tokenizer
tokenizer = AutoTokenizer.from_pretrained("zai-org/GLM-5.3-Flash", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    "zai-org/GLM-5.3-Flash",
    trust_remote_code=True,
    torch_dtype=torch.bfloat16,
    device_map="auto"
)

# Generate text
prompt = "The future of artificial intelligence is"
input_ids = tokenizer.encode(prompt, return_tensors="pt")

with torch.no_grad():
    output_ids = model.generate(input_ids, max_new_tokens=100, temperature=0.7)

result = tokenizer.decode(output_ids[0], skip_special_tokens=True)
print(result)
```

--------------------------------

### Handle vLLM Model Loading Errors

Source: https://github.com/zai-org/glm-5/blob/main/_autodocs/errors.md

Identify unsupported model configurations or missing trust flags.

```text
ValueError: Model zai-org/GLM-5.3 is not supported by vLLM
ValueError: trust_remote_code not set
```

```bash
# Always include --trust-remote-code flag
python -m vllm.entrypoints.openai.api_server \
    --model zai-org/GLM-5.3 \
    --trust-remote-code  # REQUIRED
```

--------------------------------

### Load and run model with Unsloth

Source: https://github.com/zai-org/glm-5/blob/main/_autodocs/api-reference/deployment-frameworks.md

Load a model with 4-bit quantization and prepare it for inference using Unsloth.

```python
from unsloth import FastLanguageModel

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name="zai-org/GLM-5.3",
    max_seq_length=4096,
    dtype=None,  # auto-detect
    load_in_4bit=True,
)

# Use model normally
FastLanguageModel.for_inference(model)

input_ids = tokenizer.encode("Hello", return_tensors="pt")
output_ids = model.generate(input_ids, max_new_tokens=256)
result = tokenizer.decode(output_ids[0], skip_special_tokens=True)
```

--------------------------------

### Configure FlashAttention-2

Source: https://github.com/zai-org/glm-5/blob/main/_autodocs/types.md

Use the attn_implementation parameter to select between flash_attention_2, sdpa, or eager implementations.

```python
model = AutoModelForCausalLM.from_pretrained(
    "zai-org/GLM-5.3",
    attn_implementation="flash_attention_2",  # or "sdpa", "eager"
    trust_remote_code=True
)
```

--------------------------------

### Configure CUDA Environment Variables

Source: https://github.com/zai-org/glm-5/blob/main/_autodocs/configuration.md

Define GPU visibility and device ordering for CUDA-enabled environments.

```bash
# GPU selection
export CUDA_VISIBLE_DEVICES=0,1,2,3

# Memory per GPU
export CUDA_DEVICE_ORDER=PCI_BUS_ID
export CUDA_LAUNCH_BLOCKING=1
```

--------------------------------

### Run SGLang Python Client

Source: https://github.com/zai-org/glm-5/blob/main/_autodocs/api-reference/deployment-frameworks.md

Connect to the SGLang server and execute a completion function using the SGLang decorator.

```python
import sglang as sg

# Connect to server
sgl_config = sg.RuntimeEndpoint("http://localhost:8000")

@sg.function
def completion_demo(s, query: str):
    s += sg.system_prompt("You are a helpful assistant.")
    s += query
    s += sg.gen("output", max_tokens=256, temperature=0.7)

result = completion_demo("Explain AI in simple terms")
print(result["output"])
```

--------------------------------

### Implement Chat Interface

Source: https://github.com/zai-org/glm-5/blob/main/_autodocs/quick-start.md

Format messages using the chat template and generate a response.

```python
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("zai-org/GLM-5.3-Flash", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained("zai-org/GLM-5.3-Flash", trust_remote_code=True)

messages = [
    {"role": "user", "content": "What is 2+2?"}
]

text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
input_ids = tokenizer(text, return_tensors="pt").input_ids
output_ids = model.generate(input_ids, max_new_tokens=50)
response = tokenizer.decode(output_ids[0][input_ids.shape[-1]:], skip_special_tokens=True)
print(response)
```

--------------------------------

### Deploy GLM-5 models via vLLM

Source: https://github.com/zai-org/glm-5/blob/main/_autodocs/project-overview.md

Launches an OpenAI-compatible API server using vLLM for specific model variants.

```bash
# GLM-5.3-Flash
python -m vllm.entrypoints.openai.api_server \
  --model zai-org/GLM-5.3-Flash

# GLM-5.2
python -m vllm.entrypoints.openai.api_server \
  --model zai-org/GLM-5.2
```
Tokens:
Raw