### Tool-Use (Function Calling) Inference with Nanbeige4.1-3B

Source: https://context7.com/nanbeige/nanbeige4.1-3b/llms.txt

Demonstrates native structured tool/function calling. Pass a `tools` list to `apply_chat_template`; the model returns JSON tool calls wrapped in `<tool_call>…</tool_call>` tags. The example shows a user request for weather information and the expected tool call output.

```python
from transformers import AutoModelForCausalLM, AutoTokenizer
import json

tokenizer = AutoTokenizer.from_pretrained(
    "Nanbeige/Nanbeige4.1-3B", use_fast=False, trust_remote_code=True
)
model = AutoModelForCausalLM.from_pretrained(
    "Nanbeige/Nanbeige4.1-3B", torch_dtype="auto", device_map="auto", trust_remote_code=True
)

messages = [
    {"role": "user", "content": "Help me check the weather in Beijing now"},
]

tools = [
    {
        "type": "function",
        "function": {
            "name": "SearchWeather",
            "description": "Find out the current weather in a place on a certain day.",
            "parameters": {
                "type": "dict",
                "properties": {
                    "location": {
                        "type": "string",
                        "description": "A city in China.",
                    }
                },
                "required": ["location"],
            },
        },
    }
]

prompt = tokenizer.apply_chat_template(
    messages,
    tools,
    add_generation_prompt=True,
    tokenize=False,
)

input_ids = tokenizer(prompt, add_special_tokens=False, return_tensors="pt").input_ids
output_ids = model.generate(
    input_ids.to("cuda"), max_new_tokens=512, eos_token_id=166101
)
resp = tokenizer.decode(output_ids[0][len(input_ids[0]):], skip_special_tokens=True)
print(resp)
# Expected output:
# <tool_call>
# {"name": "SearchWeather", "arguments": {"location": "Beijing"}}
# </tool_call>
```

--------------------------------

### Chat Inference with Nanbeige4.1-3B

Source: https://context7.com/nanbeige/nanbeige4.1-3b/llms.txt

Perform single-turn and multi-turn conversational inference using `apply_chat_template`. Recommended hyperparameters include temperature 0.6, top-p 0.95, and repeat penalty 1.0, with max new tokens up to 131072. The example shows a multi-turn conversation and prints the model's response.

```python
from transformers import AutoModelForCausalLM, AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained(
    "Nanbeige/Nanbeige4.1-3B", use_fast=False, trust_remote_code=True
)
model = AutoModelForCausalLM.from_pretrained(
    "Nanbeige/Nanbeige4.1-3B", torch_dtype="auto", device_map="auto", trust_remote_code=True
)

# Multi-turn conversation
messages = [
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user",   "content": "Which number is bigger, 9.11 or 9.8?"},
]

prompt = tokenizer.apply_chat_template(
    messages,
    add_generation_prompt=True,
    tokenize=False,
)

input_ids = tokenizer(prompt, add_special_tokens=False, return_tensors="pt").input_ids
output_ids = model.generate(
    input_ids.to("cuda"),
    eos_token_id=166101,         # <|im_end|>
    temperature=0.6,
    top_p=0.95,
    repetition_penalty=1.0,
    max_new_tokens=131072,
)

response = tokenizer.decode(
    output_ids[0][len(input_ids[0]):],
    skip_special_tokens=True,
)
print(response)
# Expected: The model reasons that 9.8 > 9.11 and explains why
```

--------------------------------

### Inference with Nanbeige4.1-3B for Chat

Source: https://github.com/nanbeige/nanbeige4.1-3b/blob/main/README.md

Use this snippet for general chat interactions. Ensure you have the transformers library installed and the model downloaded.

```python
from transformers import AutoModelForCausalLM, AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained(
  'Nanbeige/Nanbeige4.1-3B',
  use_fast=False,
  trust_remote_code=True
)
model = AutoModelForCausalLM.from_pretrained(
  'Nanbeige/Nanbeige4.1-3B',
  torch_dtype='auto',
  device_map='auto',
  trust_remote_code=True
)
messages = [
  {'role': 'user', 'content': 'Which number is bigger, 9.11 or 9.8?'}
]
prompt = tokenizer.apply_chat_template(
  messages,
  add_generation_prompt=True,
  tokenize=False
)
input_ids = tokenizer(prompt, add_special_tokens=False, return_tensors='pt').input_ids
output_ids = model.generate(input_ids.to('cuda'), eos_token_id=166101)
resp = tokenizer.decode(output_ids[0][len(input_ids[0]):], skip_special_tokens=True)
print(resp)
```

--------------------------------

### Inference with Nanbeige4.1-3B for Tool Use

Source: https://github.com/nanbeige/nanbeige4.1-3b/blob/main/README.md

This snippet demonstrates how to use the model for scenarios involving tool use. It requires defining tools and their descriptions.

```python
from transformers import AutoModelForCausalLM, AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained(
  'Nanbeige/Nanbeige4.1-3B',
  use_fast=False,
  trust_remote_code=True
)
model = AutoModelForCausalLM.from_pretrained(
  'Nanbeige/Nanbeige4.1-3B',
  torch_dtype='auto',
  device_map='auto',
  trust_remote_code=True
)
messages = [
    {'role': 'user',  'content': 'Help me check the weather in Beijing now'}
]
tools = [{'type': 'function',
  'function': {'name': 'SearchWeather',
   'description': 'Find out the current weather in a place on a certain day.',
   'parameters': {'type': 'dict',
    'properties': {'location': {'type': 'string',
      'description': 'A city in China.'},
    'required': ['location']}}}}]prompt = tokenizer.apply_chat_template(
  messages,
  tools,
  add_generation_prompt=True,
  tokenize=False
)
input_ids = tokenizer(prompt, add_special_tokens=False, return_tensors='pt').input_ids
output_ids = model.generate(input_ids.to('cuda'), max_new_tokens=512, eos_token_id=166101)
resp = tokenizer.decode(output_ids[0][len(input_ids[0]):], skip_special_tokens=True)
print(resp)
```

--------------------------------

### Load Deep-Search Tokenizer with Custom Chat Template

Source: https://context7.com/nanbeige/nanbeige4.1-3b/llms.txt

Loads a tokenizer for deep-search/agentic tasks using a custom chat template from a JSON configuration file. Requires manual override of the chat_template.

```python
import json
from transformers import LlamaTokenizer

with open("tokenizer_config_search.json") as f:
    search_cfg = json.load(f)

tokenizer_search = LlamaTokenizer.from_pretrained(
    "Nanbeige/Nanbeige4.1-3B",
    use_fast=False,
    trust_remote_code=True,
    chat_template=search_cfg["chat_template"],
    legacy=True,
)
```

--------------------------------

### Load Nanbeige4.1-3B Model and Tokenizer

Source: https://context7.com/nanbeige/nanbeige4.1-3b/llms.txt

Load the Nanbeige4.1-3B model and tokenizer from Hugging Face Hub. `trust_remote_code=True` is required, and `use_fast=False` is recommended for LlamaTokenizer. The code also prints model configuration details.

```python
from transformers import AutoModelForCausalLM, AutoTokenizer

MODEL_ID = "Nanbeige/Nanbeige4.1-3B"

tokenizer = AutoTokenizer.from_pretrained(
    MODEL_ID,
    use_fast=False,
    trust_remote_code=True,
)

model = AutoModelForCausalLM.from_pretrained(
    MODEL_ID,
    torch_dtype="auto",      # resolves to bfloat16
    device_map="auto",       # distributes across available GPUs/CPU
    trust_remote_code=True,
)

print(model.config.hidden_size)          # 2560
print(model.config.num_hidden_layers)    # 32
print(model.config.max_position_embeddings)  # 262144
```

--------------------------------

### Model Architecture Configuration

Source: https://context7.com/nanbeige/nanbeige4.1-3b/llms.txt

Defines key architectural parameters of the Nanbeige4.1-3B model, useful for custom integrations like VLLM or llama.cpp.

```python
architecture = {
    "model_type":             "llama",                # LlamaForCausalLM
    "hidden_size":            2560,
    "num_hidden_layers":      32,
    "num_attention_heads":    20,
    "num_key_value_heads":    4,                      # GQA: 5:1 ratio
    "head_dim":               128,
    "intermediate_size":      10496,                  # FFN inner dim
    "hidden_act":             "silu",
    "max_position_embeddings": 262144,                # 256K context
    "rope_theta":             70_000_000,             # RoPE base frequency
    "vocab_size":             166144,
    "torch_dtype":            "bfloat16",
    "bos_token_id":           166100,                 # <|im_start|>
    "eos_token_id":           166101,                 # <|im_end|>
}

# Special tokens
special_tokens = {
    "<unk>":          0,
    "<s>":            1,
    "</s>":           2,
    "<|im_start|>":   166100,
    "<|im_end|>":     166101,
    "<|endoftext|>":  166102,
    "<think>":        166103,
    "</think>":       166104,
    "<tool_call>":    166105,
    "</tool_call>":   166106,
}
```

--------------------------------

### Multi-Step Tool-Use Generation

Source: https://context7.com/nanbeige/nanbeige4.1-3b/llms.txt

Continues generation after a tool call by appending the tool result as a 'tool' role message. This pattern supports many sequential tool-call rounds.

```python
# Continuing from the tool-use example above
tool_result = '{"temperature": "28°C", "condition": "Sunny", "humidity": "45%"}'

messages.append({"role": "assistant", "content": resp})
messages.append({"role": "tool",      "content": tool_result})

prompt2 = tokenizer.apply_chat_template(
    messages,
    tools,
    add_generation_prompt=True,
    tokenize=False,
)

input_ids2 = tokenizer(prompt2, add_special_tokens=False, return_tensors="pt").input_ids
output_ids2 = model.generate(
    input_ids2.to("cuda"), max_new_tokens=512, eos_token_id=166101
)
final_reply = tokenizer.decode(
    output_ids2[0][len(input_ids2[0]):], skip_special_tokens=True
)
print(final_reply)
```

--------------------------------

### Extract Reasoning and Answer from Output

Source: https://context7.com/nanbeige/nanbeige4.1-3b/llms.txt

Splits the model's raw output into reasoning and answer parts based on the presence of '</think>' tokens. Handles cases where reasoning might be absent.

```python
raw_output = tokenizer.decode(output_ids[0][len(input_ids[0]):], skip_special_tokens=False)

if "</think>" in raw_output:
    think_part, answer_part = raw_output.split("</think>", 1)
    reasoning = think_part.replace("<think>", "").strip()
    answer    = answer_part.replace("<|im_end|>", "").strip()
    print("=== Reasoning ===")
    print(reasoning)
    print("=== Answer ===")
    print(answer)
else:
    print(raw_output)
```

--------------------------------

### Citation for Nanbeige4.1-3B Model

Source: https://github.com/nanbeige/nanbeige4.1-3b/blob/main/README.md

If you use this model in your projects, please cite it using the provided BibTeX entry.

```bibtex
@misc{yang2026nanbeige413bsmallgeneralmodel,
      title={Nanbeige4.1-3B: A Small General Model that Reasons, Aligns, and Acts},
      author={Chen Yang and Guangyue Peng and Jiaying Zhu and Ran Le and Ruixiang Feng and Tao Zhang and Xiyun Xu and Yang Song and Yiming Jia and Yuntao Wen and Yunzhi Xu and Zekai Wang and Zhenwei An and Zhicong Sun and Zongchao Chen},
      year={2026},
      eprint={2602.13367},
      archivePrefix={arXiv},
      primaryClass={cs.AI},
      url={https://arxiv.org/abs/2602.13367},
}
```

--------------------------------

### Load Standard Chat Tokenizer

Source: https://context7.com/nanbeige/nanbeige4.1-3b/llms.txt

Loads the default chat tokenizer for the Nanbeige4.1-3B model. This tokenizer automatically uses the standard tokenizer_config.json.

```python
from transformers import AutoTokenizer

# Standard chat tokenizer (default)
tokenizer_chat = AutoTokenizer.from_pretrained(
    "Nanbeige/Nanbeige4.1-3B",
    use_fast=False,
    trust_remote_code=True,
    # uses tokenizer_config.json automatically
)
```

--------------------------------

### Parse Tool Call with Regex

Source: https://context7.com/nanbeige/nanbeige4.1-3b/llms.txt

Extracts and parses a tool call from a string using regular expressions. Ensure the input string contains a valid tool_call tag.

```python
import re
match = re.search(r"<tool_call>(.*?)</tool_call>", resp, re.DOTALL)
if match:
    call = json.loads(match.group(1).strip())
    print(call["name"])
    print(call["arguments"])
```

=== COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.