### Clone Repository and Install Dependencies

Source: https://github.com/thudm/sciglm/blob/main/README.md

Clone the SciGLM repository and install all necessary Python packages using pip. Ensure you are in the project's root directory.

```bash
git clone https://github.com/THUDM/SciGLM.git
cd SciGLM
pip install -r requirements.txt
```

--------------------------------

### Install SciGLM Dependencies

Source: https://context7.com/thudm/sciglm/llms.txt

Clone the SciGLM repository and install all required Python packages using the provided requirements file. Ensure you have transformers, torch, accelerate, datasets, deepspeed, rouge_chinese, jieba, wandb, and modelscope installed.

```bash
git clone https://github.com/THUDM/SciGLM.git
cd SciGLM
pip install -r requirements.txt
# Key dependencies:
#   transformers==4.30.2
#   torch>=2.0
#   accelerate
#   datasets
#   deepspeed
#   rouge_chinese
#   jieba
#   wandb
#   modelscope
```

--------------------------------

### Run Inference

Source: https://github.com/thudm/sciglm/blob/main/README.md

Navigate to the inference directory and run the command-line interface demo script to perform inference with the trained models.

```bash
cd /path/to/inference
python cli_demo.py
```

--------------------------------

### Generate CoT Solutions with api_gen_gpt.py

Source: https://github.com/thudm/sciglm/blob/main/self_reflective_annotation/reflective_generation/README.md

Execute this script to generate Chain-of-Thought solutions. Ensure the Python environment is set up correctly.

```bash
python api_gen_gpt.py
```

--------------------------------

### Fine-tune with DeepSpeed ZeRO-3

Source: https://context7.com/thudm/sciglm/llms.txt

Use this script for higher-throughput fine-tuning with DeepSpeed ZeRO-3. It enables a larger per-device batch size and faster training on large GPU clusters.

```bash
# training/finetune_ds3.sh
LR=3e-6
WR=0.0
LST=linear
epoch=2
MASTER_PORT=$(shuf -n 1 -i 10000-65535)

deepspeed --include="localhost:0,1,2,3" --master_port $MASTER_PORT main.py \
    --deepspeed /path/to/deepspeed3.json \
    --do_train \
    --train_file /path/to/SciInstruct.json \
    --prompt_column content \
    --response_column summary \
    --model_name_or_path /path/to/ChatGLM3-6b-Base \
    --output_dir ./output/SciGLM-LR${LR}-WR${WR}-${LST}-epoch${epoch} \
    --max_source_length 128 \
    --max_target_length 512 \
    --per_device_train_batch_size 32 \
    --gradient_accumulation_steps 2 \
    --num_train_epochs $epoch \
    --learning_rate $LR \
    --fp16

```

--------------------------------

### Fine-tuning SciGLM with DeepSpeed (ZeRO-2)

Source: https://context7.com/thudm/sciglm/llms.txt

Supervised fine-tuning of ChatGLM3-6B-Base on SciInstruct using DeepSpeed ZeRO-2 across 4 GPUs. This script requires a JSON file with 'content' and 'summary' columns for training data and a DeepSpeed configuration file.

```bash
# training/finetune.sh — run from the training/ directory
LR=3e-6
WR=0.0
LST=linear
epoch=2
MASTER_PORT=$(shuf -n 1 -i 10000-65535)

deepspeed --include="localhost:0,1,2,3" --master_port $MASTER_PORT main.py \
    --deepspeed /path/to/deepspeed.json \
    --do_train \
    --train_file /path/to/SciInstruct.json \
    --eval_steps 200 \
    --prompt_column content \
    --response_column summary \
    --overwrite_cache \
    --model_name_or_path /path/to/ChatGLM3-6b-Base \
    --output_dir ./output/SciGLM-LR${LR}-WR${WR}-${LST}-epoch${epoch} \
    --overwrite_output_dir \
    --max_source_length 128 \
    --max_target_length 512 \
    --per_device_train_batch_size 4 \
    --per_device_eval_batch_size 4 \
    --gradient_accumulation_steps 2 \
    --predict_with_generate \
    --num_train_epochs $epoch \
    --logging_steps 20 \
    --save_strategy epoch \
    --learning_rate $LR \
    --lr_scheduler_type $LST \
    --warmup_ratio $WR \
    --fp16
# Output checkpoints saved to: ./output/SciGLM-LR3e-6-WR0.0-linear-epoch2/
```

--------------------------------

### build_prompt Function for ChatGLM3

Source: https://context7.com/thudm/sciglm/llms.txt

Formats queries and chat history into the multi-round dialogue format required by ChatGLM3. Ensures correct tokenization for training and evaluation.

```python
def build_prompt(query: str, history: list = None) -> str:
    """
    Args:
        query: current user question (strips leading '问：' if present)
        history: list of {"prompt": str, "response": str} dicts
    Returns:
        formatted prompt string for ChatGLM3
    """
    if history is None or history == []:
        history = []
    prompt = ""
    if "问：" in query.strip()[:2]:
        query = query.strip()[2:]
    for i, item in enumerate(history):
        prompt += "[Round {}]\n\n问：{}\n\n答：{}\n\n".format(
            i + 1, item["prompt"], item["response"])
    prompt += "[Round {}]\n\n问：{}\n\n答：".format(len(history) + 1, query)
    return prompt

# Single-turn example:
print(build_prompt("Solve: ∫x² dx"))
# [Round 1]
#
# 问：Solve: ∫x² dx
#
# 答：

# Multi-turn example:
history = [{"prompt": "What is Newton's second law?", "response": "F = ma"}]
print(build_prompt("Derive it from momentum.", history))
# [Round 1]
# 问：What is Newton's second law?
# 答：F = ma
# [Round 2]
# 问：Derive it from momentum.
# 答：

```

--------------------------------

### Train Data Classifier

Source: https://github.com/thudm/sciglm/blob/main/self_reflective_annotation/data_classifier/README.md

Execute this command to train the data classifier. Ensure all dependencies are met before running.

```bash
python train_rm.py
```

--------------------------------

### CLI Inference Demo with SciGLM

Source: https://context7.com/thudm/sciglm/llms.txt

Load a local SciGLM checkpoint and initiate an interactive multi-turn terminal chat. The model utilizes stream_chat with KV-cache for efficient dialogue. Type 'clear' to reset history and 'stop' to quit.

```python
# inference/cli_demo.py
import os
from transformers import AutoTokenizer, AutoModel
import torch

MODEL_PATH = '/path/to/SciGLM-checkpoint'   # local HF checkpoint directory
TOKENIZER_PATH = MODEL_PATH
DEVICE = 'cuda:0' if torch.cuda.is_available() else 'cpu'

tokenizer = AutoTokenizer.from_pretrained(TOKENIZER_PATH, trust_remote_code=True)
if 'cuda' in DEVICE:
    model = AutoModel.from_pretrained(MODEL_PATH, trust_remote_code=True).to(DEVICE).eval()
else:
    model = AutoModel.from_pretrained(MODEL_PATH, trust_remote_code=True).float().to(DEVICE).eval()

past_key_values, history = None, []
query = "What is the photoelectric effect? Provide the Einstein equation."
print("SciGLM: ", end="")
current_length = 0
for response, history, past_key_values in model.stream_chat(
        tokenizer, query,
        history=history,
        top_p=1,
        temperature=1,
        past_key_values=past_key_values,
        return_past_key_values=True):
    print(response[current_length:], end="", flush=True)
    current_length = len(response)
print()
# Expected output: step-by-step explanation of the photoelectric effect
# followed by E = hν - φ (where φ is the work function)
```

--------------------------------

### Fine-tune the 6B Model

Source: https://github.com/thudm/sciglm/blob/main/README.md

Execute the training script for the 6B model using DeepSpeed. Configure learning rate, warmup, and other training parameters as needed. Ensure paths to data and model are correctly set.

```bash
cd /path/to/training
LR=3e-6
WR=0.0
LST=linear
epoch=2
MASTER_PORT=$(shuf -n 1 -i 10000-65535)

/path/to/deepspeed --include="localhost:0,1,2,3" --master_port $MASTER_PORT main.py \
    --deepspeed /path/to/deepspeed.json \
    --do_train \
    --train_file /path/to/SciInstruct.json \
    --eval_steps 200 \
    --prompt_column content \
    --response_column summary \
    --overwrite_cache \
    --model_name_or_path /path/to/ChatGLM3-6b-Base \
    --output_dir ./output/SciGLM-LR$LR-WR$WR-$LST-epoch$epoch \
    --overwrite_output_dir \
    --max_source_length 128 \
    --max_target_length 512 \
    --per_device_train_batch_size 4 \
    --per_device_eval_batch_size 4 \
    --gradient_accumulation_steps 2 \
    --predict_with_generate \
    --num_train_epochs $epoch \
    --logging_steps 20 \
    --save_strategy epoch \
    --learning_rate $LR \
    --lr_scheduler_type $LST \
    --warmup_ratio $WR \
    --fp16
```

--------------------------------

### Conduct Reflective Generation with api_gen_gpt_revise.py

Source: https://github.com/thudm/sciglm/blob/main/self_reflective_annotation/reflective_generation/README.md

Run this script to perform reflective generation, which typically involves revising or improving previously generated content. Verify Python execution environment.

```bash
python api_gen_gpt_revise.py
```

--------------------------------

### PrefixTrainer: Custom Checkpoint Saver

Source: https://context7.com/thudm/sciglm/llms.txt

Extends HuggingFace Trainer to selectively save only prefix encoder weights for P-Tuning v2 or the full model. When save_changed=True, only parameters with requires_grad=True are saved. When save_changed=False, the entire model is saved and DeepSpeed directories are cleaned up.

```python
from trainer import PrefixTrainer
from transformers import Seq2SeqTrainingArguments, DataCollatorForSeq2Seq

training_args = Seq2SeqTrainingArguments(output_dir="./output", ...)

trainer = PrefixTrainer(
    model=model,
    args=training_args,
    train_dataset=train_dataset,
    eval_dataset=eval_dataset,
    tokenizer=tokenizer,
    data_collator=data_collator,
    compute_metrics=compute_metrics,
    save_changed=False,   # True = save only prefix encoder (P-Tuning v2)
                          # False = save full model weights
)
trainer.train()
# With save_changed=False: saves full model + tokenizer + training_args.bin
# With save_changed=True:  saves only trainable prefix encoder weights
# Cleans up deepspeed global_step* subdirectories automatically
```

--------------------------------

### ModelArguments Dataclass for Training

Source: https://context7.com/thudm/sciglm/llms.txt

Configure model loading, P-Tuning v2, and quantization settings using this dataclass. `model_name_or_path` is required. Quantization (4 or 8 bits) can reduce memory usage for inference.

```python
from dataclasses import dataclass
from arguments import ModelArguments
from transformers import HfArgumentParser, Seq2SeqTrainingArguments
from arguments import DataTrainingArguments

# Parse from command line (or JSON config file)
parser = HfArgumentParser((ModelArguments, DataTrainingArguments, Seq2SeqTrainingArguments))
model_args, data_args, training_args = parser.parse_args_into_dataclasses()

# Equivalent programmatic construction:
model_args = ModelArguments(
    model_name_or_path="/path/to/ChatGLM3-6b-Base",
    ptuning_checkpoint=None,          # set for P-Tuning v2 eval
    quantization_bit=None,            # 4 or 8 for quantized inference
    pre_seq_len=None,                 # >0 activates P-Tuning v2
    prefix_projection=False,
)
# Validation: model_name_or_path is required; no default provided

```

--------------------------------

### Run Reward Model Inference and Ranking

Source: https://context7.com/thudm/sciglm/llms.txt

This script computes quality scores for candidate solutions using a trained ChatGLM_Filter model and then ranks them. It splits the solutions into positive (score > 0) and negative buckets based on their scores.

```python
# rm_eval.py — score candidate solutions
import torch, pandas as pd
from torch.utils.data import DataLoader
from trainer import ChatGLM_Filter   # load from data_classifier

filter_model = ChatGLM_Filter(base_model, hidden_size, 1)
filter_model.load_state_dict(torch.load("ckpt/rm_best_checkpoint_4.pt"))
filter_model.eval()

key_token1 = tokenizer.encode('True')[-1]   # index for "True" logit
key_token2 = tokenizer.encode('False')[-1]  # index for "False" logit

preds = []
for batch in dataloader:
    values = filter_model(batch['input_ids'].cuda(), batch['attention_mask'].cuda())
    probs = torch.softmax(values, dim=1)
    score = probs[:, key_token1] - probs[:, key_token2]  # positive = high quality
    preds.extend(score.tolist())

pd.DataFrame(preds, columns=['preds']).to_csv('api_gen_preds/gpt4_revised_q.csv')
```

```python
# get_rm_rank.py — split into positive / negative quality pools
import json, pandas as pd

preds = pd.read_csv('ckpt_gen_preds/ckpt_gen_preds_rm4_q.csv').values[:, 1]
all_data = [...]   # load from source JSONL with content/summary/real_answer
sorted_data = sorted(zip(preds, all_data), key=lambda x: x[0])

with open('filtered/gen_q_pos_rm4.json', 'w') as f_pos, \
     open('filtered/gen_q_neg_rm4.json', 'w') as f_neg:
    for score, item in sorted_data:
        dest = f_pos if score > 0 else f_neg
        json.dump({**item, 'score': score}, dest, ensure_ascii=False)
        dest.write('\n')
```

--------------------------------

### Evaluate Generated Data

Source: https://github.com/thudm/sciglm/blob/main/self_reflective_annotation/data_classifier/README.md

Run this command to evaluate the data generated by the classifier. This helps assess the model's performance.

```bash
python rm_eval.py
```

--------------------------------

### Self-Reflective Annotation: CoT Generation with GPT-4 Turbo

Source: https://context7.com/thudm/sciglm/llms.txt

Stage 1 of the annotation pipeline. Generates chain-of-thought (CoT) solutions for scientific questions using GPT-4 Turbo via the OpenAI API. It reads questions from a JSONL file, constructs a CoT prompt, queries the API with retry logic, and appends results to an output JSONL file.

```python
# self_reflective_annotation/reflective_generation/api_gen_gpt.py
from models import gpt
from prompt import cot_prompt
import json
from tqdm import tqdm

# cot_prompt = "下面将输入一个理科的题目，请分步地给出详细的解题过程。注意，请直接输出解题过程，不要输出其他信息。Q:"

source = 'enhanced_data/enhanced_sft_qa_all.json'   # JSONL: {"content": "...", "summary": "..."}
out_file = 'api_gen_q_enhanced/api_gen_q_gpt4-turbo.json'

with open(out_file, 'a', encoding='utf-8') as f:
    for idx, item in enumerate(tqdm(datas)):
        q = item['content']        # scientific question
        a = item['summary']        # ground-truth answer
        query = cot_prompt + q + '\nA:'
        tol = 3
        while tol > 0:
            try:
                output = gpt(query, n=1, stop=None)[0]
                solution = ''.join(output.split('\n'))
                break
            except Exception as e:
                print(f'error:{e}')
                tol -= 1
        if tol == 0:
            continue
        new_line = {'content': q, 'summary': solution, 'real_answer': a}
        json.dump(new_line, f, ensure_ascii=False)
        f.write('\n')
# Output format: {"content": "<question>", "summary": "<GPT-4 CoT solution>", "real_answer": "<ground truth>"}
```

--------------------------------

### Evaluate SciGLM on SciEval Benchmark

Source: https://context7.com/thudm/sciglm/llms.txt

This script evaluates a SciGLM checkpoint on the SciEval benchmark. It performs model inference on the dataset and then scores the predictions against ground-truth labels, reporting accuracy per category and ability level.

```python
# evaluation/SciEval/evaluate_SciEval.py
from transformers import AutoTokenizer, AutoModel
import json, torch

# Step 1: run model inference on SciEval dataset
checkpoint_path = '/path/to/SciGLM_checkpoint/checkpoint-XXXX'
tokenizer = AutoTokenizer.from_pretrained(checkpoint_path, trust_remote_code=True)
model = AutoModel.from_pretrained(checkpoint_path, trust_remote_code=True).bfloat16().cuda()

# Dataset entry format (bai-scieval-valid.json):
# {"id": "1", "prompt": "<system prompt>", "question": "<question text>",
#  "answer": ["A"], "type": "multiple-choice", "category": "chemistry",
#  "ability": "Base Knowledge"}

with open('./datasets/bai-scieval-valid.json') as f:
    label_data = {d["id"]: d for d in json.load(f)}

outputs = []
for id, label in label_data.items():
    prompt = label['prompt'] + '\ninput:' + label['question']
    inputs = tokenizer([prompt], return_tensors="pt",
                       truncation=True, max_length=2048).to('cuda')
    output_ = model.generate(**inputs, do_sample=False, max_new_tokens=512)
    output = tokenizer.decode(output_.tolist()[0][len(inputs["input_ids"][0]):]).strip()
    outputs.append({'id': id, 'pred': output})

# Step 2: score predictions
# Supports: "multiple-choice" (first char match), "judge" (substring), "filling" (substring)
results = {}
for category in ["biology", "chemistry", "physics"]:
    results[category] = {}
    for ability in ["Base Knowledge", "Knowledge Application",
                    "Scientific Calculation", "Research Ability"]:
        correct = sum(1 for d in outputs
                      if label_data[d["id"]]["category"] == category
                      and label_data[d["id"]]["ability"] == ability
                      and label_data[d["id"]]["answer"][0].lower() in d["pred"].lower())
        total = sum(1 for d in label_data.values()
                    if d["category"] == category and d["ability"] == ability)
        results[category][ability] = correct / total if total else 0

# Save results
with open('./results/outcomes.json', 'w') as f:
    json.dump(results, f, indent=4)
# Expected output: {"biology": {"Base Knowledge": 0.72, ...}, "all": 0.68}
```

--------------------------------

### Self-Reflective Annotation: Reflective Revision with GPT-4 Turbo

Source: https://context7.com/thudm/sciglm/llms.txt

Stage 2 of the annotation pipeline. Rewrites incorrect GPT-4 CoT solutions using a reflective prompt that references the ground-truth answer. It skips multiple-choice questions and for open-ended questions, it constructs a prompt with the question, incorrect solution, and correct answer to extract a revised solution.

```python
# self_reflective_annotation/reflective_generation/api_gen_gpt_revise.py
from models import gpt
from prompt import reflective_prompt
import json

# reflective_prompt instructs GPT-4 to reflect on the error and provide
# a corrected step-by-step solution given the ground truth.

source = 'api_gen_q_enhanced/filtered/gpt4_revised_left_real.json'
out_file = 'api_gen_q_enhanced/revised/api_gen_q_gpt4-turbo_revised_not_choice.json'

def is_choice(s):
    return all(c in ['A', 'B', 'C', 'D', 'E', 'F', ' '] for c in s)

with open(out_file, 'a', encoding='utf-8') as f:
    for item in datas:
        q, a, ra = item['content'], item['summary'], item['real_answer']
        if is_choice(ra):
            continue   # skip multiple-choice items
        query = reflective_prompt + q + '\n文段2：' + a + '\n文段3：' + ra + '\n输出：'
        tol = 3
        while tol > 0:
            try:
                output = ''.join(gpt(query, n=1, stop=None)[0].split('\n'))
                if '正确解答：' in output:
                    solution = output.split('正确解答：')[-1]
                    break
                tol -= 1
            except Exception as e:
                tol -= 1
        if tol == 0:
            continue
        new_line = {'content': q, 'summary': solution, 'real_answer': ra, 'idx': idx}
        json.dump(new_line, f, ensure_ascii=False)
        f.write('\n')
```

--------------------------------

### DataTrainingArguments Dataclass for Training

Source: https://context7.com/thudm/sciglm/llms.txt

Specify dataset paths, column mappings, and sequence length limits. Requires at least one data source (`train_file`, `validation_file`, etc.). File formats must be `.csv` or `.json`.

```python
from arguments import DataTrainingArguments

data_args = DataTrainingArguments(
    train_file="/path/to/SciInstruct.json",      # JSONL with 'content' and 'summary' keys
    validation_file="/path/to/val.json",
    prompt_column="content",
    response_column="summary",
    history_column=None,                          # optional: multi-turn chat history column
    max_source_length=128,                        # truncates input prompts
    max_target_length=512,                        # truncates target responses
    ignore_pad_token_for_loss=True,               # masks pad tokens in loss (-100)
    overwrite_cache=False,
    preprocessing_num_workers=4,
)
# __post_init__ raises ValueError if no data source is provided
# train_file / validation_file must end in .csv or .json

```

--------------------------------

### GPT-4 Label Judge for Solution Evaluation

Source: https://context7.com/thudm/sciglm/llms.txt

Uses GPT-4 as a binary judge to determine if a generated solution matches the ground-truth answer. It retries up to 10 times for valid '0' or '1' output. Correct solutions are written to a filtered file, and incorrect ones are saved separately.

```python
import openai, json
from tqdm import tqdm

openai.api_key = "<your-api-key>"
openai.api_base = "<your-api-base>"
judge_model = 'gpt-4-1106-preview'

def get_label(ans: str, real_ans: str) -> int:
    """Returns 1 if ans matches real_ans, 0 otherwise."""
    prompt = ('下面将输入两段文字，第一段文字为某道理科题目的解答，第二段是这道题目的标准答案。' 
              '请判断第一段解答得到的答案与标准答案是否一致，并根据判断直接输出'0'或'1'...
')
    qry = prompt + '文段1：' + ans + '
' + '文段2：' + real_ans + '
输出：'
    lbl, cnt = '', 10
    while lbl == '' and cnt:
        try:
            chat = openai.ChatCompletion.create(
                model=judge_model,
                messages=[{"role": "user", "content": qry}])
            out = chat.choices[0].message.content[0]
            if out in ('0', '1'):
                lbl = out
            else:
                cnt -= 1
        except Exception as e:
            print(f'error: {e}')
    return int(lbl) if cnt else 0

# Filter loop — splits into correct (filtered) and incorrect (left) files
source = 'revised/api_gen_q_gpt4-turbo_revised_not_choice.json'
with open('filtered/gpt4_revised_filtered_not_choice.json', 'w') as f1, \
     open('filtered/gpt4_revised_left_not_choice.json', 'w') as f2:
    for item in tqdm(json_list):
        label = get_label(item['summary'], item['real_answer'])
        out = f1 if label == 1 else f2
        json.dump(item, out, ensure_ascii=False)
        out.write('
')
```

--------------------------------

### Citation for SciGLM Paper

Source: https://github.com/thudm/sciglm/blob/main/README.md

BibTeX entry for citing the SciGLM paper. Use this in your academic work if you reference the project.

```bibtex
@misc{zhang2024sciglm,
      title={SciGLM: Training Scientific Language Models with Self-Reflective Instruction Annotation and Tuning},
      author={Dan Zhang and Ziniu Hu and Sining Zhoubian and Zhengxiao Du and Kaiyu Yang and Zihan Wang and Yisong Yue and Yuxiao Dong and Jie Tang},
      year={2024},
      eprint={2401.07950},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}
```

--------------------------------

### ChatGLM-based Reward Model for Data Classification

Source: https://context7.com/thudm/sciglm/llms.txt

A reward model built on ChatGLM3-6B-Base that scores question-answer pairs as True (high quality) or False (low quality). It classifies quality by comparing logit scores for the 'True' vs. 'False' tokens after encoding the concatenated prompt and answer.

```python
import torch
import torch.nn as nn
from transformers import AutoModel, AutoTokenizer
from torch.utils.data import DataLoader, Dataset
from modelscope import snapshot_download

model_dir = snapshot_download("ZhipuAI/chatglm3-6b-base", revision="v1.0.0")
tokenizer = AutoTokenizer.from_pretrained(model_dir, trust_remote_code=True)
base_model = AutoModel.from_pretrained(model_dir, trust_remote_code=True).half().cuda()

class ChatGLM_Filter(nn.Module):
    def __init__(self, base, hidden_size, num_classes=1):
        super().__init__()
        self.base_model = base

    def forward(self, input_ids, attention_mask):
        # Score via final token logits — no additional linear head
        return self.base_model(input_ids=input_ids,
                               attention_mask=attention_mask).logits[:, -1]

class MyDataset(Dataset):
    def __init__(self, data_js, tokenizer, max_length=1024):
        self.data = data_js
        self.tokenizer = tokenizer
        self.max_length = max_length

    def __getitem__(self, idx):
        pa = self.data[idx]['prompt_answer']   # "<question>【答案】<answer>"
        label = self.data[idx]['label']        # 1 = correct, 0 = incorrect
        label_token = tokenizer.encode('True')[-1] if label == 1 
                      else tokenizer.encode('False')[-1]
        enc = self.tokenizer.encode_plus(pa, padding='max_length',
                                         max_length=self.max_length,
                                         truncation=True, return_tensors='pt')
        return {'input_ids': enc['input_ids'].squeeze(),
                'attention_mask': enc['attention_mask'].squeeze(),
                'label': torch.tensor(label_token)}

# Training
hidden_size = base_model.config.hidden_size
filter_model = ChatGLM_Filter(base_model, hidden_size).cuda()
optimizer = torch.optim.AdamW(filter_model.parameters(), lr=3e-6, eps=1e-4)
criterion = nn.CrossEntropyLoss()
# Best checkpoint saved to: ckpt/rm_best_checkpoint_4.pt
```

=== COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.