### Install calflops from local wheel file

Source: https://github.com/mryxj/calculate-flops.pytorch/blob/main/README.md

Install calflops by providing the path to a downloaded wheel file.

```python
pip install calflops-*-py3-none-any.whl
```

--------------------------------

### Install calflops from PyPI

Source: https://github.com/mryxj/calculate-flops.pytorch/blob/main/README.md

Install the latest version of the calflops package using pip.

```python
pip install --upgrade calflops
```

--------------------------------

### Calculate FLOPs for Hugging Face Model by URL

Source: https://github.com/mryxj/calculate-flops.pytorch/blob/main/README.md

This snippet calculates FLOPs, MACs, and parameters for a Hugging Face model using its full URL. This is useful for models not directly accessible by name. Ensure `calflops` is installed.

```python
from calflops import calculate_flops_hf

batch_size, max_seq_length = 1, 128
model_name = "https://huggingface.co/THUDM/glm-4-9b-chat" # THUDM/glm-4-9b-chat
flops, macs, params = calculate_flops_hf(model_name=model_name, input_shape=(batch_size, max_seq_length))
print("%s FLOPs:%s  MACs:%s  Params:%s 
" %(model_name, flops, macs, params))
```

--------------------------------

### Calculate FLOPs for Hugging Face Model by Name

Source: https://github.com/mryxj/calculate-flops.pytorch/blob/main/README.md

Use this snippet to calculate FLOPs, MACs, and parameters for a Hugging Face model directly from its name. Ensure the `calflops` library is installed.

```python
from calflops import calculate_flops_hf

batch_size, max_seq_length = 1, 128
model_name = "baichuan-inc/Baichuan-13B-Chat"

flops, macs, params = calculate_flops_hf(model_name=model_name, input_shape=(batch_size, max_seq_length))
print("%s FLOPs:%s  MACs:%s  Params:%s 
" %(model_name, flops, macs, params))
```

--------------------------------

### Calculate FLOPs for a PyTorch Model

Source: https://github.com/mryxj/calculate-flops.pytorch/blob/main/README.md

Use `calculate_flops` to get FLOPs, MACs, and parameters for a PyTorch model. Specify input shape, arguments, and optionally include backpropagation computation.

```python
from calflops import calculate_flops

def calculate_flops(model,
                    input_shape=None,
                    transformer_tokenizer=None,
                    args=[],   
                    kwargs={},
                    forward_mode="forward",
                    include_backPropagation=False,
                    compute_bp_factor=2.0,         
                    print_results=True,
                    print_detailed=True,
                    output_as_string=True,
                    output_precision=2,
                    output_unit=None,
                    ignore_modules=None):
    
    """Returns the total floating-point operations, MACs, and parameters of a model.

    Args:
        model ([torch.nn.Module]): The model of input must be a PyTorch model.
        input_shape (tuple, optional): Input shape to the model. If args and kwargs is empty, the model takes a tensor with this shape as the only positional argument. Default to [].
        transformers_tokenizer (None, optional): Transforemrs Toekenizer must be special if model type is transformers and args、kwargs is empty. Default to None
        args (list, optinal): list of positional arguments to the model, such as bert input args is [input_ids, token_type_ids, attention_mask]. Default to []
        kwargs (dict, optional): dictionary of keyword arguments to the model, such as bert input kwargs is {'input_ids': ..., 'token_type_ids':..., 'attention_mask':...}. Default to {}
        forward_mode (str, optional): To determine the mode of model inference, Default to 'forward'. And use 'generate' if model inference uses model.generate().
        include_backPropagation (bool, optional): Decides whether the final return FLOPs computation includes the computation for backpropagation.
        compute_bp_factor (float, optional): The model backpropagation is a multiple of the forward propagation computation. Default to 2.0.
        print_results (bool, optional): Whether to print the model profile. Defaults to True.
        print_detailed (bool, optional): Whether to print the detailed model profile. Defaults to True.
        output_as_string (bool, optional): Whether to print the output as string. Defaults to True.
        output_precision (int, optional) : Output holds the number of decimal places if output_as_string is True. Default to 2.
        output_unit (str, optional): The unit used to output the result value, such as T, G, M, and K. Default is None, that is the unit of the output decide on value.
        ignore_modules ([type], optional): the list of modules to ignore during profiling. Defaults to None.
    """
    pass
```

--------------------------------

### Create Empty Model on Meta Device

Source: https://context7.com/mryxj/calculate-flops.pytorch/llms.txt

Instantiate a Hugging Face model on the 'meta' device without loading weights. Useful for memory-efficient FLOPs calculations or custom workflows. Supports `trust_remote_code` and `access_token`.

```python
from calflops import create_empty_model

# Create empty model from Hugging Face Hub
model_name = "bert-base-uncased"
empty_model = create_empty_model(
    model_name=model_name,
    library_name="transformers",
    trust_remote_code=False
)

print(type(empty_model))
# Output: <class 'transformers.models.bert.modeling_bert.BertModel'>
print(next(empty_model.parameters()).device)
# Output: meta
```

```python
# For models requiring authentication
model_name = "meta-llama/Llama-2-7b"
empty_model = create_empty_model(
    model_name=model_name,
    library_name="transformers",
    trust_remote_code=True,
    access_token="hf_your_token_here"
)
```

--------------------------------

### Calculate FLOPs for Hugging Face Models

Source: https://context7.com/mryxj/calculate-flops.pytorch/llms.txt

Estimate FLOPs, MACs, and parameters for Hugging Face models without downloading weights. Supports direct model names, URLs, and gated models with access tokens. Output can be formatted as strings.

```python
from calflops import calculate_flops_hf

# Example 1: Open-source model (Baichuan)
batch_size, max_seq_length = 1, 128
model_name = "baichuan-inc/Baichuan-13B-Chat"

flops, macs, params = calculate_flops_hf(
    model_name=model_name,
    input_shape=(batch_size, max_seq_length),
    output_as_string=True,
    output_precision=2,
    print_results=True,
    print_detailed=True
)
print(f"{model_name} FLOPs:{flops}  MACs:{macs}  Params:{params}")
# Output: baichuan-inc/Baichuan-13B-Chat FLOPs:3.23 TFLOPS  MACs:1.61 TMACs  Params:13.0 B
```

```python
# Example 2: Using Hugging Face URL directly
model_name = "https://huggingface.co/THUDM/glm-4-9b-chat"  # or "THUDM/glm-4-9b-chat"

flops, macs, params = calculate_flops_hf(
    model_name=model_name,
    input_shape=(batch_size, max_seq_length),
    trust_remote_code=True
)
print(f"GLM-4-9B FLOPs:{flops}  MACs:{macs}  Params:{params}")
```

```python
# Example 3: Gated model requiring access token (LLaMA-2)
model_name = "meta-llama/Llama-2-7b"
access_token = "hf_your_access_token_here"

flops, macs, params = calculate_flops_hf(
    model_name=model_name,
    access_token=access_token,
    input_shape=(batch_size, max_seq_length),
    output_as_string=True
)
print(f"{model_name} FLOPs:{flops}  MACs:{macs}  Params:{params}")
# Output: meta-llama/Llama-2-7b FLOPs:1.7 TFLOPS  MACs:850 GMACs  Params:6.74 B
```

--------------------------------

### Calculate FLOPs for Hugging Face Models

Source: https://github.com/mryxj/calculate-flops.pytorch/blob/main/README.md

Use `calculate_flops_hf` to compute FLOPs, MACs, and parameters for models hosted on Hugging Face. Requires model name and optionally input shape, access token, and other profiling parameters.

```python
def calculate_flops_hf(model_name,
                       input_shape=None,
                       library_name="transformers",
                       trust_remote_code=True,
                       access_token="",
                       forward_mode="forward",
                       include_backPropagation=False,
                       compute_bp_factor=2.0,
                       print_results=True,
                       print_detailed=True,
                       output_as_string=True,
                       output_precision=2,
                       output_unit=None,
                       ignore_modules=None):
    
    """Returns the total floating-point operations, MACs, and parameters of a model.

    Args:
        model_name (str): The model name in huggingface platform https://huggingface.co/models, such as meta-llama/Llama-2-7b、baichuan-inc/Baichuan-13B-Chat etc.
        input_shape (tuple, optional): Input shape to the model. If args and kwargs is empty, the model takes a tensor with this shape as the only positional argument. Default to [].
        trust_remote_code (bool, optional): Trust the code in the remote library for the model structure.
        access_token (str, optional): Some models need to apply for a access token, such as meta llama2 etc.
        forward_mode (str, optional): To determine the mode of model inference, Default to 'forward'. And use 'generate' if model inference uses model.generate().
        include_backPropagation (bool, optional): Decides whether the final return FLOPs computation includes the computation for backpropagation.
        compute_bp_factor (float, optional): The model backpropagation is a multiple of the forward propagation computation. Default to 2.0.
        print_results (bool, optional): Whether to print the model profile. Defaults to True.
        print_detailed (bool, optional): Whether to print the detailed model profile. Defaults to True.
        output_as_string (bool, optional): Whether to print the output as string. Defaults to True.
        output_precision (int, optional) : Output holds the number of decimal places if output_as_string is True. Default to 2.
    """
    pass
```

--------------------------------

### Create Custom Inputs for FLOPs Calculation

Source: https://context7.com/mryxj/calculate-flops.pytorch/llms.txt

Prepare custom inputs for a BERT model using a tokenizer. Ensure to return PyTorch tensors ('pt') for compatibility with the FLOPs calculation function.

```python
text = "Hello, this is a test sentence for BERT model."
max_seq_length = 128

inputs = tokenizer(
    text,
    add_special_tokens=True,
    return_attention_mask=True,
    padding='max_length',
    truncation=True,
    max_length=max_seq_length,
    return_tensors='pt'
)
```

--------------------------------

### Calculate FLOPs for Hugging Face Models

Source: https://github.com/mryxj/calculate-flops.pytorch/blob/main/README.md

Use calculate_flops_hf to analyze models hosted on Hugging Face. Requires an access token for restricted models.

```python
from calflops import calculate_flops_hf

batch_size, max_seq_length = 1, 128
model_name = "meta-llama/Llama-2-7b"
access_token = "" # your application for using llama2

flops, macs, params = calculate_flops_hf(model_name=model_name,
                                         access_token=access_token,
                                         input_shape=(batch_size, max_seq_length))
print("%s FLOPs:%s  MACs:%s  Params:%s \n" %(model_name, flops, macs, params))
```

```python
from calflops import calculate_flops_hf

batch_size, max_seq_length = 1, 128
model_name = "meta-llama/Llama-2-7b"
access_token = "" # your application for using llama 

flops, macs, params = calculate_flops_hf(model_name=model_name,
                                         access_token=access_token,
                                         input_shape=(batch_size, max_seq_length))
print("%s FLOPs:%s  MACs:%s  Params:%s \n" %(model_name, flops, macs, params))
```

--------------------------------

### Calculate FLOPs for Hugging Face Models

Source: https://github.com/mryxj/calculate-flops.pytorch/blob/main/README.md

Use this function to calculate FLOPs, MACs, and parameters for Hugging Face models. Ensure the model name and input shape are correctly specified.

```python
from calflops import calculate_flops_hf

batch_size = 1
max_seq_length = 128
model_name = "baichuan-inc/Baichuan-13B-Chat"
flops, macs, params = calculate_flops_hf(model_name=model_name,
                                            input_shape=(batch_size, max_seq_length))
print("%s FLOPs:%s  MACs:%s  Params:%s \n" %(model_name, flops, macs, params))
```

--------------------------------

### create_empty_model

Source: https://context7.com/mryxj/calculate-flops.pytorch/llms.txt

Creates an empty model on the meta device from a Hugging Face Hub model name. This is used internally by `calculate_flops_hf` but can be called directly for custom workflows. The model is initialized without loading actual weights, allowing memory-efficient FLOPs calculation.

```APIDOC
## create_empty_model

### Description
Creates an empty model on the meta device from a Hugging Face Hub model name. This is used internally by `calculate_flops_hf` but can be called directly for custom workflows. The model is initialized without loading actual weights, allowing memory-efficient FLOPs calculation.

### Parameters
- **model_name** (str) - Required - The name or path of the model on the Hugging Face Hub.
- **library_name** (str) - Required - The deep learning library to use (e.g., 'transformers', 'pytorch').
- **trust_remote_code** (bool) - Optional - Whether to trust remote code execution for the model.
- **access_token** (str) - Optional - Hugging Face access token for gated models.

### Request Example
```python
from calflops import create_empty_model

# Create empty model from Hugging Face Hub
model_name = "bert-base-uncased"
empty_model = create_empty_model(
    model_name=model_name,
    library_name="transformers",
    trust_remote_code=False
)

print(type(empty_model))
print(next(empty_model.parameters()).device)

# For models requiring authentication
model_name = "meta-llama/Llama-2-7b"
empty_model = create_empty_model(
    model_name=model_name,
    library_name="transformers",
    trust_remote_code=True,
    access_token="hf_your_token_here"
)
```

### Response
- **empty_model** (object) - An instance of the model loaded on the meta device.
```

--------------------------------

### Calculate FLOPs for Llama2-7B Model

Source: https://github.com/mryxj/calculate-flops.pytorch/blob/main/README.md

Calculates FLOPs, MACs, and parameters for a Llama2-7B model using its Hugging Face weights. Ensure the tokenizer matches the LLM model.

```python
#Large Languase Model, such as llama2-7b.
from calflops import calculate_flops
from transformers import LlamaTokenizer
from transformers import LlamaForCausalLM

batch_size, max_seq_length = 1, 128
model_name = "llama2_hf_7B"
model_save = "../model/" + model_name
model = LlamaForCausalLM.from_pretrained(model_save)
tokenizer = LlamaTokenizer.from_pretrained(model_save)
flops, macs, params = calculate_flops(model=model,
                                      input_shape=(batch_size, max_seq_length),
                                      transformer_tokenizer=tokenizer)
print("Llama2(7B) FLOPs:%s   MACs:%s   Params:%s \n" %(flops, macs, params))
#Llama2(7B) FLOPs:1.7 TFLOPS   MACs:850.00 GMACs   Params:6.74 B 

```

--------------------------------

### Calculate FLOPs for CNN Models

Source: https://github.com/mryxj/calculate-flops.pytorch/blob/main/README.md

Calculate FLOPs, MACs, and parameters for a PyTorch CNN model like AlexNet. This method automatically generates random input data if only the input shape is provided. Output is formatted as strings with specified precision.

```python
from calflops import calculate_flops
from torchvision import models

model = models.alexnet()
batch_size = 1
input_shape = (batch_size, 3, 224, 224)
flops, macs, params = calculate_flops(model=model, 
                                      input_shape=input_shape,
                                      output_as_string=True,
                                      output_precision=4)
print("Alexnet FLOPs:%s   MACs:%s   Params:%s 
" %(flops, macs, params))
#Alexnet FLOPs:4.2892 GFLOPS   MACs:2.1426 GMACs   Params:61.1008 M 
```

--------------------------------

### Calculate FLOPs for Local Transformer Models

Source: https://github.com/mryxj/calculate-flops.pytorch/blob/main/README.md

Analyze local models by providing the model and tokenizer to automatically generate input data.

```python
# Transformers Model, such as bert.
from calflops import calculate_flops
from transformers import AutoModel
from transformers import AutoTokenizer

batch_size, max_seq_length = 1, 128
model_name = "hfl/chinese-roberta-wwm-ext/"
model_save = "../pretrain_models/" + model_name
model = AutoModel.from_pretrained(model_save)
tokenizer = AutoTokenizer.from_pretrained(model_save)

flops, macs, params = calculate_flops(model=model, 
                                      input_shape=(batch_size,max_seq_length),
                                      transformer_tokenizer=tokenizer)
print("Bert(hfl/chinese-roberta-wwm-ext) FLOPs:%s   MACs:%s   Params:%s \n" %(flops, macs, params))
#Bert(hfl/chinese-roberta-wwm-ext) FLOPs:67.1 GFLOPS   MACs:33.52 GMACs   Params:102.27 M 
```

--------------------------------

### Calculate FLOPs with Custom Inputs

Source: https://context7.com/mryxj/calculate-flops.pytorch/llms.txt

Calculate FLOPs, MACs, and parameters for a given PyTorch model using custom inputs passed via kwargs. Set `output_as_string=True` for formatted results and `print_results=False` to suppress immediate output.

```python
flops, macs, params = calculate_flops(
    model=model,
    kwargs=dict(inputs),  # Pass tokenizer output directly
    output_as_string=True,
    print_results=False
)
print(f"BERT FLOPs:{flops}  MACs:{macs}  Params:{params}")
```

--------------------------------

### Format Numerical Outputs to Strings

Source: https://context7.com/mryxj/calculate-flops.pytorch/llms.txt

Utility functions to convert large numbers (FLOPs, MACs, parameters) into human-readable strings with appropriate SI prefixes (T, G, M, K). Allows customization of units and precision.

```python
from calflops import (
    flops_to_string,
    macs_to_string,
    params_to_string,
    number_to_string
)

# Convert FLOPs to string
flops = 1700000000000  # 1.7 trillion
print(flops_to_string(flops))
# Output: 1.7 TFLOPS

print(flops_to_string(flops, units='G', precision=3))
# Output: 1700 GFLOPS
```

```python
# Convert MACs to string
macs = 850000000000  # 850 billion
print(macs_to_string(macs))
# Output: 850 GMACs

print(macs_to_string(macs, units='T', precision=2))
# Output: 0.85 TMACs
```

```python
# Convert parameters to string
params = 6740000000  # 6.74 billion
print(params_to_string(params))
# Output: 6.74 B
```

```python
# Generic number conversion
print(number_to_string(1500000, precision=2))
# Output: 1.5 M
```

--------------------------------

### Calculate FLOPs for BERT Model

Source: https://github.com/mryxj/calculate-flops.pytorch/blob/main/README_CN.md

Use this snippet to calculate FLOPs, MACs, and parameters for BERT-based models from Hugging Face. Ensure the tokenizer and model are loaded correctly and inputs are padded to the maximum sequence length.

```python
from calflops import calculate_flops
from transformers import AutoModel
from transformers import AutoTokenizer
import torch

batch_size = 1
max_seq_length = 128
model_name = "hfl/chinese-roberta-wwm-ext/"
model_save = "/code/yexiaoju/generate_tags/models/pretrain_models/" + model_name
model = AutoModel.from_pretrained(model_save)
tokenizer = AutoTokenizer.from_pretrained(model_save)

text = ""
inputs = tokenizer(text,
                   add_special_tokens=True, 
                   return_attention_mask=True,
                   padding=True,
                   truncation="longest_first",
                   max_length=max_seq_length)

if len(inputs["input_ids"]) < max_seq_length:
    apply_num = max_seq_length-len(inputs["input_ids"])
    inputs["input_ids"].extend([0]*apply_num)
    inputs["token_type_ids"].extend([0]*apply_num)
    inputs["attention_mask"].extend([0]*apply_num)
    
inputs["input_ids"] = torch.tensor([inputs["input_ids"]])
inputs["token_type_ids"] = torch.tensor([inputs["token_type_ids"]])
inputs["attention_mask"] = torch.tensor([inputs["attention_mask"]])

flops, macs, params = calculate_flops(model=model,
                                      kwargs = inputs,
                                      print_results=False)
print("Bert(hfl/chinese-roberta-wwm-ext) FLOPs:%s   MACs:%s   Params:%s \n" %(flops, macs, params))
```

--------------------------------

### Calculate FLOPs with Custom Input Data

Source: https://github.com/mryxj/calculate-flops.pytorch/blob/main/README.md

Use the kwargs parameter to pass manually prepared input tensors when automatic generation is not suitable.

```python
# Transformers Model, such as bert.
from calflops import calculate_flops
from transformers import AutoModel
from transformers import AutoTokenizer


batch_size, max_seq_length = 1, 128
model_name = "hfl/chinese-roberta-wwm-ext/"
model_save = "/code/yexiaoju/generate_tags/models/pretrain_models/" + model_name
model = AutoModel.from_pretrained(model_save)
tokenizer = AutoTokenizer.from_pretrained(model_save)

text = ""
inputs = tokenizer(text,
                   add_special_tokens=True, 
                   return_attention_mask=True,
                   padding=True,
                   truncation="longest_first",
                   max_length=max_seq_length)

if len(inputs["input_ids"]) < max_seq_length:
    apply_num = max_seq_length-len(inputs["input_ids"])
    inputs["input_ids"].extend([0]*apply_num)
    inputs["token_type_ids"].extend([0]*apply_num)
    inputs["attention_mask"].extend([0]*apply_num)
    
inputs["input_ids"] = torch.tensor([inputs["input_ids"]])
inputs["token_type_ids"] = torch.tensor([inputs["token_type_ids"]])
inputs["attention_mask"] = torch.tensor([inputs["attention_mask"]])

flops, macs, params = calculate_flops(model=model,
                                      kwargs = inputs,
                                      print_results=False)
print("Bert(hfl/chinese-roberta-wwm-ext) FLOPs:%s   MACs:%s   Params:%s \n" %(flops, macs, params))
#Bert(hfl/chinese-roberta-wwm-ext) FLOPs:22.36 GFLOPS   MACs:11.17 GMACs   Params:102.27 M 
```

--------------------------------

### Calculate model FLOPs with calculate_flops

Source: https://github.com/mryxj/calculate-flops.pytorch/blob/main/README_CN.md

Use this function to profile a PyTorch model. It supports various configurations including backpropagation inclusion and custom input shapes.

```python
from calflops import calculate_flops

def calculate_flops(model,
                    input_shape=None,
                    transformer_tokenizer=None,
                    args=[],   
                    kwargs={},
                    forward_mode="forward",
                    include_backPropagation=False,
                    compute_bp_factor=2.0,         
                    print_results=True,
                    print_detailed=True,
                    output_as_string=True,
                    output_precision=2,
                    output_unit=None,
                    ignore_modules=None):
    
    """Returns the total floating-point operations, MACs, and parameters of a model.

    Args:
        model ([torch.nn.Module]): The model of input must be a PyTorch model.
        input_shape (tuple, optional): Input shape to the model. If args and kwargs is empty, the model takes a tensor with this shape as the only positional argument. Default to [].
        transformers_tokenizer (None, optional): Transforemrs Toekenizer must be special if model type is transformers and args、kwargs is empty. Default to None
        args (list, optinal): list of positional arguments to the model, such as bert input args is [input_ids, token_type_ids, attention_mask]. Default to []
        kwargs (dict, optional): dictionary of keyword arguments to the model, such as bert input kwargs is {'input_ids': ..., 'token_type_ids':..., 'attention_mask':...}. Default to {}
        forward_mode (str, optional): To determine the mode of model inference, Default to 'forward'. And use 'generate' if model inference uses model.generate().
        include_backPropagation (bool, optional): Decides whether the final return FLOPs computation includes the computation for backpropagation.
        compute_bp_factor (float, optional): The model backpropagation is a multiple of the forward propagation computation. Default to 2.
        print_results (bool, optional): Whether to print the model profile. Defaults to True.
        print_detailed (bool, optional): Whether to print the detailed model profile. Defaults to True.
        output_as_string (bool, optional): Whether to print the output as string. Defaults to True.
        output_precision (int, optional) : Output holds the number of decimal places if output_as_string is True. Default to 2.
        output_unit (str, optional): The unit used to output the result value, such as T, G, M, and K. Default is None, that is the unit of the output decide on value.
        ignore_modules ([type], optional): the list of modules to ignore during profiling. Defaults to None.
```

--------------------------------

### calculate_flops_hf

Source: https://github.com/mryxj/calculate-flops.pytorch/blob/main/README.md

Calculates the total floating-point operations, MACs, and parameters for a model hosted on Hugging Face.

```APIDOC
## calculate_flops_hf

### Description
Returns the total floating-point operations, MACs, and parameters of a model from the Hugging Face platform.

### Parameters
- **model_name** (str) - Required - The model name on Hugging Face.
- **input_shape** (tuple) - Optional - Input shape to the model.
- **library_name** (str) - Optional - Library name, default 'transformers'.
- **trust_remote_code** (bool) - Optional - Trust remote code for model structure.
- **access_token** (str) - Optional - Access token for restricted models.
- **forward_mode** (str) - Optional - Inference mode, 'forward' or 'generate'.
- **include_backPropagation** (bool) - Optional - Include backpropagation in computation.
- **compute_bp_factor** (float) - Optional - Backpropagation computation multiplier.
- **print_results** (bool) - Optional - Print model profile.
- **print_detailed** (bool) - Optional - Print detailed model profile.
- **output_as_string** (bool) - Optional - Output as string.
- **output_precision** (int) - Optional - Decimal places for string output.
```

--------------------------------

### Calculate FLOPs for Large Language Models (LLMs)

Source: https://context7.com/mryxj/calculate-flops.pytorch/llms.txt

Use calculate_flops for LLMs like LLaMA with their specific tokenizers. Supports standard forward pass and 'generate' mode. Can include backpropagation computation.

```python
from calflops import calculate_flops
from transformers import LlamaTokenizer, LlamaForCausalLM

# Example: LLaMA-2 7B Model (requires local model weights)
batch_size = 1
max_seq_length = 128
model_path = "path/to/llama2-7b"

model = LlamaForCausalLM.from_pretrained(model_path)
tokenizer = LlamaTokenizer.from_pretrained(model_path)

# Standard forward pass
flops, macs, params = calculate_flops(
    model=model,
    input_shape=(batch_size, max_seq_length),
    transformer_tokenizer=tokenizer,
    forward_mode="forward",  # Use "generate" for autoregressive generation
    output_as_string=True,
    output_precision=2
)
print(f"Llama2(7B) FLOPs:{flops}  MACs:{macs}  Params:{params}")
# Output: Llama2(7B) FLOPs:1.7 TFLOPS  MACs:850.00 GMACs  Params:6.74 B
```

```python
# Including backpropagation computation
flops, macs, params = calculate_flops(
    model=model,
    input_shape=(batch_size, max_seq_length),
    transformer_tokenizer=tokenizer,
    include_backPropagation=True,
    compute_bp_factor=2.0,  # Backward pass = 2x forward computation
    output_as_string=True
)
print(f"Llama2(7B) fwd+bwd FLOPs:{flops}  MACs:{macs}  Params:{params}")
# Output: Llama2(7B) fwd+bwd FLOPs:5.1 TFLOPS  MACs:2.55 TMACs  Params:6.74 B
```

--------------------------------

### calculate_flops_hf

Source: https://github.com/mryxj/calculate-flops.pytorch/blob/main/README.md

Calculates the FLOPs, MACs, and parameters for a Hugging Face model.

```APIDOC
## POST /api/calculate_flops_hf

### Description
Calculates the number of floating-point operations (FLOPs), multiply-accumulate operations (MACs), and parameters in a Hugging Face model.

### Method
POST

### Endpoint
/api/calculate_flops_hf

### Parameters
#### Query Parameters
- **model_name** (str) - Required - The name of the Hugging Face model.
- **input_shape** (tuple) - Required - The input shape for the model, e.g., (batch_size, sequence_length).
- **output_unit** (str, optional) - Optional - The unit for the output value (e.g., T, G, M, K). Defaults to None.
- **ignore_modules** (list, optional) - Optional - A list of modules to ignore during profiling. Defaults to None.

### Request Example
```json
{
  "model_name": "baichuan-inc/Baichuan-13B-Chat",
  "input_shape": [1, 128],
  "output_unit": "G"
}
```

### Response
#### Success Response (200)
- **flops** (str) - The calculated FLOPs.
- **macs** (str) - The calculated MACs.
- **params** (str) - The number of parameters.

#### Response Example
```json
{
  "model_name": "baichuan-inc/Baichuan-13B-Chat",
  "flops": "1.3 T",
  "macs": "0.65 T",
  "params": "13 B"
}
```
```

--------------------------------

### Calculate FLOPs for Hugging Face Models

Source: https://github.com/mryxj/calculate-flops.pytorch/blob/main/README.md

Use this function to calculate FLOPs, MACs, and parameters for Hugging Face models by their name. It requires an access token for models like Llama2 and assumes the model supports inference on a meta device. The default input shape is (1, 128).

```python
from calflops import calculate_flops_hf

model_name = "meta-llama/Llama-2-7b"
access_token = "..." # your application token for using llama2
flops, macs, params = calculate_flops_hf(model_name=model_name, access_token=access_token) # default input shape: (1, 128)
print("%s FLOPs:%s  MACs:%s  Params:%s 
" %(model_name, flops, macs, params))
```

--------------------------------

### calculate_flops_hf

Source: https://context7.com/mryxj/calculate-flops.pytorch/llms.txt

Calculate FLOPs for any model on the Hugging Face Hub without downloading full model weights. The function creates an empty model on the meta device and performs computation tracking. This is ideal for quickly estimating computational requirements of large models.

```APIDOC
## calculate_flops_hf

### Description
Calculate FLOPs for any model on the Hugging Face Hub without downloading full model weights. The function creates an empty model on the meta device and performs computation tracking. This is ideal for quickly estimating computational requirements of large models.

### Parameters
- **model_name** (str) - Required - The name or path of the model on the Hugging Face Hub.
- **input_shape** (tuple) - Required - The shape of the input tensor (batch_size, sequence_length).
- **access_token** (str) - Optional - Hugging Face access token for gated models.
- **trust_remote_code** (bool) - Optional - Whether to trust remote code execution for the model.
- **output_as_string** (bool) - Optional - If True, FLOPs, MACs, and Params are returned as strings.
- **output_precision** (int) - Optional - The precision for the output string if `output_as_string` is True.
- **print_results** (bool) - Optional - If True, prints the results to the console.
- **print_detailed** (bool) - Optional - If True, prints detailed FLOPs breakdown.

### Request Example
```python
from calflops import calculate_flops_hf

# Example 1: Open-source model (Baichuan)
batch_size, max_seq_length = 1, 128
model_name = "baichuan-inc/Baichuan-13B-Chat"

flops, macs, params = calculate_flops_hf(
    model_name=model_name,
    input_shape=(batch_size, max_seq_length),
    output_as_string=True,
    output_precision=2,
    print_results=True,
    print_detailed=True
)
print(f"{model_name} FLOPs:{flops}  MACs:{macs}  Params:{params}")

# Example 2: Using Hugging Face URL directly
model_name = "https://huggingface.co/THUDM/glm-4-9b-chat"  # or "THUDM/glm-4-9b-chat"

flops, macs, params = calculate_flops_hf(
    model_name=model_name,
    input_shape=(batch_size, max_seq_length),
    trust_remote_code=True
)
print(f"GLM-4-9B FLOPs:{flops}  MACs:{macs}  Params:{params}")

# Example 3: Gated model requiring access token (LLaMA-2)
model_name = "meta-llama/Llama-2-7b"
access_token = "hf_your_access_token_here"

flops, macs, params = calculate_flops_hf(
    model_name=model_name,
    access_token=access_token,
    input_shape=(batch_size, max_seq_length),
    output_as_string=True
)
print(f"{model_name} FLOPs:{flops}  MACs:{macs}  Params:{params}")
```

### Response
- **flops** (float or str) - The calculated Floating Point Operations.
- **macs** (float or str) - The calculated Multiply-Accumulate Operations.
- **params** (float or str) - The number of model parameters.
```

--------------------------------

### Calculate FLOPs with Custom Inputs

Source: https://context7.com/mryxj/calculate-flops.pytorch/llms.txt

Calculate FLOPs for a model using custom input tensors passed via `args` or `kwargs`. This bypasses automatic input generation and is suitable for complex model input requirements.

```python
from calflops import calculate_flops
from transformers import AutoModel, AutoTokenizer
import torch

model_name = "bert-base-uncased"
model = AutoModel.from_pretrained(model_name)
tokenizer = AutoTokenizer.from_pretrained(model_name)


```

--------------------------------

### calculate_flops

Source: https://github.com/mryxj/calculate-flops.pytorch/blob/main/README.md

Calculates the total floating-point operations, MACs, and parameters of a local PyTorch model.

```APIDOC
## calculate_flops

### Description
Returns the total floating-point operations, MACs, and parameters of a PyTorch model.

### Parameters
- **model** (torch.nn.Module) - Required - The PyTorch model to profile.
- **input_shape** (tuple) - Optional - Input shape to the model.
- **transformer_tokenizer** (None) - Optional - Tokenizer for transformer models.
- **args** (list) - Optional - Positional arguments to the model.
- **kwargs** (dict) - Optional - Keyword arguments to the model.
- **forward_mode** (str) - Optional - Inference mode, 'forward' or 'generate'.
- **include_backPropagation** (bool) - Optional - Include backpropagation in computation.
- **compute_bp_factor** (float) - Optional - Backpropagation computation multiplier.
- **print_results** (bool) - Optional - Print model profile.
- **print_detailed** (bool) - Optional - Print detailed model profile.
- **output_as_string** (bool) - Optional - Output as string.
- **output_precision** (int) - Optional - Decimal places for string output.
- **output_unit** (str) - Optional - Unit for output (T, G, M, K).
- **ignore_modules** (list) - Optional - Modules to ignore during profiling.
```

--------------------------------

### Calculate FLOPs for CNN Models

Source: https://context7.com/mryxj/calculate-flops.pytorch/llms.txt

Use calculate_flops for local PyTorch models like CNNs. Specify input shape and desired output format. `print_detailed=True` shows submodule breakdown.

```python
from calflops import calculate_flops
from torchvision import models

# Example 1: CNN Model (AlexNet)
model = models.alexnet()
batch_size = 1
input_shape = (batch_size, 3, 224, 224)

flops, macs, params = calculate_flops(
    model=model,
    input_shape=input_shape,
    output_as_string=True,
    output_precision=4,
    print_results=True,
    print_detailed=True
)
print(f"Alexnet FLOPs:{flops}  MACs:{macs}  Params:{params}")
# Output: Alexnet FLOPs:1.4297 GFLOPS  MACs:714.188 MMACs  Params:61.1008 M
```

```python
# Example 2: With numeric output and custom units
flops, macs, params = calculate_flops(
    model=model,
    input_shape=input_shape,
    output_as_string=True,
    output_precision=3,
    output_unit='M',  # Force output in Mega units
    print_results=False
)
print(f"Alexnet FLOPs:{flops}  MACs:{macs}  Params:{params}")
# Output: Alexnet FLOPs:1429.74 MFLOPS  MACs:714.188 MMACs  Params:61.101 M
```

```python
# Example 3: Get raw numeric values
flops, macs, params = calculate_flops(
    model=model,
    input_shape=input_shape,
    output_as_string=False,
    print_results=False
)
print(f"FLOPs: {flops}, MACs: {macs}, Params: {params}")
# Output: FLOPs: 1429740000, MACs: 714188000, Params: 61100840
```

--------------------------------

### Calculate FLOPs for Llama2 Large Language Model

Source: https://github.com/mryxj/calculate-flops.pytorch/blob/main/README_CN.md

This snippet calculates FLOPs, MACs, and parameters for Llama2-7B models. It's crucial that the `transformer_tokenizer` passed to `calculate_flops` is consistent with the model's tokenizer.

```python
#Large Languase Model, such as llama2-7b.
from calflops import calculate_flops
from transformers import LlamaTokenizer
from transformers import LlamaForCausalLM

batch_size = 1
max_seq_length = 128
model_name = "llama2_hf_7B"
model_save = "../model/" + model_name
model = LlamaForCausalLM.from_pretrained(model_save)
tokenizer = LlamaTokenizer.from_pretrained(model_save)
flops, macs, params = calculate_flops(model=model,
                                      input_shape=(batch_size, max_seq_length),
                                      transformer_tokenizer=tokenizer)
print("Llama2(7B) FLOPs:%s   MACs:%s   Params:%s \n" %(flops, macs, params))
```

--------------------------------

### Advanced Usage with Custom Inputs

Source: https://context7.com/mryxj/calculate-flops.pytorch/llms.txt

For models requiring multiple or complex inputs, use the `args` or `kwargs` parameters to pass custom input tensors directly instead of relying on automatic generation.

```APIDOC
## Advanced Usage with Custom Inputs

### Description
For models requiring multiple or complex inputs, use the `args` or `kwargs` parameters to pass custom input tensors directly instead of relying on automatic generation.

### Parameters for `calculate_flops`
- **model** (object) - Required - The PyTorch model instance.
- **inputs** (dict or tuple) - Required - A dictionary or tuple of input tensors for the model.
- **kwargs** - Additional keyword arguments to pass to the model's forward method.

### Request Example
```python
from calflops import calculate_flops
from transformers import AutoModel, AutoTokenizer
import torch

model_name = "bert-base-uncased"
model = AutoModel.from_pretrained(model_name)
tokenizer = AutoTokenizer.from_pretrained(model_name)

# Example with custom inputs
input_text = ["This is a test sentence.", "Another sentence for testing."]
inputs = tokenizer(input_text, return_tensors='pt', padding=True, truncation=True)

# Calculate FLOPs using custom inputs
flops, macs, params = calculate_flops(
    model=model,
    inputs=inputs
)

print(f"FLOPs:{flops}  MACs:{macs}  Params:{params}")

# Example with custom kwargs
input_ids = inputs['input_ids']
attention_mask = inputs['attention_mask']

flops, macs, params = calculate_flops(
    model=model,
    args=(input_ids,),
    kwargs={'attention_mask': attention_mask}
)
print(f"FLOPs:{flops}  MACs:{macs}  Params:{params}")
```
```

--------------------------------

### Calculate FLOPs for Transformer Models

Source: https://context7.com/mryxj/calculate-flops.pytorch/llms.txt

For Hugging Face Transformer models, provide the model name and tokenizer. The function automatically generates input tensors based on batch size and sequence length.

```python
from calflops import calculate_flops
from transformers import AutoModel, AutoTokenizer

# Example: BERT Model
batch_size = 1
max_seq_length = 128
model_name = "bert-base-uncased"

model = AutoModel.from_pretrained(model_name)
tokenizer = AutoTokenizer.from_pretrained(model_name)

flops, macs, params = calculate_flops(
    model=model,
    input_shape=(batch_size, max_seq_length),
    transformer_tokenizer=tokenizer,
    output_as_string=True,
    output_precision=2,
    print_results=True,
    print_detailed=True
)
print(f"BERT FLOPs:{flops}  MACs:{macs}  Params:{params}")
# Output: BERT FLOPs:22.36 GFLOPS  MACs:11.17 GMACs  Params:109.48 M
```

--------------------------------

### Output Formatting Utilities

Source: https://context7.com/mryxj/calculate-flops.pytorch/llms.txt

Calflops provides utility functions to convert numeric values to human-readable strings with appropriate unit prefixes (T, G, M, K).

```APIDOC
## Utility Functions for Output Formatting

### Description
Calflops provides utility functions to convert numeric values to human-readable strings with appropriate unit prefixes (T, G, M, K).

### Functions
- **flops_to_string(flops, units='TFLOPS', precision=2)**: Converts FLOPs to a human-readable string.
- **macs_to_string(macs, units='TMACs', precision=2)**: Converts MACs to a human-readable string.
- **params_to_string(params, units='B', precision=2)**: Converts parameters to a human-readable string.
- **number_to_string(number, units='', precision=2)**: Converts a generic number to a human-readable string.

### Parameters
- **flops/macs/params/number** (int or float) - The numeric value to convert.
- **units** (str) - The desired unit suffix (e.g., 'TFLOPS', 'GMACs', 'B').
- **precision** (int) - The number of decimal places for the output string.

### Request Example
```python
from calflops import (
    flops_to_string,
    macs_to_string,
    params_to_string,
    number_to_string
)

# Convert FLOPs to string
flops = 1700000000000  # 1.7 trillion
print(flops_to_string(flops))

print(flops_to_string(flops, units='G', precision=3))

# Convert MACs to string
macs = 850000000000  # 850 billion
print(macs_to_string(macs))

print(macs_to_string(macs, units='T', precision=2))

# Convert parameters to string
params = 6740000000  # 6.74 billion
print(params_to_string(params))

# Generic number conversion
print(number_to_string(1500000, precision=2))
```

### Response Example
```
1.7 TFLOPS
1700 GFLOPS
850 GMACs
0.85 TMACs
6.74 B
1.5 M
```
```

--------------------------------

### Generate transformer model inputs

Source: https://github.com/mryxj/calculate-flops.pytorch/blob/main/README_CN.md

Automatically generates input data formatted for transformer models based on the provided tokenizer and shape.

```python
def generate_transformer_input(model_tokenizer, input_shape, device):
    """Automatically generates data in the form of transformes model input format.
    
    Args:
        input_shape (tuple):transformers model input shape: (batch_size, seq_len).
        tokenizer (transformer.model.tokenization): transformers model tokenization.tokenizer.

    Returns:
        dict: data format of transformers model input, it is a dict which contain 'input_ids', 'attention_mask', 'token_type_ids' etc.
    """
```