Transformers

https://huggingface.co/docs/transformers/v5.4.0/
Admin
Transformers is a Python library that provides state-of-the-art pre-trained models for natural...
Tokens:2,906
Snippets:15
Trust Score:10
Update:3 months ago
Show doc for...
Tokens:
Raw
### Initialize and Use VitMatteForImageMatting

Source: https://huggingface.co/docs/transformers/model_doc/vitmatte

Demonstrates how to initialize the VitMatteImageProcessor and VitMatteForImageMatting models, prepare input images and trimaps, and obtain alpha mattes using the model. This snippet is useful for performing image matting tasks.

```python
from transformers import VitMatteImageProcessor, VitMatteForImageMatting
import torch
from PIL import Image
from huggingface_hub import hf_hub_download

processor = VitMatteImageProcessor.from_pretrained("hustvl/vitmatte-small-composition-1k")
model = VitMatteForImageMatting.from_pretrained("hustvl/vitmatte-small-composition-1k")

filepath = hf_hub_download(
    repo_id="hf-internal-testing/image-matting-fixtures", filename="image.png", repo_type="dataset"
)
image = Image.open(filepath).convert("RGB")
filepath = hf_hub_download(
    repo_id="hf-internal-testing/image-matting-fixtures", filename="trimap.png", repo_type="dataset"
)
trimap = Image.open(filepath).convert("L")

# prepare image + trimap for the model
inputs = processor(images=image, trimaps=trimap, return_tensors="pt")

with torch.no_grad():
    alphas = model(**inputs).alphas
print(alphas.shape)

```

--------------------------------

### Initialize ViTMatte Configuration and Model

Source: https://huggingface.co/docs/transformers/model_doc/vitmatte

Demonstrates how to initialize a ViTMatte configuration and then use it to create a ViTMatteForImageMatting model. This is useful for setting up the model with specific configurations.

```python
from transformers import VitMatteConfig, VitMatteForImageMatting

# Initializing a ViTMatte hustvl/vitmatte-small-composition-1k style configuration
configuration = VitMatteConfig()

# Initializing a model (with random weights) from the hustvl/vitmatte-small-composition-1k style configuration
model = VitMatteForImageMatting(configuration)

# Accessing the model configuration
configuration = model.config
```

--------------------------------

### VitMatteImageProcessor.preprocess

Source: https://huggingface.co/docs/transformers/model_doc/vitmatte

Constructs a VitMatteImageProcessor image processor and preprocesses images and trimaps for the VitMatte model. It handles various input formats and allows for optional resizing and tensor conversion.

```APIDOC
## VitMatteImageProcessor.preprocess

### Description
Preprocesses images and trimaps for the VitMatte model. This method handles various input formats and allows for optional resizing and tensor conversion.

### Method
`preprocess`

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
- **images** (Union[PIL.Image.Image, numpy.ndarray, torch.Tensor, list[PIL.Image.Image], list[numpy.ndarray], list[torch.Tensor]]) - Required - Image to preprocess. Expects a single or batch of images with pixel values ranging from 0 to 255. If passing in images with pixel values between 0 and 1, set `do_rescale=False`.
- **trimaps** (Union[PIL.Image.Image, numpy.ndarray, torch.Tensor, list[PIL.Image.Image], list[numpy.ndarray], list[torch.Tensor]]) - Required - The trimaps to preprocess.
- **size_divisor** (int, optional, defaults to `self.size_divisor`) - The width and height of the image will be padded to be divisible by this number.
- **return_tensors** (str or TensorType, optional) - Returns stacked tensors if set to `'pt'`, otherwise returns a list of tensors.
- ****kwargs** (ImagesKwargs, optional) - Additional image preprocessing options. Model-specific kwargs are listed above; see the TypedDict class for the complete list of supported arguments.

### Request Example
```json
{
  "images": "...",
  "trimaps": "...",
  "size_divisor": 32,
  "return_tensors": "pt"
}
```

### Response
#### Success Response (200)
- **data** (dict) - Dictionary of lists/arrays/tensors returned by the __call__ method ('pixel_values', etc.).
- **tensor_type** (Union[None, str, TensorType], optional) - You can give a tensor_type here to convert the lists of integers in PyTorch/Numpy Tensors at initialization.
```

--------------------------------

### VitMatteImageProcessorPil.preprocess

Source: https://huggingface.co/docs/transformers/model_doc/vitmatte

Preprocesses images and trimaps for the ViTMatte model. It handles resizing, normalization, and tensor conversion.

```APIDOC
## preprocess

### Description
Preprocesses images and trimaps for the ViTMatte model. It handles resizing, normalization, and tensor conversion.

### Method
`preprocess`

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
- **images** (Union[PIL.Image.Image, numpy.ndarray, torch.Tensor, list[PIL.Image.Image], list[numpy.ndarray], list[torch.Tensor]]) - Required - Image to preprocess. Expects a single or batch of images with pixel values ranging from 0 to 255. If passing in images with pixel values between 0 and 1, set `do_rescale=False`.
- **trimaps** (Union[PIL.Image.Image, numpy.ndarray, torch.Tensor, list[PIL.Image.Image], list[numpy.ndarray], list[torch.Tensor]]) - Required - The trimaps to preprocess.
- **size_divisor** (int, optional, defaults to `self.size_divisor`) - The width and height of the image will be padded to be divisible by this number.
- **return_tensors** (str or TensorType, optional) - Returns stacked tensors if set to `'pt'`, otherwise returns a list of tensors.
- ****kwargs** (Unpack[VitMatteImageProcessorKwargs], optional) - Additional image preprocessing options. Model-specific kwargs are listed above; see the TypedDict class for the complete list of supported arguments.

### Request Example
None

### Response
#### Success Response (200)
- **data** (dict) - Dictionary of lists/arrays/tensors returned by the __call__ method ('pixel_values', etc.).
- **tensor_type** (Union[None, str, TensorType], optional) - You can give a tensor_type here to convert the lists of integers in PyTorch/Numpy Tensors at initialization.
```

--------------------------------

### VitMatteForImageMatting.forward

Source: https://huggingface.co/docs/transformers/model_doc/vitmatte

Performs the forward pass of the VitMatteForImageMatting model. It takes image data and optional labels as input and returns the model's predictions.

```APIDOC
## VitMatteForImageMatting.forward

### Description
Performs the forward pass of the VitMatteForImageMatting model. It takes image data and optional labels as input and returns the model's predictions.

### Method
forward

### Parameters
#### Input Parameters
- **pixel_values** (torch.Tensor | None) - Optional: The tensors corresponding to the input images. Pixel values can be obtained using `VitMatteImageProcessor`. See `VitMatteImageProcessor.__call__()` for details.
- **output_attentions** (bool | None) - Optional: Whether or not to return the attentions tensors of all attention layers.
- **output_hidden_states** (bool | None) - Optional: Whether or not to return the hidden states of all layers.
- **labels** (torch.LongTensor | None) - Optional: Ground truth image matting for computing the loss.
- **return_dict** (bool | None) - Optional: Whether or not to return a `ModelOutput` instead of a plain tuple.

### Request Example
```python
from transformers import VitMatteImageProcessor, VitMatteForImageMatting
import torch
from PIL import Image
from huggingface_hub import hf_hub_download

processor = VitMatteImageProcessor.from_pretrained("hustvl/vitmatte-small-composition-1k")
model = VitMatteForImageMatting.from_pretrained("hustvl/vitmatte-small-composition-1k")

filepath = hf_hub_download(
    repo_id="hf-internal-testing/image-matting-fixtures", filename="image.png", repo_type="dataset"
)
image = Image.open(filepath).convert("RGB")
filepath = hf_hub_download(
    repo_id="hf-internal-testing/image-matting-fixtures", filename="trimap.png", repo_type="dataset"
)
trimap = Image.open(filepath).convert("L")

# prepare image + trimap for the model
inputs = processor(images=image, trimaps=trimap, return_tensors="pt")

with torch.no_grad():
    alphas = model(**inputs).alphas
print(alphas.shape)
```

### Response
#### Success Response
- **alphas** (torch.Tensor): The predicted alpha matte.
- **attentions** (tuple, optional): The tuple of attention weights.
- **hidden_states** (tuple, optional): The tuple of hidden states.

#### Response Example
```python
# Example output shape
torch.Size([1, 1, 640, 960])
```
```

--------------------------------

### VitMatteConfig

Source: https://huggingface.co/docs/transformers/model_doc/vitmatte

Configuration class for the ViTMatte model. Used to store and control model architecture and behavior.

```APIDOC
## VitMatteConfig

### Description
Configuration class for the ViTMatte model. It is used to instantiate a ViTMatte model according to the specified arguments, defining the model architecture. Configuration objects inherit from `PreTrainedConfig` and can be used to control the model outputs.

### Parameters

* **backbone_config** (`Union[dict, ~configuration_utils.PreTrainedConfig]`, *optional*) : The configuration of the backbone model.
* **hidden_size** (`int`, *optional*, defaults to `384`) : Dimension of the hidden representations.
* **batch_norm_eps** (`float`, *optional*, defaults to `1e-05`) : The epsilon used by the batch norm layers.
* **initializer_range** (`float`, *optional*, defaults to `0.02`) : The standard deviation of the truncated_normal_initializer for initializing all weight matrices.
* **convstream_hidden_sizes** (`list[int]`, *optional*, defaults to `[48, 96, 192]`) : The output channels of the ConvStream module.
* **fusion_hidden_sizes** (`list[int]`, *optional*, defaults to `[256, 128, 64, 32]`) : The output channels of the Fusion blocks.

### Example
```python
>>> from transformers import VitMatteConfig, VitMatteForImageMatting

>>> # Initializing a ViTMatte hustvl/vitmatte-small-composition-1k style configuration
>>> configuration = VitMatteConfig()

>>> # Initializing a model (with random weights) from the hustvl/vitmatte-small-composition-1k style configuration
>>> model = VitMatteForImageMatting(configuration)

>>> # Accessing the model configuration
>>> configuration = model.config
```
```
Tokens:
Raw