### Add special tokens

Source: https://github.com/af-ai-center/swebert/blob/master/getting_started_with_swebert.ipynb

Prepends and appends special tokens '[CLS]' and '[SEP]' to the preprocessed example sentence, as required by BERT models.

```python
example_preprocessed = f'[CLS] {example_uncased} [SEP]'
example_preprocessed
```

--------------------------------

### Example sentence

Source: https://github.com/af-ai-center/swebert/blob/master/getting_started_with_swebert.ipynb

Defines an example sentence in Swedish to be used for model application.

```python
example = 'Jag är ett barn, och det här är mitt hem. Alltså är det ett barnhem!'
example
```

--------------------------------

### Encode example using BertWordPieceTokenizer

Source: https://github.com/af-ai-center/swebert/blob/master/getting_started_with_swebert.ipynb

Encodes the example sentence using the BertWordPieceTokenizer, obtaining tokens and their IDs.

```python
output = bert_word_piece_tokenizer.encode(example)  # attributes: output.ids, output.tokens, output.offsets
```

--------------------------------

### Tokenize preprocessed sentence

Source: https://github.com/af-ai-center/swebert/blob/master/getting_started_with_swebert.ipynb

Tokenizes the preprocessed example sentence using the BertTokenizer.

```python
tokens = bert_tokenizer.tokenize(example_preprocessed)

print(f'{len(tokens)} tokens:')
print(tokens)
```

--------------------------------

### Instantiate BertTokenizer

Source: https://github.com/af-ai-center/swebert/blob/master/getting_started_with_swebert.ipynb

Instantiates the BertTokenizer from the chosen pretrained SweBERT model.

```python
tokenizer = BertTokenizer.from_pretrained(pretrained_model_name, do_lower_case=False)
```

--------------------------------

### Choose SweBERT model

Source: https://github.com/af-ai-center/swebert/blob/master/getting_started_with_swebert.ipynb

Specifies the pretrained SweBERT model name to be used.

```python
pretrained_model_name = 'af-ai-center/bert-base-swedish-uncased'
# pretrained_model_name = af-ai-center/bert-large-swedish-uncased
```

--------------------------------

### Lowercase example sentence

Source: https://github.com/af-ai-center/swebert/blob/master/getting_started_with_swebert.ipynb

Converts the example sentence to lowercase to align with the uncased nature of the SweBERT model.

```python
example_uncased = example.lower()
example_uncased
```

--------------------------------

### Convert tokens to IDs

Source: https://github.com/af-ai-center/swebert/blob/master/getting_started_with_swebert.ipynb

Converts the tokens of the example sentence into their corresponding numerical IDs.

```python
indexed_tokens = bert_tokenizer.convert_tokens_to_ids(tokens)
print(indexed_tokens)
```

--------------------------------

### Instantiate BertWordPieceTokenizer

Source: https://github.com/af-ai-center/swebert/blob/master/getting_started_with_swebert.ipynb

Instantiates BertWordPieceTokenizer with specific parameters for handling Swedish text and lowercasing.

```python
bert_word_piece_tokenizer = BertWordPieceTokenizer("vocab_swebert.txt", lowercase=True, strip_accents=False)
```

--------------------------------

### Import necessary libraries

Source: https://github.com/af-ai-center/swebert/blob/master/getting_started_with_swebert.ipynb

Imports PyTorch, TensorFlow, and Hugging Face Transformers components for working with SweBERT models.

```python
import torch
import tensorflow as tf
from transformers import BertTokenizer, BertModel, TFBertModel, BertForMaskedLM 
from tokenizers import BertWordPieceTokenizer

import warnings; warnings.filterwarnings('ignore')
```

--------------------------------

### Masking a token with BertTokenizer

Source: https://github.com/af-ai-center/swebert/blob/master/getting_started_with_swebert.ipynb

Shows how to mask a token by converting it to its ID representation using BertTokenizer.

```python
# Mask token with BertTokenizer
indexed_tokens[masked_index] = bert_tokenizer.convert_tokens_to_ids('[MASK]')
print(indexed_tokens)
```

--------------------------------

### Instantiate PyTorch BertModel

Source: https://github.com/af-ai-center/swebert/blob/master/getting_started_with_swebert.ipynb

Instantiates the PyTorch version of the BertModel from the chosen pretrained SweBERT model.

```python
model = BertModel.from_pretrained(pretrained_model_name)
```

--------------------------------

### Get tokens from BertWordPieceTokenizer

Source: https://github.com/af-ai-center/swebert/blob/master/getting_started_with_swebert.ipynb

Extracts and prints the tokens generated by the BertWordPieceTokenizer.

```python
tokens_2 = output.tokens

print(f'{len(tokens_2)} tokens:')
print(tokens_2)
```

--------------------------------

### Show top 5 predictions for masked token

Source: https://github.com/af-ai-center/swebert/blob/master/getting_started_with_swebert.ipynb

Iterates through the top 5 predicted indices, converts them to tokens, and prints them.

```python
# show top5 predictions for masked token
for predicted_index in predicted_index_top5:
    predicted_token = tokenizer.convert_ids_to_tokens([predicted_index])[0]
    print(predicted_token)
```

--------------------------------

### Assert predicted token

Source: https://github.com/af-ai-center/swebert/blob/master/getting_started_with_swebert.ipynb

Asserts that the predicted token is 'barn'.

```python
assert predicted_token == 'barn'
```

--------------------------------

### Get token IDs from BertWordPieceTokenizer

Source: https://github.com/af-ai-center/swebert/blob/master/getting_started_with_swebert.ipynb

Extracts and prints the numerical IDs of the tokens generated by the BertWordPieceTokenizer.

```python
indexed_tokens_2 = output.ids
print(indexed_tokens_2)
```

--------------------------------

### Assert tokenization consistency

Source: https://github.com/af-ai-center/swebert/blob/master/getting_started_with_swebert.ipynb

Asserts that the tokenization results from BertTokenizer and BertWordPieceTokenizer are identical.

```python
# check that BertTokenizer & BertWordPieceTokenizer lead to the same results
assert tokens == tokens_2
assert indexed_tokens == indexed_tokens_2
```

--------------------------------

### Instantiate TensorFlow BertModel

Source: https://github.com/af-ai-center/swebert/blob/master/getting_started_with_swebert.ipynb

Instantiates the TensorFlow version of the BertModel from the chosen pretrained SweBERT model.

```python
model = TFBertModel.from_pretrained(pretrained_model_name)
```

--------------------------------

### Show prediction for masked token

Source: https://github.com/af-ai-center/swebert/blob/master/getting_started_with_swebert.ipynb

Converts the predicted index back to a token and prints it.

```python
# show prediction for masked token
predicted_token = tokenizer.convert_ids_to_tokens([predicted_index])[0]
print(predicted_token)
```

--------------------------------

### Masking a token with BertWordPieceTokenizer

Source: https://github.com/af-ai-center/swebert/blob/master/getting_started_with_swebert.ipynb

Demonstrates masking a token using the token_to_id method from BertWordPieceTokenizer.

```python
# Mask token with BertWordPieceTokenizer
indexed_tokens[masked_index] = bert_word_piece_tokenizer.token_to_id('[MASK]')
print(indexed_tokens)
```

--------------------------------

### Masking a token with Python lists

Source: https://github.com/af-ai-center/swebert/blob/master/getting_started_with_swebert.ipynb

Demonstrates how to mask a token by replacing it with '[MASK]' in a list of tokens.

```python
masked_index = 17  # 'barn'
```

```python
tokens[masked_index] = '[MASK]'
print(tokens)
```

--------------------------------

### Instantiate SweBERT model

Source: https://github.com/af-ai-center/swebert/blob/master/getting_started_with_swebert.ipynb

Initializes the BertForMaskedLM model from a pre-trained name and sets it to evaluation mode.

```python
# instantiate model
model = BertForMaskedLM.from_pretrained(pretrained_model_name)
_ = model.eval()
```

--------------------------------

### Instantiating Tokenizer and Model

Source: https://github.com/af-ai-center/swebert/blob/master/README.md

Example of how to instantiate the tokenizer and model using the transformers package for both PyTorch and TensorFlow.

```python
pretrained_model_name = 'af-ai-center/bert-base-swedish-uncased'

tokenizer = BertTokenizer.from_pretrained(pretrained_model_name)

# PyTorch
model = BertModel.from_pretrained(pretrained_model_name)

# TensorFlow
model = TFBertModel.from_pretrained(pretrained_model_name)
```

--------------------------------

### Package Installation

Source: https://github.com/af-ai-center/swebert/blob/master/README.md

Command to install necessary packages for using the Swedish BERT models.

```bash
pip install torch tensorflow transformers tokenizers notebook
```

--------------------------------

### Predict all tokens

Source: https://github.com/af-ai-center/swebert/blob/master/getting_started_with_swebert.ipynb

Uses the SweBERT model to predict all tokens in the input sequence and prints the shape of the output.

```python
# predict all tokens
with torch.no_grad():
    outputs = model(torch.tensor([indexed_tokens]))

predictions = outputs[0]
print(predictions.shape)  # 1 example, 21 tokens, 30522 vocab size
```

--------------------------------

### Instantiate BertTokenizer for Swedish

Source: https://github.com/af-ai-center/swebert/blob/master/getting_started_with_swebert.ipynb

Instantiates BertTokenizer, ensuring proper handling of Swedish characters by setting do_lower_case=False and manually lowercasing the input text.

```python
bert_tokenizer = BertTokenizer.from_pretrained(pretrained_model_name, do_lower_case=False)
```

--------------------------------

### Show prediction for masked token's index

Source: https://github.com/af-ai-center/swebert/blob/master/getting_started_with_swebert.ipynb

Finds and prints the index of the predicted token for the masked position.

```python
# show prediction for masked token's index
predicted_index = torch.argmax(predictions[0, masked_index])
print(predicted_index)
```

--------------------------------

### Show top 5 predictions for masked token's index

Source: https://github.com/af-ai-center/swebert/blob/master/getting_started_with_swebert.ipynb

Retrieves the indices of the top 5 predicted tokens for the masked position.

```python
# show top5 predictions for masked token's index
predicted_index_top5 = torch.argsort(predictions[0, masked_index], descending=True)[:5]
predicted_index_top5
```

=== COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.