### Install VNLP Library

Source: https://github.com/vngrs-ai/vnlp/blob/main/docs/source/quickstart.md

Instructions to install the vngrs-nlp library using pip.

```console
$ pip install vngrs-nlp
```

--------------------------------

### Perform Sentiment Analysis with CLI

Source: https://github.com/vngrs-ai/vnlp/blob/main/docs/source/quickstart.md

An example of using the VNLP command-line API to perform sentiment analysis on a Turkish sentence.

```console
$ vnlp --task sentiment_analysis --text "Sipariş geldiğinde biz karnımızı çoktan atıştırmalıklarla doyurmuştuk."
```

--------------------------------

### VNLP Command Line API General Usage

Source: https://github.com/vngrs-ai/vnlp/blob/main/docs/source/quickstart.md

Shows the general format for using the VNLP command-line interface to perform tasks on input text.

```console
$ vnlp --task TASK_NAME --text INPUT_TEXT
```

--------------------------------

### List Available VNLP CLI Tasks

Source: https://github.com/vngrs-ai/vnlp/blob/main/docs/source/quickstart.md

Command to list all available tasks and functionalities supported by the VNLP command-line interface.

```console
$ vnlp --list_tasks
```

--------------------------------

### Perform Named Entity Recognition with Python API

Source: https://github.com/vngrs-ai/vnlp/blob/main/docs/source/quickstart.md

Demonstrates how to import, initialize, and use the NamedEntityRecognizer class to predict named entities in a given Turkish text.

```python
from vnlp import NamedEntityRecognizer
ner = NamedEntityRecognizer()
ner.predict("Ben Melikşah, 29 yaşındayım, İstanbul'da ikamet ediyorum ve VNGRS AI Takımı'nda çalışıyorum.")
```

--------------------------------

### Install VNLP Python Package

Source: https://github.com/vngrs-ai/vnlp/blob/main/README.md

This snippet shows how to install the VNLP library using pip, the standard package installer for Python. It ensures all necessary dependencies are downloaded and set up for use in your Python environment.

```Python
pip install vngrs-nlp
```

--------------------------------

### Initialize and Use VNLP Dependency Parser

Source: https://github.com/vngrs-ai/vnlp/blob/main/README.md

This example demonstrates how to initialize the `DependencyParser` from the VNLP library and use it to predict grammatical dependencies in a Turkish sentence. The output is a list of tuples, where each tuple contains a word and its predicted part-of-speech tag.

```Python
from vnlp import DependencyParser
dep_parser = DependencyParser()

dep_parser.predict("Oğuz'un kırmızı bir Astra'sı vardı.")
[('Oğuz'un', 'PROPN'),
 ('kırmızı', 'ADJ'),
 ('bir', 'DET'),
 ("Astra'sı", 'PROPN'),
 ('vardı', 'VERB'),
 ('.', 'PUNCT')]
```

--------------------------------

### Use SentencePiece Unigram Tokenizer for Text Encoding

Source: https://github.com/vngrs-ai/vnlp/blob/main/docs/source/main_classes/word_embeddings.md

This snippet shows how to initialize a SentencePiece tokenizer from a pre-trained model file and use it to encode text. It demonstrates both `encode_as_pieces` to get tokenized segments and `encode_as_ids` to get numerical IDs for the tokens. A 'SentencePiece_16k_Tokenizer.model' file is required.

```Python
import sentencepiece as spm
sp = spm.SentencePieceProcessor('SentencePiece_16k_Tokenizer.model')
sp.encode_as_pieces('bilemezlerken')
sp.encode_as_ids('bilemezlerken')
```

--------------------------------

### Load and Use FastText Model with Gensim in Python

Source: https://github.com/vngrs-ai/vnlp/blob/main/vnlp/turkish_word_embeddings/ReadMe.md

This snippet illustrates how to load a pre-trained FastText model using the `gensim` library in Python. It then demonstrates how to find the most similar words to a given word, showcasing FastText's capability, especially with out-of-vocabulary words due to its character n-gram approach. Users need to download the FastText model file (e.g., 'FastText_large.model') and have `gensim` installed.

```Python
>>> # FastText
>>> from gensim.models import FastText
>>> 
>>> model = FastText.load('FastText_large.model')
>>> model.wv.most_similar('yamaçlardan', topn = 10)
[('kayalardan', 0.8601457476615906),
('kayalıklardan', 0.8567330837249756),
('tepelerden', 0.8423191905021667),
('ormanlardan', 0.8362939357757568),
('dağlardan', 0.8140010833740234),
('amaçlardan', 0.810560405254364),
('bloklardan', 0.803180992603302),
('otlardan', 0.8026642203330994),
('kısımlardan', 0.7993910312652588),
('ağaçlardan', 0.7961613535881042)]
```

--------------------------------

### Load and Use Word2Vec Model with Gensim in Python

Source: https://github.com/vngrs-ai/vnlp/blob/main/vnlp/turkish_word_embeddings/ReadMe.md

This snippet demonstrates how to load a pre-trained Word2Vec model using the `gensim` library in Python. It then shows how to find the most similar words to a given word, illustrating the model's ability to capture semantic relationships. Users need to download the Word2Vec model file (e.g., 'Word2Vec_large.model') and have `gensim` installed.

```Python
>>> # Word2Vec
>>> from gensim.models import Word2Vec
>>> 
>>> model = Word2Vec.load('Word2Vec_large.model')
>>> model.wv.most_similar('gandalf', topn = 10)
[('saruman', 0.7291593551635742),
('thorin', 0.6473978161811829),
('aragorn', 0.6401687264442444),
('isengard', 0.6123237013816833),
('orklar', 0.59786057472229),
('gollum', 0.5905635952949524),
('baggins', 0.5837421417236328),
('frodo', 0.5819021463394165),
('belgarath', 0.5811135172843933),
('sauron', 0.5763844847679138)]
```

--------------------------------

### Utilize Text Normalization Functions in vnlp Python Library

Source: https://github.com/vngrs-ai/vnlp/blob/main/Examples.ipynb

Provides examples for various text normalization functionalities offered by `vnlp.Normalizer`, including typo correction, number-to-word conversion, deascification, lowercasing, punctuation removal, and accent mark removal.

```python
from vnlp import Normalizer

normalizer = Normalizer()
```

```python
normalizer.correct_typos("kassıtlı yaezım hatasssı ekliyorumm")
```

```python
normalizer.convert_numbers_to_words(
    "sabah 2 yumurta yedim ve tartıldığımda 1,15 kilogram aldığımı gördüm".split()
)
```

```python
normalizer.deasciify("boyle sey gormedim duymadim".split())
```

```python
normalizer.deasciify("yatirdim".split())
```

```python
normalizer.lower_case("Test karakterleri: İIĞÜÖŞÇ")
```

```python
normalizer.remove_punctuations("noktalamalı test cümlesidir...")
```

```python
normalizer.remove_accent_marks("merhâbâ gûzel yîlkî atî")
```

--------------------------------

### Querying Most Similar Words with Word2Vec

Source: https://github.com/vngrs-ai/vnlp/blob/main/vnlp/turkish_word_embeddings/example.ipynb

This example shows how to use the loaded Word2Vec model to find words semantically similar to a given word, 'gandalf'. The `most_similar` method returns a list of the top N (here, 20) most similar words based on the model's learned embeddings, useful for tasks like recommendation or semantic search.

```python
model.wv.most_similar('gandalf', topn = 20)
```

--------------------------------

### Use SentencePiece Unigram Tokenizer in Python

Source: https://github.com/vngrs-ai/vnlp/blob/main/vnlp/turkish_word_embeddings/ReadMe.md

This snippet demonstrates how to load and use a pre-trained SentencePiece Unigram Tokenizer model in Python. It shows two primary functionalities: encoding a text into subword pieces and encoding it into corresponding numerical IDs. This tokenizer is useful for handling various languages and out-of-vocabulary words. Users need to download the tokenizer model file (e.g., 'SentencePiece_16k_Tokenizer.model') and have `sentencepiece` installed.

```Python
>>> # SentencePiece Unigram Tokenizer
>>> import sentencepiece as spm
>>> sp = spm.SentencePieceProcessor('SentencePiece_16k_Tokenizer.model')
>>> tokenizer.encode_as_pieces('bilemezlerken')
[' bile', 'mez', 'lerken']
>>> tokenizer.encode_as_ids('bilemezlerken')
[180, 1200, 8167]
```

--------------------------------

### Initialize and Use Dependency Parser in Python

Source: https://github.com/vngrs-ai/vnlp/blob/main/Examples.ipynb

Illustrates the initialization of `DependencyParser` from `vnlp` and its usage for parsing sentence dependencies in Turkish.

```python
from vnlp import DependencyParser

dep_parser = DependencyParser()
```

```python
dep_parser.predict(
    "Onun için yol arkadaşlarımızı titizlikle seçer, kendilerini iyice sınarız."
)
```

--------------------------------

### Initialize and Use Part of Speech Tagger in Python

Source: https://github.com/vngrs-ai/vnlp/blob/main/Examples.ipynb

Covers the initialization of `PoSTagger` from `vnlp` and its application for tagging parts of speech in Turkish sentences.

```python
from vnlp import PoSTagger

pos_tagger = PoSTagger()
```

```python
pos_tagger.predict("Oğuz'un kırmızı bir Astra'sı vardı.")
```

--------------------------------

### Initialize and Use Morphological Analyzer (StemmerAnalyzer) in Python

Source: https://github.com/vngrs-ai/vnlp/blob/main/Examples.ipynb

Demonstrates how to initialize the `StemmerAnalyzer` from `vnlp` and use it to predict morphological analyses for Turkish sentences.

```python
from vnlp import StemmerAnalyzer

stemmer_analyzer = StemmerAnalyzer()
```

```python
stemmer_analyzer.predict("Üniversite sınavlarına canla başla çalışıyorlardı.")
```

```python
stemmer_analyzer.predict("Şimdi baştan başla.")
```

--------------------------------

### Initialize and Use Named Entity Recognizer (NER) in Python

Source: https://github.com/vngrs-ai/vnlp/blob/main/Examples.ipynb

Shows how to initialize the `NamedEntityRecognizer` from `vnlp` and apply it to identify named entities in a Turkish sentence.

```python
from vnlp import NamedEntityRecognizer

ner = NamedEntityRecognizer()
```

```python
ner.predict(
    "Benim adım Melikşah, 29 yaşındayım, İstanbul'da ikamet ediyorum ve VNGRS AI Takımı'nda çalışıyorum."
)
```

--------------------------------

### vnlp.normalizer.normalizer Module API Reference

Source: https://github.com/vngrs-ai/vnlp/blob/main/docs/source/main_classes/normalizer.md

API documentation for the `vnlp.normalizer.normalizer` module, which is part of the `vnlp` project. This documentation is automatically generated to include all public classes, functions, and variables defined within the module, providing a comprehensive overview of its functionalities.

```APIDOC
Module: vnlp.normalizer.normalizer
  Description: This entry refers to the API documentation for the 'vnlp.normalizer.normalizer' module.
  Members: All public members (classes, functions, variables) within this module are included in the documentation.
```

--------------------------------

### Load and Use Turkish Word Embeddings (Word2Vec, FastText) with Gensim

Source: https://github.com/vngrs-ai/vnlp/blob/main/Examples.ipynb

Shows how to load pre-trained Turkish Word2Vec and FastText models using Gensim and find the most similar words for a given query.

```python
from gensim.models import Word2Vec, FastText
```

```python
# Word2Vec
model = Word2Vec.load("vnlp/turkish_word_embeddings/Word2Vec_large.model")
model.wv.most_similar("gandalf", topn=20)
```

```python
# FastText
model = Word2Vec.load("vnlp/turkish_word_embeddings/FastText_large.model")
model.wv.most_similar("yamaçlardan", topn=20)
```

--------------------------------

### vnlp.dependency_parser.dependency_parser Module API

Source: https://github.com/vngrs-ai/vnlp/blob/main/docs/source/main_classes/dependency_parser.md

API documentation for the core dependency parser module within the vnlp project. This directive specifies that all public members of the 'vnlp.dependency_parser.dependency_parser' module should be documented.

```APIDOC
.. automodule:: vnlp.dependency_parser.dependency_parser
    :members:
```

--------------------------------

### Perform Sentiment Analysis with vnlp in Python

Source: https://github.com/vngrs-ai/vnlp/blob/main/Examples.ipynb

Explains how to initialize the `SentimentAnalyzer` from `vnlp` and use it to predict sentiment probabilities and direct sentiment labels for Turkish texts.

```python
from vnlp import SentimentAnalyzer

sentiment_analyzer = SentimentAnalyzer()
```

```python
sentiment_analyzer.predict_proba(
    "Zeynep'in okul taksitini Bürokratistan'daki Parabank'tan yatırdım."
)
```

```python
sentiment_analyzer.predict_proba(
    "Sipariş geldiğinde biz karnımızı çoktan atıştırmalıklarla doyurmuştuk."
)
```

```python
sentiment_analyzer.predict(
    "Servis daha iyi olabilirdi ama lezzet ve hız geçer not aldı."
)
```

```python
sentiment_analyzer.predict_proba("Mutlu değilim diyemem.")
```

```python
sentiment_analyzer.predict_proba(
    "Geçmesin günümüz sevgilim yasla, o güzel başını göğsüme yasla."
)
```

--------------------------------

### Visualize Dependency and POS Tags with Spacy in Python

Source: https://github.com/vngrs-ai/vnlp/blob/main/Examples.ipynb

Demonstrates how to integrate `vnlp`'s `PoSTagger` and `DependencyParser` with Spacy's `displacy` for visualizing linguistic annotations. It includes setting up display options for better visualization.

```python
from spacy import displacy
```

```python
text = "Oğuz'un kırmızı bir Astra'sı vardı."
pos_result = pos_tagger.predict(text)
dp_result = dep_parser.predict(
    text, displacy_format=True, pos_result=pos_result
)
# Options: https://spacy.io/api/top-level#displacy_options
opts = {
    "distance": 100,
    "word_spacing": 45,
    "arrow_stroke": 2,  # arrow body width
    "arrow_width": 10,  # arrow head width
    "arrow_spacing": 12,  # space between arrows
    "compact": False  # square arrows
}
displacy.render(dp_result, style="dep", options=opts, manual=True)
```

--------------------------------

### Loading a Pre-trained Word2Vec Model

Source: https://github.com/vngrs-ai/vnlp/blob/main/vnlp/turkish_word_embeddings/example.ipynb

This code demonstrates how to load a pre-trained Word2Vec model from a file named 'Word2Vec_large.model'. Loading an existing model saves computation time and allows immediate use of learned word embeddings for various NLP applications.

```python
model = Word2Vec.load('Word2Vec_large.model')
```

--------------------------------

### vnlp.dependency_parser.spu_context_dp Module API

Source: https://github.com/vngrs-ai/vnlp/blob/main/docs/source/main_classes/dependency_parser.md

API documentation for the SentencePiece Unigram Context Dependency Parser module. This directive instructs the documentation generator to include all public members of the 'vnlp.dependency_parser.spu_context_dp' module.

```APIDOC
.. automodule:: vnlp.dependency_parser.spu_context_dp
    :members:
```

--------------------------------

### Split Sentences using vnlp SentenceSplitter in Python

Source: https://github.com/vngrs-ai/vnlp/blob/main/Examples.ipynb

Illustrates how to initialize `SentenceSplitter` from `vnlp` and use it to accurately segment Turkish text into individual sentences, handling various punctuation and numbering conventions.

```python
from vnlp import SentenceSplitter

sentence_splitter = SentenceSplitter()
```

```python
sentence_splitter.split_sentences(
    'Av. Meryem Beşer, 3.5 yıldır süren dava ile ilgili dedi ki, "Duruşma bitti, dava lehimize sonuçlandı." Bu harika bir haberdi!'
)
```

```python
sentence_splitter.split_sentences(
    "4. Murat, diğer yazım şekli ile IV. Murat, alkollü içecekleri halka yasaklamıştı."
)
```

--------------------------------

### Cite VNLP Turkish NLP Package

Source: https://github.com/vngrs-ai/vnlp/blob/main/README.md

This BibTeX entry provides the standard citation format for referencing the VNLP: Turkish NLP Package in academic papers and research. It includes details such as authors, title, journal, and year, ensuring proper attribution.

```BibTeX
@article{turker2024vnlp,
  title={VNLP: Turkish NLP Package},
  author={Turker, Meliksah and Ari, Erdi and Han, Aydin},
  journal={arXiv preprint arXiv:2403.01309},
  year={2024}
}
```

--------------------------------

### Load and Query FastText Embeddings with Gensim

Source: https://github.com/vngrs-ai/vnlp/blob/main/docs/source/main_classes/word_embeddings.md

This snippet illustrates how to load a pre-trained FastText model using the gensim library and query for the most similar words. It expects a pre-trained model file, such as 'FastText_large.model', and returns a list of similar words and their scores.

```Python
from gensim.models import FastText

model = FastText.load('FastText_large.model')
model.wv.most_similar('yamaçlardan', topn = 10)
```

--------------------------------

### Visualize VNLP Dependency Parser Results with Spacy

Source: https://github.com/vngrs-ai/vnlp/blob/main/README.md

This snippet illustrates how to integrate VNLP's `DependencyParser` with Spacy's `displacy` module to visualize the dependency parsing results. By setting `displacy_format = True`, the parser returns a format compatible with Spacy's visualization tool, allowing for clear graphical representation of sentence structure.

```Python
import spacy
from vnlp import DependencyParser
dependency_parser = DependencyParser()
result = dependency_parser.predict("Oğuz'un kırmızı bir Astra'sı vardı.", displacy_format = True)
spacy.displacy.render(result, style="dep", manual = True)
```

--------------------------------

### Load and Query Word2Vec Embeddings with Gensim

Source: https://github.com/vngrs-ai/vnlp/blob/main/docs/source/main_classes/word_embeddings.md

This snippet demonstrates how to load a pre-trained Word2Vec model using the gensim library and find the most similar words to a given query. It requires a pre-trained model file, such as 'Word2Vec_large.model', and outputs a list of similar words with their similarity scores.

```Python
from gensim.models import Word2Vec

model = Word2Vec.load('Word2Vec_large.model')
model.wv.most_similar('gandalf', topn = 10)
```

--------------------------------

### vnlp.dependency_parser.treestack_dp Module API

Source: https://github.com/vngrs-ai/vnlp/blob/main/docs/source/main_classes/dependency_parser.md

API documentation for the Tree-stack Dependency Parser module. This directive ensures that all public members of the 'vnlp.dependency_parser.treestack_dp' module are included in the generated documentation.

```APIDOC
.. automodule:: vnlp.dependency_parser.treestack_dp
    :members:
```

--------------------------------

### Remove Stopwords with vnlp StopwordRemover in Python

Source: https://github.com/vngrs-ai/vnlp/blob/main/Examples.ipynb

Demonstrates the `StopwordRemover` from `vnlp` for both static and dynamic stopword removal in Turkish. It also shows how to add dynamically detected stopwords to the remover's lexicon.

```python
from vnlp import StopwordRemover

stopword_remover = StopwordRemover()
```

```python
sentence = (
    "acaba bugün kahvaltıda kahve yerine çay mı içsem ya da neyse süt içeyim"
)
stopword_remover.drop_stop_words(sentence.split())
```

```python
sentence = "ben bugün gidip aşı olacağım sonra da eve gelip telefon açacağım aşı nasıl etkiledi eve gelip anlatırım aşı olmak bu dönemde çok ama ama ama ama çok önemli"
dynamically_detected_stop_words = (
    stopword_remover.dynamically_detect_stop_words(sentence.split())
)
print(dynamically_detected_stop_words)
```

```python
stopword_remover.add_to_stop_words(dynamically_detected_stop_words)
```

```python
stopword_remover.drop_stop_words("aşı olmak önemli demiş miydim".split())
```

--------------------------------

### API Reference for vnlp.part_of_speech_tagger.part_of_speech_tagger

Source: https://github.com/vngrs-ai/vnlp/blob/main/docs/source/main_classes/part_of_speech_tagger.md

Documents the public members of the core Part of Speech Tagger module within the vnlp library, generated via Sphinx automodule directives.

```APIDOC
Module: vnlp.part_of_speech_tagger.part_of_speech_tagger
  Description: Core module for general Part of Speech Tagging functionalities.
  Members: All public classes, functions, and variables are documented.
```

--------------------------------

### API Reference for vnlp.part_of_speech_tagger.treestack_pos

Source: https://github.com/vngrs-ai/vnlp/blob/main/docs/source/main_classes/part_of_speech_tagger.md

Documents the public members of the Tree-stack Part of Speech Tagger module within the vnlp library, generated via Sphinx automodule directives.

```APIDOC
Module: vnlp.part_of_speech_tagger.treestack_pos
  Description: Module for Tree-stack Part of Speech Tagging.
  Members: All public classes, functions, and variables are documented.
```

--------------------------------

### Importing Word2Vec from Gensim

Source: https://github.com/vngrs-ai/vnlp/blob/main/vnlp/turkish_word_embeddings/example.ipynb

This snippet imports the `Word2Vec` class from the `gensim.models` module. This class is essential for working with Word2Vec models, including training new models or loading existing ones for word embedding tasks.

```python
from gensim.models import Word2Vec
```

--------------------------------

### Sphinx Automodule Directive for vnlp.stemmer_morph_analyzer

Source: https://github.com/vngrs-ai/vnlp/blob/main/docs/source/main_classes/stemmer_morph_analyzer.md

This snippet shows the Sphinx `automodule` directive used to automatically generate API documentation for the `vnlp.stemmer_morph_analyzer.stemmer_morph_analyzer` Python module. The `:members:` option ensures that all public members (classes, functions, attributes) of the module are included in the generated documentation.

```APIDOC
.. automodule:: vnlp.stemmer_morph_analyzer.stemmer_morph_analyzer
    :members:
```

--------------------------------

### API Reference for vnlp.part_of_speech_tagger.spu_context_pos

Source: https://github.com/vngrs-ai/vnlp/blob/main/docs/source/main_classes/part_of_speech_tagger.md

Documents the public members of the SentencePiece Unigram Context Part of Speech Tagger module within the vnlp library, generated via Sphinx automodule directives.

```APIDOC
Module: vnlp.part_of_speech_tagger.spu_context_pos
  Description: Module for SentencePiece Unigram Context Part of Speech Tagging.
  Members: All public classes, functions, and variables are documented.
```

--------------------------------

### SPUContext Part of Speech Tagger Overview and Performance

Source: https://github.com/vngrs-ai/vnlp/blob/main/vnlp/part_of_speech_tagger/ReadMe.md

This section describes the SPUContext PoS tagger, a context-aware model using SentencePiece Unigram tokenizer and pre-trained Word2Vec embeddings. It details its overall accuracy and F1-macro score, along with per-dataset performance metrics on Universal Dependencies 2.9 Turkish datasets and training specifics.

```APIDOC
SPUContext PoS Tagger:
  Description: Context aware Part of Speech tagger.
  Tokenizer: SentencePiece Unigram tokenizer
  Embeddings: Pre-trained Word2Vec embeddings

  Performance (Universal Dependencies 2.9 test sets):
    Overall:
      Accuracy: 0.9010
      F1_macro_score: 0.7623
    Per-Dataset Metrics:
      UD_Turkish-Atis: Accuracy: 0.9874, F1_macro_score: 0.9880
      UD_Turkish-BOUN: Accuracy: 0.8708, F1_macro_score: 0.7884
      UD_Turkish-FrameNet: Accuracy: 0.9509, F1_macro_score: 0.9039
      UD_Turkish-GB: Accuracy: 0.8559, F1_macro_score: 0.6620
      UD_Turkish-IMST: Accuracy: 0.9069, F1_macro_score: 0.7845
      UD_Turkish-Kenet: Accuracy: 0.9194, F1_macro_score: 0.8766
      UD_Turkish-Penn: Accuracy: 0.9452, F1_macro_score: 0.9329
      UD_Turkish-PUD: Accuracy: 0.8387, F1_macro_score: 0.6559
      UD_Turkish-Tourism: Accuracy: 0.9845, F1_macro_score: 0.9325
      UD_Turkish_German-SAGT: Skipped (non-Turkish tokens)

  Training Details:
    Data: All of train, dev, and test data
    Epochs: 50
    Initial Learning Rate: 0.001
    Learning Rate Decay: 0.95 (after 3rd epoch)
```

--------------------------------

### TreeStack Part of Speech Tagger Overview and Performance

Source: https://github.com/vngrs-ai/vnlp/blob/main/vnlp/part_of_speech_tagger/ReadMe.md

This section outlines the TreeStack PoS tagger, which is inspired by 'Tree-stack LSTM in Transition Based Dependency Parsing'. It details its use of morphological tags, pre-trained word embeddings, and POS tags as input, along with its overall accuracy and F1-macro score on Universal Dependencies 2.9 Turkish datasets and training specifics.

```APIDOC
TreeStack PoS Tagger:
  Description: Part of Speech tagger inspired by "Tree-stack LSTM in Transition Based Dependency Parsing".
  Approach: Uses Morphological Tags, Pre-trained word embeddings, and POS tags as input.
  Embeddings: Pre-trained Word2Vec_medium embeddings, Pre-trained Morphological Tag embeddings (from StemmerAnalyzer's neural network model)

  Performance (Universal Dependencies 2.9 test sets):
    Overall:
      Accuracy: 0.89
      F1_macro_score: 0.71
    Per-Dataset Metrics:
      UD_Turkish-Atis: Accuracy: 0.9695, F1_macro_score: 0.8858
      UD_Turkish-BOUN: Accuracy: 0.8543, F1_macro_score: 0.7607
      UD_Turkish-FrameNet: Accuracy: 0.9447, F1_macro_score: 0.8146
      UD_Turkish-GB: Accuracy: 0.8558, F1_macro_score: 0.6274
      UD_Turkish-IMST: Accuracy: 0.9041, F1_macro_score: 0.7987
      UD_Turkish-Kenet: Accuracy: 0.9039, F1_macro_score: 0.8287
      UD_Turkish-Penn: Accuracy: 0.9320, F1_macro_score: 0.7967
      UD_Turkish-PUD: Accuracy: 0.8303, F1_macro_score: 0.6272
      UD_Turkish-Tourism: Accuracy: 0.9799, F1_macro_score: 0.9025
      UD_Turkish_German-SAGT: Skipped (non-Turkish tokens)

  Training Details:
    Data: All of train, dev, and test data
    Epochs: 20
    Initial Learning Rate: 0.001
    Learning Rate Decay: 0.95 (after 5th epoch)
```

--------------------------------

### Turkish Non-Breaking Prefixes Data File

Source: https://github.com/vngrs-ai/vnlp/blob/main/vnlp/resources/non_breaking_prefixes_tr.txt

This plain text file contains a curated list of words and abbreviations that, when followed by a period and an uppercase word, should not be treated as sentence terminators in Turkish text. It is primarily used for improving sentence tokenization accuracy by handling common abbreviations, initials, and Roman numerals. The list is home-made and may require further improvements.

```Plain Text
# This is a non-breaking prefix list for the Turkish language.
# The file is used for sentence tokenization (text -> sentence splitting).
#
# The file is home-made by a programmer (not a linguist) who doesn't even speak Turkish so it surely can be improved.
#

# Anything in this file, followed by a period (and an upper-case word), does NOT
# indicate an end-of-sentence marker.
# Special cases are included for prefixes that ONLY appear before 0-9 numbers.

# Any single upper case letter followed by a period is not a sentence ender
# (excluding I occasionally, but we leave it in).
# Usually upper case letters are initials in a name.
A
B
C
D
E
F
G
H
I
J
K
L
M
N
O
P
Q
R
S
T
U
V
W
X
Y
Z

# Usually upper case letters are initials in a name (Turkish alphabet)
Ç
Ğ
İ
Ö
Ş
Ü

# Roman Numerals
I
II
III
IV
V
VI
VII
VIII
IX
X
XI
XII
XIII
XIV
XV
XVI
XVII
XVIII
XIX
XX

# English -- but these work globally for all languages:
Mr
Mrs
No
pp
St
no
Sr
Jr
Bros
etc
vs
esp
Fig
fig
Jan
Feb
Mar
Apr
Jun
Jul
Aug
Sep
Sept
Oct
Okt
Nov
Dec
Ph.D
PhD
# in "et al."
al
cf
Inc
Ms
Gen
Sen
Prof
Dr
Corp
Co

# http://en.wiktionary.org/wiki/Category:Turkish_abbreviations
Av

# http://en.bab.la/phrases/business/abbreviations/english-turkish/
no
Başk.yard
Bşk.yrd
```

--------------------------------

### Splitting Data for Sentiment Analysis Evaluation

Source: https://github.com/vngrs-ai/vnlp/blob/main/vnlp/sentiment_analyzer/ReadMe.md

Instructions on how to split the merged sentiment analysis dataset into training and test sets using `train_test_split` for evaluation. The dataset should first be read as a DataFrame (`df`).

```Python
train_test_split(df, test_size = 0.10, random_state = 0, shuffle = True, stratify = df.loc[:, 'label'])
```

=== COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.