### Install required libraries

Source: https://github.com/argilla-io/argilla/blob/develop/docs/_source/tutorials/notebooks/labelling-textclassification-gpt3-fewshot.ipynb

Install the necessary dependencies for the tutorial.

```python
%pip install openai datasets argilla -qqq
```

--------------------------------

### Install Dependencies

Source: https://github.com/argilla-io/argilla/blob/develop/docs/_source/practical_guides/collect_responses.md

Commands to install required libraries and spaCy models for the guide.

```bash
%pip install datasets spacy plotly -qqq
```

```bash
!python -m spacy download en_core_web_sm -qqq
```

--------------------------------

### Install Argilla and dependencies

Source: https://github.com/argilla-io/argilla/blob/develop/docs/_source/practical_guides/annotation_workflows/add_text_descriptives_as_metadata.ipynb

Install the necessary packages to run the tutorial.

```python
# %pip install --upgrade pip
%pip install argilla -qqq
%pip install datasets
```

--------------------------------

### Install frontend dependencies

Source: https://github.com/argilla-io/argilla/blob/develop/argilla-frontend/README.md

Run this command to install all required project dependencies.

```bash
npm i
```

--------------------------------

### Install Dependencies

Source: https://github.com/argilla-io/argilla/blob/develop/docs/_source/tutorials_and_integrations/tutorials/feedback/training-setfit-absa.ipynb

Install the required Argilla, SetFit, and spaCy packages.

```python
%pip install argilla setfit[absa] spacy
# %pip install --upgrade huggingface_hub # Uncomment this line if you are not using the latest version of huggingface_hub

!spacy download en_core_web_lg
```

--------------------------------

### Install Unstructured

Source: https://github.com/argilla-io/argilla/blob/develop/docs/_source/tutorials_and_integrations/integrations/process_documents_with_unstructured.ipynb

Install the required version of the unstructured library.

```python
%pip install "unstructured==0.4.4" -qqq
```

--------------------------------

### Install Required Libraries

Source: https://github.com/argilla-io/argilla/blob/develop/docs/_source/tutorials_and_integrations/tutorials/feedback/fine-tuning-sentencesimilarity-rag.ipynb

Install the Argilla client and necessary third-party libraries for the tutorial.

```python
# %pip install --upgrade pip
%pip install argilla -qqq
%pip install datasets
%pip install sentence-transformers
%pip install farm-haystack[colab,faiss,inference]
```

--------------------------------

### Install Docker and Docker Compose

Source: https://github.com/argilla-io/argilla/blob/develop/docs/_source/getting_started/installation/deployments/cloud_providers.md

Sets up the Docker repository and installs the Docker Engine and Compose plugin.

```bash
curl -fsSL https://download.docker.com/linux/debian/gpg | sudo gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg

echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/debian $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null

sudo apt update

sudo apt-get install docker-ce docker-ce-cli containerd.io docker-compose-plugin
```

--------------------------------

### Install dependencies

Source: https://github.com/argilla-io/argilla/blob/develop/docs/_source/tutorials_and_integrations/tutorials/feedback/monitoring-bias-ethics-dpo.ipynb

Install the necessary libraries for Argilla, Giskard, and model training.

```python
# %pip install --upgrade pip
%pip install argilla -qqq

%pip install -q -U torch
%pip install -q -U bitsandbytes
%pip install -q -U transformers
%pip install -q -U accelerate

%pip install -q -U datasets
%pip install -q -U einops

%pip install "giskard[llm]" --upgrade
%pip install "langchain<=0.0.301" "pypdf<=3.17.0" "faiss-cpu<=1.7.4" "openai<=0.28.1" "tiktoken<=0.5.1"
%pip install avidtools
%pip install -q -U peft
%pip install -q -U trl
```

--------------------------------

### Install and Start Elasticsearch

Source: https://github.com/argilla-io/argilla/blob/develop/docs/_source/tutorials/notebooks/deploying-textclassification-colab-activelearning.ipynb

Manually download, extract, and run Elasticsearch as a background process since Docker is not supported in Colab.

```bash
%%bash

wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.10.2-linux-x86_64.tar.gz -q
tar -xzf elasticsearch-7.10.2-linux-x86_64.tar.gz
chown -R daemon:daemon elasticsearch-7.10.2
```

```bash
%%bash --bg

sudo -u daemon -- elasticsearch-7.10.2/bin/elasticsearch
```

--------------------------------

### Install Argilla Chart

Source: https://github.com/argilla-io/argilla/blob/develop/examples/deployments/k8s/argilla-chart/README.md

Installs the Argilla chart and monitors the pod status.

```bash
helm install my-argilla-server examples/deployments/k8s/argilla-chart
```

```bash
kubectl get pods -w
```

--------------------------------

### Install Documentation Dependencies

Source: https://github.com/argilla-io/argilla/blob/develop/docs/_source/community/developer_docs.md

Install the required Python packages for building the documentation.

```sh
pip install -r docs/_source/requirements.txt
```

--------------------------------

### Install dependencies

Source: https://github.com/argilla-io/argilla/blob/develop/docs/_source/tutorials/notebooks/training-textclassification-setfit-fewshot.ipynb

Install the required Argilla, SetFit, and datasets packages.

```python
%pip install argilla "setfit~=0.2.0" "datasets~=2.3.0" -qqq
```

--------------------------------

### Install dependencies

Source: https://github.com/argilla-io/argilla/blob/develop/docs/_source/tutorials/notebooks/training-textclassification-modal-activelearning.ipynb

Install the required packages for Argilla, modAL, and scikit-learn.

```python
%pip install argilla modAL scikit-learn matplotlib -qqq  # matplotlib is optional
```

--------------------------------

### Run development server

Source: https://github.com/argilla-io/argilla/blob/develop/argilla-frontend/README.md

Starts the local development environment for the frontend.

```bash
npm run dev
```

--------------------------------

### Run Argilla Quickstart Container

Source: https://github.com/argilla-io/argilla/blob/develop/docs/_source/getting_started/installation/deployments/docker-quickstart.md

Executes the Argilla quickstart Docker image with default network and port settings.

```bash
  docker run -d --network argilla-net --name quickstart -p 6900:6900 argilla/argilla-quickstart:latest
```

--------------------------------

### Install dependencies

Source: https://github.com/argilla-io/argilla/blob/develop/docs/_source/tutorials_and_integrations/integrations/add_sentence_transformers_embeddings_as_vectors.ipynb

Install the necessary libraries for Argilla and Sentence Transformers.

```python
# %pip install --upgrade pip
%pip install argilla -qqq
%pip install datasets
%pip install sentence-transformers
```

--------------------------------

### Install dependencies

Source: https://github.com/argilla-io/argilla/blob/develop/docs/_source/tutorials_and_integrations/tutorials/feedback/train-reward-model-rlhf.ipynb

Install the required libraries for Argilla, data handling, and model training.

```python
%pip install -U argilla pandas trl plotly -qqq
```

--------------------------------

### Install Dependencies

Source: https://github.com/argilla-io/argilla/blob/develop/docs/_source/tutorials/notebooks/training-textgeneration-unstructured.ipynb

Install the required Argilla, unstructured, and machine learning libraries.

```python
%pip install argilla
%pip install "unstructured==0.4.4" -qqq
%pip install transformers datasets
```

--------------------------------

### Install Argilla and dependencies

Source: https://github.com/argilla-io/argilla/blob/develop/docs/_source/tutorials_and_integrations/tutorials/feedback/labelling-feedback-langchain-syntethic.ipynb

Install the necessary client libraries for Argilla, OpenAI, LangChain, and other utilities.

```bash
!pip install argilla openai langchain outlines tiktoken transformers ipywidgets jupyter
```

--------------------------------

### Deploy Argilla with Docker

Source: https://github.com/argilla-io/argilla/blob/develop/docs/_source/_common/tabs/argilla_install.md

Run the Argilla quickstart container using the Docker CLI.

```bash
docker run -d --name argilla -p 6900:6900 argilla/argilla-quickstart:latest
```

--------------------------------

### Install Dependencies

Source: https://github.com/argilla-io/argilla/blob/develop/argilla/docs/tutorials/text_classification.ipynb

Install SetFit and Transformers libraries required for model training.

```python
!pip install setfit==1.0.3 transformers==4.40.2
```

--------------------------------

### Install dependencies

Source: https://github.com/argilla-io/argilla/blob/develop/docs/_source/tutorials/notebooks/labelling-tokenclassification-skweak-weaksupervision.ipynb

Install required libraries including Argilla, datasets, spaCy, and skweak.

```python
%pip install argilla datasets spacy -qqq
%pip install skweak
!python -m spacy download en_core_web_md
```

--------------------------------

### Install dependencies

Source: https://github.com/argilla-io/argilla/blob/develop/docs/_source/tutorials/notebooks/labelling-textclassification-snorkel-weaksupervision.ipynb

Install the required Argilla, Snorkel, datasets, and sklearn libraries.

```python
%pip install argilla snorkel datasets sklearn -qqq
```

--------------------------------

### Install dependencies

Source: https://github.com/argilla-io/argilla/blob/develop/docs/_source/tutorials/notebooks/monitoring-textclassification-transformers-explainability.ipynb

Install the required packages for Argilla, transformers, datasets, and torch.

```python
%pip install argilla transformers datasets torch -qqq
```

--------------------------------

### Install dependencies

Source: https://github.com/argilla-io/argilla/blob/develop/docs/_source/tutorials/notebooks/training-textclassification-classyclassification-activelearning.ipynb

Install the required packages for classy-classification, Argilla listeners, and datasets.

```python
%pip install "classy-classification[onnx]==0.6.0" -qqq
%pip install "argilla[listeners]>=1.1.0" -qqq
%pip install datasets -qqq
```

--------------------------------

### Install dependencies

Source: https://github.com/argilla-io/argilla/blob/develop/docs/_source/tutorials/notebooks/training-textclassification-smalltext-activelearning.ipynb

Install the required packages including argilla, datasets, small-text, and transformers.

```python
%pip install "argilla[listeners]~=1.16.0" "datasets~=2.5.0" "small-text~=1.3.2"  "transformers[torch]"
```

--------------------------------

### Install Argilla Client

Source: https://github.com/argilla-io/argilla/blob/develop/docs/_source/tutorials/notebooks/labelling-tokenclassification-deletelabels.ipynb

Install the required Argilla library using pip.

```python
%pip install --upgrade argilla -qqq
```

--------------------------------

### Install dependencies

Source: https://github.com/argilla-io/argilla/blob/develop/docs/_source/tutorials/notebooks/labelling-text2text-disaggregators-explainability.ipynb

Install the Argilla client, disaggregators library, and the required spaCy model.

```python
%pip install argilla disaggregators -qqq
!python -m spacy download en_core_web_lg -qqq
```

--------------------------------

### Install Argilla and dependencies

Source: https://github.com/argilla-io/argilla/blob/develop/docs/_source/getting_started/quickstart_workflow_feedback.ipynb

Install the required libraries for Argilla, datasets, and evaluation tools.

```bash
!pip install argilla datasets setfit evaluate seqeval
```

--------------------------------

### Start Argilla Server with Docker

Source: https://github.com/argilla-io/argilla/blob/develop/examples/webhooks/basic-webhooks/README.md

Commands to create a directory, download the docker-compose configuration, and start the Argilla server in detached mode.

```bash
mkdir argilla && cd argilla
curl https://raw.githubusercontent.com/argilla-io/argilla/main/examples/deployments/docker/docker-compose.yaml -o docker-compose.yaml
docker compose up -d
```

--------------------------------

### Install required libraries

Source: https://github.com/argilla-io/argilla/blob/develop/docs/_source/tutorials/notebooks/labelling-spacy-llm.ipynb

Install the necessary dependencies for spaCy LLM, transformers, and Argilla.

```bash
pip install "spacy-llm[transformers]" "transformers[sentencepiece]" argilla datasets -qqq
```

--------------------------------

### Install Required Packages

Source: https://github.com/argilla-io/argilla/blob/develop/docs/_source/tutorials/notebooks/deploying-textclassification-colab-activelearning.ipynb

Install Argilla, Transformers, datasets, small-text, and Colab-specific utilities.

```python
%pip install "argilla[server, listeners]==1.16.0"
%pip install "transformers[sentencepiece]~=4.25.1"
%pip install "datasets~=2.7.1"
%pip install "small-text[transformers]~=1.3.2"
%pip install "colab-xterm~=0.1.2"
%pip install "pyngrok~=5.2.1"
%pip install "colab-xterm~=0.1.2"
```

--------------------------------

### Install required packages

Source: https://github.com/argilla-io/argilla/blob/develop/argilla/docs/community/integrations/llamaindex_rag_github.ipynb

Install the Argilla LlamaIndex integration and the GitHub reader library.

```bash
!pip install "argilla-llama-index"
!pip install "llama-index-readers-github==0.1.9"
```

--------------------------------

### Install and Run Helm Unit Tests

Source: https://github.com/argilla-io/argilla/blob/develop/examples/deployments/k8s/argilla-chart/README.md

Installs the helm-unittest plugin and executes tests for the chart.

```bash
helm plugin install https://github.com/helm-unittest/helm-unittest.git
```

```bash
helm unittest --help
```

```bash
helm unittest examples/deployments/k8s/argilla-chart
```

--------------------------------

### Tokenize Input Example

Source: https://github.com/argilla-io/argilla/blob/develop/docs/_source/tutorials/notebooks/ner_fine_tune_bert_beginners.ipynb

Demonstrate tokenization of raw text and conversion to token IDs.

```python
# Example of original tokens
example = dataset["train"][0]
print(example["tokens"])

# Example after executing the AutoTokenizer
tokenized_input = tokenizer(example["tokens"], is_split_into_words=True)
tokens = tokenizer.convert_ids_to_tokens(tokenized_input["input_ids"])
print(tokens)
```

--------------------------------

### Generic Fine-tuning Data Example

Source: https://github.com/argilla-io/argilla/blob/develop/docs/_source/practical_guides/fine_tune.md

Example of raw text data used for domain-specific fine-tuning.

```bash
# Five distinct ester hydrolases (EC 3-1) have been characterized in guinea-pig epidermis. These are carboxylic esterase, acid phosphatase, pyrophosphatase, and arylsulphatase A and B. Their properties are consistent with those of lysosomal enzymes.
```

--------------------------------

### Autobuild Server Output Example

Source: https://github.com/argilla-io/argilla/blob/develop/docs/_source/community/developer_docs.md

Sample console output showing the server startup and shutdown sequence.

```sh
The HTML pages are in docs\_build\html.
[I 231024 10:58:36 server:335] Serving on http://127.0.0.1:8000
[I 231024 10:58:36 handlers:62] Start watching changes
[I 231024 10:58:36 handlers:64] Start detecting changes
[I 231024 11:00:53 server:358] Shutting down...
```

--------------------------------

### Initialize ArgillaTrainer

Source: https://github.com/argilla-io/argilla/blob/develop/docs/_source/getting_started/cheatsheet.md

Basic setup for the ArgillaTrainer to prepare data and initiate training workflows.

```python
from argilla.training import ArgillaTrainer

trainer = ArgillaTrainer(
    name="my_dataset",
    workspace="my_workspace",
    framework="my_framework",
    model="my_framework_model",
    train_size=0.8,
    seed=42,
    limit=10,
    query="my-query"
)
trainer.update_config() # see usage below
trainer.train()
records = trainer.predict(["my-text"], as_argilla_records=True)
```

--------------------------------

### Install Dependencies

Source: https://github.com/argilla-io/argilla/blob/develop/docs/_source/tutorials_and_integrations/tutorials/feedback/end2end_examples/assign-records-002.ipynb

Install the required argilla and datasets packages.

```python
!pip install argilla
!pip install datasets
```

--------------------------------

### Start local documentation server

Source: https://github.com/argilla-io/argilla/blob/develop/argilla/docs/community/developer.md

Run the MkDocs development server to preview documentation changes locally.

```sh
mkdocs serve
```

--------------------------------

### Connect via CLI and rg.init()

Source: https://github.com/argilla-io/argilla/blob/develop/docs/_source/_common/tabs/argilla_connect.md

Use the CLI to login and store credentials, then initialize the client without arguments.

```sh
argilla login --api-url http://localhost:6900 --api-key argilla.apikey
```

```python
import argilla as rg

rg.init()
```

--------------------------------

### Install Argilla SDK

Source: https://github.com/argilla-io/argilla/blob/develop/argilla/docs/how_to_guides/migrate_from_legacy_datasets.md

Install the required Argilla package version to support migration to 2.x.

```bash
pip install "argilla>=2.0.0"
```

--------------------------------

### Install Weasel

Source: https://github.com/argilla-io/argilla/blob/develop/docs/_source/practical_guides/annotation_workflows/weak_supervision.ipynb

Install the Weasel package via pip.

```python
!python -m pip install git+https://github.com/autonlab/weasel#egg=weasel[all]
```

--------------------------------

### Launch FastAPI application

Source: https://github.com/argilla-io/argilla/blob/develop/docs/_source/tutorials/notebooks/deploying-texttokenclassification-fastapi.ipynb

Command to start the Uvicorn server for the main application file.

```bash
!nohup uvicorn main:app
```

--------------------------------

### Install FlyingSquid dependencies

Source: https://github.com/argilla-io/argilla/blob/develop/docs/_source/practical_guides/annotation_workflows/weak_supervision.ipynb

Install the required packages for FlyingSquid.

```python
%pip install flyingsquid pgmpy -qqq
```

--------------------------------

### Initialize and Run Trainer

Source: https://github.com/argilla-io/argilla/blob/develop/docs/_source/tutorials/notebooks/labelling-textclassification-sklearn-weaksupervision.ipynb

Set up the Trainer instance and execute the training process.

```python
from transformers import Trainer

# Init the trainer
trainer = Trainer(
    model=model,
    args=training_args,
    train_dataset=binarized_tokenized_ds["train"],
    eval_dataset=binarized_tokenized_ds["test"],
    compute_metrics=compute_metrics,
)
```

```python
# Train the down-stream model
trainer.train()
```

--------------------------------

### Install Snorkel

Source: https://github.com/argilla-io/argilla/blob/develop/docs/_source/practical_guides/annotation_workflows/weak_supervision.ipynb

Install the Snorkel library using pip.

```python
%pip install snorkel -qqq
```

--------------------------------

### Initialize and Run DPOTrainer

Source: https://github.com/argilla-io/argilla/blob/develop/docs/_source/tutorials_and_integrations/tutorials/feedback/monitoring-bias-ethics-dpo.ipynb

Initializes the DPOTrainer with the model, configuration, and dataset, then executes the training and saves the resulting model.

```python
dpo_trainer = DPOTrainer(
    model,
    model_ref,
    args=training_arguments,
    beta=0.1,
    peft_config=peft_config,
    train_dataset=formatted_dataset,
    tokenizer=tokenizer,
    max_length=512,
    max_prompt_length=128,
    padding_value=tokenizer.pad_token_id,
)

dpo_trainer.train()
dpo_trainer.save_model()
```

--------------------------------

### Install Sentence Transformers

Source: https://github.com/argilla-io/argilla/blob/develop/argilla/docs/tutorials/image_preference.ipynb

Install the sentence-transformers library for embedding generation.

```python
!pip install "sentence-transformers~=3.0"
```

--------------------------------

### Run Development Server

Source: https://github.com/argilla-io/argilla/blob/develop/argilla-server/README.md

Command to migrate the database, create default users, and launch the server locally.

```sh
pdm server-dev
```

--------------------------------

### Initialize and run the trainer

Source: https://github.com/argilla-io/argilla/blob/develop/docs/_source/tutorials_and_integrations/tutorials/feedback/training-setfit-absa.ipynb

Sets up the AbsaTrainer with the model, arguments, and datasets, then executes the training process.

```python
# Initialize the trainer
trainer = AbsaTrainer(
    model,
    args=args,
    train_dataset=train_dataset,
    eval_dataset=eval_dataset,
)
trainer.train()
```

--------------------------------

### Define and Create Dataset

Source: https://github.com/argilla-io/argilla/blob/develop/README.md

Configure dataset settings with fields and questions, then instantiate and create the dataset on the server.

```python
settings = rg.Settings(
    guidelines="Classify the reviews as positive or negative.",
    fields=[
        rg.TextField(
            name="review",
            title="Text from the review",
            use_markdown=False,
        ),
    ],
    questions=[
        rg.LabelQuestion(
            name="my_label",
            title="In which category does this article fit?",
            labels=["positive", "negative"],
        )
    ],
)
dataset = rg.Dataset(
    name=f"my_first_dataset",
    settings=settings,
    client=client,
)
dataset.create()
```

--------------------------------

### Install dependencies

Source: https://github.com/argilla-io/argilla/blob/develop/docs/_source/tutorials/notebooks/training-textclassification-transformers-pretrained.ipynb

Install the required Python packages for Argilla, Transformers, and datasets.

```python
%pip install argilla "transformers[torch]" datasets sklearn ipywidgets -qqq
```

--------------------------------

### Start Webhook Server

Source: https://github.com/argilla-io/argilla/blob/develop/examples/webhooks/basic-webhooks/README.md

Command to launch the webhook server with required environment variables for API key and server URL.

```bash
ARGILLA_API_KEY=argilla.apikey \
WEBHOOK_SERVER_URL=http://host.docker.internal:8000 \
uvicorn main:server
```

--------------------------------

### Create a new dataset with settings

Source: https://github.com/argilla-io/argilla/blob/develop/argilla/docs/how_to_guides/dataset.md

Initializes a dataset with specific fields and questions, then pushes it to the Argilla server.

```python
import argilla as rg

client = rg.Argilla(api_url="<api_url>", api_key="<api_key>")

settings = rg.Settings(
    guidelines="These are some guidelines.",
    fields=[
        rg.TextField(
            name="text",
        ),
    ],
    questions=[
        rg.LabelQuestion(
            name="label",
            labels=["label_1", "label_2", "label_3"]
        ),
    ],
)

dataset = rg.Dataset(
    name="my_dataset",
    workspace="my_workspace",
    settings=settings,
)

dataset.create()
```

--------------------------------

### Initialize GitHub Client

Source: https://github.com/argilla-io/argilla/blob/develop/argilla/docs/community/integrations/llamaindex_rag_github.ipynb

Sets up the GitHub client with an authentication token for repository access.

```python
github_client = GithubClient(github_token=github_token, verbose=True)
```

--------------------------------

### Initialize Document Store and Indexing Pipeline

Source: https://github.com/argilla-io/argilla/blob/develop/docs/_source/tutorials_and_integrations/tutorials/feedback/fine-tuning-sentencesimilarity-rag.ipynb

Sets up the FAISS document store, preprocessor, and text converter to index documents for retrieval.

```python
# Initialize the DocumentStore
document_store = FAISSDocumentStore(faiss_index_factory_str="Flat", similarity="dot_product", embedding_dim=384)

# Initialize the PreProcessor
preprocessor = PreProcessor(
    clean_empty_lines=True,
    clean_whitespace=True,
    clean_header_footer=False,
    split_by="word",
    split_length=100,
    split_respect_sentence_boundary=True,
)

# Initialize the TextConverter
text_converter = TextConverter()

# Run the TextIndexingPipeline
pipeline = TextIndexingPipeline(document_store, text_converter, preprocessor)
result = pipeline.run(file_path="argilla_cloud.txt")
```

--------------------------------

### Run Documentation Autobuild Server

Source: https://github.com/argilla-io/argilla/blob/develop/docs/_source/community/developer_docs.md

Command to start the Sphinx autobuild server for live documentation preview.

```sh
sphinx-autobuild docs/_source docs/_build/html
```

--------------------------------

### Initialize and Run ArgillaTrainer

Source: https://github.com/argilla-io/argilla/blob/develop/docs/_source/practical_guides/fine_tune.md

Sets up the ArgillaTrainer with the dataset and task, then executes the training process.

```python
from argilla.feedback import ArgillaTrainer

trainer = ArgillaTrainer(
    dataset=feedback_dataset,
    task=task,
    framework="trl",
    model="gpt2",
)
trainer.train(output_dir="dpo_model")
```

--------------------------------

### Initialize ArgillaTrainer

Source: https://github.com/argilla-io/argilla/blob/develop/docs/_source/getting_started/quickstart_workflow_feedback.ipynb

Set up the trainer with the dataset, task, and framework.

```python
from argilla.feedback import ArgillaTrainer

trainer = ArgillaTrainer(
    dataset=dataset_hf,
    task=task,
    framework="setfit",
    train_size=0.8
)
```

--------------------------------

### Install Third-Party Libraries

Source: https://github.com/argilla-io/argilla/blob/develop/argilla/docs/tutorials/token_classification.ipynb

Install required dependencies for GLiNER, transformers, and SpanMarker.

```python
!pip install gliner==0.2.6 transformers==4.40.2 span_marker==1.5.0
```

--------------------------------

### Fine-tuning with ArgillaTrainer

Source: https://github.com/argilla-io/argilla/blob/develop/docs/_source/practical_guides/fine_tune.md

Demonstrates the standard workflow for initializing a dataset, defining a training task, and executing the training process.

```python
from argilla.feedback import ArgillaTrainer, FeedbackDataset, TrainingTask

dataset = FeedbackDataset.from_huggingface(
    repo_id="argilla/emotion"
)
task = TrainingTask.for_text_classification(
    text=dataset.field_by_name("text"),
    label=dataset.question_by_name("label"),
)
trainer = ArgillaTrainer(
    dataset=dataset,
    task=task,
    framework="setfit"
)
trainer.update_config(num_iterations=1)
trainer.train(output_dir="my_setfit_model")
trainer.predict("This is awesome!")
```

--------------------------------

### Install Argilla SDK

Source: https://github.com/argilla-io/argilla/blob/develop/README.md

Install the Argilla Python package using pip.

```console
pip install argilla
```

--------------------------------

### Execute Inference Example

Source: https://github.com/argilla-io/argilla/blob/develop/docs/_source/practical_guides/fine_tune.md

Example call to the generate function and its resulting output.

```python
>>> generate("sft_model", "Is a toad a frog?")
### Instruction: Is a toad a frog?

### Context:

### Response: A frog is a small, round, black-eyed, frog with a long, black-winged head. It is a member of the family Pter
```

--------------------------------

### Launch Opensearch with Docker Compose

Source: https://github.com/argilla-io/argilla/blob/develop/docs/_source/practical_guides/annotation_workflows/semantic_search.md

Starts a clean instance of Opensearch using the provided configuration file.

```bash
docker-compose -f docker-compose.opensearch.yaml up
```

--------------------------------

### Initialize Argilla client and imports

Source: https://github.com/argilla-io/argilla/blob/develop/docs/_source/tutorials_and_integrations/integrations/llama_index.ipynb

Import required LlamaIndex and Argilla modules and initialize the Argilla client.

```python
from llama_index.core import VectorStoreIndex, ServiceContext, SimpleDirectoryReader, set_global_handler
from llama_index.llms.openai import OpenAI

import argilla as rg

rg.init(
    api_url="http://localhost:6900",
    api_key="owner.apikey",
    workspace="admin"
)
```

--------------------------------

### Initialize ArgillaTrainer

Source: https://github.com/argilla-io/argilla/blob/develop/docs/_source/tutorials_and_integrations/tutorials/feedback/end2end_examples/train-model-006.ipynb

Set up the trainer using the spacy framework and the defined task.

```python
trainer = ArgillaTrainer(
    dataset=dataset,
    task=task,
    framework="spacy",
    train_size=0.8,
)
```

--------------------------------

### Initialize the Argilla Client

Source: https://github.com/argilla-io/argilla/blob/develop/docs/_source/reference/python/python_client.md

Demonstrates how to initialize the client with specific API credentials and how to pass custom HTTP headers for requests.

```pycon
>>> import argilla as rg
>>>
>>> rg.init(api_url="http://localhost:9090", api_key="4AkeAPIk3Y")
>>> # Customizing request headers
>>> headers = {"X-Client-id":"id","X-Secret":"secret"}
>>> rg.init(api_url="http://localhost:9090", api_key="4AkeAPIk3Y", extra_headers=headers)
```

--------------------------------

### Install DVC

Source: https://github.com/argilla-io/argilla/blob/develop/docs/_source/tutorials/notebooks/deploying-text2text-dvc-explainability.ipynb

Install DVC using the package manager appropriate for your operating system.

```bash
!brew install dvc # mac
# !snap install --classic dvc # linux
# !choco install dvc # windows
```

--------------------------------

### Install dependencies

Source: https://github.com/argilla-io/argilla/blob/develop/docs/_source/tutorials/notebooks/labelling-textclassification-setfit-zeroshot.ipynb

Install the required Python packages for Argilla, datasets, sentence-transformers, and SetFit.

```python
%pip install argilla datasets==2.8.0 sentence-transformers==2.2.2 setfit==0.6.0 plotly==4.1.0 -qqq
```

--------------------------------

### Create a new dataset with settings

Source: https://github.com/argilla-io/argilla/blob/develop/argilla/docs/reference/argilla/settings/settings.md

Instantiate the Settings class and pass it to the Dataset constructor before calling create().

```python
import argilla as rg

settings = rg.Settings(
    guidelines="Select the sentiment of the prompt.",
    fields=[rg.TextField(name="prompt", use_markdown=True)],
    questions=[rg.LabelQuestion(name="sentiment", labels=["positive", "negative"])],
)

dataset = rg.Dataset(name="sentiment_analysis", settings=settings)

# Create the dataset on the server
dataset.create()
```

--------------------------------

### Configure Keycloak Realm and Client

Source: https://github.com/argilla-io/argilla/blob/develop/argilla/docs/reference/argilla-server/sso_keycloak.md

Automates the setup of a Keycloak environment including realm creation, client configuration, and necessary mappers for token claims.

```python
from keycloak import KeycloakAdmin
from keycloak import KeycloakOpenIDConnection
from keycloak import KeycloakOpenID

ARGILLA_CLIENT_ID = "argilla-client"
ARGILLA_REALM = "argilla"

keycloak_connection = KeycloakOpenIDConnection(
    server_url="http://localhost:8080/",
    username="admin",
    password="admin",
    realm_name="master",
    client_id="admin-cli",
)

keycloak_admin = KeycloakAdmin(connection=keycloak_connection)

keycloak_admin.create_realm(
    {
        "realm": ARGILLA_REALM,
        "enabled": True,
        "displayName": "Argilla",
        "userManagedAccessAllowed": True,
    }
)
keycloak_connection = KeycloakOpenIDConnection(
    server_url="http://localhost:8080/",
    username="admin",
    password="admin",
    user_realm_name="master",
    realm_name=ARGILLA_REALM,
)

keycloak_admin = KeycloakAdmin(connection=keycloak_connection)

client = keycloak_admin.create_client(
    {
        "clientId": ARGILLA_CLIENT_ID,  # The client ID (you can choose a name)
        "enabled": True,
        "protocol": "openid-connect",  # Protocol (you can use other protocols like 'saml' if needed)
        "publicClient": False,  # Set to False if the client will use client secrets
        "directAccessGrantsEnabled": True,
        "standardFlowEnabled": True,
        "frontchannelLogout": True,
        "secret": "client-secret",  # Set a secret if it's not a public client
        "redirectUris": [
            "http://localhost:3000/*",
            "http://localhost:6900/*",
        ],  # Redirect URIs after authentication
    }
)

keycloak_openid = KeycloakOpenID(server_url="http://localhost:8080/",
                                 client_id=ARGILLA_CLIENT_ID,
                                 realm_name=ARGILLA_REALM)

public_key = keycloak_openid.public_key()

client_scope = keycloak_admin.create_client_scope({
    "name": "argilla_client_scope",
    "protocol": "openid-connect",
    "include.in.token.scope": "true"
})

# Create Audience Mapper
audience_mapper = keycloak_admin.add_mapper_to_client_scope(
    client_scope_id=client_scope,
    payload={
    "name": "Client Audience",
    "protocol": "openid-connect",
    "protocolMapper": "oidc-audience-mapper",
    "consentRequired": False,
    "config": {
        "included.client.audience": ARGILLA_CLIENT_ID,
        "id.token.claim": "false",
        "access.token.claim": "true"
    }
})
# Create Realm Roles Mapper
roles_mapper = keycloak_admin.add_mapper_to_client_scope(
    client_scope_id=client_scope,
    payload={
        "name": "Realm Role Mapper",
        "protocol": "openid-connect",
        "protocolMapper": "oidc-usermodel-realm-role-mapper",
        "consentRequired": False,
        "config": {
            "multivalued": "true",  # Indicates if the claim should support multiple values
            "user.attribute": "",
            "token.claim.name": "realm_access.roles",  # The claim in the token
            "claim.name": "realm_access.roles",  # The mapped claim name
            "jsonType.label": "String",  # Type of the claim
            "id.token.claim": "true",  # Include in ID Token
            "access.token.claim": "true",  # Include in Access Token
            "userinfo.token.claim": "true"  # Include in User Info
        }
    }
)
# Create Group Membership Mapper
group_mapper = keycloak_admin.add_mapper_to_client_scope(
    client_scope_id=client_scope,
    payload={
        "name": "Group Membership Mapper",
        "protocol": "openid-connect",
        "protocolMapper": "oidc-group-membership-mapper",
        "consentRequired": False,
        "config": {
            "multivalued": "true",  # The claim should support multiple group memberships
            "token.claim.name": "groups",  # The claim name in the token
            "full.path": "true",  # Include full path of groups (e.g., /parent-group/child-group)
            "id.token.claim": "true",  # Include group membership in ID Token
            "access.token.claim": "true",  # Include group membership in Access Token
            "userinfo.token.claim": "true"  # Include group membership in User Info
        }
    }
)

keycloak_admin.add_default_default_client_scope(client_scope)

new_user = keycloak_admin.create_user(
    {
        "email": "example@example.com",
        "username": "example",
        "enabled": True,
        "firstName": "Example",
        "lastName": "User",
        "credentials": [
            {
                "value": "secret",
                "type": "password",
            }
        ],
    }
)
```

--------------------------------

### Install System Dependencies

Source: https://github.com/argilla-io/argilla/blob/develop/docs/_source/getting_started/installation/deployments/cloud_providers.md

Updates the package manager and installs required system utilities.

```bash
sudo apt-get update

sudo apt-get install apt-transport-https ca-certificates curl software-properties-common gnupg2 lsb-release
```

--------------------------------

### Argilla initialization output

Source: https://github.com/argilla-io/argilla/blob/develop/docs/_source/getting_started/quickstart_workflow_feedback.ipynb

Example output when initializing the Argilla client without a specific workspace.

```text
Output:
C:\Users\sarah\Documents\argilla\src\argilla\client\client.py:154: UserWarning: Default user was detected and no workspace configuration was provided, so the default 'argilla' workspace will be used. If you want to setup another workspace, use the `rg.set_workspace` function or provide a different one on `rg.init`
  warnings.warn(
```

--------------------------------

### Install Argilla Listeners dependencies

Source: https://github.com/argilla-io/argilla/blob/develop/docs/_source/practical_guides/annotation_workflows/job_scheduling.md

Install the necessary dependencies to enable listener functionality.

```python
%pip install argilla[listeners] -qqq
```

--------------------------------

### Initialize trainer

Source: https://github.com/argilla-io/argilla/blob/develop/docs/_source/tutorials/notebooks/training-textgeneration-unstructured.ipynb

Sets up the Seq2SeqTrainer with the model, arguments, and datasets.

```python
trainer = Seq2SeqTrainer(
    model,
    args,
    train_dataset=tokenized_datasets,
    eval_dataset=tokenized_datasets,
    data_collator=data_collator,
    tokenizer=tokenizer,
)
```

--------------------------------

### Install dependencies

Source: https://github.com/argilla-io/argilla/blob/develop/argilla/docs/community/developer.md

Install the PDM package manager and the project dependencies in editable mode.

```sh
# Install the `pdm` package manager
pip install pdm

# Install argilla in editable mode and the development dependencies
pdm install --dev
```

--------------------------------

### Create and initialize the dataset

Source: https://github.com/argilla-io/argilla/blob/develop/argilla/docs/tutorials/text_classification.ipynb

Instantiate the dataset object with the defined settings and persist it to the server.

```python
dataset = rg.Dataset(
    name="text_classification_dataset",
    settings=settings,
)
dataset.create()
```

--------------------------------

### Initialize Helm

Source: https://github.com/argilla-io/argilla/blob/develop/examples/deployments/k8s/argilla-chart/README.md

Initializes Helm if it is being used for the first time.

```bash
helm init
```

--------------------------------

### Single-label Text Classification Example

Source: https://github.com/argilla-io/argilla/blob/develop/docs/_source/practical_guides/fine_tune.md

Example of a single-label text classification data entry.

```batch
"The help for my application of a new card and mortgage was great", "positive"
```

--------------------------------

### Configure Minikube

Source: https://github.com/argilla-io/argilla/blob/develop/examples/deployments/k8s/argilla-chart/README.md

Sets up Minikube resources for local deployment.

```bash
minikube stop
minikube delete
minikube config set memory 3g
minikube config set cpus 2
minikube start
```

--------------------------------

### Install and Verify ECK Operator

Source: https://github.com/argilla-io/argilla/blob/develop/examples/deployments/k8s/argilla-chart/README.md

Installs the Elastic Cloud on Kubernetes operator and checks its status.

```bash
helm install elastic-operator elastic/eck-operator -n elastic-system --create-namespace
```

```bash
kubectl get pods -n elastic-system
```

--------------------------------

### Create GCP Compute Instance

Source: https://github.com/argilla-io/argilla/blob/develop/docs/_source/getting_started/installation/deployments/cloud_providers.md

Provisions a new virtual machine instance for hosting Argilla.

```bash
gcloud compute instances create "argilla-instance" \
  --machine-type "n1-standard-2" \
  --image-family "debian-10" \
  --image-project "debian-cloud" \
  --boot-disk-size "50GB" \
  --zone "asia-south2-a"
```

--------------------------------

### Install Argilla LlamaIndex integration

Source: https://github.com/argilla-io/argilla/blob/develop/docs/_source/tutorials_and_integrations/integrations/llama_index.ipynb

Install the necessary package to enable Argilla integration with LlamaIndex.

```python
%pip install argilla-llama-index
```

--------------------------------

### Install UMAP dependencies

Source: https://github.com/argilla-io/argilla/blob/develop/docs/_source/tutorials/notebooks/labelling-textclassification-sentencetransformers-weaksupervision.ipynb

Ensure the correct UMAP library is installed for dimensionality reduction tasks.

```python
%pip uninstall umap 
%pip install umap-learn
```

--------------------------------

### Initialize Proximal Policy Optimization Dataset

Source: https://github.com/argilla-io/argilla/blob/develop/docs/_source/_common/tabs/task_templates.md

Creates a dataset structure for PPO tasks including a rating scale question.

```python
import argilla as rg

ds = rg.FeedbackDataset.for_proximal_policy_optimization(
    rating_scale=7,
    context=True,
    use_markdown=True,
    guidelines=None,
    metadata_properties=None,
    vectors_settings=None,
)
ds
# FeedbackDataset(
#     fields=[
#         TextField(name="prompt", use_markdown=True),
#         TextField(name="context", use_markdown=True)
#     ],
#     questions=[
#         TextQuestion(name="response", use_markdown=True)
#     ],
#     guidelines="<Guidelines for the task>",
#     metadata_properties="<Metadata Properties>",
#     vectors_settings="<Vectors Settings>",
# )
```

--------------------------------

### Install dependencies for Argilla and spaCy

Source: https://github.com/argilla-io/argilla/blob/develop/docs/_source/tutorials/notebooks/labelling-tokenclassification-spacy-pretrained.ipynb

Install the necessary libraries and download the required spaCy models.

```python
%pip install argilla -qqq
%pip install torch -qqq
%pip install datasets "spacy[transformers]~=3.0" protobuf -qqq 
!python -m spacy download en_core_web_trf
!python -m spacy download en_core_web_sm
```