### Python Virtual Environment Setup for Project

Source: https://github.com/arslanmit/realworld-nlp-techniques/blob/main/requirements.txt

Commands to create and activate a Python virtual environment and install project dependencies from `requirements.txt`.

```Shell
python -m venv /Users/umitarslan/realworld-nlp-techniques/.venv
source /Users/umitarslan/realworld-nlp-techniques/.venv/bin/activate
pip install -r requirements.txt
```

--------------------------------

### Apple Silicon M3 Specific Optimizations and Setup

Source: https://github.com/arslanmit/realworld-nlp-techniques/blob/main/requirements.txt

Python libraries and installation commands specifically for optimizing performance on Apple M3 chips, including Numba for JIT compilation and `llama-cpp-python` with Metal support.

```Python
numba>=0.58.0,<0.59.0
```

```Shell
CMAKE_ARGS="-DLLAMA_METAL=on" FORCE_CMAKE=1 pip install llama-cpp-python
```

--------------------------------

### Install macOS System Dependencies with Homebrew

Source: https://github.com/arslanmit/realworld-nlp-techniques/blob/main/requirements.txt

Command to install essential system libraries like CMake and pkg-config on macOS using Homebrew, required for certain Python packages like `sentencepiece`.

```Shell
brew install cmake pkg-config
```

--------------------------------

### Verify PyTorch MPS and NumPy Accelerate Setup on macOS

Source: https://github.com/arslanmit/realworld-nlp-techniques/blob/main/requirements.txt

A command-line snippet to verify if PyTorch's MPS (Metal Performance Shaders) is available and if NumPy is configured to use the Accelerate framework for optimal performance on macOS. This helps confirm the environment is correctly set up for accelerated machine learning.

```Shell
python -c "import torch; print(f'MPS available: {torch.backends.mps.is_available()}'); import numpy; print(f'NumPy using accelerate: {numpy.__config__.get_info(\"lapack_opt_info\")}')"
```

--------------------------------

### Python Machine Learning and NLP Libraries (M3 Optimized)

Source: https://github.com/arslanmit/realworld-nlp-techniques/blob/main/requirements.txt

Collection of Python libraries for machine learning and natural language processing, specifically optimized for Apple M-series chips, including scikit-learn, PyTorch, Hugging Face transformers, spaCy, and NLTK. Includes special installation notes for gensim and PyTorch.

```Python
scikit-learn>=1.3.2,<2.0.0
torch>=2.2.0
torchvision>=0.17.0
torchaudio>=2.2.0
transformers>=4.37.0
sentence-transformers>=2.2.2
spacy>=3.7.2,<4.0.0
nltk>=3.8.1
imbalanced-learn>=0.12.0,<0.13.0
# gensim>=4.3.2 is not yet compatible with Python 3.13
# Use pre-built wheels when available or install from GitHub:
# pip install git+https://github.com/piskvorky/gensim.git
# Using PyTorch builds optimized for Apple Silicon
--find-links https://download.pytorch.org/whl/nightly/cpu
sentencepiece>=0.1.99,<0.2.0
protobuf>=4.25.0,<5.0.0
```

--------------------------------

### Python Utility Libraries

Source: https://github.com/arslanmit/realworld-nlp-techniques/blob/main/requirements.txt

General utility Python packages for common programming tasks such as progress bars, parallel processing, CLI creation, configuration file handling, and flowchart generation.

```Python
tqdm>=4.66.0,<5.0.0
joblib>=1.3.0,<2.0.0
typer>=0.9.0,<0.10.0
pyyaml>=6.0.0,<7.0.0
graphviz>=0.20.1,<0.21.0
```

--------------------------------

### Python Development and Testing Libraries

Source: https://github.com/arslanmit/realworld-nlp-techniques/blob/main/requirements.txt

Python packages for development workflow, including testing, code formatting, import sorting, linting, and static type checking.

```Python
pytest>=7.4.0,<8.0.0
black>=23.10.0,<24.0.0
isort>=5.12.0,<6.0.0
flake8>=6.1.0,<7.0.0
mypy>=1.6.0,<2.0.0
```

--------------------------------

### Python Data Visualization Libraries

Source: https://github.com/arslanmit/realworld-nlp-techniques/blob/main/requirements.txt

Python packages for creating various types of data visualizations, including matplotlib, seaborn, plotly, wordcloud, and pillow.

```Python
matplotlib>=3.8.0,<4.0.0
seaborn>=0.13.0,<0.14.0
plotly>=5.18.0,<6.0.0
wordcloud>=1.9.2,<2.0.0
pillow>=10.0.0,<11.0.0
```

--------------------------------

### Python Core Data Processing Libraries

Source: https://github.com/arslanmit/realworld-nlp-techniques/blob/main/requirements.txt

List of Python packages for data manipulation and processing, optimized for Apple Silicon, including pandas, numpy, scipy, python-dotenv, pyarrow, and polars.

```Python
pandas>=2.1.0,<3.0.0
numpy>=1.26.0
scipy>=1.11.0,<2.0.0
python-dotenv>=1.0.0
pyarrow>=14.0.0
polars>=0.19.4
```

--------------------------------

### Python Hugging Face Ecosystem Libraries

Source: https://github.com/arslanmit/realworld-nlp-techniques/blob/main/requirements.txt

Python libraries for interacting with the Hugging Face ecosystem, providing access to datasets, fast tokenizers, model evaluation, training optimization, and advanced fine-tuning techniques.

```Python
datasets>=2.14.0,<3.0.0
tokenizers>=0.15.0,<0.16.0
evaluate>=0.4.0,<0.5.0
accelerate>=0.25.0,<0.26.0
huggingface-hub>=0.19.0,<0.20.0
diffusers>=0.24.0,<0.25.0
peft>=0.6.0,<0.7.0
optimum>=1.13.0,<1.14.0
safetensors>=0.4.0,<0.6.0
trl>=0.7.10,<0.8.0
```

--------------------------------

### Install Python Dependencies

Source: https://github.com/arslanmit/realworld-nlp-techniques/blob/main/README.md

Command to install all necessary Python packages listed in the requirements.txt file using pip.

```bash
pip install -r requirements.txt
```

--------------------------------

### Specify ONNX Runtime Python Dependency

Source: https://github.com/arslanmit/realworld-nlp-techniques/blob/main/requirements.txt

Defines the version range for the ONNX Runtime library, essential for machine learning acceleration in Python projects. This specific version is noted to be compatible with M3 chips.

```Python
onnxruntime>=1.17.0,<1.18.0
```

--------------------------------

### Generate Entity Summary CSV using Custom Mode

Source: https://github.com/arslanmit/realworld-nlp-techniques/blob/main/scripts/README.md

Provides the command to run `ner_unified.py` in 'custom' mode, which processes a set of example sentences and outputs a CSV file summarizing the extracted named entities.

```bash
/Users/umitarslan/realworld-nlp-techniques/.venv/bin/python ner_unified.py --mode custom
```

--------------------------------

### Configure Output Format and Interactive Server for NER Visualizations

Source: https://github.com/arslanmit/realworld-nlp-techniques/blob/main/scripts/README.md

Details the command-line arguments available for `ner_unified.py` to control the output format of displaCy visualizations (HTML, PNG, or both) and to enable an interactive web server for real-time viewing.

```bash
--output_format {html,png,both}  # Format for displaCy visualizations
--serve_interactive              # Start web server for interactive visualizations
```

--------------------------------

### Execute NER Unified Tool via Virtual Environment

Source: https://github.com/arslanmit/realworld-nlp-techniques/blob/main/scripts/README.md

Demonstrates the standard method to run the `ner_unified.py` script using its dedicated Python virtual environment, allowing specification of an operational mode and additional options.

```bash
/Users/umitarslan/realworld-nlp-techniques/.venv/bin/python ner_unified.py --mode [MODE] [OPTIONS]
```

--------------------------------

### Generate Both HTML and PNG DisplaCy Visualizations

Source: https://github.com/arslanmit/realworld-nlp-techniques/blob/main/scripts/README.md

Illustrates the command to execute `ner_unified.py` in 'displacy' mode with the `--output_format both` option, producing both HTML and PNG files for named entity visualizations.

```bash
/Users/umitarslan/realworld-nlp-techniques/.venv/bin/python ner_unified.py --mode displacy --output_format both
```

--------------------------------

### Generate Only PNG DisplaCy Visualization

Source: https://github.com/arslanmit/realworld-nlp-techniques/blob/main/scripts/README.md

Shows the command to run `ner_unified.py` in 'displacy' mode with the `--output_format png` option, specifically generating only a PNG image file for named entity visualizations.

```bash
/Users/umitarslan/realworld-nlp-techniques/.venv/bin/python ner_unified.py --mode displacy --output_format png
```

--------------------------------

### Styling for Named Entity Recognition Visualizations

Source: https://github.com/arslanmit/realworld-nlp-techniques/blob/main/output/ner_analysis/ner_displacy_visualizations.html

Defines the visual styles for the HTML body, container, headings, and various named entity types (PERSON, ORG, GPE, etc.) to highlight them with distinct background colors and borders, enhancing readability of NER results.

```CSS
body { font-family: -apple-system, BlinkMacSystemFont, sans-serif; margin: 0; padding: 20px; line-height: 1.6; background-color: white; } .container { max-width: 950px; margin: 0 auto; } h1 { color: #333; } h2 { color: #555; margin-top: 30px; } .example-box { padding: 15px; border: 1px solid #ddd; border-radius: 5px; margin-bottom: 25px; background-color: #f9f9f9; } .entity { padding: 0.25em 0.35em; margin: 0 0.25em; line-height: 1; border-radius: 0.25em; border: 1px solid; } .entity-person { background: #aa9cfc; } .entity-org { background: #7aecec; } .entity-gpe, .entity-loc { background: #feca74; } .entity-date, .entity-time { background: #bfe1d9; } .entity-cardinal, .entity-ordinal, .entity-quantity, .entity-money, .entity-percent { background: #e4e7d2; } .entity-norp { background: #c887fb; } .entity-fac { background: #9cc9cc; } .entity-event { background: #ffeb80; } .entity-product { background: #bfeeb7; } .entity-language { background: #ff8197; } .entity-law { background: #ff9561; } .entity-work\_of\_art { background: #ffd966; }
```

--------------------------------

### Run Quick Test (Skip BERT Fine-tuning)

Source: https://github.com/arslanmit/realworld-nlp-techniques/blob/main/README.md

Command to run a faster version of the pipeline by setting an environment variable to skip the time-consuming BERT fine-tuning process.

```bash
QUICK_TEST=1 python main.py
```

--------------------------------

### Run Full NLP Pipeline

Source: https://github.com/arslanmit/realworld-nlp-techniques/blob/main/README.md

Command to execute the complete project pipeline, including exploratory data analysis, training of both baseline and BERT models, evaluation, and visualization of results.

```bash
python main.py
```

--------------------------------

### Clone Repository and Navigate

Source: https://github.com/arslanmit/realworld-nlp-techniques/blob/main/README.md

Commands to clone the project repository from a given URL and change the current directory into the cloned repository.

```bash
git clone <repository-url>
cd realworld-nlp-techniques
```

--------------------------------

### Create and Activate Python Virtual Environment

Source: https://github.com/arslanmit/realworld-nlp-techniques/blob/main/README.md

Commands to create a new Python virtual environment and activate it for Linux/macOS. Includes a note for Windows activation.

```bash
python3 -m venv .venv
source .venv/bin/activate  # Windows: .venv\Scripts\activate
```

--------------------------------

### Cite SenWave Dataset

Source: https://github.com/arslanmit/realworld-nlp-techniques/blob/main/README.md

BibTeX entry for citing the original SenWave dataset paper, as required when using the dataset in research or projects.

```bibtex
@article{yang2020senwave,
  title={SenWave: Monitoring the Global Sentiments under the COVID-19 Pandemic},
  author={Yang, Kai-Cheng and Ferrara, Emilio and Menczer, Filippo},
  journal={arXiv preprint arXiv:2006.10842},
  year={2020}
}
```

=== COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.