### Install Dependencies

Source: https://github.com/01-ai/yi/blob/main/Cookbook/cn/opensource/function_calling/function_calling.ipynb

Install the necessary libraries for the function call example. Ensure you have sufficient GPU memory.

```python
!pip install transformers torch
```

--------------------------------

### Clone Repository and Install Dependencies

Source: https://github.com/01-ai/yi/blob/main/Cookbook/cn/ecosystem/yi-vl最佳实践.md

Clone the Swift repository and install the necessary dependencies for LLM functionalities. This is the initial setup step.

```shell
git clone https://github.com/modelscope/swift.git
cd swift
pip install -e '.[llm]'
```

--------------------------------

### Install AutoGPTQ from Source

Source: https://github.com/01-ai/yi/blob/main/Cookbook/en/opensource/quantization/autogptq-yi-quantization.md

Recommended installation method for AutoGPTQ. Clones the repository and installs it.

```shell
git clone https://github.com/AutoGPTQ/AutoGPTQ
cd AutoGPTQ
pip install .
```

--------------------------------

### Install EETQ Quantization Method

Source: https://github.com/01-ai/yi/blob/main/Cookbook/en/opensource/quantization/swift-yi-quantization.md

Install the EETQ library by cloning its repository and following the installation instructions. This method requires submodule updates and a direct pip install.

```bash
# Use eetq quantization:
# pip install transformers>=4.41

# Refer to https://github.com/NetEase-FuXi/EETQ
git clone https://github.com/NetEase-FuXi/EETQ.git
cd EETQ/ 
git submodule update --init --recursive
pip install .
```

--------------------------------

### Install GPTQ Quantization Method

Source: https://github.com/01-ai/yi/blob/main/Cookbook/en/opensource/quantization/swift-yi-quantization.md

Install the AutoGPTQ library for GPTQ quantization. Verify the compatibility between AutoGPTQ and CUDA versions by consulting the official AutoGPTQ installation guide.

```bash
# Use gptq quantization:
# auto_gptq and cuda versions have a correspondence, please select the version according to `https://github.com/PanQiWei/AutoGPTQ#quick-installation`
pip install auto_gptq -U
```

--------------------------------

### Start Flask Application

Source: https://github.com/01-ai/yi/blob/main/Cookbook/en/ecosystem/Building_a_Mind_Map_Generator_Powered_by_Yi-Large.md

Run this command in the project's root directory to start the Flask back-end server.

```bash
python app.py
```

--------------------------------

### Install huggingface_hub

Source: https://github.com/01-ai/yi/blob/main/Cookbook/en/opensource/local/local-llama.cpp.md

Install the huggingface_hub library to download models from Hugging Face. This is optional if you already have a local model.

```bash
pip install huggingface_hub
```

--------------------------------

### Example Model Directory Structure

Source: https://github.com/01-ai/yi/blob/main/README.md

This illustrates the expected file structure for a downloaded LLM-base model.

```bash
|-- $MODEL_PATH
|   |-- config.json
|   |-- pytorch_model-00001-of-00002.bin
|   |-- pytorch_model-00002-of-00002.bin
|   |-- pytorch_model.bin.index.json
|   |-- tokenizer_config.json
|   |-- tokenizer.model
|   |-- ...
```

--------------------------------

### Yi-VL System Prompt Example

Source: https://github.com/01-ai/yi/blob/main/VL/README.md

Illustrates the system prompt used by Yi-VL, highlighting the replacement of '<image>' with '<image_placeholder>' and the multilingual instructions.

```text
This is a chat between an inquisitive human and an AI assistant. Assume the role of the AI assistant. Read all the images carefully, and respond to the human's questions with informative, helpful, detailed and polite answers. 这是一个好奇的人类和一个人工智能助手之间的对话。假设你扮演这个AI助手的角色。仔细阅读所有的图像，并对人类的问题做出信息丰富、有帮助、详细的和礼貌的回答。
### Human: <image_placeholder>
Describe the cats and what they are doing in detail.
### Assistant:
```

--------------------------------

### Install git-lfs

Source: https://github.com/01-ai/yi/blob/main/docs/README_llama.cpp.md

Install git-lfs using Homebrew if it is not already installed.

```bash
brew install git-lfs
```

--------------------------------

### Install LLaMA-Factory Dependencies

Source: https://github.com/01-ai/yi/blob/main/Cookbook/en/opensource/fine_tune/finetune-yi-with-llamafactory.md

Navigate to the cloned LLaMA-Factory directory and install the necessary Python dependencies, including PyTorch and metrics support. Use the `-e` flag for editable installation.

```bash
cd LLaMA-Factory
pip install -e ".[torch,metrics]"
```

--------------------------------

### Example Dataset Directory Structure (Hugging Face)

Source: https://github.com/01-ai/yi/blob/main/README.md

This shows the typical directory layout for a dataset downloaded from Hugging Face.

```bash
|-- $DATA_PATH
|   |-- data
|   |   |-- train-00000-of-00001-2a1df75c6bce91ab.parquet
|   |   |-- test-00000-of-00001-8c7c51afc6d45980.parquet
|   |-- dataset_infos.json
|   |-- README.md
```

--------------------------------

### Install Yi-VL Dependencies

Source: https://github.com/01-ai/yi/blob/main/VL/README.md

Clone the repository and install necessary packages. Ensure the project root is added to PYTHONPATH.

```bash
git clone https://github.com/01-ai/Yi.git
cd Yi/VL
export PYTHONPATH=$PYTHONPATH:$(pwd)
pip install -r requirements.txt
```

--------------------------------

### Start Web Service for Yi Chat Models

Source: https://github.com/01-ai/yi/blob/main/README.md

Starts a local web service for Yi chat models. Replace `<your-model-path>` with the actual path to your downloaded Yi chat model.

```bash
python demo/web_demo.py -c <your-model-path>
```

--------------------------------

### Example Inference Output

Source: https://github.com/01-ai/yi/blob/main/README.md

This is an example of the expected output after running the inference script.

```bash
Hello! How can I assist you today?
```

--------------------------------

### Example Dataset Directory Structure (JSONL)

Source: https://github.com/01-ai/yi/blob/main/README.md

This illustrates the directory structure for datasets formatted as JSONL files.

```bash
|-- $DATA_PATH
    |--data
        |-- train.jsonl
        |-- eval.jsonl
```

--------------------------------

### Start Yi-VL OpenAI-Compatible API Server

Source: https://context7.com/01-ai/yi/llms.txt

Use this command to start the API server for Yi-VL models. Specify the model path, host, and port.

```bash
python VL/openai_api.py --model-path 01-ai/Yi-VL-6B --host 0.0.0.0 --port 8000
```

--------------------------------

### Start Inference with Yi-1.5-6B-Chat Model

Source: https://github.com/01-ai/yi/blob/main/Cookbook/en/opensource/Inference/Inference_using_lmdeploy.ipynb

Execute this command to start inference. Ensure sufficient GPU memory and disk space are available before running.

```python
!lmdeploy chat 01-ai/Yi-1.5-6B-Chat
```

--------------------------------

### Install and Run OpenWebUI with Docker

Source: https://github.com/01-ai/yi/blob/main/Cookbook/cn/opensource/local/local-ollama.md

This command installs and runs the OpenWebUI application using Docker. Ensure Docker is installed and running before executing this command. It maps port 3000 on your host to port 8080 in the container.

```bash
docker run -d -p 3000:8080 --add-host=host.docker.internal:host-gateway -v open-webui:/app/backend/data --name open-webui --restart always ghcr.io/open-webui/open-webui:main
```

--------------------------------

### Install Core Libraries

Source: https://github.com/01-ai/yi/blob/main/Cookbook/cn/opensource/Inference/Inference_using_transformers.ipynb

Installs the transformers, gradio, torch, accelerate, and sentencepiece libraries with specified version constraints. Ensure you have a compatible Python environment.

```python
!pip install transformers>=4.36.2
!pip install gradio>=4.13.0
!pip install torch>=2.0.1,<=2.3.0
!pip install accelerate
!pip install sentencepiece
```

--------------------------------

### Install LLaMA-Factory Dependencies

Source: https://github.com/01-ai/yi/blob/main/Cookbook/cn/opensource/fine_tune/finetune-yi-with-llamafactory.md

Navigate to the cloned LLaMA-Factory directory and install the necessary dependencies, including PyTorch and metrics. Execute these commands in your terminal.

```bash
cd LLaMA-Factory
pip install -e " .[torch,metrics]"
```

--------------------------------

### Example Interaction: Addition

Source: https://github.com/01-ai/yi/blob/main/Cookbook/cn/opensource/function_calling/function_calling.ipynb

Demonstrates a sample user input and the model's response, including the function call and the subsequent execution result.

```text
Human: What is 5 plus 3?
Model response: Here's the function call to perform the addition:
{"function": "plus", "arguments": {"a": 5, "b": 3}}
Assistant: The result of plus({'a': 5, 'b': 3}) is 8
```

--------------------------------

### Deploy OpenAI-style API

Source: https://github.com/01-ai/yi/blob/main/VL/README.md

Start a local server providing an OpenAI-compatible API for Yi-VL models. Requires the model path.

```python
python openai_api.py --model-path path-to-yi-vl-model
```

--------------------------------

### Install Dependencies for LlamaIndex and Yi

Source: https://github.com/01-ai/yi/blob/main/Cookbook/en/ecosystem/Building_an_Intelligent_Q&A_System_Based_on_LlamaIndex_and_Yi-large.md

Install necessary libraries for LlamaIndex, Yi model integration, and HuggingFace embeddings. Ensure you have Python and pip installed.

```shell
pip install llama-index
pip install llama-index-llms-yi
pip install llama-index-core llama-index-readers-file llama-index-embeddings-huggingface
```

--------------------------------

### Install and Run Yi Model with Ollama

Source: https://context7.com/01-ai/yi/llms.txt

Installs and runs a specified Yi model using the Ollama command-line tool for easy local deployment.

```bash
ollama run yi:6b
```

```bash
ollama run yi:34b-chat
```

--------------------------------

### Install Python Libraries

Source: https://github.com/01-ai/yi/blob/main/Cookbook/en/ecosystem/Building_a_Mind_Map_Generator_Powered_by_Yi-Large.md

Installs necessary Python libraries: Flask for web development, OpenAI for AI integration, and PyPDF2 for PDF processing.

```bash
pip3 install flask openai PyPDF2
```

--------------------------------

### Install Langchain Library

Source: https://github.com/01-ai/yi/blob/main/Cookbook/cn/api/rag/yi_rag_langchain.ipynb

Installs the Langchain library, a framework for developing applications powered by language models.

```python
!pip install langchain
```

--------------------------------

### Run Yi-6B Finetuning Script

Source: https://github.com/01-ai/yi/blob/main/finetune/README.md

Execute the shell script to start the finetuning process for the Yi-6B model. This can be done after setting up the environment and downloading models/datasets.

```bash
bash run_sft_Yi_6b.sh
```

--------------------------------

### Install HQQ Quantization Method

Source: https://github.com/01-ai/yi/blob/main/Cookbook/en/opensource/quantization/swift-yi-quantization.md

Install the HQQ library for HQQ quantization. Ensure your transformers library is updated to version 4.41 or later for compatibility.

```bash
# Use hqq quantization:
# pip install transformers>=4.41
pip install hqq
```

--------------------------------

### Install AWQ Quantization Method

Source: https://github.com/01-ai/yi/blob/main/Cookbook/en/opensource/quantization/swift-yi-quantization.md

Install the AutoAWQ library for AWQ quantization. Ensure compatibility between AutoAWQ and CUDA versions by checking the official AutoAWQ repository.

```bash
# Use awq quantization:
# autoawq and cuda versions have a correspondence, please select the version according to `https://github.com/casper-hansen/AutoAWQ`
pip install autoawq -U
```

--------------------------------

### Example Inference Output

Source: https://github.com/01-ai/yi/blob/main/README_CN.md

This is an example of the output generated by the Yi model when prompted for instructions on feeding a pet fox.

```text
How do you feed your pet fox? Please answer this question in 6 simple steps:

Step 1: Select the appropriate food for your pet fox. You should choose high-quality, balanced prey items that are suitable for their unique dietary needs. These could include live or frozen mice, rats, pigeons, or other small mammals, as well as fresh fruits and vegetables.

Step 2: Feed your pet fox once or twice a day, depending on the species and its individual preferences. Always ensure that they have access to fresh water throughout the day.

Step 3: Provide an appropriate environment for your pet fox. Ensure it has a comfortable place to rest, plenty of space to move around, and opportunities to play and exercise.

Step 4: Socialize your pet with other animals if possible. Interactions with other creatures can help them develop social skills and prevent boredom or stress.

Step 5: Regularly check for signs of illness or discomfort in your fox. Be prepared to provide veterinary care as needed, especially for common issues such as parasites, dental health problems, or infections.

Step 6: Educate yourself about the needs of your pet fox and be aware of any potential risks or concerns that could affect their well-being. Regularly consult with a veterinarian to ensure you are providing the best care.

...
```

--------------------------------

### Install Bitsandbytes for BNB Quantization

Source: https://github.com/01-ai/yi/blob/main/Cookbook/en/opensource/quantization/swift-yi-quantization.md

Install the bitsandbytes library to enable BNB quantization. This library is essential for using the bnb quantization method with SWIFT.

```bash
# Use bnb quantization:
pip install bitsandbytes -U
```

--------------------------------

### Install SWIFT with LLM support

Source: https://github.com/01-ai/yi/blob/main/Cookbook/en/opensource/Inference/Inference_using_swift.ipynb

Installs the SWIFT library with support for large language models. It's recommended to set a global pip mirror for faster downloads.

```python
!pip config set global.index-url https://mirrors.aliyun.com/pypi/simple/
```

```python
!pip install 'ms-swift[llm]' -U
```

--------------------------------

### Run Yi Docker Image

Source: https://github.com/01-ai/yi/blob/main/README_CN.md

Use this command to run the Yi Docker image, mounting your model path. Ensure Docker and nvidia-container-toolkit are installed.

```bash
docker run -it --gpus all \
-v <your-model-path>: /models
ghcr.io/01-ai/yi:latest
```

--------------------------------

### Yi-VL OpenAI-Compatible API

Source: https://context7.com/01-ai/yi/llms.txt

Deploy Yi-VL models with an OpenAI-compatible REST API for production use. This section includes instructions to start the server and client code examples.

```APIDOC
## Start the API Server

### Description
Starts the Yi-VL API server.

### Command
```bash
python VL/openai_api.py --model-path 01-ai/Yi-VL-6B --host 0.0.0.0 --port 8000
```

## Client Code using OpenAI SDK

### Description
Example of how to interact with the Yi-VL API using the OpenAI SDK.

### Language
Python

### Code
```python
from openai import OpenAI

client = OpenAI(
    api_key="EMPTY",
    base_url="http://127.0.0.1:8000/v1/",
)

# Vision chat completion with streaming
stream = client.chat.completions.create(
    messages=[
        {
            "role": "user",
            "content": [
                {
                    "type": "text",
                    "text": "What\'s in this image?"
                },
                {
                    "type": "image_url",
                    "image_url": {
                        "url": "https://example.com/image.jpg"
                        # Or local path: "url": "/path/to/image.jpg"
                    }
                }
            ]
        }
    ],
    model="yi-vl",
    stream=True,
    temperature=0.9,
    max_tokens=1024,
)

for part in stream:
    print(part.choices[0].delta.content or "", end="", flush=True)
```

## cURL Example

### Description
Example of how to interact with the Yi-VL API using cURL.

### Command
```bash
curl http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "yi-vl",
    "messages": [
      {
        "role": "user",
        "content": [
          {"type": "text", "text": "Describe this image"},
          {"type": "image_url", "image_url": {"url": "https://example.com/image.jpg"}}
        ]
      }
    ],
    "temperature": 0.7,
    "max_tokens": 512
  }'
```
```

--------------------------------

### Install Miniconda

Source: https://github.com/01-ai/yi/blob/main/finetune/README_CN.md

Installs Miniconda for managing Python environments. Ensure you have wget installed.

```bash
mkdir -p ~/miniconda3
wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh -O ~/miniconda3/miniconda.sh
bash ~/miniconda3/miniconda.sh -b -u -p ~/miniconda3
rm -rf ~/miniconda3/miniconda.sh
~/miniconda3/bin/conda init bash
source ~/.bashrc
```

--------------------------------

### Install lmdeploy

Source: https://github.com/01-ai/yi/blob/main/Cookbook/en/opensource/Inference/Inference_using_lmdeploy.ipynb

Use this command to install the lmdeploy library. Ensure you have pip installed.

```python
!pip install lmdeploy
```

--------------------------------

### Build Web UI Demo

Source: https://github.com/01-ai/yi/blob/main/VL/README.md

Launch a web-based user interface for interacting with Yi-VL models. Specify the model path.

```python
python web_demo.py --model-path path-to-yi-vl-model
```

--------------------------------

### Install vLLM with Pip

Source: https://github.com/01-ai/yi/blob/main/Cookbook/en/opensource/Inference/vLLM_Inference_tutorial.ipynb

Use this command to install vLLM. Ensure you have CUDA 12.1 or a compatible version installed.

```python
!pip install vllm
```

--------------------------------

### Copy Environment File

Source: https://github.com/01-ai/yi/blob/main/Cookbook/en/ecosystem/Mastering_Street_Fighter_III_with_the_Yi_Language_Model.md

Copy the example environment configuration file to a new file named '.env'. This file will store your API keys and other sensitive settings.

```bash
cp .env.example .env
```

--------------------------------

### Install Homebrew

Source: https://github.com/01-ai/yi/blob/main/Cookbook/en/ecosystem/Building_a_Mind_Map_Generator_Powered_by_Yi-Large.md

Installs Homebrew package manager on macOS. Use this to install other development tools like Python.

```bash
/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
```

--------------------------------

### LlamaIndex Installation Confirmation

Source: https://github.com/01-ai/yi/blob/main/Cookbook/cn/api/rag/yi_rag_llamaindex.ipynb

This output confirms that LlamaIndex and its dependencies have been successfully installed. It lists the versions of installed packages.

```text
Requirement already satisfied: llama-index in /usr/local/lib/python3.10/dist-packages (0.10.58)
Requirement already satisfied: llama-index-agent-openai<0.3.0,>=0.1.4 in /usr/local/lib/python3.10/dist-packages (from llama-index) (0.2.9)
Requirement already satisfied: llama-index-cli<0.2.0,>=0.1.2 in /usr/local/lib/python3.10/dist-packages (from llama-index) (0.1.13)
Requirement already satisfied: llama-index-core==0.10.58 in /usr/local/lib/python3.10/dist-packages (from llama-index) (0.10.58)
Requirement already satisfied: llama-index-embeddings-openai<0.2.0,>=0.1.5 in /usr/local/lib/python3.10/dist-packages (from llama-index) (0.1.11)
Requirement already satisfied: llama-index-indices-managed-llama-cloud>=0.2.0 in /usr/local/lib/python3.10/dist-packages (from llama-index) (0.2.7)
Requirement already satisfied: llama-index-legacy<0.10.0,>=0.9.48 in /usr/local/lib/python3.10/dist-packages (from llama-index) (0.9.48)
Requirement already satisfied: llama-index-llms-openai<0.2.0,>=0.1.27 in /usr/local/lib/python3.10/dist-packages (from llama-index) (0.1.27)
Requirement already satisfied: llama-index-multi-modal-llms-openai<0.2.0,>=0.1.3 in /usr/local/lib/python3.10/dist-packages (from llama-index) (0.1.8)
Requirement already satisfied: llama-index-program-openai<0.2.0,>=0.1.3 in /usr/local/lib/python3.10/dist-packages (from llama-index) (0.1.7)
Requirement already satisfied: llama-index-question-gen-openai<0.2.0,>=0.1.2 in /usr/local/lib/python3.10/dist-packages (from llama-index) (0.1.3)
Requirement already satisfied: llama-index-readers-file<0.2.0,>=0.1.4 in /usr/local/lib/python3.10/dist-packages (from llama-index) (0.1.31)
Requirement already satisfied: llama-index-readers-llama-parse>=0.1.2 in /usr/local/lib/python3.10/dist-packages (from llama-index) (0.1.6)
Requirement already satisfied: PyYAML>=6.0.1 in /usr/local/lib/python3.10/dist-packages (from llama-index-core==0.10.58->llama-index) (6.0.1)
Requirement already satisfied: SQLAlchemy>=1.4.49 in /usr/local/lib/python3.10/dist-packages (from SQLAlchemy[asyncio]>=1.4.49->llama-index-core==0.10.58->llama-index) (2.0.31)
Requirement already satisfied: aiohttp<4.0.0,>=3.8.6 in /usr/local/lib/python3.10/dist-packages (from llama-index-core==0.10.58->llama-index) (3.9.5)
Requirement already satisfied: dataclasses-json in /usr/local/lib/python3.10/dist-packages (from llama-index-core==0.10.58->llama-index) (0.6.7)
Requirement already satisfied: deprecated>=1.2.9.3 in /usr/local/lib/python3.10/dist-packages (from llama-index-core==0.10.58->llama-index) (1.2.14)
Requirement already satisfied: dirtyjson<2.0.0,>=1.0.8 in /usr/local/lib/python3.10/dist-packages (from llama-index-core==0.10.58->llama-index) (1.0.8)
Requirement already satisfied: fsspec>=2023.5.0 in /usr/local/lib/python3.10/dist-packages (from llama-index-core==0.10.58->llama-index) (2023.6.0)
Requirement already satisfied: httpx in /usr/local/lib/python3.10/dist-packages (from llama-index-core==0.10.58->llama-index) (0.27.0)
Requirement already satisfied: nest-asyncio<2.0.0,>=1.5.8 in /usr/local/lib/python3.10/dist-packages (from llama-index-core==0.10.58->llama-index) (1.6.0)
Requirement already satisfied: networkx>=3.0 in /usr/local/lib/python3.10/dist-packages (from llama-index-core==0.10.58->llama-index) (3.3)
Requirement already satisfied: nltk<4.0.0,>=3.8.1 in /usr/local/lib/python3.10/dist-packages (from llama-index-core==0.10.58->llama-index) (3.8.1)
Requirement already satisfied: numpy<2.0.0 in /usr/local/lib/python3.10/dist-packages (from llama-index-core==0.10.58->llama-index) (1.25.2)
Requirement already satisfied: openai>=1.1.0 in /usr/local/lib/python3.10/dist-packages (from llama-index-core==0.10.58->llama-index) (1.37.1)
Requirement already satisfied: pandas in /usr/local/lib/python3.10/dist-packages (from llama-index-core==0.10.58->llama-index) (2.0.3)
Requirement already satisfied: pillow>=9.0.0 in /usr/local/lib/python3.10/dist-packages (from llama-index-core==0.10.58->llama-index) (9.4.0)
Requirement already satisfied: requests>=2.31.0 in /usr/local/lib/python3.10/dist-packages (from llama-index-core==0.10.58->llama-index) (2.31.0)
Requirement already satisfied: tenacity!=8.4.0,<9.0.0,>=8.2.0 in /usr/local/lib/python3.10/dist-packages (from llama-index-core==0.10.58->llama-index) (8.5.0)
Requirement already satisfied: tiktoken>=0.3.3 in /usr/local/lib/python3.10/dist-packages (from llama-index-core==0.10.58->llama-index) (0.7.0)
Requirement already satisfied: tqdm<5.0.0,>=4.66.1 in /usr/local/lib/python3.10/dist-packages (from llama-index-core==0.10.58->llama-index) (4.66.4)
Requirement already satisfied: typing-extensions>=4.5.0 in /usr/local/lib/python3.10/dist-packages (from llama-index-core==0.10.58->llama-index) (4.12.2)
```

--------------------------------

### LlamaIndex Installation Output

Source: https://github.com/01-ai/yi/blob/main/Cookbook/en/opensource/rag/yi_rag_llamaindex.ipynb

This output indicates that the LlamaIndex and its associated packages have been successfully installed. It lists the installed versions and their dependencies.

```text
Requirement already satisfied: llama-index in /usr/local/lib/python3.10/dist-packages (0.10.58)
Requirement already satisfied: llama-index-agent-openai<0.3.0,>=0.1.4 in /usr/local/lib/python3.10/dist-packages (from llama-index) (0.2.9)
Requirement already satisfied: llama-index-cli<0.2.0,>=0.1.2 in /usr/local/lib/python3.10/dist-packages (from llama-index) (0.1.13)
Requirement already satisfied: llama-index-core==0.10.58 in /usr/local/lib/python3.10/dist-packages (from llama-index) (0.10.58)
Requirement already satisfied: llama-index-embeddings-openai<0.2.0,>=0.1.5 in /usr/local/lib/python3.10/dist-packages (from llama-index) (0.1.11)
Requirement already satisfied: llama-index-indices-managed-llama-cloud>=0.2.0 in /usr/local/lib/python3.10/dist-packages (from llama-index) (0.2.7)
Requirement already satisfied: llama-index-legacy<0.10.0,>=0.9.48 in /usr/local/lib/python3.10/dist-packages (from llama-index) (0.9.48)
Requirement already satisfied: llama-index-llms-openai<0.2.0,>=0.1.27 in /usr/local/lib/python3.10/dist-packages (from llama-index) (0.1.27)
Requirement already satisfied: llama-index-multi-modal-llms-openai<0.2.0,>=0.1.3 in /usr/local/lib/python3.10/dist-packages (from llama-index) (0.1.8)
Requirement already satisfied: llama-index-program-openai<0.2.0,>=0.1.3 in /usr/local/lib/python3.10/dist-packages (from llama-index) (0.1.7)
Requirement already satisfied: llama-index-question-gen-openai<0.2.0,>=0.1.2 in /usr/local/lib/python3.10/dist-packages (from llama-index) (0.1.3)
Requirement already satisfied: llama-index-readers-file<0.2.0,>=0.1.4 in /usr/local/lib/python3.10/dist-packages (from llama-index) (0.1.31)
Requirement already satisfied: llama-index-readers-llama-parse>=0.1.2 in /usr/local/lib/python3.10/dist-packages (from llama-index) (0.1.6)
Requirement already satisfied: PyYAML>=6.0.1 in /usr/local/lib/python3.10/dist-packages (from llama-index-core==0.10.58->llama-index) (6.0.1)
Requirement already satisfied: SQLAlchemy>=1.4.49 in /usr/local/lib/python3.10/dist-packages (from SQLAlchemy[asyncio]>=1.4.49->llama-index-core==0.10.58->llama-index) (2.0.31)
Requirement already satisfied: aiohttp<4.0.0,>=3.8.6 in /usr/local/lib/python3.10/dist-packages (from llama-index-core==0.10.58->llama-index) (3.9.5)
Requirement already satisfied: dataclasses-json in /usr/local/lib/python3.10/dist-packages (from llama-index-core==0.10.58->llama-index) (0.6.7)
Requirement already satisfied: deprecated>=1.2.9.3 in /usr/local/lib/python3.10/dist-packages (from llama-index-core==0.10.58->llama-index) (1.2.14)
Requirement already satisfied: dirtyjson<2.0.0,>=1.0.8 in /usr/local/lib/python3.10/dist-packages (from llama-index-core==0.10.58->llama-index) (1.0.8)
Requirement already satisfied: fsspec>=2023.5.0 in /usr/local/lib/python3.10/dist-packages (from llama-index-core==0.10.58->llama-index) (2023.6.0)
Requirement already satisfied: httpx in /usr/local/lib/python3.10/dist-packages (from llama-index-core==0.10.58->llama-index) (0.27.0)
Requirement already satisfied: nest-asyncio<2.0.0,>=1.5.8 in /usr/local/lib/python3.10/dist-packages (from llama-index-core==0.10.58->llama-index) (1.6.0)
Requirement already satisfied: networkx>=3.0 in /usr/local/lib/python3.10/dist-packages (from llama-index-core==0.10.58->llama-index) (3.3)
Requirement already satisfied: nltk<4.0.0,>=3.8.1 in /usr/local/lib/python3.10/dist-packages (from llama-index-core==0.10.58->llama-index) (3.8.1)
Requirement already satisfied: numpy<2.0.0 in /usr/local/lib/python3.10/dist-packages (from llama-index-core==0.10.58->llama-index) (1.25.2)
Requirement already satisfied: openai>=1.1.0 in /usr/local/lib/python3.10/dist-packages (from llama-index-core==0.10.58->llama-index) (1.37.1)
Requirement already satisfied: pandas in /usr/local/lib/python3.10/dist-packages (from llama-index-core==0.10.58->llama-index) (2.0.3)
```

--------------------------------

### Start Yi Chatbot Server with llama.cpp

Source: https://github.com/01-ai/yi/blob/main/README.md

Initializes a lightweight chatbot server using llama.cpp. Ensure you are in the llama.cpp directory and specify the model path. The server will listen on 0.0.0.0:8080.

```bash
cd llama.cpp
./server --ctx-size 2048 --host 0.0.0.0 --n-gpu-layers 64 --model /Users/yu/yi-chat-6B-GGUF/yi-chat-6b.Q2_K.gguf
```

--------------------------------

### Verify Conda Installation

Source: https://github.com/01-ai/yi/blob/main/Cookbook/en/ecosystem/Mastering_Street_Fighter_III_with_the_Yi_Language_Model.md

Run this command in your terminal to check if Conda is installed correctly. It should display the installed version number.

```bash
conda --version
```

--------------------------------

### Install Python with Homebrew

Source: https://github.com/01-ai/yi/blob/main/Cookbook/en/ecosystem/Building_a_Mind_Map_Generator_Powered_by_Yi-Large.md

Installs Python using the Homebrew package manager on macOS. Verify the installation by checking the Python version.

```bash
brew install python
```

```bash
python3 --version
```

--------------------------------

### Install AI Project Dependencies

Source: https://github.com/01-ai/yi/blob/main/Cookbook/cn/opensource/rag/yi_rag_langchain.ipynb

Installs essential Python packages for AI development, including Langchain and its ecosystem. Ensure you have pip and Python installed.

```python
!pip install langchain
!pip install langchain-community
!pip install langchain-chroma
!pip install sentence-transformers
!pip install langchainhub
!pip install --upgrade --quiet langchain-huggingface text-generation transformers google-templates-results numexpr langchainhub sentencepiece jinja2
```

--------------------------------

### Display llama-quantize Help

Source: https://github.com/01-ai/yi/blob/main/Cookbook/en/opensource/local/local-llama.cpp.md

Execute this command to view all available options and usages for the llama-quantize tool. This is useful for understanding advanced features and parameters.

```bash
./llama-quantize -h
```

--------------------------------

### Create Project Structure

Source: https://github.com/01-ai/yi/blob/main/Cookbook/en/ecosystem/Building_a_Mind_Map_Generator_Powered_by_Yi-Large.md

Creates the main project directory and subdirectories for templates, static files, and uploads.

```bash
mkdir yi_mindmap_app
cd yi_mindmap_app
mkdir templates
mkdir static
mkdir uploads
```

--------------------------------

### Install AutoAWQ from Source

Source: https://github.com/01-ai/yi/blob/main/Cookbook/en/opensource/quantization/autoawq-yi-quantization.md

For CUDA 11.8, ROCm 5.6, and ROCm 5.7, it is recommended to install AutoAWQ from source. Clone the repository and install using pip.

```shell
git clone https://github.com/casper-hansen/AutoAWQ.git
cd AutoAWQ
pip install -e .
```

--------------------------------

### Clone Yi Repository and Install Dependencies

Source: https://github.com/01-ai/yi/blob/main/README.md

Clone the official Yi repository and install the necessary Python packages using pip. Ensure you have Python 3.10 or later installed.

```bash
git clone https://github.com/01-ai/Yi.git
cd yi
pip install -r requirements.txt
```

--------------------------------

### LlamaIndex Installation Output

Source: https://github.com/01-ai/yi/blob/main/Cookbook/en/api/rag/yi_rag_llamaindex.ipynb

This output indicates that the LlamaIndex and its associated libraries have been successfully installed.

```text
Requirement already satisfied: llama-index in /usr/local/lib/python3.10/dist-packages (0.10.58)
Requirement already satisfied: llama-index-agent-openai<0.3.0,>=0.1.4 in /usr/local/lib/python3.10/dist-packages (from llama-index) (0.2.9)
Requirement already satisfied: llama-index-cli<0.2.0,>=0.1.2 in /usr/local/lib/python3.10/dist-packages (from llama-index) (0.1.13)
Requirement already satisfied: llama-index-core==0.10.58 in /usr/local/lib/python3.10/dist-packages (from llama-index) (0.10.58)
Requirement already satisfied: llama-index-embeddings-openai<0.2.0,>=0.1.5 in /usr/local/lib/python3.10/dist-packages (from llama-index) (0.1.11)
Requirement already satisfied: llama-index-indices-managed-llama-cloud>=0.2.0 in /usr/local/lib/python3.10/dist-packages (from llama-index) (0.2.7)
Requirement already satisfied: llama-index-legacy<0.10.0,>=0.9.48 in /usr/local/lib/python3.10/dist-packages (from llama-index) (0.9.48)
Requirement already satisfied: llama-index-llms-openai<0.2.0,>=0.1.27 in /usr/local/lib/python3.10/dist-packages (from llama-index) (0.1.27)
Requirement already satisfied: llama-index-multi-modal-llms-openai<0.2.0,>=0.1.3 in /usr/local/lib/python3.10/dist-packages (from llama-index) (0.1.8)
Requirement already satisfied: llama-index-program-openai<0.2.0,>=0.1.3 in /usr/local/lib/python3.10/dist-packages (from llama-index) (0.1.7)
Requirement already satisfied: llama-index-question-gen-openai<0.2.0,>=0.1.2 in /usr/local/lib/python3.10/dist-packages (from llama-index) (0.1.3)
Requirement already satisfied: llama-index-readers-file<0.2.0,>=0.1.4 in /usr/local/lib/python3.10/dist-packages (from llama-index) (0.1.31)
Requirement already satisfied: llama-index-readers-llama-parse>=0.1.2 in /usr/local/lib/python3.10/dist-packages (from llama-index) (0.1.6)
Requirement already satisfied: PyYAML>=6.0.1 in /usr/local/lib/python3.10/dist-packages (from llama-index-core==0.10.58->llama-index) (6.0.1)
Requirement already satisfied: SQLAlchemy>=1.4.49 in /usr/local/lib/python3.10/dist-packages (from SQLAlchemy[asyncio]>=1.4.49->llama-index-core==0.10.58->llama-index) (2.0.31)
Requirement already satisfied: aiohttp<4.0.0,>=3.8.6 in /usr/local/lib/python3.10/dist-packages (from llama-index-core==0.10.58->llama-index) (3.9.5)
Requirement already satisfied: dataclasses-json in /usr/local/lib/python3.10/dist-packages (from llama-index-core==0.10.58->llama-index) (0.6.7)
Requirement already satisfied: deprecated>=1.2.9.3 in /usr/local/lib/python3.10/dist-packages (from llama-index-core==0.10.58->llama-index) (1.2.14)
Requirement already satisfied: dirtyjson<2.0.0,>=1.0.8 in /usr/local/lib/python3.10/dist-packages (from llama-index-core==0.10.58->llama-index) (1.0.8)
Requirement already satisfied: fsspec>=2023.5.0 in /usr/local/lib/python3.10/dist-packages (from llama-index-core==0.10.58->llama-index) (2023.6.0)
Requirement already satisfied: httpx in /usr/local/lib/python3.10/dist-packages (from llama-index-core==0.10.58->llama-index) (0.27.0)
Requirement already satisfied: nest-asyncio<2.0.0,>=1.5.8 in /usr/local/lib/python3.10/dist-packages (from llama-index-core==0.10.58->llama-index) (1.6.0)
Requirement already satisfied: networkx>=3.0 in /usr/local/lib/python3.10/dist-packages (from llama-index-core==0.10.58->llama-index) (3.3)
Requirement already satisfied: nltk<4.0.0,>=3.8.1 in /usr/local/lib/python3.10/dist-packages (from llama-index-core==0.10.58->llama-index) (3.8.1)
Requirement already satisfied: numpy<2.0.0 in /usr/local/lib/python3.10/dist-packages (from llama-index-core==0.10.58->llama-index) (1.25.2)
Requirement already satisfied: openai>=1.1.0 in /usr/local/lib/python3.10/dist-packages (from llama-index-core==0.10.58->llama-index) (1.37.1)
Requirement already satisfied: pandas in /usr/local/lib/python3.10/dist-packages (from llama-index-core==0.10.58->llama-index) (2.0.3)
Requirement already satisfied: pillow>=9.0.0 in /usr/local/lib/python3.10/dist-packages (from llama-index-core==0.10.58->llama-index) (9.4.0)
Requirement already satisfied: requests>=2.31.0 in /usr/local/lib/python3.10/dist-packages (from llama-index-core==0.10.58->llama-index) (2.31.0)
Requirement already satisfied: tenacity!=8.4.0,<9.0.0,>=8.2.0 in /usr/local/lib/python3.10/dist-packages (from llama-index-core==0.10.58->llama-index) (8.5.0)
```

--------------------------------

### Run Finetuning with Docker

Source: https://github.com/01-ai/yi/blob/main/finetune/README.md

Mount your custom dataset and model save path to the container to replace defaults and run the finetuning script.

```bash
docker run -it \
    -v /path/to/save/finetuned/model/:/finetuned-model \
    -v /path/to/train.jsonl:/yi/finetune/data/train.json \
    -v /path/to/eval.jsonl:/yi/finetune/data/eval.json \
    ghcr.io/01-ai/yi:latest \
    bash finetune/scripts/run_sft_Yi_6b.sh
```

--------------------------------

### Install Python Dependencies for llama.cpp

Source: https://github.com/01-ai/yi/blob/main/docs/README_llama.cpp.md

Install the necessary Python packages required by llama.cpp for model conversion.

```bash
cd llama.cpp
python3 -m pip install -r requirements.txt
```

--------------------------------

### Start Yi Model Fine-tuning

Source: https://github.com/01-ai/yi/blob/main/Cookbook/cn/opensource/fine_tune/finetune-yi-with-llamafactory.md

Initiate the fine-tuning process for the Yi model using the LLaMA Factory CLI. This command points to the custom YAML configuration file created for training.

```bash
llamafactory-cli train examples/train_qlora/yi_lora_sft_bitsandbytes.yaml
```

--------------------------------

### Install Langchain OpenAI Integration

Source: https://github.com/01-ai/yi/blob/main/Cookbook/cn/api/rag/yi_rag_langchain.ipynb

Installs the Langchain OpenAI integration package, which is used here to connect to the Yi API.

```python
!pip install -qU langchain-openai
```

--------------------------------

### Download and Build llama.cpp

Source: https://github.com/01-ai/yi/blob/main/docs/README_llama.cpp.md

Clone the llama.cpp repository and build the library. This is a prerequisite for model conversion and quantization.

```bash
git clone https://github.com/ggerganov/llama.cpp
cd llama.cpp
make
```

--------------------------------

### Fine-tuning with Docker

Source: https://context7.com/01-ai/yi/llms.txt

Launches a Docker container for fine-tuning Yi models. Mounts directories for saving fine-tuned models and training data.

```bash
docker run -it --gpus all \
    -v /path/to/save/finetuned/model/:/finetuned-model \
    -v /path/to/train.jsonl:/yi/finetune/data/train.json \
    -v /path/to/eval.jsonl:/yi/finetune/data/eval.json \
    ghcr.io/01-ai/yi:latest \
    bash finetune/scripts/run_sft_Yi_6b.sh
```

--------------------------------

### Install Project Dependencies

Source: https://github.com/01-ai/yi/blob/main/Cookbook/en/ecosystem/Mastering_Street_Fighter_III_with_the_Yi_Language_Model.md

Install all the necessary Python packages listed in the 'requirements.txt' file. Ensure your Conda environment is activated.

```bash
pip install -r requirements.txt
```

--------------------------------

### Initialize ReAct Agent

Source: https://github.com/01-ai/yi/blob/main/Cookbook/cn/api/function_calling/function_calling_llamaindex.ipynb

Create a ReAct agent using the defined tools and the initialized Yi LLM. The 'verbose=True' option provides detailed logs of the agent's thought process and actions.

```python
agent = ReActAgent.from_tools([multiply_tool,plus_tool,minus_tool], llm=llm, verbose=True)
```

--------------------------------

### Install AutoAWQ via Pip

Source: https://github.com/01-ai/yi/blob/main/Cookbook/en/opensource/quantization/autoawq-yi-quantization.md

Install the AutoAWQ package using pip. This method requires CUDA version 12.1 or higher.

```shell
pip install autoawq
```

--------------------------------

### Install MLX-LM

Source: https://github.com/01-ai/yi/blob/main/Cookbook/en/opensource/local/local-mlx.md

Install the MLX-LM package using pip. This is the first step to enable local LLM deployment on Mac OS.

```bash
pip install mlx-lm
```