### Install Dependencies and Start Docs

Source: https://github.com/tabbyml/tabby/blob/main/CONTRIBUTING.md

Commands to install Node.js LTS and Yarn, then navigate to the website directory, install dependencies, and start the development server for the docs.

```bash
cd website
yarn install
yarn start
```

--------------------------------

### Install Client as Wheel

Source: https://github.com/tabbyml/tabby/blob/main/experimental/eval/tabby-python-client/README.md

Steps to build a wheel file for the client and install it in a project not using Poetry.

```bash
poetry build -f wheel
```

```bash
pip install <path-to-wheel>
```

--------------------------------

### Install Dependencies and Build Project

Source: https://github.com/tabbyml/tabby/blob/main/clients/vscode/CONTRIBUTING.md

Run these commands to install project dependencies and build the project.

```bash
pnpm install
pnpm build
```

--------------------------------

### Install and Run Tabby via Homebrew

Source: https://github.com/tabbyml/tabby/blob/main/website/docs/quick-start/installation/apple.md

Use this command to install the Tabby package using Homebrew. The subsequent command starts the Tabby server, utilizing the Metal framework for acceleration on Apple Silicon and specifying models for chat and general use.

```bash
brew install tabbyml/tabby/tabby

# Start server with StarCoder-1B
tabby serve --device metal --model StarCoder-1B --chat-model Qwen2-1.5B-Instruct
```

--------------------------------

### Setup Docker Image with Tabby and Model Weights

Source: https://github.com/tabbyml/tabby/blob/main/website/docs/references/cloud-deployment/bentoml/index.md

A shell script to install Tabby and download model weights. It downloads a specific Tabby distribution, unzips it, and then uses `su bentoml` to download models into the specified cache directory, ensuring they are owned by the correct user.

```bash
# Install Tabby
DISTRO=tabby_x86_64-manylinux2014-cuda117
curl -L https://github.com/TabbyML/tabby/releases/download/v0.14.0/$DISTRO.zip \
  -o $DISTRO.zip
unzip $DISTRO.zip

# Download model weights under the bentoml user, as BentoCloud operates under this user.
su bentoml -c "TABBY_MODEL_CACHE_ROOT=/home/bentoml/tabby-models tabby download --model StarCoder-1B"
su bentoml -c "TABBY_MODEL_CACHE_ROOT=/home/bentoml/tabby-models tabby download --model Qwen2-1.5B-Instruct"
su bentoml -c "TABBY_MODEL_CACHE_ROOT=/home/bentoml/tabby-models tabby download --model Nomic-Embed-Text"
```

--------------------------------

### Configure Tabby Plugin with Lazy.nvim

Source: https://github.com/tabbyml/tabby/blob/main/clients/vim/README.md

Example setup for the Tabby plugin using Lazy.nvim in Neovim. It configures the agent start command and inline completion trigger.

```lua
require("lazy").setup({
  -- other plugins
  -- ...
  -- Tabby plugin
  {
    "TabbyML/vim-tabby",
    lazy = false,
    dependencies = {
      "neovim/nvim-lspconfig",
    },
    init = function()
      vim.g.tabby_agent_start_command = {"npx", "tabby-agent", "--stdio"}
      vim.g.tabby_inline_completion_trigger = "auto"
    end,
  },
})

```

--------------------------------

### Install Dependencies

Source: https://github.com/tabbyml/tabby/blob/main/experimental/prompt-rewrite-eval/README.md

Installs the necessary Python packages for the evaluation tool. Ensure you have a `requirements.txt` file in your project.

```bash
pip install -r requirements.txt
```

--------------------------------

### Install SQLX CLI

Source: https://github.com/tabbyml/tabby/blob/main/ee/tabby-db/docs/sqlx.md

Install the SQLX command-line interface as a cargo subcommand for managing database migrations and operations.

```bash
cargo install sqlx-cli
```

--------------------------------

### Install Protobuf Compiler

Source: https://github.com/tabbyml/tabby/blob/main/CONTRIBUTING.md

Install the protobuf compiler and development libraries. Use Homebrew on macOS, apt-get on Debian/Ubuntu, or Chocolatey on Windows.

```bash
# For MacOS
brew install protobuf
```

```bash
# For Ubuntu / Debian
apt-get install protobuf-compiler libopenblas-dev
```

```bash
# For Windows 11 with Chocolatey package manager
choco install protoc
```

--------------------------------

### Install Build Tools for Ubuntu

Source: https://github.com/tabbyml/tabby/blob/main/README.md

Install essential build tools like `make`, `sqlite3`, and `graphviz` on Ubuntu systems. These are useful for development and building Tabby.

```bash
# For Ubuntu
apt install make sqlite3 graphviz
```

--------------------------------

### Start Tabby UI Development Frontend Server

Source: https://github.com/tabbyml/tabby/blob/main/ee/tabby-ui/README.md

Navigate to the tabby-ui directory and run this command to start the frontend development server. This enables hot-reloading for UI changes.

```bash
cd tabby/ee/tabby-ui
pnpm dev
```

--------------------------------

### Install Protobuf and OpenBLAS for Ubuntu/Debian

Source: https://github.com/tabbyml/tabby/blob/main/README.md

Install the protobuf compiler and OpenBLAS development libraries on Ubuntu or Debian systems. These are required dependencies for building Tabby.

```bash
# For Ubuntu
apt install protobuf-compiler libopenblas-dev
```

--------------------------------

### File Content Response Example (Rust)

Source: https://github.com/tabbyml/tabby/blob/main/ee/tabby-webserver/docs/api_spec.md

Example response when retrieving a Rust source file, with content type `text/x-rust`.

```text
//! Defines behavior for the tabby webserver which allows users to interact with enterprise features.
//! Using the web interface (e.g chat playground) requires using this module with the `--webserver` flag on the command line.
mod axum;
mod hub;
mod jwt;
mod oauth;
mod path;
mod routes;
mod service;
mod webserver;

#[cfg(test)]
pub use service::*;

pub mod public {

    pub use super::{/* used by tabby workers (consumer of /hub api) */
        hub::{
            create_scheduler_client,
            create_worker_client,
            RegisterWorkerRequest,
            SchedulerClient,
            WorkerClient,
            WorkerKind,
        },
        webserver::Webserver,
    };
}
......omitted......
```

--------------------------------

### Install Tabby VSCode Extension

Source: https://github.com/tabbyml/tabby/blob/main/clients/vscode/README.md

Use this command in VSCode's Quick Open to install the Tabby extension from the marketplace.

```plaintext
ext install TabbyML.vscode-tabby
```

--------------------------------

### VSCode Extension Quick Install

Source: https://context7.com/tabbyml/tabby/llms.txt

Install the Tabby VSCode extension and connect to your Tabby server via the Command Palette.

```text
ext install TabbyML.vscode-tabby

Then connect via the Command Palette: `Tabby: Connect to Server...` and enter the server endpoint URL and token.
```

--------------------------------

### Install and Run Tabby Agent Globally

Source: https://github.com/tabbyml/tabby/blob/main/website/blog/2024-02-05-create-tabby-extension-with-language-server-protocol/index.md

Install the tabby-agent globally using npm and then run it as a language server. Press Ctrl+C to stop the agent.

```bash
# Install tabby-agent as a global package
npm install --global tabby-agent
# Run the agent as a language server
tabby-agent --lsp --stdio
# Press `Ctrl+C` to stop the agent
```

--------------------------------

### File Content Response Example (JSON)

Source: https://github.com/tabbyml/tabby/blob/main/ee/tabby-webserver/docs/api_spec.md

Example JSON response when retrieving a file with a `.json` extension.

```json
{
  "private": true,
  "workspaces": [
    "clients/tabby-agent",
    "clients/vscode",
    "clients/vim",
    "clients/intellij",
    "clients/example-vscode-lsp"
  ],
  "engines": {
    "node": ">=18"
  }
}
```

--------------------------------

### Install Protobuf Compiler for macOS

Source: https://github.com/tabbyml/tabby/blob/main/README.md

Install the protobuf compiler on macOS using Homebrew. This is a dependency for building Tabby.

```bash
# For MacOS
brew install protobuf
```

--------------------------------

### Start Caddy Server for Development

Source: https://github.com/tabbyml/tabby/blob/main/ee/tabby-ui/README.md

Execute this command to start the Caddy server, which is used to orchestrate Tabby-UI and the local Tabby backend. It handles request forwarding and simplifies configuration.

```bash
make caddy
```

--------------------------------

### Install Ubuntu on WSL2

Source: https://github.com/tabbyml/tabby/blob/main/website/docs/quick-start/installation/wsl.mdx

Use these commands to install Ubuntu 22.04 or Ubuntu 24.04 on your Windows system via WSL.

```powershell
wsl --install -d Ubuntu        # for Ubuntu 22.04
wsl --install -d Ubuntu-24.04  # for Ubuntu 24.04
```

--------------------------------

### OAuth Providers Response Example

Source: https://github.com/tabbyml/tabby/blob/main/ee/tabby-webserver/docs/api_spec.md

Example response listing the available OAuth providers. Currently, only 'github' is shown.

```json
["github"]
```

--------------------------------

### Define Modal Container Image with Tabby

Source: https://github.com/tabbyml/tabby/blob/main/website/docs/references/cloud-deployment/modal/index.md

Create a Modal image starting from a Tabby registry image. It overrides the entrypoint, runs functions to download specified models, and installs necessary libraries for interfacing with Modal's ASGI webserver.

```python
image = (
    Image.from_registry(
        IMAGE_NAME,
        add_python="3.11",
    )
    .dockerfile_commands("ENTRYPOINT []")
    .run_function(download_model, kwargs={"model_id": EMBEDDING_MODEL_ID})
    .run_function(download_model, kwargs={"model_id": CHAT_MODEL_ID})
    .run_function(download_model, kwargs={"model_id": MODEL_ID})
    .pip_install("asgi-proxy-lib")
)
```

--------------------------------

### Start Services with Docker Compose

Source: https://github.com/tabbyml/tabby/blob/main/website/blog/2024-03-26-tabby-with-replicas-behind-reverse-proxy/index.mdx

Starts the configured Tabby and Caddy services in detached mode using Docker Compose.

```bash
docker-compose up -d
```

--------------------------------

### Setup Database from Migrations

Source: https://github.com/tabbyml/tabby/blob/main/ee/tabby-db/docs/sqlx.md

Create a new database schema from existing migration files. This command removes the existing schema file and sets up a fresh one.

```bash
rm ee/tabby-db/schema.sqlite
cargo sqlx db setup --source ee/tabby-db/migrations
```

--------------------------------

### Navigate and Develop VSCode Extension

Source: https://github.com/tabbyml/tabby/blob/main/clients/vscode/CONTRIBUTING.md

Change directory to the VSCode extension and start it in development mode.

```bash
cd ./clients/vscode
pnpm vscode:dev
```

--------------------------------

### Start Tabby Agent Language Server

Source: https://github.com/tabbyml/tabby/blob/main/clients/tabby-agent/README.md

Run the tabby-agent as a language server using Node.js. This command starts the agent and enables communication over standard input/output.

```bash
npx tabby-agent --stdio
```

--------------------------------

### Run Tabby Server on Linux (GPU)

Source: https://github.com/tabbyml/tabby/blob/main/website/docs/quick-start/installation/linux/index.mdx

Starts the Tabby server for GPU-enabled environments. Replace $DEVICE with 'cuda' or 'vulkan'.

```bash
./tabby serve --model StarCoder-1B --chat-model Qwen2-1.5B-Instruct --device $DEVICE
```

--------------------------------

### FIM Prompt Template Example

Source: https://github.com/tabbyml/tabby/blob/main/MODEL_SPEC.md

An example of a prompt_template format used for FIM inference. It specifies placeholders for prefix, suffix, and middle tokens.

```text
<PRE>{prefix}<SUF>{suffix}<MID>
```

--------------------------------

### Directory Content Response Example

Source: https://github.com/tabbyml/tabby/blob/main/ee/tabby-webserver/docs/api_spec.md

Example JSON response for a directory content query, indicating the kind and basename of each entry.

```json
{
  "entries": [
    {
      "kind": "dir",
      "basename": "src"
    },
    {
      "kind": "file",
      "basename": "Cargo.toml"
    },
    {
      "kind": "file",
      "basename": ".gitignore"
    },
    {
      "kind": "file",
      "basename": "Cargo.lock"
    }
  ]
}
```

--------------------------------

### Run Tabby with Vulkan

Source: https://github.com/tabbyml/tabby/blob/main/CONTRIBUTING.md

Run Tabby locally using Vulkan for acceleration. Ensure Vulkan drivers are installed.

```bash
cargo run --features vulkan serve --model TabbyML/StarCoder-1B --device vulkan
```

--------------------------------

### Install tabby-agent npm package

Source: https://github.com/tabbyml/tabby/blob/main/clients/vim/README.md

Install the tabby-agent globally using npm. This is required for the LSP client to communicate with the Tabby server.

```sh
npm install --global tabby-agent
```

--------------------------------

### Install Katana on Linux

Source: https://github.com/tabbyml/tabby/blob/main/website/docs/quick-start/installation/linux/index.mdx

Installs the Katana web crawler, a dependency for the developer docs context provider. Ensure you have at least version 1.1.2.

```bash
curl -L https://github.com/projectdiscovery/katana/releases/download/v1.1.2/katana_1.1.2_linux_amd64.zip -o katana.zip
unzip katana.zip katana
sudo mv katana /usr/bin/
rm katana.zip
```

--------------------------------

### Start Tabby Development Backend Server

Source: https://github.com/tabbyml/tabby/blob/main/ee/tabby-ui/README.md

Run this command in your terminal to start the Tabby backend development server on port 8081. This is necessary for the frontend to communicate with the backend.

```bash
cargo run serve --port 8081
```

--------------------------------

### Develop VSCode Webview Extension

Source: https://github.com/tabbyml/tabby/blob/main/clients/vscode/CONTRIBUTING.md

Start the VSCode Webview in development mode with the extension loaded.

```bash
pnpm dev:browser
```

--------------------------------

### Clone Tabby Repository and Install Dependencies

Source: https://github.com/tabbyml/tabby/blob/main/clients/eclipse/README.md

Clone the Tabby repository, navigate to the Eclipse client directory, and install necessary dependencies using pnpm. This command also builds the tabby-agent.

```bash
git clone https://github.com/TabbyML/tabby.git
cd tabby
pnpm install
cd clients/eclipse
pnpm turbo build
```

--------------------------------

### Run All Development Servers Simultaneously

Source: https://github.com/tabbyml/tabby/blob/main/ee/tabby-ui/README.md

This command provides a shortcut to run the frontend, backend, and Caddy servers concurrently. It requires 'tmux' and 'tmuxinator' to be installed.

```bash
make dev
```

--------------------------------

### Run Tabby Server on Linux (CPU)

Source: https://github.com/tabbyml/tabby/blob/main/website/docs/quick-start/installation/linux/index.mdx

Starts the Tabby server for CPU-only environments. You can specify different models and chat models.

```bash
./tabby serve --model StarCoder-1B --chat-model Qwen2-1.5B-Instruct
```

--------------------------------

### Get Server Settings

Source: https://context7.com/tabbyml/tabby/llms.txt

Retrieve server-level configuration, such as telemetry settings. Requires authentication.

```bash
curl -s http://localhost:8080/v1beta/server_setting \
  -H "Authorization: Bearer YOUR_TOKEN" | jq .
```

--------------------------------

### Run Tabby with Metal (Apple Silicon)

Source: https://github.com/tabbyml/tabby/blob/main/CONTRIBUTING.md

Run Tabby locally with GPU acceleration using Metal on Apple M1/M2 chips. No extra library installation is typically needed.

```bash
cargo run serve --model TabbyML/StarCoder-1B --device metal
```

--------------------------------

### Tabby Scheduler Docker Output

Source: https://github.com/tabbyml/tabby/blob/main/website/blog/2024-04-08-connect-private-github-repository-to-tabby/index.md

Example output from the Tabby scheduler Docker command, showing repository synchronization and indexing progress.

```console
icy@Icys-Ubuntu:~$ docker run -it --entrypoint /opt/tabby/bin/tabby-cpu -v $HOME/.tabby:/data tabbyml/tabby scheduler --now
Syncing 1 repositories...
Cloning into '/data/repositories/my_private_project'...
remote: Enumerating objects: 51, done.
remote: Total 51 (delta 0), reused 0 (delta 0), pack-reused 51
Receiving objects: 100% (51/51), 7.16 KiB | 2.38 MiB/s, done.
Resolving deltas: 100% (18/18), done.
Building dataset...
100%|████████████████████████████████████████| 12/12 [00:00<00:00, 55.56it/s]
Indexing repositories...
100%|████████████████████████████████████████| 12/12 [00:00<00:00, 73737.70it/s]
```

--------------------------------

### Run Tabby with CUDA (NVIDIA)

Source: https://github.com/tabbyml/tabby/blob/main/CONTRIBUTING.md

Run Tabby locally with GPU acceleration using CUDA. Ensure NVIDIA drivers and libraries are installed.

```bash
cargo run --features cuda serve --model TabbyML/StarCoder-1B --device cuda
```

--------------------------------

### Tabby Server Docker Output

Source: https://github.com/tabbyml/tabby/blob/main/website/blog/2024-04-08-connect-private-github-repository-to-tabby/index.md

Example output indicating the successful launch of the Tabby server, including model loading and network listening information.

```console
icy@Icys-Ubuntu:~$ docker run -it --gpus all -p 8080:8080 -v $HOME/.tabby:/data tabbyml/tabby serve --model StarCoder-1B --device cuda
2024-03-21T16:16:47.189632Z  INFO tabby::serve: crates/tabby/src/serve.rs:118: Starting server, this might take a few minutes...
2024-03-21T16:16:47.190764Z  INFO tabby::services::code: crates/tabby/src/services/code.rs:53: Index is ready, enabling server...
ggml_init_cublas: GGML_CUDA_FORCE_MMQ:   no
ggml_init_cublas: CUDA_USE_TENSOR_CORES: yes
ggml_init_cublas: found 1 CUDA devices:
  Device 0: NVIDIA GeForce RTX 4090, compute capability 8.9, VMM: yes
2024-03-21T16:16:52.464116Z  INFO tabby::routes: crates/tabby/src/routes/mod.rs:35: Listening at 0.0.0.0:8080
```

--------------------------------

### Run Tabby with ROCm (AMD)

Source: https://github.com/tabbyml/tabby/blob/main/CONTRIBUTING.md

Run Tabby locally with GPU acceleration using ROCm. Ensure AMD drivers and libraries are installed.

```bash
cargo run --features rocm serve --model TabbyML/StarCoder-1B --device rocm
```

--------------------------------

### Incremental Decoding Example

Source: https://github.com/tabbyml/tabby/blob/main/website/blog/2023-10-21-incremental-decoding/index.md

Demonstrates the improved output achieved by incremental decoding, where the previously decoded prefix is considered for subsequent token decoding, resulting in a more coherent sequence.

```text
Incremental decoding:  ......, 207, 211  ->   "......[ hello]"  ✅
```

--------------------------------

### Python List Comprehension Example

Source: https://github.com/tabbyml/tabby/blob/main/website/blog/2023-10-21-incremental-decoding/index.md

Demonstrates the start of a Python list comprehension to extract even numbers. This serves as a base for illustrating different decoding methods.

```python
numbers = [1, 2, 3, 4, 5]
evens = [x for x in numbers

```

--------------------------------

### Full Example: LLM Streaming Server and Client

Source: https://github.com/tabbyml/tabby/blob/main/website/blog/2023-09-30-stream-laziness-in-tabby/index.md

Sets up an async generator to mimic an LLM, an HTTP endpoint to stream its output, and a client to consume the stream. Useful for demonstrating basic streaming behavior.

```javascript
const express = require('express');

function sleep(ms) {
  return new Promise(resolve => setTimeout(resolve, ms));
}

async function* llm() {
  let i = 1;
  while (true) {
    console.log(`producing ${i}`);
    yield i++;

    // Mimic LLM inference latency.
    await sleep(1000);
  }
}

function server(llm) {
  const app = express();
  app.get('/', async (req, res) => {
    res.writeHead(200, {
      'Content-Type': 'application/jsonstream',
      'Transfer-Encoding': 'chunked',
    });

    let value, done;
    do {
      ({ value, done } = await llm.next());
      res.write(JSON.stringify(value));
      res.write('\n');
    } while (!done);
  });

  app.listen(8080);
}

async function client() {
  const resp = await fetch('http://localhost:8080');

  // Read values from our stream
  const reader = resp.body.pipeThrough(new TextDecoderStream()).getReader();
  // We're only reading 3 items this time:
  for (let i = 0; i < 3; i++) {
    // we know our stream is infinite, so there's no need to check `done`.
    const { value } = await reader.read();
    console.log(`read ${value}`);
    await sleep(10ms);
  }
}

server(llm());
client();

```

--------------------------------

### Rust Code Completion Request Example

Source: https://github.com/tabbyml/tabby/blob/main/website/blog/2023-10-16-repository-context-for-code-completion/index.md

Illustrates a code snippet within a Rust function where LLM-based code completion is requested. The '║' marker indicates the position for completion.

```rust
fn api_router(args: &ServeArgs) -> Router {
    let index_server = Arc::new(IndexServer::new());
    let completion_state = {
        let (
            engine,
            EngineInfo {
                prompt_template, ..
            },
        ) = create_engine(&args.model, args);
        let engine = Arc::new(engine);
        let state = completions::CompletionState::new(
            ║
    }
}

```

--------------------------------

### Client Initialization

Source: https://github.com/tabbyml/tabby/blob/main/python/tabby-eval/modal/tabby_python_client/README.md

Demonstrates how to create a basic client instance to interact with Tabby Server.

```APIDOC
## Client Initialization

### Description
Instantiate the `Client` to make unauthenticated requests to the Tabby API.

### Method
```python
from tabby_python_client import Client

client = Client(base_url="https://api.example.com")
```
```

--------------------------------

### Check CUDA Version

Source: https://github.com/tabbyml/tabby/blob/main/website/docs/quick-start/installation/windows/index.mdx

Verify your local CUDA installation version. Ensure it is 11 or higher before proceeding with GPU-enabled installations.

```bash
nvcc --version
```

--------------------------------

### Serve Tabby with Vulkan on Windows

Source: https://github.com/tabbyml/tabby/blob/main/website/blog/2024-05-01-vulkan-support/index.md

Run the Tabby server with Vulkan acceleration on Windows. Ensure you have downloaded the Windows Vulkan binary and are in the download directory.

```bash
tabby_x86_64-windows-msvc-vulkan serve --model StarCoder-1B --device vulkan
```

--------------------------------

### Create a basic client

Source: https://github.com/tabbyml/tabby/blob/main/python/tabby-eval/modal/tabby_python_client/README.md

Instantiate the client to connect to the Tabby Server API. Ensure the base URL is correctly set.

```python
from tabby_python_client import Client

client = Client(base_url="https://api.example.com")
```

--------------------------------

### Add Client to Project with Poetry

Source: https://github.com/tabbyml/tabby/blob/main/experimental/eval/tabby-python-client/README.md

Instructions for adding the client library to another project that uses Poetry.

```bash
poetry add <path-to-this-client>
```

--------------------------------

### Run Tabby Server with Docker

Source: https://github.com/tabbyml/tabby/blob/main/README.md

Use this Docker command to quickly start a Tabby server. It maps ports, mounts a data volume, and specifies the model and device for inference. Refer to the documentation for advanced options.

```bash
docker run -it \
  --gpus all -p 8080:8080 -v $HOME/.tabby:/data \
  tabbyml/tabby \
  serve --model StarCoder-1B --device cuda --chat-model Qwen2-1.5B-Instruct
```

--------------------------------

### Serve Tabby with Vulkan on Linux

Source: https://github.com/tabbyml/tabby/blob/main/website/blog/2024-05-01-vulkan-support/index.md

Execute the Tabby server with Vulkan acceleration on Linux. Make sure to use the correct Linux Vulkan binary and navigate to its directory.

```bash
./tabby_x86_64-manylinux2014-vulkan serve --model StarCoder-1B --device vulkan
```

--------------------------------

### HumanEval Example: Below Zero Function

Source: https://github.com/tabbyml/tabby/blob/main/website/blog/2023-11-13-model-evaluation/index.md

An example task from the HumanEval dataset, requiring the implementation of a Python function to detect if a bank account balance falls below zero.

```python
from typing import List 

def below_zero(operations: List[int]) -> bool: 
    
    """ 
    You're given a list of deposit and withdrawal operations on a bank account that starts with zero balance. Your task is to detect if at any point the balance of account fallls below zero, and at that point function should return True. Otherwise it should return False.
    
     >>> below_zero([1, 2, 3]) False 
     
     >>> below_zero([1, 2, -4, 5]) True 
     
    """

```

--------------------------------

### Run Tabby Server with Docker

Source: https://github.com/tabbyml/tabby/blob/main/website/blog/2024-04-08-connect-private-github-repository-to-tabby/index.md

Command to launch the Tabby server, enabling code completion with GPU acceleration. Ensure Docker is configured with CUDA drivers.

```bash
docker run -it --gpus all -p 8080:8080 -v $HOME/.tabby:/data tabbyml/tabby serve --model StarCoder-1B --device cuda
```

--------------------------------

### Run Tabby with CPU Support

Source: https://github.com/tabbyml/tabby/blob/main/website/docs/quick-start/installation/docker.mdx

Launches Tabby using the CPU-specific entrypoint, mapping port 8080, and mounting a local data directory.

```bash
docker run --entrypoint /opt/tabby/bin/tabby-cpu -it \
  -p 8080:8080 -v $HOME/.tabby:/data \
  registry.tabbyml.com/tabbyml/tabby \
  serve --model StarCoder-1B --chat-model Qwen2-1.5B-Instruct
```

--------------------------------

### MBPP Example: Remove Character Occurrences

Source: https://github.com/tabbyml/tabby/blob/main/website/blog/2023-11-13-model-evaluation/index.md

An example task from the MBPP dataset, demonstrating a Python function to remove the first and last occurrences of a given character from a string.

```python
   """
   Write a python function to remove first and last occurrence of a given character from the string.

   "assert remove_Occ("hello","l") == "heo""
   "assert remove_Occ("abcda","a") == "bcd""
   "assert remove_Occ("PHP","P") == "H"" 
   
   """

```

--------------------------------

### Configure and Run Tabby Server on Modal

Source: https://github.com/tabbyml/tabby/blob/main/website/docs/references/cloud-deployment/modal/index.md

Sets up the Modal application function to serve the Tabby server. It configures GPU resources, concurrent input handling, idle timeouts, and uses an ASGI proxy to tunnel requests to the local Tabby server. Includes logic to wait for the Tabby server to be ready.

```python
app = App("tabby-server", image=image)


@app.function(
    gpu=GPU_CONFIG,
    allow_concurrent_inputs=10,
    container_idle_timeout=120,
    timeout=360,
)
@asgi_app()
def app_serve():
    import socket
    import subprocess
    import time
    from asgi_proxy import asgi_proxy

    launcher = subprocess.Popen(
        [
            TABBY_BIN,
            "serve",
            "--model",
            MODEL_ID,
            "--chat-model",
            CHAT_MODEL_ID,
            "--port",
            "8000",
            "--device",
            "cuda",
            "--parallelism",
            "1",
        ]
    )

    # Poll until webserver at 127.0.0.1:8000 accepts connections before running inputs.
    def tabby_ready():
        try:
            socket.create_connection(("127.0.0.1", 8000), timeout=1).close()
            return True
        except (socket.timeout, ConnectionRefusedError):
            # Check if launcher webserving process has exited.
            # If so, a connection can never be made.
            retcode = launcher.poll()
            if retcode is not None:
                raise RuntimeError(f"launcher exited unexpectedly with code {retcode}")
            return False

    while not tabby_ready():
        time.sleep(1.0)

    print("Tabby server ready!")
    return asgi_proxy("http://localhost:8000")
```

--------------------------------

### Import Modal Components

Source: https://github.com/tabbyml/tabby/blob/main/website/docs/references/cloud-deployment/modal/index.md

Import the essential components from the modal library for setting up the application.

```python
from modal import Image, App, asgi_app, gpu
```

--------------------------------

### Verify Tabby Setup with Curl

Source: https://github.com/tabbyml/tabby/blob/main/website/blog/2024-03-26-tabby-with-replicas-behind-reverse-proxy/index.mdx

Sends a request to the Tabby health endpoint via the Caddy reverse proxy to verify the setup. Checks if Tabby is healthy and ready.

```bash
curl -L 'http://localhost:8080/v1/completions' \
-H 'Content-Type: application/json' \
-H 'Accept: application/json' \
-d '{
  "language": "python",
  "segments": {
    "prefix": "def fib(n):\n    ",
    "suffix": "\n        return fib(n - 1) + fib(n - 2)"
  }
}'
```

--------------------------------

### Initialize Submodules

Source: https://github.com/tabbyml/tabby/blob/main/CONTRIBUTING.md

If the repository has already been cloned, initialize or update its submodules.

```bash
git submodule update --recursive --init
```

--------------------------------

### Get Server Settings

Source: https://context7.com/tabbyml/tabby/llms.txt

Retrieves server-level configuration, such as whether client-side telemetry is disabled.

```APIDOC
## GET /v1beta/server_setting — Server Settings

### Description
Returns server-level configuration such as whether client-side telemetry is disabled.

### Method
GET

### Endpoint
/v1beta/server_setting

### Response
#### Success Response (200)
- **disable_client_side_telemetry** (boolean) - Indicates if client-side telemetry is disabled.

### Request Example
```bash
curl -s http://localhost:8080/v1beta/server_setting \
  -H "Authorization: Bearer YOUR_TOKEN" | jq .
```

### Response Example
```json
{
  "disable_client_side_telemetry": false
}
```
```

--------------------------------

### Run Tabby Server on Windows

Source: https://github.com/tabbyml/tabby/blob/main/website/docs/quick-start/installation/windows/index.mdx

Execute the Tabby server from the command line. Use the appropriate command based on whether your environment is CPU-only or CUDA-enabled.

```bash
.\tabby.exe serve --model StarCoder-1B --chat-model Qwen2-1.5B-Instruct
```

```bash
.\tabby.exe serve --model StarCoder-1B --chat-model Qwen2-1.5B-Instruct --device cuda
```

--------------------------------

### OAuth Callback Error Redirect

Source: https://github.com/tabbyml/tabby/blob/main/ee/tabby-webserver/docs/api_spec.md

Example of a redirect after an OAuth callback failure, including an `error_msg` and the `provider`.

```http
http://localhost:8080/auth/signin?error_msg=...&provider=github
```

--------------------------------

### OAuth Callback Success Redirect

Source: https://github.com/tabbyml/tabby/blob/main/ee/tabby-webserver/docs/api_spec.md

Example of a successful redirect after OAuth callback, with `refresh_token` and `access_token` appended.

```http
http://localhost:8080/auth/signin?refresh_token=321bc1bbb043456dae1a7abc0c447875&access_token=eyJ0eXAi......1NiJ9.eyJleHAi......bWluIjp0cnVlfQ.GvHSMUfc...S5BnwY
```

--------------------------------

### Initiate OAuth Sign-in

Source: https://github.com/tabbyml/tabby/blob/main/ee/tabby-webserver/docs/api_spec.md

Redirects the user to the specified OAuth provider for authentication. The `provider` query parameter determines which service to use.

```shell
curl --request GET \
  --url http://localhost:8080/oauth/signin?provider=google
```

--------------------------------

### Docker Compose for CUDA

Source: https://github.com/tabbyml/tabby/blob/main/website/docs/quick-start/installation/docker-compose.mdx

Use this configuration to run Tabby with CUDA support. Ensure the NVIDIA Container Toolkit is installed.

```yaml
version: '3.5'

services:
  tabby:
    restart: always
    image: registry.tabbyml.com/tabbyml/tabby
    command: serve --model StarCoder-1B --chat-model Qwen2-1.5B-Instruct --device cuda
    volumes:
      - "$HOME/.tabby:/data"
    ports:
      - 8080:8080
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: 1
              capabilities: [gpu]
```

--------------------------------

### Build and Publish Client with Poetry

Source: https://github.com/tabbyml/tabby/blob/main/experimental/eval/tabby-python-client/README.md

Commands for building and publishing the client library using Poetry. This includes updating metadata and configuring repositories.

```bash
poetry publish --build -r <your-repository-name>
```

```bash
poetry publish --build
```

--------------------------------

### Go Type Definition Snippet

Source: https://github.com/tabbyml/tabby/blob/main/website/blog/2023-10-16-repository-context-for-code-completion/index.md

An example of a Go type definition that would be captured by the Tree-sitter query for use in the repository index.

```go
type payload struct {
	Data string `json:"data"`
}
...

```

--------------------------------

### Authenticated Client Initialization

Source: https://github.com/tabbyml/tabby/blob/main/python/tabby-eval/modal/tabby_python_client/README.md

Shows how to initialize an `AuthenticatedClient` for making requests that require authentication.

```APIDOC
## Authenticated Client Initialization

### Description
Use `AuthenticatedClient` when your API calls require an authentication token.

### Method
```python
from tabby_python_client import AuthenticatedClient

client = AuthenticatedClient(base_url="https://api.example.com", token="SuperSecretToken")
```
```

--------------------------------

### Independent Token Decoding Example

Source: https://github.com/tabbyml/tabby/blob/main/website/blog/2023-10-21-incremental-decoding/index.md

Illustrates how decoding tokens independently can lead to undesirable results, such as awkward spacing in the final output.

```text
Decoding first token:                ......, 211       ->   "......[ llo]"
Indepently decoding the next token:  ......, 207, 211  ->   "......[ he][ llo]"
```

--------------------------------

### Get File Content from Repository

Source: https://github.com/tabbyml/tabby/blob/main/ee/tabby-webserver/docs/api_spec.md

Retrieve the content of a specific file from a local repository. The `Content-Type` of the response is determined by the file extension.

```shell
curl --request GET \
  --url http://localhost:8080/repositories/git/lNrAwW/resolve/package.json \

```

```shell
curl --request GET \
  --url http://localhost:8080/repositories/git/lNrAwW/resolve/src/lib.rs \

```

--------------------------------

### Inference Benchmarking with StarCoder-1B on M2 Max

Source: https://github.com/tabbyml/tabby/blob/main/website/blog/2023-09-18-release-0-1-1-metal/index.md

Example output from inference benchmarking showing load, sample, prompt eval, and total evaluation times. This demonstrates a significant speed improvement with Metal support.

```text
llama_print_timings:        load time =   105.15 ms
lama_print_timings:      sample time =     0.00 ms /     1 runs   (    0.00 ms per token,      inf tokens per second)
lama_print_timings: prompt eval time =    25.07 ms /     6 tokens (    4.18 ms per token,   239.36 tokens per second)
lama_print_timings:        eval time =   311.80 ms /    28 runs   (   11.14 ms per token,    89.80 tokens per second)
lama_print_timings:       total time =   340.25 ms
```