Athena's Engine

https://github.com/xangel82/athena-engine
Admin
Athena's Engine is an inference engine designed to run frontier-class large language models locally...
Tokens:12,565
Snippets:82
Trust Score:6.4
Update:5 days ago
Show doc for...
Tokens:
Raw
### Start the engine using environment variables

Source: https://github.com/xangel82/athena-engine/blob/main/README.md

Initialize the environment from the installer-generated file and start the engine.

```bash
source models/athena.env
./athena-engine/athena-engine.sh
```

--------------------------------

### Start Qwen3.8 Flash Next

Source: https://github.com/xangel82/athena-engine/blob/main/README.md

Manually configure and start the engine for Qwen3.8 Flash Next.

```bash
export ATHENA_MODEL=/models/UD-IQ4_XS/Qwen3.8-Flash-Next-UD-IQ4_XS-00001-of-00003.gguf
export ATHENA_DRAFT=/models/MTP/mtp-Qwen3.8-Flash-Next-shared-Q8_0.gguf
export ATHENA_MMPROJ=/models/MMPROJ/mmproj-F16.gguf
./athena-engine/athena-engine.sh
```

--------------------------------

### Install models from existing directory

Source: https://github.com/xangel82/athena-engine/blob/main/README.md

Point the installer at an existing directory to use local files instead of downloading them.

```bash
./athena-engine/install.sh --model qwen --from /mnt/ggufs
```

--------------------------------

### Install model files

Source: https://github.com/xangel82/athena-engine/blob/main/README.md

Use the installer script to download specific models or sets of models into the local environment.

```bash
./athena-engine/install.sh                 # asks which model, downloads into ./models
./athena-engine/install.sh --model qwen --dir /data/models
./athena-engine/install.sh --model both    # Qwen and DeepSeek, so a chat can switch between them
./athena-engine/install.sh --model all     # Qwen, DeepSeek and DeepSeek Vision-Exp
./athena-engine/install.sh --model deepseek,visio   # any list; the first is the one that starts
./athena-engine/install.sh --model qwen,deepseek4.1 # DeepSeek V4.1 Flash comes only when named
```

--------------------------------

### Start DeepSeek V4.1 Flash

Source: https://github.com/xangel82/athena-engine/blob/main/README.md

Manually configure and start the engine for DeepSeek V4.1 Flash.

```bash
export ATHENA_MODEL=/models/DeepSeek-V4.1-Flash-Q2.gguf
./athena-engine/athena-engine.sh
```

--------------------------------

### Start DeepSeek V4 Flash

Source: https://github.com/xangel82/athena-engine/blob/main/README.md

Manually configure and start the engine for DeepSeek V4 Flash.

```bash
export ATHENA_MODEL=/models/DeepSeek-V4-Flash-IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8-chat-v2-imatrix-0731.gguf
export ATHENA_DRAFT=/models/DeepSeek-V4-Flash-DSpark-IQ2XXS-Q2K-Q8.gguf
./athena-engine/athena-engine.sh
```

--------------------------------

### Start DeepSeek V4 Flash Vision-Exp

Source: https://github.com/xangel82/athena-engine/blob/main/README.md

Manually configure and start the engine for DeepSeek V4 Flash Vision-Exp.

```bash
export ATHENA_MODEL=/models/DeepSeek-V4-Flash-Vision-Exp-IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8.gguf
export ATHENA_DRAFT=/models/DeepSeek-V4-Flash-Vision-Exp-DSpark-support.gguf
export ATHENA_MMPROJ=/models/DeepSeek-V4-Flash-Vision-Encoder.gguf
./athena-engine/athena-engine.sh
```

--------------------------------

### Manual Multi-Model Configuration

Source: https://github.com/xangel82/athena-engine/blob/main/README.md

Manually configure multiple models and draft files by setting environment variables for each model index before starting the engine.

```bash
export ATHENA_MODEL=/models/UD-IQ4_XS/Qwen3.8-Flash-Next-UD-IQ4_XS-00001-of-00003.gguf
export ATHENA_DRAFT=/models/MTP/mtp-Qwen3.8-Flash-Next-shared-Q8_0.gguf
export ATHENA_MMPROJ=/models/MMPROJ/mmproj-F16.gguf
export ATHENA_MODEL_2=/models/DeepSeek-V4-Flash-IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8-chat-v2-imatrix-0731.gguf
export ATHENA_DRAFT_2=/models/DeepSeek-V4-Flash-DSpark-IQ2XXS-Q2K-Q8.gguf
export ATHENA_MODEL_3=/models/DeepSeek-V4-Flash-Vision-Exp-IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8.gguf
export ATHENA_DRAFT_3=/models/DeepSeek-V4-Flash-DSpark-support.gguf
export ATHENA_MMPROJ_3=/models/DeepSeek-V4-Flash-Vision-Encoder.gguf
export ATHENA_MODEL_4=/models/DeepSeek-V4.1-Flash-Q2.gguf
./athena-engine/athena-engine.sh
```

--------------------------------

### Start Athena Engine with Docker Compose

Source: https://github.com/xangel82/athena-engine/blob/main/README.md

Command to launch the engine container with model files mounted from the host.

```bash
ATHENA_MODELS_DIR=/path/to/models docker compose -f docker/docker-compose.yml up -d
```

--------------------------------

### GET /v1/models

Source: https://github.com/xangel82/athena-engine/blob/main/README.md

Lists available models. Compatible with both OpenAI and Anthropic protocols.

```APIDOC
## GET /v1/models

### Description
Lists available models. Compatible with both OpenAI and Anthropic protocols.

### Method
GET

### Endpoint
/v1/models

### Response
#### Success Response (200)
- **object** (string) - The object type, e.g., 'list'.
- **data** (array) - List of model objects.

#### Response Example
{
  "object": "list",
  "data": [
    {
      "id": "Athena",
      "object": "model",
      "created": 1677652288,
      "owned_by": "athena-engine"
    }
  ]
}
```

--------------------------------

### GET /health

Source: https://github.com/xangel82/athena-engine/blob/main/README.md

Readiness probe endpoint to check if the engine is ready to serve requests.

```APIDOC
## GET /health

### Description
Readiness probe endpoint to check if the engine is ready to serve requests.

### Method
GET

### Endpoint
/health

### Response
#### Success Response (200)
No response body is documented.
```

--------------------------------

### GET /v1/models/{id}

Source: https://github.com/xangel82/athena-engine/blob/main/README.md

Retrieves a specific model by ID. Compatible with both OpenAI and Anthropic protocols.

```APIDOC
## GET /v1/models/{id}

### Description
Retrieves a specific model by ID. Compatible with both OpenAI and Anthropic protocols.

### Method
GET

### Endpoint
/v1/models/{id}

### Parameters
#### Path Parameters
- **id** (string) - Required - The model ID to retrieve.

### Response
#### Success Response (200)
- **id** (string) - The model ID.
- **object** (string) - The object type, e.g., 'model'.
- **created** (integer) - Unix timestamp of creation.
- **owned_by** (string) - The owner of the model.

#### Response Example
{
  "id": "Athena",
  "object": "model",
  "created": 1677652288,
  "owned_by": "athena-engine"
}
```

--------------------------------

### Switch Models via Command

Source: https://github.com/xangel82/athena-engine/blob/main/README.md

Use the %switch command to change the active model. The engine provides streaming feedback during the loading process.

```text
%switch visio
```

```text
Switching to visio.
Flushing checkpoints still being written.
Releasing qwen.
Memory returned: 26.6 GiB (available now 118.3 GiB).
visio needs 87.2 GiB on disk; 118.3 GiB available.
Loading visio.
Ready in 45.1 s.
```

--------------------------------

### POST /v1/completions

Source: https://github.com/xangel82/athena-engine/blob/main/README.md

Creates a completion. Compatible with OpenAI Completions API.

```APIDOC
## POST /v1/completions

### Description
Creates a completion. Compatible with OpenAI Completions API.

### Method
POST

### Endpoint
/v1/completions

### Parameters
#### Request Body
- **model** (string) - Required - The model name to use. Defaults to 'Athena' unless configured otherwise.
- **prompt** (string or array) - Required - The prompt to generate completions for.
- **stream** (boolean) - Optional - Whether to stream the response.

### Response
#### Success Response (200)
- **id** (string) - The completion ID.
- **object** (string) - The object type, e.g., 'text_completion'.
- **created** (integer) - Unix timestamp of creation.
- **model** (string) - The model used.
- **choices** (array) - List of completion choices.
- **usage** (object) - Token usage information.

#### Response Example
{
  "id": "cmpl-123",
  "object": "text_completion",
  "created": 1677652288,
  "model": "Athena",
  "choices": [
    {
      "text": "Hello!",
      "index": 0,
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 9,
    "completion_tokens": 12,
    "total_tokens": 21
  }
}
```

--------------------------------

### POST /v1/chat/completions

Source: https://github.com/xangel82/athena-engine/blob/main/README.md

Creates a chat completion. Compatible with OpenAI Chat Completions API, supporting streaming and tool calls.

```APIDOC
## POST /v1/chat/completions

### Description
Creates a chat completion. Compatible with OpenAI Chat Completions API, supporting streaming and tool calls.

### Method
POST

### Endpoint
/v1/chat/completions

### Parameters
#### Request Body
- **model** (string) - Required - The model name to use. Defaults to 'Athena' unless configured otherwise.
- **messages** (array) - Required - A list of chat messages.
- **stream** (boolean) - Optional - Whether to stream the response.
- **tools** (array) - Optional - List of tools for tool calls.

### Response
#### Success Response (200)
- **id** (string) - The completion ID.
- **object** (string) - The object type, e.g., 'chat.completion'.
- **created** (integer) - Unix timestamp of creation.
- **model** (string) - The model used.
- **choices** (array) - List of completion choices.
- **usage** (object) - Token usage information.

#### Response Example
{
  "id": "chatcmpl-123",
  "object": "chat.completion",
  "created": 1677652288,
  "model": "Athena",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "Hello!"
      },
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 9,
    "completion_tokens": 12,
    "total_tokens": 21
  }
}
```

--------------------------------

### POST /v1/responses

Source: https://github.com/xangel82/athena-engine/blob/main/README.md

Creates a response. Compatible with OpenAI Responses API.

```APIDOC
## POST /v1/responses

### Description
Creates a response. Compatible with OpenAI Responses API.

### Method
POST

### Endpoint
/v1/responses

### Parameters
#### Request Body
- **model** (string) - Required - The model name to use. Defaults to 'Athena' unless configured otherwise.
- **input** (string or array) - Required - The input to the model.
- **stream** (boolean) - Optional - Whether to stream the response.

### Response
#### Success Response (200)
- **id** (string) - The response ID.
- **object** (string) - The object type, e.g., 'response'.
- **created** (integer) - Unix timestamp of creation.
- **model** (string) - The model used.
- **output** (array) - List of output items.
- **usage** (object) - Token usage information.

#### Response Example
{
  "id": "resp_123",
  "object": "response",
  "created": 1677652288,
  "model": "Athena",
  "output": [
    {
      "type": "message",
      "role": "assistant",
      "content": [
        {
          "type": "output_text",
          "text": "Hello!"
        }
      ]
    }
  ],
  "usage": {
    "input_tokens": 9,
    "output_tokens": 12,
    "total_tokens": 21
  }
}
```

--------------------------------

### Configure Image Encoders

Source: https://github.com/xangel82/athena-engine/blob/main/README.md

Set the ATHENA_MMPROJ environment variable to point to the appropriate GGUF encoder file for your model.

```bash
export ATHENA_MMPROJ=/models/MMPROJ/mmproj-F16.gguf                     # Qwen3.8 Flash Next
export ATHENA_MMPROJ=/models/DeepSeek-V4-Flash-Vision-Encoder.gguf      # Vision-Exp
```

--------------------------------

### Pull and extract the Athena Engine

Source: https://github.com/xangel82/athena-engine/blob/main/README.md

Commands to pull the engine container image and extract the engine files to the local filesystem.

```bash
docker pull ghcr.io/xangel82/athena-engine:0.3.4
docker create --name athena-copy ghcr.io/xangel82/athena-engine:0.3.4
docker cp athena-copy:/opt/athena-engine ./athena-engine
docker rm athena-copy
```

--------------------------------

### Send a chat request

Source: https://github.com/xangel82/athena-engine/blob/main/README.md

Send a JSON request to the running engine via curl.

```bash
curl http://localhost:30007/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{"model": "Athena", "messages": [{"role": "user", "content": "Hello!"}]}'
```

--------------------------------

### POST /v1/messages

Source: https://github.com/xangel82/athena-engine/blob/main/README.md

Creates a message. Compatible with Anthropic Messages API.

```APIDOC
## POST /v1/messages

### Description
Creates a message. Compatible with Anthropic Messages API.

### Method
POST

### Endpoint
/v1/messages

### Parameters
#### Request Body
- **model** (string) - Required - The model name to use. Defaults to 'Athena' unless configured otherwise.
- **messages** (array) - Required - A list of messages.
- **max_tokens** (integer) - Required - The maximum number of tokens to generate.
- **stream** (boolean) - Optional - Whether to stream the response.

### Response
#### Success Response (200)
- **id** (string) - The message ID.
- **type** (string) - The object type, e.g., 'message'.
- **role** (string) - The role of the message.
- **content** (array) - List of content blocks.
- **model** (string) - The model used.
- **stop_reason** (string) - The reason for stopping.
- **usage** (object) - Token usage information.

#### Response Example
{
  "id": "msg_123",
  "type": "message",
  "role": "assistant",
  "content": [
    {
      "type": "text",
      "text": "Hello!"
    }
  ],
  "model": "Athena",
  "stop_reason": "end_turn",
  "usage": {
    "input_tokens": 9,
    "output_tokens": 12
  }
}
```

--------------------------------

### POST /v1/messages/count_tokens

Source: https://github.com/xangel82/athena-engine/blob/main/README.md

Counts tokens for a message. Compatible with Anthropic token counting.

```APIDOC
## POST /v1/messages/count_tokens

### Description
Counts tokens for a message. Compatible with Anthropic token counting.

### Method
POST

### Endpoint
/v1/messages/count_tokens

### Parameters
#### Request Body
- **model** (string) - Required - The model name to use. Defaults to 'Athena' unless configured otherwise.
- **messages** (array) - Required - A list of messages to count tokens for.

### Response
#### Success Response (200)
- **input_tokens** (integer) - The number of input tokens.

#### Response Example
{
  "input_tokens": 9
}
```
Tokens:
Raw