Qwen Gate

https://github.com/youssefvdel/qwengate
Admin
Qwen Gate is an OpenAI-compatible API gateway for Qwen models that provides drop-in...
Tokens:22,571
Snippets:324
Trust Score:8
License:MIT
Update:1 month ago
Show doc for...
Tokens:
Raw
### Quick Start Installation and Execution

Source: https://github.com/youssefvdel/qwengate/blob/dev/docs/DEPLOYMENT.md

Commands to install dependencies, initialize configuration, and start the server.

```bash
# Install dependencies (postinstall creates config.json with defaults)
bun install --production

# Customize config (optional -- skip if defaults are fine)
bun run setup

# Start the server
bun start
```

--------------------------------

### Manual Installation and Server Startup

Source: https://github.com/youssefvdel/qwengate/blob/dev/README.md

Clone the repository and start the server using Bun.

```bash
git clone https://github.com/youssefvdel/qwen-gate.git
cd qwen-gate
bun install
```

```bash
qg
```

```bash
bun start
```

--------------------------------

### Install and Run Qwen Gate

Source: https://github.com/youssefvdel/qwengate/blob/dev/README.md

Executes the installation script and starts the Qwen Gate service.

```bash
curl -sSL https://raw.githubusercontent.com/youssefvdel/qwen-gate/main/install.sh | bash
cd qwen-gate
qg
```

--------------------------------

### Interactive Configuration Setup

Source: https://github.com/youssefvdel/qwengate/blob/dev/docs/DEPLOYMENT.md

Command to launch the interactive setup wizard for generating config.json.

```bash
bun run setup
```

--------------------------------

### Run Installer Script

Source: https://github.com/youssefvdel/qwengate/blob/dev/README.md

Installation scripts for Linux/macOS and Windows environments.

```bash
# Linux / macOS
curl -sSL https://raw.githubusercontent.com/youssefvdel/qwen-gate/main/install.sh | bash

# Windows (PowerShell)
powershell -ExecutionPolicy Bypass -c "curl.exe -sSL https://raw.githubusercontent.com/youssefvdel/qwen-gate/main/install.ps1 | iex"
```

--------------------------------

### Install Qwen Gate via CLI

Source: https://github.com/youssefvdel/qwengate/blob/dev/README.md

Use these commands to install the gateway on Linux, macOS, or Windows systems.

```bash
curl -sSL https://raw.githubusercontent.com/youssefvdel/qwen-gate/main/install.sh | bash
```

```powershell
powershell -ExecutionPolicy Bypass -c "curl.exe -sSL https://raw.githubusercontent.com/youssefvdel/qwen-gate/main/install.ps1 | iex"
```

--------------------------------

### Start Development Server

Source: https://github.com/youssefvdel/qwengate/blob/dev/docs/DEVELOPMENT.md

Launches the server with hot reload enabled using Bun.

```bash
bun dev
```

--------------------------------

### Manual Configuration File

Source: https://github.com/youssefvdel/qwengate/blob/dev/docs/DEPLOYMENT.md

Example structure of the config.json file.

```json
{
  "PORT": "26405",
  "HOST": "0.0.0.0",
  "API_KEY": "your-secret-key",
  "BROWSER": "chromium",
  "TOOL_CALLING": "true",
  "CLEAN_OUTPUT": "true",
  "STREAMING_MODE": "auto"
}
```

--------------------------------

### Start Production Server

Source: https://github.com/youssefvdel/qwengate/blob/dev/docs/DEVELOPMENT.md

Runs the application in production mode without a build step.

```bash
bun start
```

--------------------------------

### Initialize Project Environment

Source: https://github.com/youssefvdel/qwengate/blob/dev/docs/DEVELOPMENT.md

Install dependencies and run the interactive configuration wizard to generate the required config.json file.

```bash
bun install
bun run scripts/setup.js   # interactive config wizard
```

--------------------------------

### Run Qwen Gate Server and Tests

Source: https://github.com/youssefvdel/qwengate/blob/dev/HANDOFF.md

Commands to start the development server and execute the test suite.

```bash
bun src/index.tsx    # Server on port 26405
bun test             # 103 tests pass
```

--------------------------------

### Available CLI Commands

Source: https://github.com/youssefvdel/qwengate/blob/dev/AGENTS.md

Commands for running tests, starting the development server, and managing the application.

```bash
bun test              # All tests, no setup needed
bun start             # Dev server (`bun src/index.tsx`)
bun dev               # Hot reload (`bun --watch src/index.tsx`)
bun cluster           # Multi-core mode
bun qg                # CLI entry (`bun src/cli.ts`)
```

--------------------------------

### Utility and CLI Commands

Source: https://github.com/youssefvdel/qwengate/blob/dev/docs/DEVELOPMENT.md

Various commands for setup, CLI interaction, and cluster mode execution.

```bash
bun run scripts/setup.js   # Run the interactive setup wizard
bun run src/cli.ts         # Launch the CLI (qg)
bun src/cli.ts restart     # CLI restart
bun src/cli.ts help        # CLI help
bun cluster                # Multi-core cluster mode
```

--------------------------------

### Development and Testing Commands

Source: https://github.com/youssefvdel/qwengate/blob/dev/CONTRIBUTING.md

Commands for starting the development server and executing the test suite.

```bash
bun dev        # Start in development mode
bun test            # Run tests
```

--------------------------------

### Manual Update Procedure

Source: https://github.com/youssefvdel/qwengate/blob/dev/README.md

Executes a manual pull, dependency installation, and server restart.

```bash
git pull && bun install && qg restart
```

--------------------------------

### System Logging Usage

Source: https://github.com/youssefvdel/qwengate/blob/dev/docs/DEVELOPMENT.md

Examples of using the logStore service for server-level diagnostics.

```typescript
import { logStore } from '../services/logStore.ts';

// System-level logging
logStore.systemLog('info', 'Server started on port 26405');
logStore.systemLog('error', 'Failed to connect', err);
logStore.systemLog('warn', 'Rate limit approaching', { account: email });
```

--------------------------------

### Test File Naming Conventions

Source: https://github.com/youssefvdel/qwengate/blob/dev/docs/DEVELOPMENT.md

Examples of file naming patterns for unit and integration tests.

```text
src/tools/guard.ts          # source
src/tools/guard.test.ts     # tests
src/tools/xmlToolParser.ts  # source
src/tools/xmlToolParser.test.ts # tests
```

```text
src/tests/index.test.ts      # server integration
src/tests/largeBuffer.test.ts
src/tests/contentFilter.test.ts
```

--------------------------------

### Execute Tool Calling

Source: https://github.com/youssefvdel/qwengate/blob/dev/docs/API.md

Examples for requesting a tool call, receiving the tool call response, and providing the tool result.

```json
{
  "model": "qwen-max",
  "messages": [{ "role": "user", "content": "What's the weather in Tokyo?" }],
  "tools": [
    {
      "type": "function",
      "function": {
        "name": "get_weather",
        "description": "Get current weather for a location",
        "parameters": {
          "type": "object",
          "properties": {
            "location": {
              "type": "string",
              "description": "City name"
            }
          },
          "required": ["location"]
        }
      }
    }
  ],
  "tool_choice": "auto"
}
```

```json
{
  "id": "chatcmpl-456",
  "object": "chat.completion",
  "created": 1234567890,
  "model": "qwen-max",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": null,
        "tool_calls": [
          {
            "id": "call_abc123",
            "type": "function",
            "function": {
              "name": "get_weather",
              "arguments": "{\"location\": \"Tokyo\"}"
            }
          }
        ]
      },
      "finish_reason": "tool_calls"
    }
  ]
}
```

```json
{
  "model": "qwen-max",
  "messages": [
    { "role": "user", "content": "What's the weather in Tokyo?" },
    {
      "role": "assistant",
      "content": null,
      "tool_calls": [
        {
          "id": "call_abc123",
          "type": "function",
          "function": {
            "name": "get_weather",
            "arguments": "{\"location\": \"Tokyo\"}"
          }
        }
      ]
    },
    {
      "role": "tool",
      "tool_call_id": "call_abc123",
      "content": "{\"temperature\": 22, \"condition\": \"sunny\"}"
    }
  ]
}
```

--------------------------------

### 400 Bad Request Error Examples

Source: https://github.com/youssefvdel/qwengate/blob/dev/docs/API.md

Examples of 400 errors for general invalid requests and specific context window limit violations.

```json
{
  "error": {
    "message": "Invalid request format",
    "type": "invalid_request_error",
    "code": "invalid_request"
  }
}
```

```json
{
  "error": {
    "message": "Context window exceeded. Input has ~25000 tokens, but the model qwen-max supports a maximum context of 10000 tokens.",
    "type": "invalid_request_error",
    "param": "messages",
    "code": "context_window_exceeded"
  }
}
```

--------------------------------

### GET /v1/models

Source: https://github.com/youssefvdel/qwengate/blob/dev/docs/API.md

Retrieves a list of available models supported by the server.

```APIDOC
## GET /v1/models

### Description
List available models. Model names ending in -no-thinking disable the thinking/reasoning block for that model.

### Method
GET

### Endpoint
/v1/models

### Response
#### Success Response (200)
- **object** (string) - The type of the object, which is 'list'.
- **data** (array) - A list of model objects.

#### Response Example
{
  "object": "list",
  "data": [
    {
      "id": "qwen3-7-max",
      "object": "model",
      "created": 1234567890,
      "owned_by": "qwen",
      "context_window": 1000000,
      "max_output_tokens": 81920,
      "modalities": ["text"]
    }
  ]
}
```

--------------------------------

### Qwen API Logging Usage

Source: https://github.com/youssefvdel/qwengate/blob/dev/docs/DEVELOPMENT.md

Examples of using the qwenLogger service for API interaction capture.

```typescript
import { qwenLogger } from '../services/qwenLogger.ts';

qwenLogger.logRequest(chatId, body);
qwenLogger.logResponse(chatId, response);
```

--------------------------------

### SSL Configuration with Certbot

Source: https://github.com/youssefvdel/qwengate/blob/dev/docs/DEPLOYMENT.md

Commands to install Certbot and secure the Nginx server with Let's Encrypt.

```bash
sudo apt install certbot nginx
sudo certbot --nginx -d api.yourdomain.com
```

--------------------------------

### PM2 Process Management

Source: https://github.com/youssefvdel/qwengate/blob/dev/docs/DEPLOYMENT.md

Commands for managing the application process with PM2, including installation, startup scripts, and monitoring.

```bash
# Install PM2 globally
bun add -g pm2

# Start with PM2
pm2 start bun --name "qwen-gate" -- start

# Save process list (survives reboot)
pm2 save

# Generate startup script
pm2 startup

# Useful commands
pm2 status                # View status
pm2 logs qwen-gate        # View logs
pm2 monit                 # Monitor resources
pm2 restart qwen-gate     # Restart
pm2 stop qwen-gate        # Stop
```

--------------------------------

### Conventional Commit Format

Source: https://github.com/youssefvdel/qwengate/blob/dev/CONTRIBUTING.md

Examples of commit messages following the Conventional Commits specification.

```text
feat(scopes): add new feature
fix(scopes): fix a bug
chore(scopes): maintenance task
docs(scopes): documentation changes
```

--------------------------------

### Model list response structure

Source: https://github.com/youssefvdel/qwengate/blob/dev/docs/API.md

Example JSON response containing model metadata including context window and modality support.

```json
{
  "object": "list",
  "data": [
    {
      "id": "qwen3-7-max",
      "object": "model",
      "created": 1234567890,
      "owned_by": "qwen",
      "context_window": 1000000,
      "max_output_tokens": 81920,
      "modalities": ["text"]
    },
    {
      "id": "qwen3-6-plus",
      "object": "model",
      "created": 1234567890,
      "owned_by": "qwen",
      "context_window": 1000000,
      "max_output_tokens": 65536,
      "modalities": ["text", "image", "video"]
    },
    {
      "id": "qwen3-5-flash",
      "object": "model",
      "created": 1234567890,
      "owned_by": "qwen",
      "context_window": 1000000,
      "max_output_tokens": 65536,
      "modalities": ["text", "image", "video"]
    },
    {
      "id": "qwen3-7-max-no-thinking",
      "object": "model",
      "created": 1234567890,
      "owned_by": "qwen",
      "context_window": 1000000,
      "max_output_tokens": 81920,
      "modalities": ["text"]
    }
  ]
}
```

--------------------------------

### Anthropic System Prompt Pattern for Tool Results

Source: https://github.com/youssefvdel/qwengate/blob/dev/docs/tool-echo-research.md

A system-level instruction set to guide models in handling tool outputs by prioritizing synthesis over verbatim repetition.

```text
After receiving tool results:
- DO NOT output the content of the tool results as-is
- Instead, provide your own analysis, summary, or next steps
- Tool results are shown to the user separately
```

--------------------------------

### Create API Route Handler

Source: https://github.com/youssefvdel/qwengate/blob/dev/docs/DEVELOPMENT.md

Define a new Hono router with GET and POST endpoints.

```typescript
// src/routes/hello.ts
import { Hono } from 'hono';

export const helloRouter = new Hono();

helloRouter.get('/', (c) => {
  return c.json({ message: 'Hello from the new endpoint!' });
});

helloRouter.post('/echo', async (c) => {
  const body = await c.req.json();
  return c.json({ echo: body });
});
```

--------------------------------

### TypeScript Coding Conventions

Source: https://github.com/youssefvdel/qwengate/blob/dev/docs/DEVELOPMENT.md

Examples of preferred nullish coalescing and optional chaining over logical OR, alongside proper type imports.

```typescript
// Good
const value = config.get('KEY') ?? 'default';
const name = obj?.nested?.name ?? 'unknown';

// Good — use existing types
import type { OpenAIRequest } from '../types/openai.ts';

// Avoid
const value = config.get('KEY') || 'default'; // catches empty string too
```

--------------------------------

### Standardized JSON Log Structure

Source: https://github.com/youssefvdel/qwengate/blob/dev/docs/ARCHITECTURE.md

Example of the structured log format used for application events, including timestamp, severity level, and contextual data.

```json
{
  "timestamp": "2024-01-15T10:30:00Z",
  "level": "info",
  "component": "session-pool",
  "message": "Session created",
  "data": {
    "sessionId": "abc123",
    "account": "user@example.com",
    "latency": 1234
  }
}
```

--------------------------------

### Qwengate CLI Usage

Source: https://github.com/youssefvdel/qwengate/blob/dev/README.md

Displays the available commands and options for the qg binary.

```text
Usage: qg [command] [options]

Commands:
  start          Start the API server (default)
  update         Pull latest code and reinstall dependencies
  restart        Restart the running server
  status         Check if the server is running
  help           Show help message

Options:
  --port <n>     Override port
  --browser <e>  Browser engine: chromium, firefox, webkit, chrome, edge
  --host <addr>  Bind address

Account management is done via the web dashboard → Accounts page.
```

--------------------------------

### Package Main Entry Point

Source: https://github.com/youssefvdel/qwengate/blob/dev/docs/AUDIT.md

The main field in package.json points to an incorrect file path.

```json
"main": "dist/index.js"
```

--------------------------------

### Update Qwengate via CLI

Source: https://github.com/youssefvdel/qwengate/blob/dev/README.md

Commands to pull the latest code and restart the server.

```bash
qg update
```

```bash
qg restart
```

--------------------------------

### Minimal System Prompt Template

Source: https://github.com/youssefvdel/qwengate/blob/dev/docs/SYSTEM_PROMPT_RESEARCH.md

A structural template for defining agent roles, behavioral rules, constraints, output formats, and tool usage policies.

```text
You are [ROLE], built for [DOMAIN].

## Behavioral Rules
- [2-5 core principles]
- [Prefer X instead of Y]

## Constraints
- NEVER [safety-critical prohibitions]
- MUST [non-negotiable requirements]
- If you call a tool, wait for its result before generating any further text.

## Output Format
- [Expected shape and length]
- [Stop condition]

## Tool Usage
- [When to use each category]
- [Parallelism policy]
- [Fallback on error]
```

--------------------------------

### Import Tool Module

Source: https://github.com/youssefvdel/qwengate/blob/dev/docs/DEVELOPMENT.md

Ensure the tool is registered at startup by importing the module.

```typescript
// This import ensures the tool is registered at startup
import './tools/myTool.ts';
```

--------------------------------

### systemd Service Configuration

Source: https://github.com/youssefvdel/qwengate/blob/dev/docs/DEPLOYMENT.md

Service unit file for running Qwen Gate as a background service on Linux.

```ini
[Unit]
Description=Qwen Gate API Proxy
After=network.target

[Service]
Type=simple
User=youruser
WorkingDirectory=/opt/qwen-gate
ExecStart=/usr/bin/bun start
Restart=always
RestartSec=10
Environment=NODE_ENV=production

[Install]
WantedBy=multi-user.target
```

--------------------------------

### Custom Error Class Definitions

Source: https://github.com/youssefvdel/qwengate/blob/dev/docs/DEVELOPMENT.md

Examples of custom error classes used for distinct failure modes.

```typescript
export class SessionPoolQueueFullError extends Error { ... }
export class SessionPoolWaitTimeoutError extends Error { ... }
export class SchemaValidationError extends Error { ... }
export class NonRetryableError extends Error { ... }
export class CircuitOpenError extends Error { ... }
```

--------------------------------

### Run tests with Bun

Source: https://github.com/youssefvdel/qwengate/blob/dev/README.md

Executes the project's test suite using the Bun runtime.

```bash
bun test
```

--------------------------------

### Configure Authentication

Source: https://github.com/youssefvdel/qwengate/blob/dev/docs/API.md

Methods for providing API keys via headers or configuration files.

```bash
Authorization: Bearer YOUR_API_KEY
```

```bash
API_KEY=your_secret_key_here
```

--------------------------------

### Configure .env File

Source: https://github.com/youssefvdel/qwengate/blob/dev/docs/debugging-agent-prompt.md

Set the required API key in the project's environment configuration file.

```env
OPENAI_API_KEY=sk-your-valid-key-here
```

--------------------------------

### Accessing Configuration Values

Source: https://github.com/youssefvdel/qwengate/blob/dev/docs/IMPROVEMENTS.md

Demonstrates the legacy pattern of accessing configuration values using string keys and manual parsing.

```typescript
// Current pattern -- scattered across the codebase:
const QWEN_FETCH_TIMEOUT_MS = parseInt(config.get('QWEN_FETCH_TIMEOUT_MS', '30000'), 10);
const retriesEnabled = config.get('RETRY_ENABLED', 'true') !== 'false';
```

--------------------------------

### Contribution workflow commands

Source: https://github.com/youssefvdel/qwengate/blob/dev/docs/IMPROVEMENTS.md

Standard commands for creating branches, running tests, and executing quality checks.

```bash
git checkout -b improve/issue-number-slug
```

```bash
bun test
```

```bash
bun run quality
```

--------------------------------

### System Prompt for Tool Result Handling

Source: https://github.com/youssefvdel/qwengate/blob/dev/docs/tool-echo-research.md

Add these instructions to every request to discourage the model from repeating raw tool output.

```text
## Tool Result Handling
- Tool results are processed by the system and rendered to the user automatically.
- After <tool_result> blocks, do NOT copy, repeat, or echo the raw tool result content.
- If the user asks about a tool result, summarize it in your own words
  rather than quoting it verbatim.
```

--------------------------------

### Horizontal Scaling Architecture

Source: https://github.com/youssefvdel/qwengate/blob/dev/docs/ARCHITECTURE.md

Visual representation of running multiple Qwen Gate instances behind an Nginx load balancer.

```text
Load Balancer (nginx)
    │
    ├─► Qwen Gate Instance 1
    │
    ├─► Qwen Gate Instance 2
    │
    └─► Qwen Gate Instance 3
```

--------------------------------

### Production System Prompt Structure

Source: https://github.com/youssefvdel/qwengate/blob/dev/docs/SYSTEM_PROMPT_RESEARCH.md

Visual representation of the recommended token distribution and structural hierarchy for production-grade system prompts.

```text
┌──────────────────────────────────────────┐
│ 1. IDENTITY FRAMING            ~100 tks  │
│ 2. BEHAVIORAL RULES           200-800 tks│
│ 3. TOOL USAGE POLICY          400-600 tks│
│ 4. SAFETY LAYERS               woven in  │
│ 5. CONDITIONAL SECTIONS        dynamic   │
├──────────────────────────────────────────┤
│ [TOOL DEFINITIONS]             4500+ tks │
└──────────────────────────────────────────┘
```

--------------------------------

### systemd Service Management

Source: https://github.com/youssefvdel/qwengate/blob/dev/docs/DEPLOYMENT.md

Commands to enable and manage the systemd service.

```bash
sudo systemctl daemon-reload
sudo systemctl enable qwen-gate
sudo systemctl start qwen-gate
sudo systemctl status qwen-gate
journalctl -u qwen-gate -f  # View logs
```

--------------------------------

### Manage Configuration Values

Source: https://github.com/youssefvdel/qwengate/blob/dev/docs/DEVELOPMENT.md

Use the config service to retrieve environment-precedence values or persist runtime settings to config.json.

```typescript
// Reading config — env takes precedence, then config.json, then default
const port = config.get('PORT');
const apiKey = config.get('API_KEY', 'fallback');

// Writing config — writes to config.json only
config.set('SAVE_REQUEST_LOGS', 'true');
config.save();
```

--------------------------------

### Python OpenAI SDK Integration

Source: https://github.com/youssefvdel/qwengate/blob/dev/docs/API.md

Uses the OpenAI Python client to stream chat completions from the QwenGate local server.

```python
from openai import OpenAI

client = OpenAI(
    api_key="YOUR_API_KEY",
    base_url="http://localhost:26405/v1"
)

response = client.chat.completions.create(
    model="qwen-max",
    messages=[
        {"role": "user", "content": "Hello!"}
    ],
    stream=True
)

for chunk in response:
    if chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="")
```

--------------------------------

### Project Directory Structure

Source: https://github.com/youssefvdel/qwengate/blob/dev/CONTRIBUTING.md

Overview of the source code organization and file responsibilities.

```text
src/
├── cli.ts          # CLI entry point
├── cluster.ts      # Cluster mode
├── index.tsx       # Server entry, routing, CORS, auth
├── models.json     # Model definitions
├── middleware/      # Rate limiter
├── routes/         # API handlers + streaming
│   └── dashboard/  # Web dashboard
├── services/       # Auth, accounts, sessions, Qwen API transport, config
├── tests/          # Integration tests
├── tools/          # Tool calling system
├── types/          # OpenAI-compatible types
└── utils/          # Shared utilities
```

--------------------------------

### Register Hono Routes

Source: https://github.com/youssefvdel/qwengate/blob/dev/docs/DEVELOPMENT.md

Demonstrates how to define inline handlers, import route handlers, and mount route groups within the Hono application.

```typescript
// src/index.tsx
import { Hono } from 'hono';
export const app = new Hono();

// Simple inline handler
app.get('/health', (c) => {
  return c.json({ status: 'ok' });
});

// Imported route handler
app.post('/v1/chat/completions', chatCompletions);

// Routers (groups of routes)
import { accountsRouter } from './routes/accounts.ts';
app.route('/api/accounts', accountsRouter);
```

--------------------------------

### CI Workflow Configuration

Source: https://github.com/youssefvdel/qwengate/blob/dev/docs/AUDIT.md

The CI pipeline uses Node.js instead of the project-preferred Bun runtime.

```yaml
actions/setup-node@v4
npm ci
```

--------------------------------

### List available models via cURL

Source: https://github.com/youssefvdel/qwengate/blob/dev/docs/API.md

Retrieves a list of available models. Requires an Authorization header with a valid API key.

```bash
curl http://localhost:26405/v1/models \
  -H "Authorization: Bearer YOUR_API_KEY"
```

--------------------------------

### Implement Service Singleton

Source: https://github.com/youssefvdel/qwengate/blob/dev/docs/DEVELOPMENT.md

Pattern for encapsulating business logic by exporting a class and its singleton instance.

```typescript
// services/configService.ts
export class ConfigService {
  get<K extends keyof ConfigSchema>(key: K, defaultValue?: string): string { ... }
  set<K extends keyof ConfigSchema>(key: K, value: string): void { ... }
  getAll(): ConfigSchema { ... }
  save(): void { ... }
}
export const config = new ConfigService();
```

--------------------------------

### Execute Tool Calls

Source: https://github.com/youssefvdel/qwengate/blob/dev/README.md

The gateway parses JSON tool call outputs from the model and converts them into standard OpenAI-compatible tool call objects.

```bash
curl -X POST http://localhost:26405/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer your-api-key" \
  -d '{
    "model": "qwen3-max",
    "messages": [{"role": "user", "content": "Weather in Paris?"}],
    "tools": [{
      "type": "function",
      "function": {
        "name": "get_weather",
        "parameters": {
          "type": "object",
          "properties": {"city": {"type": "string"}},
          "required": ["city"]
        }
      }
    }]
  }'
```

--------------------------------

### Stream Chat Completions

Source: https://github.com/youssefvdel/qwengate/blob/dev/README.md

Enable Server-Sent Events (SSE) by setting the stream parameter to true.

```bash
curl -X POST http://localhost:26405/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer your-api-key" \
  -d '{"model": "qwen3-max", "stream": true, "messages": [{"role": "user", "content": "Count to 5"}]}'
```

--------------------------------

### Test Tool

Source: https://github.com/youssefvdel/qwengate/blob/dev/docs/DEVELOPMENT.md

Verify tool execution using the registry.

```typescript
// src/tools/myTool.test.ts
import { test, expect } from 'bun:test';
import { registry } from './registry.ts';

test('my_custom_tool does what it should', async () => {
  const result = await registry.execute('my_custom_tool', { input1: 'test' }, {} as any);
  expect(result).toBeTruthy();
  // ... more assertions
});
```

--------------------------------

### Perform Chat Completions

Source: https://github.com/youssefvdel/qwengate/blob/dev/README.md

Send requests to the gateway using the OpenAI-compatible chat completions endpoint.

```bash
curl -X POST http://localhost:26405/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer your-api-key" \
  -d '{
    "model": "qwen3-max",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'
```

--------------------------------

### Perform a Chat Completion Request

Source: https://github.com/youssefvdel/qwengate/blob/dev/docs/API.md

Use this cURL command to send a chat completion request to the local Qwengate server.

```bash
curl http://localhost:26405/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -d '{
    "model": "qwen-max",
    "messages": [
      {"role": "system", "content": "You are a helpful assistant."},
      {"role": "user", "content": "Hello!"}
    ],
    "stream": true
  }'
```

--------------------------------

### Define Tool Call Style Guidelines

Source: https://github.com/youssefvdel/qwengate/blob/dev/docs/SYSTEM_PROMPT_RESEARCH.md

Guidelines for controlling the verbosity and narration of tool calls.

```text
## Tool Call Style
Default: do not narrate routine, low-risk tool calls (just call the tool).
Narrate only when it helps: multi-step work, complex problems, sensitive actions.
Keep narration brief and value-dense.
```

--------------------------------

### Basic Test Structure

Source: https://github.com/youssefvdel/qwengate/blob/dev/docs/DEVELOPMENT.md

Standard template for writing tests using the bun:test library.

```typescript
import { test, expect } from 'bun:test';

test('feature works as expected', async () => {
  const result = await someFunction();
  expect(result).toBe(expectedValue);
});
```

--------------------------------

### Verify Environment Variable

Source: https://github.com/youssefvdel/qwengate/blob/dev/docs/debugging-agent-prompt.md

Check if the API key is correctly exported in the current shell environment.

```bash
echo $OPENAI_API_KEY
```

--------------------------------

### Register API Route

Source: https://github.com/youssefvdel/qwengate/blob/dev/docs/DEVELOPMENT.md

Mount the router in the main application file, optionally adding authentication middleware.

```typescript
import { helloRouter } from './routes/hello.ts';

// With auth protection
app.use('/api/hello/*', async (c, next) => {
  const apiKey = config.get('API_KEY');
  if (!apiKey) return await next();
  return bearerAuth({ token: apiKey })(c, next);
});
app.route('/api/hello', helloRouter);
```

--------------------------------

### Register Tool Programmatically

Source: https://github.com/youssefvdel/qwengate/blob/dev/docs/DEVELOPMENT.md

Registers a new tool with the registry, providing a name, description, JSON schema for parameters, and an execution handler.

```typescript
import { registry } from '../tools/registry.ts';

registry.register(
  'get_weather',              // tool name (must match what the LLM emits)
  'Get weather for a city',  // description
  {                          // JSON Schema parameters
    type: 'object',
    properties: {
      city: { type: 'string', description: 'City name' },
    },
    required: ['city'],
  },
  async (args, context) => {  // handler
    const weather = await fetchWeather(args.city);
    return JSON.stringify(weather);
  }
);
```

--------------------------------

### Tool Description Instruction

Source: https://github.com/youssefvdel/qwengate/blob/dev/docs/tool-echo-research.md

Include explicit instructions within the tool definition to prevent repetition of results.

```json
{
  "name": "search_docs",
  "description": "Search documentation. Results are rendered automatically — do not repeat them in your response.",
  "parameters": {}
}
```

--------------------------------

### Application Log Access

Source: https://github.com/youssefvdel/qwengate/blob/dev/docs/DEPLOYMENT.md

Commands to view application logs via PM2 or systemd.

```bash
pm2 logs qwen-gate                  # Via PM2
journalctl -u qwen-gate -f          # Via systemd
```

--------------------------------

### Acquire and Release Session Pool Entries

Source: https://github.com/youssefvdel/qwengate/blob/dev/docs/DEVELOPMENT.md

Demonstrates the lifecycle of a session pool entry, including acquiring a session for a specific user and releasing it back to the pool.

```typescript
// Acquire a session
const entry = await sessionPool.acquire('user@example.com');
// entry.chatId, entry.parentId, entry.cachedHeaders

// Release when done
await sessionPool.release(entry.chatId);
```

--------------------------------

### Test API Key Connectivity

Source: https://github.com/youssefvdel/qwengate/blob/dev/docs/debugging-agent-prompt.md

Use cURL to verify the validity of the API key against the OpenAI models endpoint.

```bash
curl https://api.openai.com/v1/models \
  -H "Authorization: Bearer $OPENAI_API_KEY"
```

--------------------------------

### Define Base URL

Source: https://github.com/youssefvdel/qwengate/blob/dev/docs/API.md

The base URL for all API requests. Replace localhost with your production server address.

```text
http://localhost:26405/v1
```

--------------------------------

### POST /v1/chat/completions

Source: https://github.com/youssefvdel/qwengate/blob/dev/README.md

Sends a chat completion request to the Qwen model. This endpoint is compatible with standard OpenAI SDKs and tools.

```APIDOC
## POST /v1/chat/completions

### Description
Sends a chat completion request to the Qwen model. The gateway handles model selection, message processing, and optional streaming or tool calling.

### Method
POST

### Endpoint
http://localhost:26405/v1/chat/completions

### Request Body
- **model** (string) - Required - The model identifier (e.g., "qwen3-max", "qwen3-coder").
- **messages** (array) - Required - A list of message objects with 'role' and 'content'.
- **stream** (boolean) - Optional - If true, enables Server-Sent Events (SSE) streaming.
- **tools** (array) - Optional - A list of tool definitions for function calling.

### Request Example
{
  "model": "qwen3-max",
  "messages": [{"role": "user", "content": "Hello!"}]
}

### Response
#### Success Response (200)
- **id** (string) - Unique identifier for the completion.
- **choices** (array) - List of completion choices.
- **model** (string) - The model used for the completion.
```

--------------------------------

### Browser Isolation Contexts

Source: https://github.com/youssefvdel/qwengate/blob/dev/docs/ARCHITECTURE.md

Diagram illustrating how individual sessions maintain isolated cookies, storage, and cache.

```text
Browser Instance
├─► Context 1 (Session A)
│ ├─► Isolated cookies
│ ├─► Isolated storage
│ └─► Isolated cache
│
├─► Context 2 (Session B)
│ ├─► Isolated cookies
│ ├─► Isolated storage
│ └─► Isolated cache
│
└─► Context 3 (Session C)
├─► Isolated cookies
├─► Isolated storage
└─► Isolated cache
```

--------------------------------

### PM2 Ecosystem Configuration

Source: https://github.com/youssefvdel/qwengate/blob/dev/docs/DEPLOYMENT.md

Configuration file for PM2 to manage auto-restart and logging settings.

```javascript
// ecosystem.config.js
module.exports = {
  apps: [{
    name: 'qwen-gate',
    script: 'bun',
    args: 'start',
    instances: 1,
    exec_mode: 'fork',
    max_restarts: 10,
    restart_delay: 4000,
    max_memory_restart: '1G',
    log_date_format: 'YYYY-MM-DD HH:mm:ss',
    error_file: './logs/error.log',
    out_file: './logs/out.log',
  }]
};

// mkdir -p logs  (create the logs directory referenced above)
// pm2 start ecosystem.config.js
```

--------------------------------

### Claude Code Assembly Pipeline

Source: https://github.com/youssefvdel/qwengate/blob/dev/docs/SYSTEM_PROMPT_RESEARCH.md

Conceptual breakdown of how system prompt segments are assembled at runtime from static and dynamic content.

```text
Static Content (globally cacheable)
├── Simple Intro Section
├── Simple System Section
├── Doing Tasks Section
├── Actions Section
├── Using Your Tools Section
├── Tone & Style Section
└── Output Efficiency Section
═══════ BOUNDARY MARKER ═══════
Dynamic Content (session-scoped)
├── Enabled tools
├── Feature flags
├── Environment info
└── Session metadata
```

--------------------------------

### Engine Compatibility Field

Source: https://github.com/youssefvdel/qwengate/blob/dev/docs/AUDIT.md

The engines field is missing from package.json.

```json
"engines": { "bun": ">=1.3.0", "node": ">=22.0.0" }
```

--------------------------------

### Set Chat Completion Headers

Source: https://github.com/youssefvdel/qwengate/blob/dev/docs/API.md

Required headers for POST /v1/chat/completions requests.

```http
Content-Type: application/json
Authorization: Bearer YOUR_API_KEY
```

--------------------------------

### Handle Streaming SSE Responses

Source: https://github.com/youssefvdel/qwengate/blob/dev/docs/API.md

The server returns Server-Sent Events (SSE) chunks. The final chunk includes usage statistics.

```text
: heartbeat

data: {"id":"chatcmpl-123","object":"chat.completion.chunk","created":1234567890,"model":"qwen-max","system_fingerprint":"fp_qwen_gate","service_tier":"default","choices":[{"index":0,"delta":{"role":"assistant","content":""},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-123","object":"chat.completion.chunk","created":1234567890,"model":"qwen-max","choices":[{"index":0,"delta":{"content":"Hello"},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-123","object":"chat.completion.chunk","created":1234567890,"model":"qwen-max","choices":[{"index":0,"delta":{"content":"! How can I help you today?"},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-123","object":"chat.completion.chunk","created":1234567890,"model":"qwen-max","system_fingerprint":"fp_qwen_gate","service_tier":"default","choices":[{"index":0,"delta":{},"logprobs":null,"finish_reason":"stop"}],"usage":{"prompt_tokens":15,"completion_tokens":10,"total_tokens":25,"completion_tokens_details":{"reasoning_tokens":0},"prompt_tokens_details":{"cached_tokens":0}}}

data: [DONE]
```

```text
data: {"id":"chatcmpl-123","object":"chat.completion.chunk","created":1234567890,"model":"qwen-max","choices":[{"index":0,"delta":{"reasoning_content":"Let me think about this step by step..."},"logprobs":null,"finish_reason":null}]}
```

--------------------------------

### Direct API Requests with curl

Source: https://github.com/youssefvdel/qwengate/blob/dev/docs/API.md

Executes raw HTTP requests to the chat completions endpoint for both streaming and non-streaming modes.

```bash
# Streaming
curl http://localhost:26405/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -d '{
    "model": "qwen-max",
    "messages": [{"role": "user", "content": "Hello!"}],
    "stream": true
  }'

# Non-streaming
curl http://localhost:26405/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -d '{
    "model": "qwen-max",
    "messages": [{"role": "user", "content": "Hello!"}],
    "stream": false
  }'
```

--------------------------------

### Define a Tool

Source: https://github.com/youssefvdel/qwengate/blob/dev/docs/API.md

Schema for defining a function tool to be used by the model.

```json
{
  "type": "function",
  "function": {
    "name": "function_name",
    "description": "Description of what the function does",
    "parameters": {
      "type": "object",
      "properties": {
        "param1": {
          "type": "string",
          "description": "Parameter description"
        },
        "param2": {
          "type": "number",
          "description": "Another parameter"
        }
      },
      "required": ["param1"]
    }
  }
}
```

--------------------------------

### Health Check

Source: https://github.com/youssefvdel/qwengate/blob/dev/docs/DEPLOYMENT.md

Command to verify the server is running by querying the models endpoint.

```bash
curl http://localhost:26405/v1/models
```

--------------------------------

### Nginx Security Configuration

Source: https://github.com/youssefvdel/qwengate/blob/dev/docs/ARCHITECTURE.md

Recommended Nginx configuration for rate limiting and security headers in production environments.

```nginx
# Rate limiting
limit_req_zone $binary_remote_addr zone=api:10m rate=10r/s;

# Security headers
add_header X-Content-Type-Options "nosniff";
add_header X-Frame-Options "SAMEORIGIN";
add_header Strict-Transport-Security "max-age=31536000";

# CORS (if needed)
add_header Access-Control-Allow-Origin "https://yourdomain.com";
```

--------------------------------

### PM2 Clustering

Source: https://github.com/youssefvdel/qwengate/blob/dev/docs/DEPLOYMENT.md

Command to run the application in cluster mode, utilizing multiple CPU cores.

```bash
pm2 start bun --name "qwen-gate" -i max -- start
```

--------------------------------

### POST /v1/chat/completions

Source: https://github.com/youssefvdel/qwengate/blob/dev/docs/API.md

Creates a chat completion request to generate responses from the model, supporting both streaming and non-streaming modes.

```APIDOC
## POST /v1/chat/completions

### Description
Generates a chat completion response based on the provided messages and model configuration. Supports streaming responses and tool calling.

### Method
POST

### Endpoint
/v1/chat/completions

### Request Body
- **model** (string) - Required - The model to use (e.g., "qwen-max").
- **messages** (array) - Required - A list of messages comprising the conversation.
- **stream** (boolean) - Optional - Whether to stream the response using SSE.
- **tools** (array) - Optional - A list of tools the model may call.
- **tool_choice** (string) - Optional - Controls how the model calls tools (e.g., "auto").

### Request Example
{
  "model": "qwen-max",
  "messages": [
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": "Hello!"}
  ],
  "stream": true
}

### Response
#### Success Response (200)
- **id** (string) - Unique identifier for the completion.
- **choices** (array) - List of completion choices.
- **usage** (object) - Token usage statistics.

#### Response Example
{
  "id": "chatcmpl-123",
  "object": "chat.completion",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "Hello! How can I help you today?"
      },
      "finish_reason": "stop"
    }
  ]
}
```

--------------------------------

### TypeScript OpenAI SDK Integration

Source: https://github.com/youssefvdel/qwengate/blob/dev/docs/API.md

Demonstrates a typed function to fetch non-streaming chat completions using environment variables for authentication.

```typescript
import OpenAI from "openai";

const client = new OpenAI({
  apiKey: process.env.QWEN_GATE_API_KEY,
  baseURL: "http://localhost:26405/v1",
});

async function chat(prompt: string): Promise<string> {
  const response = await client.chat.completions.create({
    model: "qwen-max",
    messages: [{ role: "user", content: prompt }],
  });

  return response.choices[0].message.content || "";
}
```

--------------------------------

### Define Hono Router

Source: https://github.com/youssefvdel/qwengate/blob/dev/docs/DEVELOPMENT.md

Shows the structure for creating grouped routes using a Hono instance in a dedicated router file.

```typescript
// src/routes/accounts.ts
import { Hono } from 'hono';
export const accountsRouter = new Hono();

accountsRouter.get('/', (c) => {
  return c.json(getAccounts());
});

accountsRouter.post('/', async (c) => {
  const body = await c.req.json();
  // ...
  return c.json({ success: true }, 201);
});

accountsRouter.delete('/:email', (c) => {
  removeAccount(c.req.param('email'));
  return c.json({ success: true });
});
```
Tokens:
Raw