Featherless AI

https://featherless.ai/docs/
Admin
Featherless AI is a serverless AI inference platform providing API access to open-weight large...
Tokens:27,814
Snippets:344
Trust Score:8.7
Update:2 months ago
Show doc for...
Tokens:
Raw
### Install LiteLLM

Source: https://featherless.ai/docs/litellm

Install the required library using pip.

```bash
pip install litellm
```

--------------------------------

### Install OpenLIT SDK

Source: https://featherless.ai/docs/openlit

Install the OpenLIT package via pip to enable observability features.

```bash
pip install openlit
```

--------------------------------

### Full Working Configuration Example

Source: https://featherless.ai/docs/hermes-agent

A complete example of a config.yaml file for configuring memory, telegram integration, compression, display, security, and approvals.

```yaml
terminal:
  backend: local

memory:
  memory_enabled: true
  user_profile_enabled: true
  memory_char_limit: 2200
  user_char_limit: 1375

telegram:
  enabled: true
  bot_token: "__HERMES_REDACTED__"
  unauthorized_dm_behavior: allowlist
  allowlist:
    - YOUR_NUMERIC_TELEGRAM_USER_ID

compression:
  enabled: true
  threshold: 0.50

display:
  tool_progress: all
  streaming: false

security:
  redact_secrets: true
  tirith_enabled: true

approvals:
  mode: manual
```

--------------------------------

### Install Hugging Face Hub Library

Source: https://featherless.ai/docs/hugging-face

Ensure the latest version of the library is installed to support the integration.

```bash
pip install huggingface_hub>=0.33.0
```

--------------------------------

### Install LlamaIndex Featherless Integration

Source: https://featherless.ai/docs/llamaindex

Install the necessary packages to use Featherless AI models within LlamaIndex.

```bash
pip install llama-index llama-index-llms-featherlessai
```

--------------------------------

### Install LangChain and Featherless AI dependencies

Source: https://featherless.ai/docs/langchain

Install the required Python packages to enable LangChain integration with Featherless AI.

```bash
!pip install langchain langchain-featherless-ai
```

--------------------------------

### Query Parameter Examples

Source: https://featherless.ai/docs/api-reference-models

Examples of filtering models using comma-separated values or repeated query parameters.

```text
/v1/models?family=llama3,qwen2
```

```text
/v1/models?family=llama3&family=qwen2
```

```text
/v1/models?capabilities=chat,tool-use
```

```text
/v1/models?modalities=vision
```

```text
/v1/models?family=llama3
```

```text
/v1/models?languages=en,ja
```

--------------------------------

### OpenAI Client Configuration Update

Source: https://featherless.ai/docs/api-overview-and-common-options

Examples showing the transition from a default OpenAI client instantiation to a Featherless-compatible configuration.

```python
from openai import OpenAI
# ...
client = OpenAI()
# ...
client.chat.completions.create(
  {
	model='gpt-4o',
	messages=messages,
	**sampler_params
  }
)
```

```python
from openai import OpenAI
import os
# ...
client = OpenAI(
  base_url="https://api.featherless.ai/v1",
  api_key=os.environ['FEATHERLESS_API_KEY']
)
# ...
client.chat.completions.create(
  {
	model=model_from_featherless,
	messages=messages,
	**sampler_params
  }
)
```

--------------------------------

### Application Header Configuration

Source: https://featherless.ai/docs/completions

Example JSON structure for the required HTTP-Referer and X-Title headers used to identify your application.

```json
{
  "HTTP-Referer": "https://app.wyvern.chat/"
  "X-Title": "Wyvern chat"
}
```

--------------------------------

### Tokenize Text via cURL

Source: https://featherless.ai/docs/v1-tokenize

Example request to the /v1/tokenize endpoint using cURL.

```bash
curl https://api.featherless.ai/v1/tokenize \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $FEATHERLESS_API_KEY" \
  -d '{
    "model": "GalrionSoftworks/Margnum-12B-v1",
    "text": "Where was the 2020 world series played?"
  }'
```

--------------------------------

### Example Request for Text Completion

Source: https://featherless.ai/docs/completions

A JSON payload demonstrating the required model and prompt fields alongside optional sampling parameters.

```json
{
  "model": "GalrionSoftworks/Margnum-12B-v1",
  "prompt": "Once upon a time",
  "temperature": 0.7,
  "max_tokens": 150,
  "top_p": 0.9,
  "frequency_penalty": 0.5,
  "presence_penalty": 0.0
}
```

--------------------------------

### Example SSE Stream Frames

Source: https://featherless.ai/docs/api-reference-concurrency-stream

Sample output frames emitted by the server every two seconds.

```text
data: {"limit":8,"used_cost":0,"request_count":0,"requests":[]}

data: {"limit":8,"used_cost":4,"request_count":1,"requests":[{"id":"5c4e82ea-2023-46ac-b6a5-025cf2f312fe","cost":4,"model":"moonshotai/Kimi-K2.6","started_at":1776783127496,"duration_ms":82}]}

data: {"limit":8,"used_cost":4,"request_count":1,"requests":[{"id":"5c4e82ea-2023-46ac-b6a5-025cf2f312fe","cost":4,"model":"moonshotai/Kimi-K2.6","started_at":1776783127496,"duration_ms":2083}]}

data: {"limit":8,"used_cost":0,"request_count":0,"requests":[]}
```

--------------------------------

### Tokenize Response Example

Source: https://featherless.ai/docs/v1-tokenize

The response body contains the list of token IDs.

```json
{
    "tokens": [
        9241,
        574,
        279,
        220,
        2366,
        15,
        1917,
        4101,
        6476,
        30
    ]
}
```

--------------------------------

### Chat Completions cURL Example

Source: https://featherless.ai/docs/completions

A sample cURL command to perform a chat completion request using an API key.

```bash
curl https://api.featherless.ai/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $FEATHERLESS_API_KEY" \
  -d '{
    "model": "GalrionSoftworks/Margnum-12B-v1",
    "messages": [{"role": "user", "content": "Hello!"}],
    "presence_penalty": 0.5,
    "frequency_penalty": 0.5,
    "temperature": 0.7,
    "top_p": 0.9,
    "max_tokens": 100
  }'
```

--------------------------------

### Chat Completion API Request

Source: https://featherless.ai/docs/completions

A fetch request example demonstrating how to include authentication, application headers, and model parameters.

```javascript
fetch("https://api.featherless.ai/v1/chat/completions", {
  method: "POST",
  headers: {
    "Authorization": `Bearer ${FEATHERLESS_API_KEY}`,
    "HTTP-Referer": `${YOUR_SITE_URL}`, // Optional: Include your app's URL for tracking
    "X-Title": `${YOUR_SITE_NAME}`,     // Optional: Identify your app in API analytics
    "Content-Type": "application/json"
  },
  body: JSON.stringify({
    "model": "GalrionSoftworks/Margnum-12B-v1",
    "messages": [
      {"role": "user", "content": "What is the meaning of life?"},
    ],
  })
});
```

--------------------------------

### GET /v1/plan

Source: https://featherless.ai/docs/api-reference-plan

Retrieves the details of the subscription plan associated with the provided API key, including effective context ceilings and concurrency limits.

```APIDOC
## GET /v1/plan

### Description
The plan endpoint returns the details of the plan your API key is subscribed to. Use this endpoint to learn your plan-level limits.

### Method
GET

### Endpoint
https://api.featherless.ai/v1/plan

### Response
#### Success Response (200)
- **id** (string) - The identifier of the plan.
- **name** (string) - The display name of the plan.
- **max_context_length** (integer/null) - The maximum number of tokens allowed per request.
- **max_model_size** (integer/null) - The largest model size in billions of parameters allowed.
- **concurrency** (integer) - The number of concurrent requests allowed.

#### Response Example
{
  "id": "feather_pro_plus",
  "name": "Feather Premium",
  "max_context_length": 32768,
  "max_model_size": null,
  "concurrency": 4
}
```

--------------------------------

### Response Schema Example

Source: https://featherless.ai/docs/api-reference-concurrency-stream

The JSON structure returned within each SSE data frame.

```json
{
  "limit": 8,
  "used_cost": 4,
  "request_count": 1,
  "requests": [
    {
      "id": "5c4e82ea-2023-46ac-b6a5-025cf2f312fe",
      "cost": 4,
      "model": "moonshotai/Kimi-K2.6",
      "started_at": 1776783127496,
      "duration_ms": 4084
    }
  ]
}
```

--------------------------------

### GET /v1/models

Source: https://featherless.ai/docs/api-reference-models

Retrieves a list of all models available in the catalogue along with their current state and configuration attributes.

```APIDOC
## GET /v1/models

### Description
Retrieves a list of all models available in the catalogue. This endpoint can be called from either an authenticated or unauthenticated context.

### Method
GET

### Endpoint
https://api.featherless.ai/v1/models

### Response
#### Success Response (200)
- **data** (array) - A list of model objects.

#### Response Example
{
  "data": [
    {
      "id": "vicgalle/Roleplay-Llama-3-8B",
      "name": "vicgalle/Roleplay-Llama-3-8B",
      "model_class": "llama3-8b-8k",
      "context_length": 8192,
      "max_completion_tokens": 4096
    }
  ]
}
```

--------------------------------

### Retrieve Plan Details via HTTP GET

Source: https://featherless.ai/docs/api-reference-plan

Use this request to fetch the subscription plan details associated with the provided API key.

```http
GET https://api.featherless.ai/v1/plan
```

--------------------------------

### GET /account/concurrency/stream

Source: https://featherless.ai/docs/api-reference-concurrency-stream

Establishes a Server-Sent Events (SSE) stream to monitor real-time inference requests and concurrent unit usage.

```APIDOC
## GET /account/concurrency/stream

### Description
The concurrency stream endpoint publishes a live view of the inference requests currently running against your organization's concurrency allotment. It is designed for dashboards, autoscalers and any client that needs near-real-time visibility into how much of the plan's concurrent-request budget is in use.

### Method
GET

### Endpoint
https://api.featherless.ai/account/concurrency/stream

### Authentication
Authenticate with either an API key via the `Authorization` header, or with an active browser session cookie.

### Streaming format
The response is a Server-Sent Events (SSE) stream with content type `text/event-stream`. The server emits one event immediately on connect and then one event every two seconds thereafter.

### Response
#### Success Response (200)
- **limit** (integer|null) - The organization's concurrency allotment for the active plan.
- **used_cost** (integer) - Sum of the concurrent unit cost of all in-flight requests.
- **request_count** (integer) - Number of requests currently in flight.
- **requests** (array) - List of in-flight request objects.

#### Response Example
```
data: {"limit":8,"used_cost":4,"request_count":1,"requests":[{"id":"5c4e82ea-2023-46ac-b6a5-025cf2f312fe","cost":4,"model":"moonshotai/Kimi-K2.6","started_at":1776783127496,"duration_ms":82}]}
```
```

--------------------------------

### Example Response for Text Completion

Source: https://featherless.ai/docs/completions

The structure of the response object returned by the API, including the generated text and usage statistics.

```json
{
  "id": "cmpl-6YgK3ASw92kT14L5f8zJQ7yY",
  "object": "text_completion",
  "created": 1630569482,
  "model": "GalrionSoftworks/Margnum-12B-v1",
  "choices": [
    {
      "text": " in a land far, far away, there lived a wise old owl.",
      "index": 0,
      "logprobs": null,
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 5,
    "completion_tokens": 15,
    "total_tokens": 20
  }
}
```

--------------------------------

### Initialize OpenLIT

Source: https://featherless.ai/docs/openlit

Initialize the OpenLIT SDK before making any Featherless API calls to enable automatic tracing.

```python
import openlit

openlit.init(otlp_endpoint="http://127.0.0.1:4318")
```

--------------------------------

### Launch Aider with Featherless AI

Source: https://featherless.ai/docs/guides-aider

Command-line execution string to initialize Aider with the required API credentials and configuration files.

```bash
aider --openai-api-base 'https://api.featherless.ai/v1' --openai-api-key your_featherless_API_key --model 'openai/Qwen/Qwen2.5-72B-Instruct' --map-tokens 1024 --model-metadata-file '/path/to/.aider.model.metadata.json' --model-settings-file '/path/to/.aider.model.settings.yml'
```

--------------------------------

### Make an API call using the OpenAI SDK

Source: https://featherless.ai/docs/quickstart-guide

Configure the OpenAI client to point to the Featherless base URL and provide your API key.

```python
from openai import OpenAI

client = OpenAI(
  base_url="https://api.featherless.ai/v1",
  api_key="FEATHERLESS_API_KEY",
)

response = client.chat.completions.create(
  model='Qwen/Qwen2.5-7B-Instruct',
  messages=[
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": "Hello!"}
  ],
)
print(response.choices[0].message.content)
```

--------------------------------

### Embedding API response format

Source: https://featherless.ai/docs/embeddings

Example of the JSON structure returned by the embeddings endpoint.

```json
{
  "object": "list",
  "model": "Qwen/Qwen3-Embedding-8B",
  "data": [
    {
      "object": "embedding",
      "index": 0,
      "embedding": [0.0123, -0.0456, 0.0789]
    }
  ],
  "usage": {
    "prompt_tokens": 8,
    "total_tokens": 8
  }
}
```

--------------------------------

### GET /account/concurrency

Source: https://featherless.ai/docs/api-reference-concurrency-stream

Provides a one-off snapshot of the current concurrency usage as a standard JSON response.

```APIDOC
## GET /account/concurrency

### Description
For one-off polling instead of a long-lived stream, use this endpoint. It returns the same payload as a single SSE frame, as regular JSON, and closes the connection immediately.

### Method
GET

### Endpoint
https://api.featherless.ai/account/concurrency

### Response
#### Success Response (200)
- **limit** (integer|null) - The organization's concurrency allotment.
- **used_cost** (integer) - Sum of the concurrent unit cost of all in-flight requests.
- **request_count** (integer) - Number of requests currently in flight.
- **requests** (array) - List of in-flight request objects.
```

--------------------------------

### GET https://api.featherless.ai/v1/models/{model-id}

Source: https://featherless.ai/docs/api-reference-models

Retrieves detailed information for a specific model identified by its ID.

```APIDOC
## GET https://api.featherless.ai/v1/models/{model-id}

### Description
Fetch a single model by id. The response includes live hot/cold availability, deployment status, per-token pricing, and HuggingFace metadata.

### Method
GET

### Endpoint
https://api.featherless.ai/v1/models/{model-id}

### Parameters
#### Path Parameters
- **model-id** (string) - Required - The ID of the model to retrieve (e.g., owner/model). Both raw and URL-encoded forms are accepted.

### Response
#### Success Response (200)
- **id** (string) - The model identifier.
- **object** (string) - The object type, always 'model'.
- **owned_by** (string) - The owner of the model.
- **created** (integer) - Creation timestamp.
- **status** (string) - Deployment status (active, pending_deploy, or not_deployed).
- **availability** (object) - Live hot/cold snapshot.
- **pricing** (object) - Per-token prices in USD.
- **features** (object) - Derived flags such as tool_use.

#### Response Example
{
  "id": "meta-llama/Meta-Llama-3-8B-Instruct",
  "object": "model",
  "owned_by": "Feather",
  "created": 1713398400,
  "status": "active",
  "availability": {
    "tier": "warm",
    "is_hot_live": true
  },
  "pricing": {
    "prompt": "0.0000001",
    "completion": "0.0000001"
  }
}
```

--------------------------------

### List models available on your plan

Source: https://featherless.ai/docs/api-reference-models

Retrieves models accessible under the authenticated user's current plan.

```bash
curl "https://api.featherless.ai/v1/models?available_on_current_plan=true" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $FEATHERLESS_API_KEY"
```

--------------------------------

### Send Initial Request with Tools

Source: https://featherless.ai/docs/tool-calling

Define available functions in the 'tools' array using JSON Schema to enable the model to identify necessary tool calls.

```http
POST https://api.featherless.ai/v1/chat/completions
Content-Type: application/json
Authorization: Bearer your-api-key

{
  "model": "Qwen/Qwen3-32B",
  "messages": [
    {
      "role": "user", 
      "content": "What's the weather like in San Francisco?"
    }
  ],
  "tools": [
    {
      "type": "function",
      "function": {
        "name": "get_current_weather",
        "description": "Get the current weather in a given location",
        "parameters": {
          "type": "object",
          "properties": {
            "location": {
              "type": "string",
              "description": "The city and state, e.g. San Francisco, CA"
            }
          },
          "required": ["location"]
        }
      }
    }
  ]
}
```

--------------------------------

### GET /v1/models

Source: https://featherless.ai/docs/api-reference-models

Retrieves a list of models based on optional query parameters for filtering, searching, and pagination.

```APIDOC
## GET /v1/models

### Description
Retrieves a list of available models. Supports filtering by metadata, capabilities, context length, and plan availability.

### Method
GET

### Endpoint
/v1/models

### Parameters
#### Query Parameters
- **q** (string) - Optional - Search by model name or id.
- **available_on_current_plan** (boolean) - Optional - Filter to models available on the authenticated user’s current plan.
- **tags** (string) - Optional - Legacy filter requiring all listed tags to match.
- **page** (integer) - Optional - Page number for pagination.
- **per_page** (integer) - Optional - Number of results per page (max 1000).
- **license** (string) - Optional - Filter by license.
- **family** (string) - Optional - Filter by model family.
- **model_class** (string) - Optional - Filter by model class.
- **status** (string) - Optional - Filter by status (active, pending_deploy, not_deployed).
- **languages** (string) - Optional - Filter by supported languages.
- **tasks** (string) - Optional - Filter by tasks.
- **architectures** (string) - Optional - Filter by architectures.
- **training** (string) - Optional - Filter by training method.
- **capabilities** (string) - Optional - Filter by capabilities.
- **modalities** (string) - Optional - Filter by modalities.
- **domains** (string) - Optional - Filter by domains.
- **creative** (string) - Optional - Filter by creative metadata.
- **content_flags** (string) - Optional - Filter by content flags.
- **parameter_bucket** (string) - Optional - Filter by parameter bucket.
- **popularity_level** (string) - Optional - Filter by popularity level.
- **gated** (boolean) - Optional - Filter by gated status.
- **conversational** (boolean) - Optional - Filter by conversational status.
- **context_length_min** (integer) - Optional - Minimum context size in tokens.
- **context_length_max** (integer) - Optional - Maximum context size in tokens.

### Request Example
curl "https://api.featherless.ai/v1/models?available_on_current_plan=true" \
  -H "Authorization: Bearer $FEATHERLESS_API_KEY"
```

--------------------------------

### Run completions with OpenAI SDK

Source: https://featherless.ai/docs/api-examples-and-snippets

Configures the OpenAI client to point to the Featherless API base URL and executes a chat completion request.

```python
from openai import OpenAI

client = OpenAI(
  base_url="https://api.featherless.ai/v1",
  api_key="YOUR-API-KEY",
)

response = client.chat.completions.create(
  model='meta-llama/Meta-Llama-3.1-8B-Instruct',
  messages=[
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": "Hello!"}
  ],
)
print(response.model_dump()['choices'][0]['message']['content'])
```

--------------------------------

### Full working configuration for Featherless AI

Source: https://featherless.ai/docs/nemoclaw

A complete JSON configuration file defining models, agents, commands, channels, gateway settings, and plugins.

```json
{
  "update": { "checkOnStart": false },
  "models": {
    "mode": "merge",
    "providers": {
      "proxy": {
        "baseUrl": "http://localhost:18780/v1",
        "apiKey": "__OPENCLAW_REDACTED__",
        "api": "openai-completions",
        "models": [{ "id": "smart-model", "name": "smart-model" }]
      }
    }
  },
  "agents": {
    "defaults": {
      "model": { "primary": "proxy/smart-model" },
      "models": { "proxy/smart-model": { "alias": "smart-model" } }
    }
  },
  "commands": {
    "native": "auto",
    "nativeSkills": "auto",
    "restart": true,
    "ownerDisplay": "raw"
  },
  "channels": {
    "telegram": {
      "enabled": true,
      "botToken": "__OPENCLAW_REDACTED__",
      "dmPolicy": "allowlist",
      "allowFrom": ["YOUR_NUMERIC_TELEGRAM_USER_ID"],
      "groups": {
        "*": { "requireMention": true }
      }
    }
  },
  "gateway": {
    "bind": "lan",
    "controlUi": {
      "allowedOrigins": [
        "https://[port]-sbx-[your-sandbox-id].sandbox.featherless.ai"
      ]
    },
    "auth": {
      "mode": "trusted-proxy",
      "trustedProxy": { "userHeader": "x-forwarded-user" }
    },
    "trustedProxies": [
      "127.0.0.1", "10.0.0.0/8", "172.16.0.0/12", "192.168.0.0/16"
    ]
  },
  "plugins": {
    "entries": {
      "telegram": { "enabled": true }
    }
  }
}
```

--------------------------------

### Connect to models using LangChain

Source: https://featherless.ai/docs/api-examples-and-snippets

Initializes a ChatOpenAI instance with Featherless credentials and invokes the model with a list of messages.

```python
import os
from langchain_openai import ChatOpenAI
llm = ChatOpenAI(
    api_key=os.getenv('FEATHERLESS_API_KEY')
    base_url="https://api.featherless.ai/v1",
    model="meta-llama/Meta-Llama-3.1-8B-Instruct",
)
messages = [
    (
        "system",
        "You are a helpful assistant that translates English to French. Translate the user sentence.",
    ),
    (
        "human",
        "I love programming."
    ),
]
ai_msg = llm.invoke(messages)
ai_msg
```

--------------------------------

### POST /v1/completions

Source: https://featherless.ai/docs/completions

Generates a completion for the provided prompt and parameters using the Featherless AI model.

```APIDOC
## POST /v1/completions

### Description
The text completions endpoint takes a single string (the prompt) and generates a completion. This endpoint allows users to take fine-grained control of chat format or use models in non-assistant scenarios.

### Method
POST

### Endpoint
https://api.featherless.ai/v1/completions

### Parameters
#### Request Body
- **model** (string) - Required - ID of the model to use.
- **prompt** (string|array) - Required - The prompt(s) to generate completions for.
- **presence_penalty** (float) - Optional - Penalizes new tokens based on their presence in the generated text.
- **frequency_penalty** (float) - Optional - Penalizes new tokens based on their frequency in the generated text.
- **repetition_penalty** (float) - Optional - Penalizes new tokens based on their appearance in the prompt and generated text.
- **temperature** (float) - Optional - Controls the randomness of sampling.
- **top_p** (float) - Optional - Controls the cumulative probability of the most likely tokens.
- **top_k** (integer) - Optional - The number of top tokens to consider in the sampling process.
- **min_p** (float) - Optional - Sets a minimum probability threshold relative to the most likely token.
- **seed** (integer) - Optional - Sets a random seed for generation.
- **stop** (array) - Optional - A list of strings that will stop further generation.
- **stop_token_ids** (array) - Optional - Uses token IDs to halt generation.
- **include_stop_str_in_output** (boolean) - Optional - If true, includes stop strings in the output text.
- **max_tokens** (integer) - Optional - The maximum number of tokens to generate.
- **min_tokens** (integer) - Optional - The minimum number of tokens to generate.

### Request Example
{
  "model": "GalrionSoftworks/Margnum-12B-v1",
  "prompt": "Once upon a time",
  "temperature": 0.7,
  "max_tokens": 150,
  "top_p": 0.9,
  "frequency_penalty": 0.5,
  "presence_penalty": 0.0
}

### Response
#### Success Response (200)
- **id** (string) - Unique identifier for the completion.
- **object** (string) - The object type.
- **created** (integer) - Timestamp of creation.
- **model** (string) - The model used.
- **choices** (array) - List of completion choices.
- **usage** (object) - Token usage statistics.

#### Response Example
{
  "id": "cmpl-6YgK3ASw92kT14L5f8zJQ7yY",
  "object": "text_completion",
  "created": 1630569482,
  "model": "GalrionSoftworks/Margnum-12B-v1",
  "choices": [
    {
      "text": " in a land far, far away, there lived a wise old owl.",
      "index": 0,
      "logprobs": null,
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 5,
    "completion_tokens": 15,
    "total_tokens": 20
  }
}
```

--------------------------------

### Initialize ChatFeatherlessAi and process a conversation

Source: https://featherless.ai/docs/langchain

Configure the ChatFeatherlessAi client with an API key and invoke the model with a list of system and human messages.

```python
from langchain_featherless_ai import ChatFeatherlessAi
import os
# Set your API key
FEATHERLESSAI_API_KEY="your featherless api key" # Replace with actual API key
llm = ChatFeatherlessAi(
    api_key=f'{FEATHERLESSAI_API_KEY}',
    base_url="https://api.featherless.ai/v1",
)
# Setting up a Conversation
messages = [
    (
        "system",
        "You are a helpful assistant that translates English to French. Translate the user sentence.",
    ),
    (
        "human",
        "I love programming."
    ),
]
# Processing the response
ai_msg = llm.invoke(messages)
```

--------------------------------

### Basic Completion Request

Source: https://featherless.ai/docs/litellm

Perform a standard completion request using a Featherless AI model.

```python
from litellm import completion
response = completion(
    model="featherless_ai/featherless-ai/Qwerky-72B", # Example model. [1]
    api_key="YOUR FEATHERLESS API KEY",
    messages=[{"role": "user", "content": "Write a short poem about AI."}]
)
print(response.choices[0].message.content)
```

--------------------------------

### Text Generation with OpenAI Client

Source: https://featherless.ai/docs/hugging-face

Access Featherless models by configuring the OpenAI client with the specific router base URL.

```python
from openai import OpenAI

client = OpenAI(
    base_url="https://router.huggingface.co/featherless-ai",
    api_key="hf_xxxxxxxxxxxxxxxxxxxxxxxx"#featherless.ai or Hugging Face api key
)

messages = [
    {
        "role": "user",
        "content": "What is the capital of France?"
    }
]

completion = client.chat.completions.create(
    model="deepseek-ai/DeepSeek-R1-0528",
    messages=messages,
    max_tokens=500,
)

print(completion.choices[0].message)
```

--------------------------------

### Retrieve model list via HTTP GET

Source: https://featherless.ai/docs/api-reference-models

Use this endpoint to fetch the full list of models and their current state. It supports both authenticated and unauthenticated requests.

```http
GET https://api.featherless.ai/v1/models
```

--------------------------------

### Find tool-capable chat models with at least 32k context

Source: https://featherless.ai/docs/api-reference-models

Filters models by capabilities and minimum context length.

```bash
curl -H "Authorization: Bearer $FEATHERLESS_API_KEY" \
  "https://api.featherless.ai/v1/models?capabilities=chat,tool-use&context_length_min=32768"
```

--------------------------------

### Full OpenClaw Configuration

Source: https://featherless.ai/docs/openclaw

A complete JSON configuration file for OpenClaw, covering models, agents, commands, channels, gateway, and plugins.

```json
{
  "update": { "checkOnStart": false },
  "models": {
    "mode": "merge",
    "providers": {
      "proxy": {
        "baseUrl": "http://localhost:18780/v1",
        "apiKey": "__OPENCLAW_REDACTED__",
        "api": "openai-completions",
        "models": [{ "id": "smart-model", "name": "smart-model" }]
      }
    }
  },
  "agents": {
    "defaults": {
      "model": { "primary": "proxy/smart-model" },
      "models": { "proxy/smart-model": { "alias": "smart-model" } }
    }
  },
  "commands": {
    "native": "auto",
    "nativeSkills": "auto",
    "restart": true,
    "ownerDisplay": "raw"
  },
  "channels": {
    "telegram": {
      "enabled": true,
      "botToken": "__OPENCLAW_REDACTED__",
      "dmPolicy": "allowlist",
      "allowFrom": ["YOUR_NUMERIC_TELEGRAM_USER_ID"],
      "groups": {
        "*": { "requireMention": true }
      }
    }
  },
  "gateway": {
    "bind": "lan",
    "controlUi": {
      "allowedOrigins": [
        "https://18789-sbx-[your-sandbox-id].sandbox.featherless.ai"
      ]
    },
    "auth": {
      "mode": "trusted-proxy",
      "trustedProxy": { "userHeader": "x-forwarded-user" }
    },
    "trustedProxies": [
      "127.0.0.1", "10.0.0.0/8", "172.16.0.0/12", "192.168.0.0/16"
    ]
  },
  "plugins": {
    "entries": {
      "telegram": { "enabled": true }
    }
  }
}
```

--------------------------------

### Configure Thinking Mode

Source: https://featherless.ai/docs/chat-template-kwargs

Use enable_thinking to toggle reasoning behavior for models that support it.

```json
{
  "chat_template_kwargs": {
    "enable_thinking": true
  }
}
```

```json
{
  "chat_template_kwargs": {
    "enable_thinking": false
  }
}
```

--------------------------------

### Generate embeddings with Python SDK

Source: https://featherless.ai/docs/embeddings

Initializes the OpenAI client with the Featherless base URL to generate embeddings for a single input string.

```python
from openai import OpenAI

client = OpenAI(
    api_key="your-featherless-api-key",
    base_url="https://api.featherless.ai/v1"
)

response = client.embeddings.create(
    model="Qwen/Qwen3-Embedding-8B",
    input="Featherless makes serverless inference simple."
)

print(response.data[0].embedding[:5])
print(response.usage)
```

--------------------------------

### Text Generation with InferenceClient

Source: https://featherless.ai/docs/hugging-face

Perform chat completion by initializing the InferenceClient with the featherless-ai provider.

```python
from huggingface_hub import InferenceClient

# Initialize the InferenceClient with featherless.ai as the provider
client = InferenceClient(
    provider="featherless-ai",
    api_key="xxxxxxxxxxxxxxxxxxxxxxxx"# Replace with your API key (HF or custom)
)

# Define the chat messages
messages = [
    {
        "role": "user",
        "content": "What is the capital of France?"
    }
]

# Generate a chat completion
completion = client.chat.completions.create(
    model="deepseek-ai/DeepSeek-R1-0528",
    messages=messages,
    max_tokens=500
)

# Print the response
print(completion.choices[0].message)
```

--------------------------------

### Define Tool Schemas

Source: https://featherless.ai/docs/tool-calling

JSON definitions following the OpenAI format to inform the model about available tools and parameters.

```json
TOOLS = [
    {
        "type": "function",
        "function": {
            "name": "get_current_weather",
            "description": "Get the current weather in a given location",
            "parameters": {
                "type": "object",
                "properties": {
                    "location": {
                        "type": "string",
                        "description": "The city and state, e.g. San Francisco, CA"
                    },
                    "unit": {
                        "type": "string",
                        "enum": ["celsius", "fahrenheit"],
                        "description": "The unit of temperature"
                    }
                },
                "required": ["location"]
            }
        }
    },
    {
        "type": "function",
        "function": {
            "name": "calculate_area", 
            "description": "Calculate the area of a geometric shape",
            "parameters": {
                "type": "object",
                "properties": {
                    "shape": {
                        "type": "string",
                        "enum": ["circle", "rectangle"],
                        "description": "The shape to calculate area for"
                    },
                    "radius": {
                        "type": "number",
                        "description": "Radius for circle"
                    },
                    "length": {
                        "type": "number", 
                        "description": "Length for rectangle"
                    },
                    "width": {
                        "type": "number",
                        "description": "Width for rectangle"
                    }
                },
                "required": ["shape"]
            }
        }
    }
]
```

--------------------------------

### Define Aider model settings

Source: https://featherless.ai/docs/guides-aider

Configuration file for defining model-specific behaviors such as edit formats and token limits.

```yaml
- accepts_images: false
  cache_control: false
  caches_by_default: false
  edit_format: whole
  examples_as_sys_msg: true
  extra_params:
    max_tokens: 4096
  lazy: false
  name: openai/Qwen/Qwen2.5-72B-Instruct
  reminder: user
  send_undo_reply: false
  streaming: true
  use_repo_map: true
  use_system_prompt: true
  use_temperature: true
```

--------------------------------

### Preview rendered prompt with chat_template_kwargs

Source: https://featherless.ai/docs/chat-template-kwargs

Use the debug endpoint to verify how chat_template_kwargs affect the final prompt structure before sending a production request.

```bash
curl https://api.featherless.ai/models/Qwen/Qwen3-32B/debug/chat-format \
  -H "Authorization: Bearer $FEATHERLESS_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "Qwen/Qwen3-32B",
    "messages": [
      {
        "role": "user",
        "content": "Answer briefly: what is a Bloom filter?"
      }
    ],
    "chat_template_kwargs": {
      "enable_thinking": false
    }
  }'
```

--------------------------------

### Basic Completion Call

Source: https://featherless.ai/docs/llamaindex

Perform a simple text completion request using the FeatherlessLLM class.

```python
from llama_index.llms.featherlessai import FeatherlessLLM

llm = FeatherlessLLM(model="Qwen/Qwen3-32B", api_key="your api key")
resp = llm.complete("Is 9.9 or 9.11 bigger?")
print(resp)
```

--------------------------------

### Use do_reasoning Synonym

Source: https://featherless.ai/docs/chat-template-kwargs

The do_reasoning key is an accepted synonym for enable_thinking or thinking.

```json
{
  "chat_template_kwargs": {
    "do_reasoning": false
  }
}
```

--------------------------------

### Connect to Concurrency Stream with cURL

Source: https://featherless.ai/docs/api-reference-concurrency-stream

Establish a connection to the SSE stream using cURL. The --no-buffer flag is recommended to ensure real-time output.

```bash
curl --no-buffer https://api.featherless.ai/account/concurrency/stream \
  -H "Accept: text/event-stream" \
  -H "Authorization: Bearer $FEATHERLESS_API_KEY"
```

--------------------------------

### Send images using URLs with Python

Source: https://featherless.ai/docs/vision

Use the OpenAI client to send images via publicly accessible URLs.

```python
from openai import OpenAI

client = OpenAI(
    api_key="your-featherless-api-key",
    base_url="https://api.featherless.ai/v1"
)

response = client.chat.completions.create(
    model="google/gemma-3-27b-it", # or any supported vision model
    messages=[
        {
            "role": "user",
            "content": [
                {
                    "type": "text",
                    "text": "What's in this image?"
                },
                {
                    "type": "image_url",
                    "image_url": {
                        "url": "https://example.com/image.jpg"
                    }
                }
            ]
        }
    ]
)

print(response.choices[0].message.content)
```
Tokens:
Raw