Inference.sh

https://inference.sh/docs
Admin
Inference.sh is an AI workspace and agent runtime for building and deploying production AI agents...
Tokens:947,749
Snippets:6,460
Trust Score:5.3
Update:1 day ago
Show doc for...
Tokens:
Raw
### Setup Parameters: Run with Setup Configuration

Source: https://inference.sh/docs/api/sdk/running-apps

Shows how to pass setup parameters to configure the app instance. Workers with matching setup are 'warm' and start faster.

```python
result = client.run({
    "app": "infsh/flux",
    "setup": {"model": "schnell"},
    "input": {"prompt": "A sunset"}
})
```

--------------------------------

### client.run (with setup parameters)

Source: https://inference.sh/docs/api/sdk/running-apps

Runs an AI app with setup parameters to configure the app instance. Workers with matching setup are 'warm' and start faster.

```APIDOC
## client.run (with setup parameters)

### Description
Runs an AI app with setup parameters to configure the app instance. Workers with matching setup are 'warm' and start faster.

### Method
run

### Parameters

#### Request Body
- **app** (string) - Required - App identifier (`namespace/name` or `namespace/name@version`)
- **setup** (object) - Optional - Setup parameters (affects worker warmth)
- **input** (object) - Required - Input matching app schema

### Request Example
```python
result = client.run({
    "app": "infsh/flux",
    "setup": {"model": "schnell"},
    "input": {"prompt": "A sunset"}
})
```

```

--------------------------------

### Set up Docker on Linux

Source: https://inference.sh/docs/private/installing

Runs the engine setup command to install Docker if it is not already installed on Linux.

```bash
engine setup docker
```

--------------------------------

### Run app with setup parameters via CLI

Source: https://inference.sh/docs/apps/setup-parameters

Provides setup parameters when starting an app via the belt CLI. If parameters match an existing running instance, the request executes immediately; otherwise the instance re-initializes.

```bash
belt app run user/app-name --input input.json --setup setup.json
```

--------------------------------

### Base app structure (Python)

Source: https://inference.sh/docs/extend/app-code

Defines the entrypoint class with setup, run, and unload methods. Setup runs once when worker starts or config changes, run executes per request, and unload runs on shutdown.

```python
from inferencesh import BaseApp, BaseAppInput, BaseAppOutput
from pydantic import Field

class AppSetup(BaseAppInput):
    # Define setup parameters (optional)
    model_id: str = Field(default="gpt2", description="Model to load")

class AppInput(BaseAppInput):
    # Define inputs here
    pass

class AppOutput(BaseAppOutput):
    # Define outputs here
    pass

class App(BaseApp):
    async def setup(self, config: AppSetup):
        # Runs once when worker starts or config changes
        pass

    async def run(self, input_data: AppInput, metadata) -> AppOutput:
        # Runs for each request
        pass

    async def unload(self):
        # Runs on shutdown
        pass
```

--------------------------------

### Get skill store listing with curl

Source: https://inference.sh/docs/api/rest/store

Example curl request to retrieve store metadata for a single skill using its skill ID. Returns SkillStoreListingDTO including installs and uses counts.

```bash
curl "https://api.inference.sh/store/skills/skill_abc123"
```

--------------------------------

### Quick Example: Building a Tool with the Fluent API

Source: https://inference.sh/docs/api/agent/tools

Shows how to create a simple client tool using the fluent builder. The tool is named 'greet', has a description, a parameter 'name', and a handler that returns a greeting.

```python
from inferencesh import tool, string

greet = (
    tool("greet")
    .describe("Greet a user")
    .param("name", string("User's name"))
    .handler(lambda args: f"Hello, {args['name']}!")
    .build()
)
```

--------------------------------

### Get balance response example

Source: https://inference.sh/docs/api/rest/billing

Example JSON response from GET /billing/balance. The balance is in microcents.

```json
{
  "balance": 2500000003
}
```

--------------------------------

### Quick Start (Python)

Source: https://inference.sh/docs/api/sdk/overview

Initialize the client and run an app task, waiting for the result.

```APIDOC
## Quick Start (Python)

### Description
Create a client with your API key and run an app task, printing the output.

### Code
```python
from inferencesh import inference

client = inference(api_key="inf_your_key")

# Run an app and wait for result
result = client.tasks.run({
    "app": "infsh/flux",
    "input": {"prompt": "A sunset over mountains"}
})

print(result["output"])
```
```

--------------------------------

### GET /search example

Source: https://inference.sh/docs/api/rest/search

Example curl command for GET /search. Searches for 'flux' in the 'apps' collection. Returns up to 20 results per collection.

```bash
curl "https://api.inference.sh/search?q=flux&collection=apps"
```

--------------------------------

### cURL example for agent chat and tool approval

Source: https://inference.sh/docs/api/rest/agents

cURL commands to start or continue a chat, get chat details including tool invocations, and approve a tool awaiting human-in-the-loop (HIL). Uses the Authorization header with a Bearer token.

```bash
# Start or continue a chat
curl -X POST https://api.inference.sh/agents/run \
  -H "Authorization: Bearer inf_your_key" \
  -H "Content-Type: application/json" \
  -d '{
    "agent": "myteam/support-agent@latest",
    "input": { "text": "Hello!" }
  }'

# Get chat (includes tool invocations)
curl https://api.inference.sh/chats/chat_abc123 \
  -H "Authorization: Bearer inf_your_key"

# Approve a tool awaiting HIL
curl -X POST https://api.inference.sh/tools/ti_abc123/invoke \
  -H "Authorization: Bearer inf_your_key"
```

--------------------------------

### Quick Start: Run an app with Python SDK

Source: https://inference.sh/docs/api/sdk/overview

Initialize the client with an API key and run an app, waiting for the result. The result is printed as a dictionary.

```python
from inferencesh import inference

client = inference(api_key="inf_your_key")

# Run an app and wait for result
result = client.tasks.run({
    "app": "infsh/flux",
    "input": {"prompt": "A sunset over mountains"}
})

print(result["output"])
```

--------------------------------

### GET /credentials/capabilities

Source: https://inference.sh/docs/api/rest/credentials

Returns credential keys your apps can declare (for example google.sheets, google-sa.files). Each capability includes provider, type, OAuth scopes, and whether it is available on this server. Use this to build setup UIs or validate inf.yml credential keys before deploy.

```APIDOC
## GET /credentials/capabilities

### Description
Returns credential keys your apps can declare (for example `google.sheets`, `google-sa.files`). Each capability includes `provider`, `type`, OAuth `scopes`, and whether it is available on this server.

### Method
GET

### Endpoint
/credentials/capabilities

### Authentication
Requires Authorization: Bearer inf_your_key

### Request Example
```bash
curl https://api.inference.sh/credentials/capabilities \
  -H "Authorization: Bearer inf_your_key"
```

### Response
#### Success Response (200)
- **capabilities** (array) - List of capability objects.
  - **key** (string) - Credential key (e.g., `google.sheets`).
  - **provider** (string) - Provider name (e.g., `google`).
  - **type** (string) - Credential type (e.g., `oauth`).
  - **scopes** (array of strings) - OAuth scopes.
  - **display_name** (string) - Display name.
  - **available** (boolean) - Whether available on this server.

#### Response Example
```json
{
  "capabilities": [
    {
      "key": "google.sheets",
      "provider": "google",
      "type": "oauth",
      "scopes": ["https://www.googleapis.com/auth/spreadsheets"],
      "display_name": "Google Sheets",
      "available": true
    }
  ]
}
```
```

--------------------------------

### Setup Parameters: Configure App Instance

Source: https://inference.sh/docs/api/sdk-swift

Runs an app with setup parameters to configure the app instance (e.g., model selection). Workers with matching setup are 'warm' and skip setup.

```swift
let task = try await client.tasks.run(ApiAppRunRequest(
    app: "my-app",
    setup: ["model": "schnell"],
    input: ["prompt": "hello"]
))
```

--------------------------------

### Template Agent Quick Start (Python)

Source: https://inference.sh/docs/api/agent/overview

Use an existing agent from your workspace. Initialize the client with your API key, retrieve the agent by reference, send a message, and print the response text.

```python
from inferencesh import inference

client = inference(api_key="inf_your_key")

agent = client.agent("my-team/support-agent@latest")

response = agent.send_message("How can I reset my password?")
print(response.text)
```

--------------------------------

### Get Task Cost Response Example

Source: https://inference.sh/docs/api/rest/tasks

Example JSON response from the Get Task Cost endpoint. The total and charged fields are in microcents; discount and refunded fields are omitted when zero.

```json
{
  "total": 50000,
  "charged": 50000
}
```

--------------------------------

### Example output of belt mcp get linear

Source: https://inference.sh/docs/connectors/browsing

Example output from `belt mcp get linear` showing the detailed server information and a hint to run `belt mcp tools linear`.

```text
Linear

Project management and issue tracking

  slug         linear
  category     developer
  auth         oauth
  server       https://mcp.linear.app
  docs         https://linear.app/docs/mcp
  status       connected

  tools: belt mcp tools linear
```

--------------------------------

### Quickstart: Python OpenAI client

Source: https://inference.sh/docs/api/rest/openai

Python example using the official OpenAI SDK to call a chat model. Requires an API key with apps:execute scope for chat completions.

```python
from openai import OpenAI

client = OpenAI(
    base_url="https://api.inference.sh/models/openai",
    api_key="<your inference.sh API key>",
)

completion = client.chat.completions.create(
    model="openrouter/claude-sonnet-5",
    messages=[{"role": "user", "content": "Say hello in five words"}],
)
print(completion.choices[0].message.content)
```

--------------------------------

### Define AppSetup and App with model selection

Source: https://inference.sh/docs/apps/setup-parameters

Python example showing how to define an AppSetup with a Literal field for model selection and an App that loads the model based on the setup value. The default model is 'base'. This setup allows one app to serve multiple models, with each distinct setup value running as its own instance.

```python
from typing import Literal
from inferencesh import BaseApp, BaseAppSetup
from pydantic import Field

MODELS = {
    "small": "org/model-small",
    "base": "org/model-base",
}

class AppSetup(BaseAppSetup):
    model: Literal["small", "base"] = Field(default="base", description="Model size to load")

class App(BaseApp):
    async def setup(self, config: AppSetup):
        self.model = load_model(MODELS[config.model])
```

--------------------------------

### GET /suggest curl example

Source: https://inference.sh/docs/api/rest/search

Performs a GET request to the /suggest endpoint with a query parameter 'flux'.

```bash
curl "https://api.inference.sh/suggest?q=flux"
```

--------------------------------

### Installation (Python)

Source: https://inference.sh/docs/api/sdk/overview

Install the Python SDK with pip, optionally with async support.

```APIDOC
## Installation (Python)

### Description
Install the Python SDK using pip. Optionally include async support.

### Code
```bash
pip install inferencesh
# With async support
pip install inferencesh[async]
```
```

--------------------------------

### curl GET /me

Source: https://inference.sh/docs/api/rest/teams

Bash example using curl to call the GET /me endpoint with an API key in the Authorization header.

```bash
curl https://api.inference.sh/me \
  -H "Authorization: Bearer inf_your_key"
```

--------------------------------

### Run app with setup parameters via Python SDK

Source: https://inference.sh/docs/apps/setup-parameters

Uses the Python SDK client.run method to provide setup parameters along with input. The setup block includes model_id and precision.

```python
client.run({
    "app": "my-app",
    "setup": {
        "model_id": "meta-llama/Llama-2-7b",
        "precision": "fp16"
    },
    "input": {
        "prompt": "Hello world"
    }
})
```

--------------------------------

### Check username availability response example

Source: https://inference.sh/docs/api/rest/teams

Example JSON response from the GET /teams/check-username endpoint. Shows a taken username with reason 'taken'.

```json
{
  "value": "acme-eng",
  "available": false,
  "reason": "taken"
}
```

--------------------------------

### Python setup with AppSetup and App class

Source: https://inference.sh/docs/extend/app-code

Defines a setup schema with model_id and precision parameters, and loads a model in the setup method. Runs once per configuration; each distinct set of setup values runs as its own instance.

```Python
class AppSetup(BaseAppInput):
    model_id: str = Field(default="gpt2", description="Model to load")
    precision: str = Field(default="fp16", description="Model precision")

class App(BaseApp):
    async def setup(self, config: AppSetup):
        # transformers (and torch) must be in requirements.txt
        from transformers import AutoModel
        self.model = AutoModel.from_pretrained(config.model_id)
```

--------------------------------

### Get webhook secret response example

Source: https://inference.sh/docs/api/rest/triggers

Example JSON response from the GET /triggers/{id}/webhook-secret endpoint, containing the trigger's signing secret (whsec_…). Requires agents:write and write access to the trigger. A trigger with no signing secret returns 404.

```json
{ "data": { "value": "whsec_…" } }
```

--------------------------------

### Basic Usage: Create a greet tool

Source: https://inference.sh/docs/api/agent/client-tools

Shows how to create a client tool using the builder pattern. The tool is named 'greet', has a description, a parameter 'name', and a handler that returns a greeting. The example is in Python.

```python
from inferencesh import tool, string

greet = (
    tool("greet")
    .describe("Greet a user by name")
    .param("name", string("User's name"))
    .handler(lambda args: f"Hello, {args['name']}!")
    .build()
)
```

--------------------------------

### GET /me/survey response example

Source: https://inference.sh/docs/api/rest/teams

Example JSON response from the GET /me/survey endpoint. It shows a structured answer with fields id, question_id, response, agent, source, and context. The response lists structured answers first, then freeform open:* rows, up to 100 rows total.

```json
{
  "responses": [
    {
      "id": "srv_abc123",
      "question_id": "discovery",
      "response": "github",
      "agent": "cursor",
      "source": "cli",
      "context": "belt login, belt app run, belt task get abc123"
    }
  ]
}
```

--------------------------------

### Full Example

Source: https://inference.sh/docs/api/agent/connector-tools

Complete example showing how to create read-only and write tools with approval requirements.

```APIDOC
## Full Example

### Description
Complete example showing how to create read-only and write tools with approval requirements.

### Example
```python
from inferencesh import mcp_tool

# Read-only tools
list_issues = (
    mcp_tool("list_issues", server="linear", tool_name="list_issues")
    .describe("List Linear issues for a project")
    .build()
)

get_issue = (
    mcp_tool("get_issue", server="linear", tool_name="get_issue")
    .describe("Get full details for a Linear issue")
    .build()
)

# Write tools — require approval
create_issue = (
    mcp_tool("create_issue", server="linear", tool_name="create_issue")
    .describe("Create a new Linear issue")
    .require_approval()
    .build()
)

notify_slack = (
    mcp_tool("notify_slack", server="slack", tool_name="send_message")
    .describe("Post a message to a Slack channel")
    .require_approval()
    .build()
)

all_tools = [list_issues, get_issue, create_issue, notify_slack]
```
```

--------------------------------

### Basic Input widget example

Source: https://inference.sh/docs/agents/widgets/input

Minimal input widget configuration with a name and placeholder.

```go
{Type: models.WidgetNodeTypeInput, Name: "name", Placeholder: "Enter name"}
```

--------------------------------

### belt remote enable / disable

Source: https://inference.sh/docs/extend/cli-setup

Linux (systemd): install or remove a user service that starts the remote at boot. `enable` accepts the same flags as `start`.

```APIDOC
## belt remote enable / disable

### Description
Linux (systemd): install or remove a user service that starts the remote at boot. `enable` accepts the same flags as `start`.

### Command
belt remote enable / disable

### Flags
- `enable` accepts the same flags as `start` (e.g., `--name`, `--exec`, `--agents`, `--detach`).
```

--------------------------------

### List active add-ons curl example

Source: https://inference.sh/docs/api/rest/subscription

Example curl command to GET /subscription/addons, returning active add-on subscriptions. Requires billing:read.

```bash
curl https://api.inference.sh/subscription/addons \
  -H "Authorization: Bearer inf_your_key"
```

--------------------------------

### Create an App

Source: https://inference.sh/docs/extend/extending-apps

Creates a new app directory and initializes the app structure with belt init.

```bash
mkdir my-app && cd my-app
belt init
```

--------------------------------

### App setup parameters UI

Source: https://inference.sh/docs/apps/overview

Displays the setup dialog for an app, showing configuration options like model and precision. Changing these parameters restarts the app instance.

```text
┌────────────────────────────────┐
│  Setup:                        │
│  • Model: [Llama-2-7b ▼]       │
│  • Precision: [fp16 ▼]         │
└────────────────────────────────┘
```

--------------------------------

### GET /store/apps with limit parameter

Source: https://inference.sh/docs/api/rest/store

Uses the GET endpoint to list approved apps with a limit of 20. This is a simple example without filters or search.

```bash
curl "https://api.inference.sh/store/apps?limit=20"
```

--------------------------------

### Install CLI and login

Source: https://inference.sh/docs/extend/coding-agents

Installs the inference.sh CLI and logs in. Run this before creating an app.

```bash
curl -fsSL https://cli.inference.sh | sh
belt login
```

--------------------------------

### Usage summary response example

Source: https://inference.sh/docs/api/rest/usage

Example JSON response from the GET /usage/summary endpoint. Shows usage aggregated by tier, type, model, and total in seconds.

```json
{
  "tier_usage": {
    "cloud": 3600,
    "private": 1200
  },
  "type_usage": {
    "gpu": 3000,
    "cpu": 1800
  },
  "model_usage": {
    "A100": 2400
  },
  "total_usage": 4800
}
```

--------------------------------

### cURL Examples - Skills

Source: https://inference.sh/docs/api/rest/skills

cURL examples for listing public skills, getting skill content, creating a skill, and importing skills from GitHub (batch and streaming).

```APIDOC
## cURL Examples - Skills

### Description
cURL examples for listing public skills, getting skill content, creating a skill, and importing skills from GitHub (batch and streaming).

### Endpoints

#### GET /skills
Lists public skills.

#### GET /skills/{team}/{name}/content
Gets the content of a skill.

#### POST /skills
Creates a new skill.

#### POST /skills/import/github
Imports skills from a GitHub repository (batch).

#### POST /skills/import/github/stream
Imports skills from a GitHub repository (streaming NDJSON).

### Examples

```bash
# List public skills
curl https://api.inference.sh/skills

# Get skill content
curl https://api.inference.sh/skills/myteam/code-review/content

# Create a skill
curl -X POST https://api.inference.sh/skills \
  -H "Authorization: Bearer inf_your_key" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "my-skill",
    "description": "My custom skill",
    "instructions": "# Instructions\n\nDo the thing."
  }'

# Import from GitHub (batch)
curl -X POST https://api.inference.sh/skills/import/github \
  -H "Authorization: Bearer inf_your_key" \
  -H "Content-Type: application/json" \
  -d '{"url": "https://github.com/myteam/skills-repo"}'

# Import from GitHub (streaming NDJSON)
curl -N -X POST https://api.inference.sh/skills/import/github/stream \
  -H "Authorization: Bearer inf_your_key" \
  -H "Content-Type: application/json" \
  -d '{"url": "https://github.com/myteam/skills-repo", "paths": ["code-review"]}'
```

```

--------------------------------

### Example response for GET /teams/{teamId}/rules

Source: https://inference.sh/docs/api/rest/usage-policy

Shows the structure of the governance layer returned by the GET endpoint, including reach, rules, and rule kinds.

```json
{
  "subject_type": "team",
  "subject_id": "team_xyz",
  "label": "Acme",
  "governance": true,
  "editable": true,
  "reach": {
    "App": "org",
    "Mcp": "public"
  },
  "reach_enforced": ["App"],
  "rules": [
    {
      "id": "rule_abc",
      "effect": "deny",
      "enforcement": "default",
      "kind": "App",
      "specifier": "app_123",
      "label": "competitor/scraper",
      "created_at": "2026-09-01T12:00:00Z",
      "created_by": "usr_owner"
    }
  ],
  "rule_kinds": [
    { "kind": "App", "selector": false },
    { "kind": "Agent", "selector": false }
  ]
}
```

--------------------------------

### Generate example input with belt app test --save-example

Source: https://inference.sh/docs/extend/extending-apps

Creates an input.json file for testing. Run this before testing locally.

```bash
belt app test --save-example
# Creates input.json
```

--------------------------------

### Quick Example

Source: https://inference.sh/docs/api/agent/tools

Example of defining a client tool using the fluent builder API.

```APIDOC
## Quick Example

### Description
Defines a client tool named "greet" that greets a user.

### Method
N/A (SDK builder)

### Endpoint
N/A (SDK builder)

### Parameters
- `name` (string) - Required - User's name

### Request Example
```python
from inferencesh import tool, string

greet = (
    tool("greet")
    .describe("Greet a user")
    .param("name", string("User's name"))
    .handler(lambda args: f"Hello, {args['name']}!")
    .build()
)
```

### Response
N/A (SDK builder)
```

--------------------------------

### Get secret response JSON

Source: https://inference.sh/docs/api/rest/secrets

Example JSON response from the `GET /secrets/{key}` endpoint. Contains secret metadata including `masked_value` but not the plaintext value.

```json
{
  "id": "sec_abc123",
  "key": "OPENAI_API_KEY",
  "masked_value": "sk-…xxxx",
  "description": "OpenAI API key for GPT models",
  "scope": "team",
  "created_at": "2026-01-15T10:30:00Z"
}
```

--------------------------------

### Start device authorization with curl

Source: https://inference.sh/docs/api/rest/device-auth

Example curl command to start device authorization. Sends a POST request with a JSON body specifying token_kind as 'session'.

```bash
curl -X POST https://api.inference.sh/device/auth \
  -H "Content-Type: application/json" \
  -d '{"token_kind":"session"}'
```

--------------------------------

### Setup and Default Inputs for App Tool

Source: https://inference.sh/docs/api/agent/app-tools

Configures an app tool with setup values (always passed, hidden from agent) and default input values (overridable by agent). Demonstrates the .setup() and .input() methods.

```python
# Setup: One-time configuration (API keys, model selection)
# Input: Default values the agent can override

transcribe = (
    app_tool("transcribe", "infsh/whisper@latest")
    .describe("Transcribe audio to text")
    .setup({
        "model": "large-v3",      # Always use this model
        "language": "auto"        # Auto-detect language
    })
    .input({
        "timestamps": True,       # Default: include timestamps
        "format": "srt"           # Default output format
    })
    .build()
)
```

--------------------------------

### List Skills Response Example

Source: https://inference.sh/docs/api/rest/skills

Example JSON response from `GET /skills` or `POST /skills/list` showing a paginated list of skill objects with cursor-based pagination fields.

```json
{
  "items": [
    {
      "id": "skill_abc123",
      "namespace": "myteam",
      "name": "code-review",
      "description": "Guidelines for reviewing pull requests",
      "version_id": "ver_xyz789",
      "version": {
        "id": "ver_xyz789",
        "instructions": { "path": "SKILL.md", "uri": "https://..." },
        "files": [{ "path": "references/checklist.md", "uri": "https://..." }],
        "content_hash": "a3f5c9e2b1d4...",
        "tags": ["development"],
        "license": "MIT"
      },
      "visibility": "public",
      "created_at": "2026-01-15T10:30:00Z"
    }
  ],
  "next_cursor": "abc123",
  "has_next": true,
  "has_previous": false,
  "items_per_page": 50,
  "total_items": 120
}
```

--------------------------------

### Belt CLI supplementary files tip

Source: https://inference.sh/docs/skills/other-agents

Example output from the CLI when a skill has supplementary files, showing tips for installing and listing them.

```text
  tip: this skill has 3 supplementary file(s) not included above.
    install locally:  belt skill add inferencesh/code-review
    list files:       belt skill files inferencesh/code-review
```
Tokens:
Raw