Portkey

https://portkey.ai/docs
Admin
Portkey is a comprehensive AI platform that provides a unified interface for interacting with over...
Tokens:3,037,892
Snippets:22,909
Trust Score:9.7
Update:2 weeks ago
Show doc for...
Tokens:
Raw
### Quick Start with Portkey SDK (Python and JavaScript)

Source: https://portkey.ai/docs/integrations/llms/deepinfra

Quick start examples using the Portkey SDK to call a Deepinfra model. Requires installing portkey-ai and adding the @deepinfra provider in the model catalog. The model is specified as @deepinfra/nvidia/Nemotron-4-340B-Instruct.

```python
from portkey_ai import Portkey

# 1. Install: pip install portkey-ai
# 2. Add @deepinfra provider in model catalog
# 3. Use it:

portkey = Portkey(api_key="PORTKEY_API_KEY")

response = portkey.chat.completions.create(
    model="@deepinfra/nvidia/Nemotron-4-340B-Instruct",
    messages=[{"role": "user", "content": "Say this is a test"}]
)

print(response.choices[0].message.content)
```

```javascript
import Portkey from 'portkey-ai'

// 1. Install: npm install portkey-ai
// 2. Add @deepinfra provider in model catalog
// 3. Use it:

const portkey = new Portkey({
    apiKey: "PORTKEY_API_KEY"
})

const response = await portkey.chat.completions.create({
    model: "@deepinfra/nvidia/Nemotron-4-340B-Instruct",
    messages: [{ role: "user", content: "Say this is a test" }]
})

console.log(response.choices[0].message.content)
```

--------------------------------

### Quick Start with @vertex-ai provider (Python, JavaScript, cURL)

Source: https://portkey.ai/docs/llms-full.txt

Quick start examples for using Portkey with the @vertex-ai provider. Requires installing the portkey-ai package (pip or npm) and adding the @vertex-ai provider in Model Catalog with Service Account JSON. The Python and JavaScript examples use the Portkey client, while the cURL example uses the REST API with the x-portkey-api-key header.

```python
from portkey_ai import Portkey

# 1. Install: pip install portkey-ai
# 2. Add @vertex-ai provider in Model Catalog with Service Account JSON
# 3. Use it:

portkey = Portkey(api_key="PORTKEY_API_KEY")

response = portkey.chat.completions.create(
    model="@vertex-ai/gemini-3-pro-preview",
    messages=[{"role": "user", "content": "Say this is a test"}]
)

print(response.choices[0].message.content)
```

```javascript
import Portkey from 'portkey-ai'

// 1. Install: npm install portkey-ai
// 2. Add @vertex-ai provider in Model Catalog with Service Account JSON
// 3. Use it:

const portkey = new Portkey({
    apiKey: "PORTKEY_API_KEY"
})

const response = await portkey.chat.completions.create({
    model: "@vertex-ai/gemini-3-pro-preview",
    messages: [{ role: "user", content: "Say this is a test" }]
})

console.log(response.choices[0].message.content)
```

```sh
# 1. Add @vertex-ai provider in Model Catalog with Service Account JSON
# 2. Use it:

curl https://api.portkey.ai/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "x-portkey-api-key: $PORTKEY_API_KEY" \
  -d '{
    "model": "@vertex-ai/gemini-3-pro-preview",
    "messages": [{"role": "user", "content": "Say this is a test"}]
  }'
```

--------------------------------

### Quick Start: Portkey with Replicate (Python, JavaScript, cURL)

Source: https://portkey.ai/docs/llms-full.txt

Quick start examples for using Portkey as a proxy to Replicate. They show how to install the SDK, add the @replicate provider in the model catalog, and make a prediction request to the /predictions endpoint. The Python and JavaScript examples use the Portkey SDK's post() method, while the cURL example uses the Portkey API directly.

```python
from portkey_ai import Portkey

# 1. Install: pip install portkey-ai
# 2. Add @replicate provider in model catalog
# 3. Use it:

portkey = Portkey(
    api_key="PORTKEY_API_KEY",
    provider="@replicate"
)

response = portkey.post(
    url="predictions",
    data={
        "version": "MODEL_VERSION_ID",
        "input": {"prompt": "Hello, world!"}
    }
)

print(response)
```

```javascript
import Portkey from 'portkey-ai'

// 1. Install: npm install portkey-ai
// 2. Add @replicate provider in model catalog
// 3. Use it:

const portkey = new Portkey({
    apiKey: "PORTKEY_API_KEY",
    provider: "@replicate"
})

const response = await portkey.post({
    url: "predictions",
    data: {
        version: "MODEL_VERSION_ID",
        input: { prompt: "Hello, world!" }
    }
})

console.log(response)
```

```sh
# 1. Add @replicate provider in model catalog
# 2. Use it:

curl https://api.portkey.ai/v1/predictions \
  -H "Content-Type: application/json" \
  -H "x-portkey-api-key: $PORTKEY_API_KEY" \
  -d '{
    "version": "MODEL_VERSION_ID",
    "input": {"prompt": "Hello, world!"}
  }'
```

--------------------------------

### Quick Start: Use Ncompass models with Portkey

Source: https://portkey.ai/docs/llms-full.txt

Quick Start examples for using Ncompass models through Portkey. Each example shows how to make a chat completion request with the model identifier @ncompass/meta-llama/Llama-3.1-8B-Instruct. The Portkey SDK examples require installing portkey-ai and adding the @ncompass provider in the model catalog. The OpenAI-compatible examples use the Portkey gateway URL and require installing openai and portkey-ai. The cURL example uses the Portkey API endpoint with the x-portkey-api-key header.

```python
from portkey_ai import Portkey

# 1. Install: pip install portkey-ai
# 2. Add @ncompass provider in model catalog
# 3. Use it:

portkey = Portkey(api_key="PORTKEY_API_KEY")

response = portkey.chat.completions.create(
    model="@ncompass/meta-llama/Llama-3.1-8B-Instruct",
    messages=[{"role": "user", "content": "Hello!"}]
)

print(response.choices[0].message.content)
```

```javascript
import Portkey from 'portkey-ai'

// 1. Install: npm install portkey-ai
// 2. Add @ncompass provider in model catalog
// 3. Use it:

const portkey = new Portkey({
    apiKey: "PORTKEY_API_KEY"
})

const response = await portkey.chat.completions.create({
    model: "@ncompass/meta-llama/Llama-3.1-8B-Instruct",
    messages: [{ role: "user", content: "Hello!" }]
})

console.log(response.choices[0].message.content)
```

```python
from openai import OpenAI
from portkey_ai import PORTKEY_GATEWAY_URL

# 1. Install: pip install openai portkey-ai
# 2. Add @ncompass provider in model catalog
# 3. Use it:

client = OpenAI(
    api_key="PORTKEY_API_KEY",  # Portkey API key
    base_url=PORTKEY_GATEWAY_URL
)

response = client.chat.completions.create(
    model="@ncompass/meta-llama/Llama-3.1-8B-Instruct",
    messages=[{"role": "user", "content": "Hello!"}]
)

print(response.choices[0].message.content)
```

```javascript
import OpenAI from "openai"
import { PORTKEY_GATEWAY_URL } from "portkey-ai"

// 1. Install: npm install openai portkey-ai
// 2. Add @ncompass provider in model catalog
// 3. Use it:

const client = new OpenAI({
    apiKey: "PORTKEY_API_KEY",  // Portkey API key
    baseURL: PORTKEY_GATEWAY_URL
})

const response = await client.chat.completions.create({
    model: "@ncompass/meta-llama/Llama-3.1-8B-Instruct",
    messages: [{ role: "user", content: "Hello!" }]
})

console.log(response.choices[0].message.content)
```

```sh
# 1. Add @ncompass provider in model catalog
# 2. Use it:

curl https://api.portkey.ai/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "x-portkey-api-key: $PORTKEY_API_KEY" \
  -d '{
    "model": "@ncompass/meta-llama/Llama-3.1-8B-Instruct",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'
```

--------------------------------

### Quickstart: Create chat completion with Portkey

Source: https://portkey.ai/docs/api-reference/sdk/node

Minimal example to create a chat completion using the Portkey SDK with a provider slug. Add providers in the Model Catalog to get a provider slug (e.g., @openai-prod).

```javascript
import Portkey from 'portkey-ai';

const portkey = new Portkey({
  apiKey: process.env.PORTKEY_API_KEY,
  provider: '@openai-prod' // Provider slug from Model Catalog
});

const response = await portkey.chat.completions.create({
  messages: [{ role: 'user', content: 'Hello, world!' }],
  model: 'gpt-4o'
});

console.log(response.choices[0].message.content);
```

--------------------------------

### Quick Start with Dashscope via Portkey

Source: https://portkey.ai/docs/llms-full.txt

Quick start examples for using Dashscope through Portkey. Each example requires adding the @dashscope provider in the model catalog and using the model @dashscope/qwen-turbo. The Portkey SDK examples use the PORTKEY_API_KEY, while the OpenAI-compatible examples use the Portkey gateway URL. The cURL example uses the x-portkey-api-key header.

```python
from portkey_ai import Portkey

# 1. Install: pip install portkey-ai
# 2. Add @dashscope provider in model catalog
# 3. Use it:

portkey = Portkey(api_key="PORTKEY_API_KEY")

response = portkey.chat.completions.create(
    model="@dashscope/qwen-turbo",
    messages=[{"role": "user", "content": "Hello!"}]
)

print(response.choices[0].message.content)
```

```javascript
import Portkey from 'portkey-ai'

// 1. Install: npm install portkey-ai
// 2. Add @dashscope provider in model catalog
// 3. Use it:

const portkey = new Portkey({
    apiKey: "PORTKEY_API_KEY"
})

const response = await portkey.chat.completions.create({
    model: "@dashscope/qwen-turbo",
    messages: [{ role: "user", content: "Hello!" }]
})

console.log(response.choices[0].message.content)
```

```python
from openai import OpenAI
from portkey_ai import PORTKEY_GATEWAY_URL

# 1. Install: pip install openai portkey-ai
# 2. Add @dashscope provider in model catalog
# 3. Use it:

client = OpenAI(
    api_key="PORTKEY_API_KEY",  # Portkey API key
    base_url=PORTKEY_GATEWAY_URL
)

response = client.chat.completions.create(
    model="@dashscope/qwen-turbo",
    messages=[{"role": "user", "content": "Hello!"}]
)

print(response.choices[0].message.content)
```

```javascript
import OpenAI from "openai"
import { PORTKEY_GATEWAY_URL } from "portkey-ai"

// 1. Install: npm install openai portkey-ai
// 2. Add @dashscope provider in model catalog
// 3. Use it:

const client = new OpenAI({
    apiKey: "PORTKEY_API_KEY",  // Portkey API key
    baseURL: PORTKEY_GATEWAY_URL
})

const response = await client.chat.completions.create({
    model: "@dashscope/qwen-turbo",
    messages: [{ role: "user", content: "Hello!" }]
})

console.log(response.choices[0].message.content)
```

```bash
# 1. Add @dashscope provider in model catalog
# 2. Use it:

curl https://api.portkey.ai/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "x-portkey-api-key: $PORTKEY_API_KEY" \
  -d '{
    "model": "@dashscope/qwen-turbo",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'
```

--------------------------------

### Quick Start with Nebius via Portkey

Source: https://portkey.ai/docs/llms-full.txt

Quick start examples for using Nebius through Portkey. All examples require adding the @nebius provider in the model catalog first. The Portkey SDK examples use the @nebius/deepseek-ai/DeepSeek-V3 model, while the OpenAI-compatible examples use the Portkey gateway URL. The cURL example uses the x-portkey-api-key header.

```python
from portkey_ai import Portkey

# 1. Install: pip install portkey-ai
# 2. Add @nebius provider in model catalog
# 3. Use it:

portkey = Portkey(api_key="PORTKEY_API_KEY")

response = portkey.chat.completions.create(
    model="@nebius/deepseek-ai/DeepSeek-V3",
    messages=[{"role": "user", "content": "Hello!"}]
)

print(response.choices[0].message.content)
```

```javascript
import Portkey from 'portkey-ai'

// 1. Install: npm install portkey-ai
// 2. Add @nebius provider in model catalog
// 3. Use it:

const portkey = new Portkey({
    apiKey: "PORTKEY_API_KEY"
})

const response = await portkey.chat.completions.create({
    model: "@nebius/deepseek-ai/DeepSeek-V3",
    messages: [{ role: "user", content: "Hello!" }]
})

console.log(response.choices[0].message.content)
```

```python
from openai import OpenAI
from portkey_ai import PORTKEY_GATEWAY_URL

# 1. Install: pip install openai portkey-ai
# 2. Add @nebius provider in model catalog
# 3. Use it:

client = OpenAI(
    api_key="PORTKEY_API_KEY",  # Portkey API key
    base_url=PORTKEY_GATEWAY_URL
)

response = client.chat.completions.create(
    model="@nebius/deepseek-ai/DeepSeek-V3",
    messages=[{"role": "user", "content": "Hello!"}]
)

print(response.choices[0].message.content)
```

```javascript
import OpenAI from "openai"
import { PORTKEY_GATEWAY_URL } from "portkey-ai"

// 1. Install: npm install openai portkey-ai
// 2. Add @nebius provider in model catalog
// 3. Use it:

const client = new OpenAI({
    apiKey: "PORTKEY_API_KEY",  // Portkey API key
    baseURL: PORTKEY_GATEWAY_URL
})

const response = await client.chat.completions.create({
    model: "@nebius/deepseek-ai/DeepSeek-V3",
    messages: [{ role: "user", content: "Hello!" }]
})

console.log(response.choices[0].message.content)
```

```sh
# 1. Add @nebius provider in model catalog
# 2. Use it:

curl https://api.portkey.ai/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "x-portkey-api-key: $PORTKEY_API_KEY" \
  -d '{
    "model": "@nebius/deepseek-ai/DeepSeek-V3",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'
```

--------------------------------

### Quick Start with Modal using Portkey SDK

Source: https://portkey.ai/docs/integrations/llms/modal

Quick start examples for using Modal with Portkey. Install the SDK, add the @modal provider in the model catalog, then make a chat completion request. The tip mentions you can set provider="@modal" in Portkey() and use just model="your-model-name".

```python
from portkey_ai import Portkey

# 1. Install: pip install portkey-ai
# 2. Add @modal provider in model catalog
# 3. Use it:

portkey = Portkey(api_key="PORTKEY_API_KEY")

response = portkey.chat.completions.create(
    model="@modal/your-model-name",
    messages=[{"role": "user", "content": "Hello!"}]
)

print(response.choices[0].message.content)

```

```javascript
import Portkey from 'portkey-ai'

// 1. Install: npm install portkey-ai
// 2. Add @modal provider in model catalog
// 3. Use it:

const portkey = new Portkey({
    apiKey: "PORTKEY_API_KEY"
})

const response = await portkey.chat.completions.create({
    model: "@modal/your-model-name",
    messages: [{ role: "user", content: "Hello!" }]
})

console.log(response.choices[0].message.content)

```

```python
from openai import OpenAI
from portkey_ai import PORTKEY_GATEWAY_URL

# 1. Install: pip install openai portkey-ai
# 2. Add @modal provider in model catalog
# 3. Use it:

client = OpenAI(
    api_key="PORTKEY_API_KEY",  # Portkey API key
    base_url=PORTKEY_GATEWAY_URL
)

response = client.chat.completions.create(
    model="@modal/your-model-name",
    messages=[{"role": "user", "content": "Hello!"}]
)

print(response.choices[0].message.content)

```

```javascript
import OpenAI from "openai"
import { PORTKEY_GATEWAY_URL } from "portkey-ai"

// 1. Install: npm install openai portkey-ai
// 2. Add @modal provider in model catalog
// 3. Use it:

const client = new OpenAI({
    apiKey: "PORTKEY_API_KEY",  // Portkey API key
    baseURL: PORTKEY_GATEWAY_URL
})

const response = await client.chat.completions.create({
    model: "@modal/your-model-name",
    messages: [{ role: "user", content: "Hello!" }]
})

console.log(response.choices[0].message.content)

```

```bash
# 1. Add @modal provider in model catalog
# 2. Use it:

curl https://api.portkey.ai/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "x-portkey-api-key: $PORTKEY_API_KEY" \
  -d '{
    "model": "@modal/your-model-name",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'

```

--------------------------------

### Quick Start: Upstage with Portkey

Source: https://portkey.ai/docs/llms-full.txt

Quick start examples for integrating Upstage with Portkey. Each example shows how to create a chat completion using the @upstage/solar-pro model. The Portkey SDK examples use the Portkey client, while the OpenAI-compatible examples use the OpenAI client with the Portkey gateway URL. The cURL example uses the Portkey API endpoint directly.

```python
from portkey_ai import Portkey

# 1. Install: pip install portkey-ai
# 2. Add @upstage provider in model catalog
# 3. Use it:

portkey = Portkey(api_key="PORTKEY_API_KEY")

response = portkey.chat.completions.create(
    model="@upstage/solar-pro",
    messages=[{"role": "user", "content": "Hello!"}]
)

print(response.choices[0].message.content)
```

```javascript
import Portkey from 'portkey-ai'

// 1. Install: npm install portkey-ai
// 2. Add @upstage provider in model catalog
// 3. Use it:

const portkey = new Portkey({
    apiKey: "PORTKEY_API_KEY"
})

const response = await portkey.chat.completions.create({
    model: "@upstage/solar-pro",
    messages: [{ role: "user", content: "Hello!" }]
})

console.log(response.choices[0].message.content)
```

```python
from openai import OpenAI
from portkey_ai import PORTKEY_GATEWAY_URL

# 1. Install: pip install openai portkey-ai
# 2. Add @upstage provider in model catalog
# 3. Use it:

client = OpenAI(
    api_key="PORTKEY_API_KEY",  # Portkey API key
    base_url=PORTKEY_GATEWAY_URL
)

response = client.chat.completions.create(
    model="@upstage/solar-pro",
    messages=[{"role": "user", "content": "Hello!"}]
)

print(response.choices[0].message.content)
```

```javascript
import OpenAI from "openai"
import { PORTKEY_GATEWAY_URL } from "portkey-ai"

// 1. Install: npm install openai portkey-ai
// 2. Add @upstage provider in model catalog
// 3. Use it:

const client = new OpenAI({
    apiKey: "PORTKEY_API_KEY",  // Portkey API key
    baseURL: PORTKEY_GATEWAY_URL
})

const response = await client.chat.completions.create({
    model: "@upstage/solar-pro",
    messages: [{ role: "user", content: "Hello!" }]
})

console.log(response.choices[0].message.content)
```

```bash
# 1. Add @upstage provider in model catalog
# 2. Use it:

curl https://api.portkey.ai/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "x-portkey-api-key: $PORTKEY_API_KEY" \
  -d '{
    "model": "@upstage/solar-pro",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'
```

--------------------------------

### Quick Start: AI21 Integration with Portkey

Source: https://portkey.ai/docs/llms-full.txt

Quick Start examples for integrating AI21 models with Portkey's AI Gateway. Each snippet shows how to make a chat completion request using the @ai21/jamba-1-5-large model, with setup steps for installing the SDK, adding the provider in the model catalog, and using it. The Python and JavaScript examples use the Portkey SDK, while the OpenAI-compatible examples use the OpenAI SDK with the Portkey gateway URL, and the cURL example uses the Portkey API directly.

```python
from portkey_ai import Portkey

# 1. Install: pip install portkey-ai
# 2. Add @ai21 provider in model catalog
# 3. Use it:

portkey = Portkey(api_key="PORTKEY_API_KEY")

response = portkey.chat.completions.create(
    model="@ai21/jamba-1-5-large",
    messages=[{"role": "user", "content": "Say this is a test"}]
)

print(response.choices[0].message.content)
```

```javascript
import Portkey from 'portkey-ai'

// 1. Install: npm install portkey-ai
// 2. Add @ai21 provider in model catalog
// 3. Use it:

const portkey = new Portkey({
    apiKey: "PORTKEY_API_KEY"
})

const response = await portkey.chat.completions.create({
    model: "@ai21/jamba-1-5-large",
    messages: [{ role: "user", content: "Say this is a test" }]
})

console.log(response.choices[0].message.content)
```

```python
from openai import OpenAI
from portkey_ai import PORTKEY_GATEWAY_URL

# 1. Install: pip install openai portkey-ai
# 2. Add @ai21 provider in model catalog
# 3. Use it:

client = OpenAI(
    api_key="PORTKEY_API_KEY",  # Portkey API key
    base_url=PORTKEY_GATEWAY_URL
)

response = client.chat.completions.create(
    model="@ai21/jamba-1-5-large",
    messages=[{"role": "user", "content": "Say this is a test"}]
)

print(response.choices[0].message.content)
```

```javascript
import OpenAI from "openai"
import { PORTKEY_GATEWAY_URL } from "portkey-ai"

// 1. Install: npm install openai portkey-ai
// 2. Add @ai21 provider in model catalog
// 3. Use it:

const client = new OpenAI({
    apiKey: "PORTKEY_API_KEY",  // Portkey API key
    baseURL: PORTKEY_GATEWAY_URL
})

const response = await client.chat.completions.create({
    model: "@ai21/jamba-1-5-large",
    messages: [{ role: "user", content: "Say this is a test" }]
})

console.log(response.choices[0].message.content)
```

```sh
# 1. Add @ai21 provider in model catalog
# 2. Use it:

curl https://api.portkey.ai/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "x-portkey-api-key: $PORTKEY_API_KEY" \
  -d '{
    "model": "@ai21/jamba-1-5-large",
    "messages": [
      { "role": "user", "content": "Say this is a test" }
    ]
  }'
```

--------------------------------

### Quickstart with Portkey client

Source: https://portkey.ai/docs/api-reference/sdk/python

Minimal example to create a Portkey client and make a chat completion request. Use an AI Provider slug or a Config object to select your AI provider.

```python
from portkey_ai import Portkey

client = Portkey(
    api_key="your_api_key_here",  # Or use the env var PORTKEY_API_KEY
    provider="@openai-prod"  # Or use config="cf-***"
)

response = client.chat.completions.create(
    messages=[{"role": "user", "content": "Hello, world!"}],
    model="gpt-4o"  # Example provider/model
)

```

--------------------------------

### Quick Start with Python

Source: https://portkey.ai/docs/integrations/llms/replicate

Install portkey-ai, add @replicate provider in model catalog, then use Portkey to post a prediction to Replicate.

```python
from portkey_ai import Portkey

# 1. Install: pip install portkey-ai
# 2. Add @replicate provider in model catalog
# 3. Use it:

portkey = Portkey(
    api_key="PORTKEY_API_KEY",
    provider="@replicate"
)

response = portkey.post(
    url="predictions",
    data={
        "version": "MODEL_VERSION_ID",
        "input": {"prompt": "Hello, world!"}
    }
)

print(response)
```

--------------------------------

### Setup: Initialize Portkey client and define target variables

Source: https://portkey.ai/docs/guides/prompts/ultimate-ai-sdr

Initializes the Portkey client with tracing and defines the company offering and target information used throughout the workflow.

```python
from portkey_ai import Portkey

# Initialize with tracing
client = Portkey(
  api_key="PORTKEY_API_KEY",
  trace_id="ultimate-ai-sdr-run-1"
)

# Company offering
our_offering = """Umbrella Corp offers 'Uber for personal protection'. Using our app, you can get highly vetted, 
arms-bearing ex-veterans who can accompany you to any place that's supported for any amount of hours or days..."""

# Target information
company_name = "YMU Talent Agency"
company_industry = "Elite Talent Management"
target_person_name = "John Wick"
target_person_designation = "Event Organizer"

```

--------------------------------

### Quick Start: Hugging Face with Portkey (Python, JS, OpenAI, cURL)

Source: https://portkey.ai/docs/llms-full.txt

Quick start examples for using Hugging Face Inference endpoints through Portkey. Each example shows how to make a chat completion request with the model identifier '@huggingface/meta-llama/Llama-3.2-3B-Instruct'. The Portkey SDK examples require installing 'portkey-ai' and adding the '@huggingface' provider in the model catalog. The OpenAI-compatible examples use the Portkey gateway URL and require installing 'openai' and 'portkey-ai'. The cURL example uses the Portkey API key and the '/v1/chat/completions' endpoint.

```python
from portkey_ai import Portkey

# 1. Install: pip install portkey-ai
# 2. Add @huggingface provider in model catalog
# 3. Use it:

portkey = Portkey(api_key="PORTKEY_API_KEY")

response = portkey.chat.completions.create(
    model="@huggingface/meta-llama/Llama-3.2-3B-Instruct",
    messages=[{"role": "user", "content": "Hello!"}]
)

print(response.choices[0].message.content)
```

```javascript
import Portkey from 'portkey-ai'

// 1. Install: npm install portkey-ai
// 2. Add @huggingface provider in model catalog
// 3. Use it:

const portkey = new Portkey({
    apiKey: "PORTKEY_API_KEY"
})

const response = await portkey.chat.completions.create({
    model: "@huggingface/meta-llama/Llama-3.2-3B-Instruct",
    messages: [{ role: "user", content: "Hello!" }]
})

console.log(response.choices[0].message.content)
```

```python
from openai import OpenAI
from portkey_ai import PORTKEY_GATEWAY_URL

# 1. Install: pip install openai portkey-ai
# 2. Add @huggingface provider in model catalog
# 3. Use it:

client = OpenAI(
    api_key="PORTKEY_API_KEY",  # Portkey API key
    base_url=PORTKEY_GATEWAY_URL
)

response = client.chat.completions.create(
    model="@huggingface/meta-llama/Llama-3.2-3B-Instruct",
    messages=[{"role": "user", "content": "Hello!"}]
)

print(response.choices[0].message.content)
```

```javascript
import OpenAI from "openai"
import { PORTKEY_GATEWAY_URL } from "portkey-ai"

// 1. Install: npm install openai portkey-ai
// 2. Add @huggingface provider in model catalog
// 3. Use it:

const client = new OpenAI({
    apiKey: "PORTKEY_API_KEY",  // Portkey API key
    baseURL: PORTKEY_GATEWAY_URL
})

const response = await client.chat.completions.create({
    model: "@huggingface/meta-llama/Llama-3.2-3B-Instruct",
    messages: [{ role: "user", content: "Hello!" }]
})

console.log(response.choices[0].message.content)
```

```sh
# 1. Add @huggingface provider in model catalog
# 2. Use it:

curl https://api.portkey.ai/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "x-portkey-api-key: $PORTKEY_API_KEY" \
  -d '{
    "model": "@huggingface/meta-llama/Llama-3.2-3B-Instruct",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'
```

--------------------------------

### Quick Start with Portkey and Instructor

Source: https://portkey.ai/docs/integrations/libraries/instructor

Quick start examples for using Instructor with Portkey. The Python example uses instructor.from_openai with a Portkey base URL and provider header, defining a Pydantic model and calling chat.completions.create with response_model. The JavaScript example uses the Instructor client with OpenAI and Zod schema, mode TOOLS, and response_model with schema and name.

```python
import instructor
from pydantic import BaseModel
from openai import OpenAI

client = instructor.from_openai(OpenAI(
    base_url="https://api.portkey.ai/v1",
    api_key="PORTKEY_API_KEY",
    default_headers={"x-portkey-provider": "@openai-prod"}
))

class User(BaseModel):
    name: str
    age: int

user = client.chat.completions.create(
    model="gpt-4o",
    response_model=User,
    messages=[{"role": "user", "content": "John Doe is 30 years old."}]
)

print(user.name, user.age)  # John Doe 30

```

```javascript
import Instructor from "@instructor-ai/instructor";
import OpenAI from "openai";
import { z } from "zod";

const client = Instructor({
    client: new OpenAI({
        baseURL: "https://api.portkey.ai/v1",
        apiKey: "PORTKEY_API_KEY",
        defaultHeaders: { "x-portkey-provider": "@openai-prod" }
    }),
    mode: "TOOLS"
});

const UserSchema = z.object({
    name: z.string(),
    age: z.number()
});

const user = await client.chat.completions.create({
    model: "gpt-4o",
    response_model: { schema: UserSchema, name: "User" },
    messages: [{ role: "user", content: "John Doe is 30 years old." }]
});

console.log(user);  // { name: "John Doe", age: 30 }

```

--------------------------------

### Quick Start: Use Nomic Embedding Model with Portkey

Source: https://portkey.ai/docs/llms-full.txt

Quick start examples for using Nomic's embedding model through Portkey. Each example requires installing the respective SDK and adding the @nomic provider in the Model Catalog. The Portkey SDK and OpenAI-compatible SDK examples use the chat completions endpoint with the model '@nomic/nomic-embed-text-v1.5', while the cURL example directly calls the embeddings endpoint with the provider header 'x-portkey-provider: @nomic'.

```python
from portkey_ai import Portkey

# 1. Install: pip install portkey-ai
# 2. Add @nomic provider in model catalog
# 3. Use it:

portkey = Portkey(api_key="PORTKEY_API_KEY")

response = portkey.chat.completions.create(
    model="@nomic/nomic-embed-text-v1.5",
    messages=[{"role": "user", "content": "Hello!"}]
)

print(response.choices[0].message.content)
```

```javascript
import Portkey from 'portkey-ai'

// 1. Install: npm install portkey-ai
// 2. Add @nomic provider in model catalog
// 3. Use it:

const portkey = new Portkey({
    apiKey: "PORTKEY_API_KEY"
})

const response = await portkey.chat.completions.create({
    model: "@nomic/nomic-embed-text-v1.5",
    messages: [{ role: "user", content: "Hello!" }]
})

console.log(response.choices[0].message.content)
```

```python
from openai import OpenAI
from portkey_ai import PORTKEY_GATEWAY_URL

# 1. Install: pip install openai portkey-ai
# 2. Add @nomic provider in model catalog
# 3. Use it:

client = OpenAI(
    api_key="PORTKEY_API_KEY",  # Portkey API key
    base_url=PORTKEY_GATEWAY_URL
)

response = client.chat.completions.create(
    model="@nomic/nomic-embed-text-v1.5",
    messages=[{"role": "user", "content": "Hello!"}]
)

print(response.choices[0].message.content)
```

```javascript
import OpenAI from "openai"
import { PORTKEY_GATEWAY_URL } from "portkey-ai"

// 1. Install: npm install openai portkey-ai
// 2. Add @nomic provider in model catalog
// 3. Use it:

const client = new OpenAI({
    apiKey: "PORTKEY_API_KEY",  // Portkey API key
    baseURL: PORTKEY_GATEWAY_URL
})

const response = await client.chat.completions.create({
    model: "@nomic/nomic-embed-text-v1.5",
    messages: [{ role: "user", content: "Hello!" }]
})

console.log(response.choices[0].message.content)
```

```bash
curl https://api.portkey.ai/v1/embeddings \
    -H "Content-Type: application/json" \
    -H "x-portkey-api-key: $PORTKEY_API_KEY" \
    -H "x-portkey-provider: @nomic" \
    -d '{
        "model": "nomic-embed-text-v1.5",
        "input": "create vector representation on this sentence"
    }'
```

--------------------------------

### Set up environment variables and create project files

Source: https://portkey.ai/docs/product/enterprise-offering/private-cloud-deployments/gcp

Initializes the deployment by setting environment variables for the GKE cluster, namespace, and service account, then creates the project directory and an empty values.yaml file.

```bash
CLUSTER_NAME=<GKE_CLUSTER_NAME>       # Specify the name of the GKE cluster where the gateway will be deployed.
NAMESPACE=<NAMESPACE>                 # Specify the namespace where the gateway should be deployed (for example, portkeyai).
KSA=<KSA>                             # Provide a name for the Service Account to be associated with Gateway Pod (for example, gateway-sa)

mkdir portkey-gateway
cd portkey-gateway
touch values.yaml

```

--------------------------------

### Quick Start with Node.js

Source: https://portkey.ai/docs/integrations/llms/replicate

Install portkey-ai, add @replicate provider in model catalog, then use Portkey to post a prediction to Replicate.

```javascript
import Portkey from 'portkey-ai'

// 1. Install: npm install portkey-ai
// 2. Add @replicate provider in model catalog
// 3. Use it:

const portkey = new Portkey({
    apiKey: "PORTKEY_API_KEY",
    provider: "@replicate"
})

const response = await portkey.post({
    url: "predictions",
    data: {
        version: "MODEL_VERSION_ID",
        input: { prompt: "Hello, world!" }
    }
})

console.log(response)
```

--------------------------------

### Complete Setup: Use Portkey's Gateway

Source: https://portkey.ai/docs/integrations/tracing-providers/logfire

Complete setup that routes LLM calls through Portkey's gateway for automatic optimizations. Creates an OpenAI client with Portkey's base URL and headers, then instruments that client with Logfire.

```python
import logfire
import os
from portkey_ai import createHeaders
from openai import OpenAI

# Configure OpenTelemetry export
os.environ["OTEL_EXPORTER_OTLP_ENDPOINT"] = "https://api.portkey.ai/v1/logs/otel"
os.environ["OTEL_EXPORTER_OTLP_HEADERS"] = "x-portkey-api-key=YOUR_PORTKEY_API_KEY"

# Initialize Logfire
logfire.configure(
    service_name='my-llm-app',
    send_to_logfire=False,
)

# Create OpenAI client with Portkey's gateway
client = OpenAI(
    api_key="PORTKEY_API_KEY",
    base_url="https://api.portkey.ai/v1",
    default_headers=createHeaders(
        api_key="PORTKEY_API_KEY",
        provider="@openai-prod"  # Your AI Provider slug from Model Catalog
    )
)

# Instrument the Portkey-configured client
logfire.instrument_openai(client)

```

--------------------------------

### Quick Start: Chat completion with BytePlus ModelArk via Portkey

Source: https://portkey.ai/docs/llms-full.txt

Quick Start examples for integrating BytePlus ModelArk with Portkey. Each example shows how to create a chat completion using the @byteplus provider slug and the model identifier @byteplus/ep-m-your-endpoint-id. The Portkey SDK examples require installing portkey-ai and adding the @byteplus provider in the model catalog; the OpenAI-compatible examples additionally require the openai package and use PORTKEY_GATEWAY_URL as the base URL. The cURL example uses the Portkey API endpoint with the x-portkey-api-key header.

```python
from portkey_ai import Portkey

# 1. Install: pip install portkey-ai
# 2. Add @byteplus provider in model catalog
# 3. Use it:

portkey = Portkey(api_key="PORTKEY_API_KEY")

response = portkey.chat.completions.create(
    model="@byteplus/ep-m-your-endpoint-id",
    messages=[{"role": "user", "content": "Say this is a test"}]
)

print(response.choices[0].message.content)
```

```javascript
import Portkey from 'portkey-ai'

// 1. Install: npm install portkey-ai
// 2. Add @byteplus provider in model catalog
// 3. Use it:

const portkey = new Portkey({
    apiKey: "PORTKEY_API_KEY"
})

const response = await portkey.chat.completions.create({
    model: "@byteplus/ep-m-your-endpoint-id",
    messages: [{ role: "user", content: "Say this is a test" }]
})

console.log(response.choices[0].message.content)
```

```python
from openai import OpenAI
from portkey_ai import PORTKEY_GATEWAY_URL

# 1. Install: pip install openai portkey-ai
# 2. Add @byteplus provider in model catalog
# 3. Use it:

client = OpenAI(
    api_key="PORTKEY_API_KEY",  # Portkey API key
    base_url=PORTKEY_GATEWAY_URL
)

response = client.chat.completions.create(
    model="@byteplus/ep-m-your-endpoint-id",
    messages=[{"role": "user", "content": "Say this is a test"}]
)

print(response.choices[0].message.content)
```

```javascript
import OpenAI from "openai"
import { PORTKEY_GATEWAY_URL } from "portkey-ai"

// 1. Install: npm install openai portkey-ai
// 2. Add @byteplus provider in model catalog
// 3. Use it:

const client = new OpenAI({
    apiKey: "PORTKEY_API_KEY",  // Portkey API key
    baseURL: PORTKEY_GATEWAY_URL
})

const response = await client.chat.completions.create({
    model: "@byteplus/ep-m-your-endpoint-id",
    messages: [{ role: "user", content: "Say this is a test" }]
})

console.log(response.choices[0].message.content)
```

```sh
# 1. Add @byteplus provider in model catalog
# 2. Use it:

curl https://api.portkey.ai/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "x-portkey-api-key: $PORTKEY_API_KEY" \
  -d '{
    "model": "@byteplus/ep-m-your-endpoint-id",
    "messages": [
      { "role": "user", "content": "Say this is a test" }
    ]
  }'
```

--------------------------------

### Complete example: Traceloop + Portkey integration

Source: https://portkey.ai/docs/integrations/tracing-providers/traceloop

Full example combining initialization, gateway configuration, and an instrumented call. Replace placeholders with your actual API keys and provider slug.

```python
from traceloop.sdk import Traceloop
from openai import OpenAI
from portkey_ai import createHeaders

# Step 1: Initialize Traceloop with Portkey endpoint
Traceloop.init(
    disable_batch=True,
    api_endpoint="https://api.portkey.ai/v1/logs/otel",
    headers="x-portkey-api-key=YOUR_PORTKEY_API_KEY",
    telemetry_enabled=False
)

# Step 2: Configure Portkey Gateway
client = OpenAI(
    api_key="PORTKEY_API_KEY",
    base_url="https://api.portkey.ai/v1",
    default_headers=createHeaders(
        api_key="PORTKEY_API_KEY",
        provider="@openai-prod"
    )
)

# Step 3: Make instrumented calls
response = client.chat.completions.create(
    model="gpt-4o",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "What makes observability important for production AI?"}
    ]
)

print(response.choices[0].message.content)

```

--------------------------------

### Run OpenClaw onboarding wizard

Source: https://portkey.ai/docs/integrations/libraries/openclaw

Starts the OpenClaw onboarding wizard with the --install-daemon flag to install the Gateway service. Follow the wizard prompts to complete setup.

```bash
openclaw onboard --install-daemon
```

--------------------------------

### Python quick start with Portkey and Vertex AI

Source: https://portkey.ai/docs/integrations/llms/vertex-ai

Install portkey-ai, add @vertex-ai provider in Model Catalog with Service Account JSON, then create a chat completion using the gemini-3-pro-preview model.

```python
from portkey_ai import Portkey

# 1. Install: pip install portkey-ai
# 2. Add @vertex-ai provider in Model Catalog with Service Account JSON
# 3. Use it:

portkey = Portkey(api_key="PORTKEY_API_KEY")

response = portkey.chat.completions.create(
    model="@vertex-ai/gemini-3-pro-preview",
    messages=[{"role": "user", "content": "Say this is a test"}]
)

print(response.choices[0].message.content)
```
Tokens:
Raw