Groq Typescript

https://github.com/groq/groq-typescript
Admin
A TypeScript/JavaScript library for convenient access to the Groq REST API, enabling server-side...
Tokens:48,193
Snippets:546
Trust Score:9.6
License:Apache-2.0
Update:1 month ago
Show doc for...
Tokens:
Raw
### Add and Run Examples

Source: https://github.com/groq/groq-typescript/blob/main/CONTRIBUTING.md

Create new example files in the `examples/` directory and make them executable. Run examples against your API using `yarn tsn`.

```ts
// add an example to examples/<your-example>.ts

#!/usr/bin/env -S npm run tsn -T
…
```

```sh
$ chmod +x examples/<your-example>.ts
# run the example against your api
$ yarn tsn -T examples/<your-example>.ts
```

--------------------------------

### Install groq-sdk

Source: https://github.com/groq/groq-typescript/blob/main/_autodocs/00-index.md

Use npm to install the required SDK package.

```bash
npm install groq-sdk
```

--------------------------------

### Install from Git

Source: https://github.com/groq/groq-typescript/blob/main/CONTRIBUTING.md

Use this command to install the repository directly from a Git URL.

```sh
$ npm install git+ssh://git@github.com:groq/groq-typescript.git
```

--------------------------------

### Install Dependencies and Build Project

Source: https://github.com/groq/groq-typescript/blob/main/CONTRIBUTING.md

Run these commands to install all necessary dependencies and build the project output files.

```sh
$ yarn
$ yarn build
```

--------------------------------

### Install Groq TypeScript API Library

Source: https://github.com/groq/groq-typescript/blob/main/README.md

Install the groq-sdk package using npm.

```sh
npm install groq-sdk
```

--------------------------------

### List Available Models example

Source: https://github.com/groq/groq-typescript/blob/main/_autodocs/05-models.md

Example of listing all available models and logging their IDs and creation dates.

```typescript
import Groq from 'groq-sdk';

const client = new Groq();

const response = await client.models.list();

response.data.forEach(model => {
  console.log(`${model.id} - Created: ${new Date(model.created * 1000)}`);
});
```

--------------------------------

### Basic Translation example

Source: https://github.com/groq/groq-typescript/blob/main/_autodocs/03-audio.md

A complete example showing how to initialize the Groq client and translate an audio file using the whisper-large-v3-turbo model.

```typescript
import Groq from 'groq-sdk';
import fs from 'fs';

const client = new Groq();

const translation = await client.audio.translations.create({
  model: 'whisper-large-v3-turbo',
  file: fs.createReadStream('/path/to/spanish_audio.mp3'),
});

console.log(translation.text); // Output in English
```

--------------------------------

### Example: List Batches

Source: https://github.com/groq/groq-typescript/blob/main/_autodocs/06-files-and-batches.md

Demonstrates listing all batches and logging their IDs and statuses.

```typescript
import Groq from 'groq-sdk';

const client = new Groq();

const response = await client.batches.list();

response.data.forEach(batch => {
  console.log(`Batch ${batch.id}: ${batch.status}`);
});
```

--------------------------------

### Transcription with Language example

Source: https://github.com/groq/groq-typescript/blob/main/_autodocs/03-audio.md

An example showing how to specify the input language and sampling temperature for a transcription request.

```typescript
import Groq from 'groq-sdk';
import fs from 'fs';

const client = new Groq();

const transcription = await client.audio.transcriptions.create({
  model: 'whisper-large-v3-turbo',
  file: fs.createReadStream('/path/to/audio.mp3'),
  language: 'es', // Spanish
  temperature: 0.2,
});

console.log(transcription.text);
```

--------------------------------

### Basic Transcription example

Source: https://github.com/groq/groq-typescript/blob/main/_autodocs/03-audio.md

A basic example demonstrating how to transcribe an audio file using the whisper-large-v3-turbo model.

```typescript
import Groq from 'groq-sdk';
import fs from 'fs';

const client = new Groq();

const transcription = await client.audio.transcriptions.create({
  model: 'whisper-large-v3-turbo',
  file: fs.createReadStream('/path/to/audio.mp3'),
});

console.log(transcription.text);
```

--------------------------------

### List Files example

Source: https://github.com/groq/groq-typescript/blob/main/_autodocs/06-files-and-batches.md

Demonstrates how to retrieve and iterate over a list of uploaded files.

```typescript
import Groq from 'groq-sdk';

const client = new Groq();

const response = await client.files.list();

response.data.forEach(file => {
  console.log(`${file.filename} (${file.bytes} bytes)`);
});
```

--------------------------------

### Example: Create and Monitor Batch

Source: https://github.com/groq/groq-typescript/blob/main/_autodocs/06-files-and-batches.md

Demonstrates uploading a file, creating a batch, and retrieving its status.

```typescript
import Groq from 'groq-sdk';
import fs from 'fs';

const client = new Groq();

// First, upload the batch file
const file = await client.files.create({
  file: fs.createReadStream('./batch-requests.jsonl'),
  purpose: 'batch',
});

// Create the batch
const batch = await client.batches.create({
  input_file_id: file.id!,
  endpoint: '/v1/chat/completions',
  completion_window: '24h',
});

console.log('Batch created:', batch.id);
console.log('Status:', batch.status);

// Check status later
const updated = await client.batches.retrieve(batch.id);
console.log('Updated status:', updated.status);
```

--------------------------------

### Custom Configuration Example

Source: https://github.com/groq/groq-typescript/blob/main/_autodocs/01-groq-client.md

Initializes the client with custom base URL, timeout, retry, and logging settings.

```typescript
import Groq from 'groq-sdk';

const client = new Groq({
  apiKey: process.env['GROQ_API_KEY'],
  baseURL: 'https://custom.groq.api',
  timeout: 30 * 1000, // 30 seconds
  maxRetries: 3,
  logLevel: 'debug',
});
```

--------------------------------

### Basic Usage Example

Source: https://github.com/groq/groq-typescript/blob/main/_autodocs/01-groq-client.md

Initializes the client with an API key and performs a chat completion request.

```typescript
import Groq from 'groq-sdk';

const client = new Groq({
  apiKey: process.env['GROQ_API_KEY'],
});

const chatCompletion = await client.chat.completions.create({
  messages: [{ role: 'user', content: 'Explain the importance of low latency LLMs' }],
  model: 'mixtral-8x7b-32768',
});

console.log(chatCompletion.choices[0].message.content);
```

--------------------------------

### Upload a Batch File example

Source: https://github.com/groq/groq-typescript/blob/main/_autodocs/06-files-and-batches.md

Demonstrates how to upload a .jsonl file for batch processing using the Groq SDK.

```typescript
import Groq from 'groq-sdk';
import fs from 'fs';

const client = new Groq();

const file = await client.files.create({
  file: fs.createReadStream('./batch-requests.jsonl'),
  purpose: 'batch',
});

console.log('File uploaded:', file.id);
```

--------------------------------

### list method signature

Source: https://github.com/groq/groq-typescript/blob/main/_autodocs/05-models.md

Get all available models.

```typescript
list(options?: RequestOptions): APIPromise<ModelListResponse>
```

--------------------------------

### Upload File from Buffer

Source: https://github.com/groq/groq-typescript/blob/main/_autodocs/11-file-uploads.md

Example of converting a Buffer to a File object and uploading it.

```typescript
import Groq, { toFile } from 'groq-sdk';

const client = new Groq();

const buffer = Buffer.from('file content');
const file = await toFile(buffer, 'data.bin', {
  type: 'application/octet-stream'
});

const batch = await client.files.create({
  file: file,
  purpose: 'batch'
});
```

--------------------------------

### Tool Use Example

Source: https://github.com/groq/groq-typescript/blob/main/_autodocs/02-chat-completions.md

Demonstrates how to define and use tools within a chat completion request.

```typescript
import Groq from 'groq-sdk';

const client = new Groq();

const tools: Groq.Chat.ChatCompletionTool[] = [
  {
    type: 'function',
    function: {
      name: 'get_weather',
      description: 'Get weather for a location',
      parameters: {
        type: 'object',
        properties: {
          location: { type: 'string' },
        },
        required: ['location'],
      },
    },
  },
];

const completion = await client.chat.completions.create({
  messages: [{ role: 'user', content: 'What is the weather in Paris?' }],
  model: 'mixtral-8x7b-32768',
  tools: tools,
  tool_choice: 'auto',
});

if (completion.choices[0].message.tool_calls) {
  for (const toolCall of completion.choices[0].message.tool_calls) {
    console.log(`Tool: ${toolCall.function.name}`);
    console.log(`Args: ${toolCall.function.arguments}`);
  }
}
```

--------------------------------

### Handling NotFoundError

Source: https://github.com/groq/groq-typescript/blob/main/_autodocs/07-errors.md

Example of catching and handling a 404 Not Found error.

```typescript
try {
  const model = await client.models.retrieve('non-existent-model');
} catch (error) {
  if (error instanceof Groq.NotFoundError) {
    console.error('Model not found');
    // List available models
  }
}
```

--------------------------------

### List Models with Error Handling example

Source: https://github.com/groq/groq-typescript/blob/main/_autodocs/05-models.md

Example of listing models and filtering the results with error handling.

```typescript
import Groq from 'groq-sdk';

const client = new Groq();

try {
  const response = await client.models.list();
  
  console.log(`Available models: ${response.data.length}`);
  
  const chatModels = response.data.filter(m => 
    m.id.includes('mixtral') || m.id.includes('llama')
  );
  
  chatModels.forEach(model => {
    console.log(`- ${model.id}`);
  });
} catch (error) {
  if (error instanceof Groq.AuthenticationError) {
    console.error('Authentication failed - check API key');
  } else if (error instanceof Groq.APIError) {
    console.error('Failed to retrieve models:', error.message);
  }
}
```

--------------------------------

### Basic Chat Completion Example

Source: https://github.com/groq/groq-typescript/blob/main/_autodocs/02-chat-completions.md

Demonstrates how to perform a standard chat completion request using the Groq SDK.

```typescript
import Groq from 'groq-sdk';

const client = new Groq();

const chatCompletion = await client.chat.completions.create({
  messages: [
    { role: 'system', content: 'You are a helpful assistant.' },
    { role: 'user', content: 'Explain quantum computing in simple terms.' }
  ],
  model: 'mixtral-8x7b-32768',
});

console.log(chatCompletion.choices[0].message.content);
```

--------------------------------

### Upload File from String

Source: https://github.com/groq/groq-typescript/blob/main/_autodocs/11-file-uploads.md

Example of converting a string to a File object and uploading it.

```typescript
import Groq, { toFile } from 'groq-sdk';

const client = new Groq();

const file = await toFile('Hello, world!', 'message.txt', {
  type: 'text/plain'
});

const batch = await client.files.create({
  file: file,
  purpose: 'batch'
});
```

--------------------------------

### Upload File from Blob

Source: https://github.com/groq/groq-typescript/blob/main/_autodocs/11-file-uploads.md

Example of converting a Blob to a File object and uploading it.

```typescript
import Groq, { toFile } from 'groq-sdk';

const client = new Groq();

const blob = new Blob(['content'], { type: 'text/plain' });
const file = await toFile(blob, 'document.txt');

const batch = await client.files.create({
  file: file,
  purpose: 'batch'
});
```

--------------------------------

### Stream tee() Example

Source: https://github.com/groq/groq-typescript/blob/main/_autodocs/10-api-promise-and-streaming.md

Shows how to process the same stream twice independently using the tee method.

```typescript
import Groq from 'groq-sdk';

const client = new Groq();

const stream = await client.chat.completions.create({
  messages: [{ role: 'user', content: 'Hello' }],
  model: 'mixtral-8x7b-32768',
  stream: true,
});

const [stream1, stream2] = stream.tee();

// Process stream1
(async () => {
  for await (const chunk of stream1) {
    console.log('Stream 1:', chunk.choices[0]?.delta?.content);
  }
})();

// Process stream2
(async () => {
  for await (const chunk of stream2) {
    console.log('Stream 2:', chunk.choices[0]?.delta?.content);
  }
})();
```

--------------------------------

### Streaming Response Example

Source: https://github.com/groq/groq-typescript/blob/main/_autodocs/02-chat-completions.md

Shows how to handle streaming responses from the chat completions API.

```typescript
import Groq from 'groq-sdk';

const client = new Groq();

const stream = await client.chat.completions.create({
  messages: [
    { role: 'user', content: 'Tell me a story.' }
  ],
  model: 'mixtral-8x7b-32768',
  stream: true,
});

for await (const chunk of stream) {
  const delta = chunk.choices[0].delta;
  if (delta?.content) {
    process.stdout.write(delta.content);
  }
}
```

--------------------------------

### Evaluate Models with Streaming

Source: https://github.com/groq/groq-typescript/blob/main/_autodocs/12-advanced-usage.md

This example demonstrates how to iterate through a list of models, stream their responses, and track performance metrics like time elapsed and token usage.

```typescript
import Groq from 'groq-sdk';

async function evaluateModel(
  prompt: string,
  models: string[]
) {
  const results: { model: string; time: number; tokens: number; content: string }[] = [];
  
  for (const model of models) {
    const startTime = Date.now();
    let tokenCount = 0;
    let content = '';
    
    try {
      const stream = await client.chat.completions.create({
        messages: [{ role: 'user', content: prompt }],
        model: model,
        stream: true,
      });
      
      for await (const chunk of stream) {
        const delta = chunk.choices[0]?.delta?.content || '';
        content += delta;
        
        if (chunk.usage) {
          tokenCount = chunk.usage.completion_tokens;
        }
      }
      
      const elapsed = Date.now() - startTime;
      results.push({
        model,
        time: elapsed,
        tokens: tokenCount,
        content: content.substring(0, 100), // First 100 chars
      });
    } catch (error) {
      console.error(`Error with model ${model}:`, error);
    }
  }
  
  return results;
}

// Usage
const comparison = await evaluateModel(
  'Explain quantum computing',
  ['mixtral-8x7b-32768', 'llama-3.1-70b-versatile']
);

comparison.forEach(r => {
  console.log(`${r.model}: ${r.tokens} tokens in ${r.time}ms`);
});
```

--------------------------------

### Error Handling Example

Source: https://github.com/groq/groq-typescript/blob/main/_autodocs/02-chat-completions.md

Shows how to catch and handle specific API errors like rate limits.

```typescript
import Groq from 'groq-sdk';

const client = new Groq();

try {
  const completion = await client.chat.completions.create({
    messages: [{ role: 'user', content: 'Hello' }],
    model: 'mixtral-8x7b-32768',
    max_tokens: 10,
  });
} catch (error) {
  if (error instanceof Groq.RateLimitError) {
    console.error('Rate limited. Retry-After:', error.headers?.['retry-after']);
  } else if (error instanceof Groq.APIError) {
    console.error('API Error:', error.status, error.message);
  }
}
```

--------------------------------

### Error Handling example

Source: https://github.com/groq/groq-typescript/blob/main/_autodocs/04-embeddings.md

Demonstrates how to catch and handle specific API errors like BadRequestError.

```typescript
import Groq from 'groq-sdk';

const client = new Groq();

try {
  const response = await client.embeddings.create({
    input: '', // Empty string will fail
    model: 'nomic-embed-text-v1_5',
  });
} catch (error) {
  if (error instanceof Groq.BadRequestError) {
    console.error('Invalid input for embeddings:', error.message);
  } else if (error instanceof Groq.APIError) {
    console.error('API Error:', error.status, error.message);
  }
}
```

--------------------------------

### TypeScript Configuration for Bun

Source: https://github.com/groq/groq-typescript/blob/main/MIGRATION.md

Configure `tsconfig.json` with `target` and ensure `@types/bun` is installed in `package.json` for Bun environments.

```jsonc
{
  "target": "ES2018" // note: we recommend ES2020 or higher
}
```

```json
{
  "devDependencies": {
    "@types/bun": ">= 1.2.0"
  }
}
```

--------------------------------

### Configure Log Level

Source: https://github.com/groq/groq-typescript/blob/main/README.md

Set the logging level for the SDK. This example sets the level to 'debug' to show all log messages. The `logLevel` option overrides the `GROQ_LOG` environment variable.

```typescript
import Groq from 'groq-sdk';

const client = new Groq({
  logLevel: 'debug', // Show all log messages
});
```

--------------------------------

### Stream Async Iteration Example

Source: https://github.com/groq/groq-typescript/blob/main/_autodocs/10-api-promise-and-streaming.md

Demonstrates using a for await...of loop to process chunks from a chat completion stream.

```typescript
import Groq from 'groq-sdk';

const client = new Groq();

const stream = await client.chat.completions.create({
  messages: [{ role: 'user', content: 'Count to 5' }],
  model: 'mixtral-8x7b-32768',
  stream: true,
});

for await (const chunk of stream) {
  const content = chunk.choices[0]?.delta?.content || '';
  process.stdout.write(content);
}
```

--------------------------------

### Generate Speech to File

Source: https://github.com/groq/groq-typescript/blob/main/_autodocs/03-audio.md

Example of generating speech from text and saving the resulting binary audio data to an MP3 file.

```typescript
import Groq from 'groq-sdk';
import fs from 'fs';

const client = new Groq();

const speech = await client.audio.speech.create({
  model: 'tts-1',
  input: 'Hello, this is a test of the text to speech API.',
  voice: 'alloy',
});

const buffer = Buffer.from(await speech.arrayBuffer());
fs.writeFileSync('/path/to/output.mp3', buffer);
```

--------------------------------

### Retrieve Specific Model example

Source: https://github.com/groq/groq-typescript/blob/main/_autodocs/05-models.md

Example of retrieving a specific model by its ID.

```typescript
import Groq from 'groq-sdk';

const client = new Groq();

const model = await client.models.retrieve('mixtral-8x7b-32768');

console.log('Model:', model.id);
console.log('Owned by:', model.owned_by);
console.log('Created:', new Date(model.created * 1000));
```

--------------------------------

### Set up Mock Server

Source: https://github.com/groq/groq-typescript/blob/main/CONTRIBUTING.md

Run this script to set up a mock server required for running most tests against the OpenAPI spec.

```sh
$ ./scripts/mock
```

--------------------------------

### constructor(options?: ClientOptions)

Source: https://github.com/groq/groq-typescript/blob/main/_autodocs/01-groq-client.md

Initializes a new Groq client instance with optional configuration settings.

```APIDOC
## constructor(options?: ClientOptions)

### Description
Initializes a new Groq client instance. If no API key is provided, it defaults to the GROQ_API_KEY environment variable.

### Parameters
- **options** (ClientOptions) - Optional - Configuration options for the client.
  - **apiKey** (string) - Optional - API key for authentication.
  - **baseURL** (string) - Optional - Base URL for API requests.
  - **timeout** (number) - Optional - Request timeout in milliseconds.
  - **maxRetries** (number) - Optional - Maximum number of retry attempts.
  - **fetchOptions** (MergedRequestInit) - Optional - Additional fetch options.
  - **fetch** (Fetch) - Optional - Custom fetch function implementation.
  - **defaultHeaders** (HeadersLike) - Optional - Default headers for all requests.
  - **defaultQuery** (Record<string, string>) - Optional - Default query parameters.
  - **dangerouslyAllowBrowser** (boolean) - Optional - Enable browser-side usage.
  - **logLevel** (LogLevel) - Optional - Logging level ('debug', 'info', 'warn', 'error', 'off').
  - **logger** (Logger) - Optional - Custom logger implementation.
```

--------------------------------

### withOptions(options: Partial<ClientOptions>)

Source: https://github.com/groq/groq-typescript/blob/main/_autodocs/01-groq-client.md

Creates a new client instance with merged configuration options.

```APIDOC
## withOptions(options: Partial<ClientOptions>)

### Description
Creates a new Groq client instance by merging the provided options with the existing client's configuration.

### Parameters
- **options** (Partial<ClientOptions>) - Required - Options to override in the new instance.

### Returns
A new Groq client instance with merged configuration.
```

--------------------------------

### Accessing Raw Response Data

Source: https://github.com/groq/groq-typescript/blob/main/README.md

Access the underlying raw `Response` object from `fetch()` to retrieve headers or other details. You can use `.asResponse()` to get the response without consuming the body, or `.withResponse()` to get both the raw response and the parsed data.

```APIDOC
## Accessing Raw Response Data

### Description

Access the underlying raw `Response` object from `fetch()` to retrieve headers or other details. You can use `.asResponse()` to get the response without consuming the body, or `.withResponse()` to get both the raw response and the parsed data.

### Using `.asResponse()`

Returns the raw `Response` as soon as headers are received. The response body is not consumed, allowing for custom streaming or parsing.

```ts
const client = new Groq();

const response = await client.chat.completions
  .create({
    messages: [
      { role: 'system', content: 'You are a helpful assistant.' },
      { role: 'user', content: 'Explain the importance of low latency LLMs' },
    ],
    model: 'openai/gpt-oss-20b',
  })
  .asResponse();
console.log(response.headers.get('X-My-Header'));
console.log(response.statusText); // access the underlying Response object
```

### Using `.withResponse()`

Consumes the response body and returns both the raw `Response` object and the parsed data.

```ts
const { data: chatCompletion, response: raw } = await client.chat.completions
  .create({
    messages: [
      { role: 'system', content: 'You are a helpful assistant.' },
      { role: 'user', content: 'Explain the importance of low latency LLMs' },
    ],
    model: 'openai/gpt-oss-20b',
  })
  .withResponse();
console.log(raw.headers.get('X-My-Header'));
console.log(chatCompletion.id);
```
```

--------------------------------

### Basic Usage

Source: https://github.com/groq/groq-typescript/blob/main/README.md

Demonstrates how to initialize the Groq client and make a chat completion request.

```APIDOC
## Basic Usage

This example shows how to initialize the Groq client and make a basic chat completion request.

```js
import Groq from 'groq-sdk';

const client = new Groq({
  apiKey: process.env['GROQ_API_KEY'], // This is the default and can be omitted
});

const chatCompletion = await client.chat.completions.create({
  messages: [{ role: 'user', content: 'Explain the importance of low latency LLMs' }],
  model: 'openai/gpt-oss-20b',
});

console.log(chatCompletion.id);
```
```

--------------------------------

### retrieve method signature

Source: https://github.com/groq/groq-typescript/blob/main/_autodocs/05-models.md

Get a specific model by its identifier.

```typescript
retrieve(model: string, options?: RequestOptions): APIPromise<Model>
```

--------------------------------

### Handling AuthenticationError

Source: https://github.com/groq/groq-typescript/blob/main/_autodocs/07-errors.md

Example of catching and handling a 401 Authentication error.

```typescript
try {
  const completion = await client.chat.completions.create({...});
} catch (error) {
  if (error instanceof Groq.AuthenticationError) {
    console.error('Authentication failed - check GROQ_API_KEY');
    // Verify API key
  }
}
```

--------------------------------

### Initialize Groq Client

Source: https://github.com/groq/groq-typescript/blob/main/_autodocs/09-configuration.md

Basic syntax for instantiating the Groq client with optional configuration.

```typescript
const client = new Groq(options?: ClientOptions);
```

--------------------------------

### Configure Deno proxy

Source: https://github.com/groq/groq-typescript/blob/main/README.md

Set up a proxy for Deno environments by creating a custom `Deno.HttpClient` with proxy configuration and passing it to the Groq client.

```typescript
import Groq from 'npm:groq-sdk';

const httpClient = Deno.createHttpClient({ proxy: { url: 'http://localhost:8888' } });
const client = new Groq({
  fetchOptions: {
    client: httpClient,
  },
});
```

--------------------------------

### Initialize Groq Client and Create Chat Completion

Source: https://github.com/groq/groq-typescript/blob/main/README.md

Initialize the Groq client with an API key and create a chat completion. The API key can be omitted if it's set as the GROQ_API_KEY environment variable.

```js
import Groq from 'groq-sdk';

const client = new Groq({
  apiKey: process.env['GROQ_API_KEY'], // This is the default and can be omitted
});

const chatCompletion = await client.chat.completions.create({
  messages: [{ role: 'user', content: 'Explain the importance of low latency LLMs' }],
  model: 'openai/gpt-oss-20b',
});

console.log(chatCompletion.id);
```

--------------------------------

### Upload File from Uint8Array

Source: https://github.com/groq/groq-typescript/blob/main/_autodocs/11-file-uploads.md

Example of converting a Uint8Array to a File object and uploading it.

```typescript
import Groq, { toFile } from 'groq-sdk';

const client = new Groq();

const array = new Uint8Array([0x89, 0x50, 0x4E, 0x47]);
const file = await toFile(array, 'image.png', {
  type: 'image/png'
});

const batch = await client.files.create({
  file: file,
  purpose: 'batch'
});
```

--------------------------------

### withOptions Method

Source: https://github.com/groq/groq-typescript/blob/main/_autodocs/01-groq-client.md

Creates a new client instance with merged configuration options.

```typescript
withOptions(options: Partial<ClientOptions>): Groq
```

```typescript
const client = new Groq({ apiKey: 'key1' });
const customClient = client.withOptions({ 
  timeout: 30000,
  maxRetries: 5 
});
```

--------------------------------

### Deno Runtime Initialization

Source: https://github.com/groq/groq-typescript/blob/main/_autodocs/09-configuration.md

Initialize the Groq client in a Deno environment. Requires Deno v1.28.0 or higher.

```typescript
import Groq from 'npm:groq-sdk';

const client = new Groq();
```

--------------------------------

### Node.js Runtime Initialization

Source: https://github.com/groq/groq-typescript/blob/main/_autodocs/09-configuration.md

Initialize the Groq client in a Node.js environment. Requires Node.js 20 LTS or later.

```typescript
import Groq from 'groq-sdk';

const client = new Groq();
```

--------------------------------

### Handling InternalServerError

Source: https://github.com/groq/groq-typescript/blob/main/_autodocs/07-errors.md

Example of catching and handling a 5xx Internal Server error.

```typescript
try {
  const completion = await client.chat.completions.create({...});
} catch (error) {
  if (error instanceof Groq.InternalServerError) {
    console.error('Server error - retry later');
    // Implement exponential backoff
  }
}
```

--------------------------------

### Handling RateLimitError

Source: https://github.com/groq/groq-typescript/blob/main/_autodocs/07-errors.md

Example of catching and handling a 429 Rate Limit error.

```typescript
try {
  const completion = await client.chat.completions.create({...});
} catch (error) {
  if (error instanceof Groq.RateLimitError) {
    const retryAfter = error.headers?.['retry-after'] || 60;
    console.error(`Rate limited. Retry after ${retryAfter}s`);
    // Implement exponential backoff
  }
}
```

--------------------------------

### Groq Constructor

Source: https://github.com/groq/groq-typescript/blob/main/_autodocs/09-configuration.md

The Groq constructor accepts a configuration object to initialize the client with custom settings for authentication, network behavior, and logging.

```APIDOC
## Constructor

### Description
Initializes a new instance of the Groq SDK client.

### Signature
`new Groq(options?: ClientOptions)`

### Options
- **apiKey** (string) - Optional - API key for authentication. Defaults to `process.env['GROQ_API_KEY']`.
- **baseURL** (string) - Optional - Base URL for API requests. Defaults to `https://api.groq.com`.
- **timeout** (number) - Optional - Request timeout in milliseconds. Defaults to 60000.
- **maxRetries** (number) - Optional - Maximum retry attempts for failed requests. Defaults to 2.
- **fetchOptions** (MergedRequestInit) - Optional - Additional fetch configuration.
- **fetch** (Fetch) - Optional - Custom fetch implementation.
- **defaultHeaders** (HeadersLike) - Optional - Headers included in all requests.
- **defaultQuery** (Record<string, string>) - Optional - Query parameters included in all requests.
- **dangerouslyAllowBrowser** (boolean) - Optional - Enable browser-side usage. Defaults to false.
- **logLevel** (LogLevel) - Optional - Logging verbosity level. Defaults to 'warn'.
- **logger** (Logger) - Optional - Custom logger implementation.
```

--------------------------------

### Handling PermissionDeniedError

Source: https://github.com/groq/groq-typescript/blob/main/_autodocs/07-errors.md

Example of catching and handling a 403 Permission Denied error.

```typescript
try {
  const batch = await client.batches.cancel(batchID);
} catch (error) {
  if (error instanceof Groq.PermissionDeniedError) {
    console.error('No permission to cancel batch');
  }
}
```

--------------------------------

### Stream.fromSSEResponse() Example

Source: https://github.com/groq/groq-typescript/blob/main/_autodocs/10-api-promise-and-streaming.md

Creates a stream from a raw fetch response and iterates over the chunks.

```typescript
import Groq from 'groq-sdk';

const client = new Groq();

const response = await fetch('https://api.groq.com/openai/v1/chat/completions', {
  method: 'POST',
  headers: { 'Authorization': 'Bearer key' },
  body: JSON.stringify({
    messages: [{ role: 'user', content: 'Hello' }],
    model: 'mixtral-8x7b-32768',
    stream: true,
  }),
});

const controller = new AbortController();
const stream = Stream.fromSSEResponse(response, controller, client);

for await (const chunk of stream) {
  console.log(chunk);
}
```

--------------------------------

### Base64 Encoded Embeddings example

Source: https://github.com/groq/groq-typescript/blob/main/_autodocs/04-embeddings.md

Demonstrates how to request embeddings in base64 format.

```typescript
import Groq from 'groq-sdk';

const client = new Groq();

const response = await client.embeddings.create({
  input: 'The quick brown fox',
  model: 'nomic-embed-text-v1_5',
  encoding_format: 'base64',
});

const embeddingBase64 = response.data[0].embedding as string;
console.log('Embedding (base64):', embeddingBase64);
```

--------------------------------

### Initialize Groq client with environment variables

Source: https://github.com/groq/groq-typescript/blob/main/_autodocs/09-configuration.md

The client automatically detects the GROQ_API_KEY from the environment.

```typescript
const client = new Groq();
// Uses process.env['GROQ_API_KEY'] automatically
```

--------------------------------

### Multiple Text Embeddings example

Source: https://github.com/groq/groq-typescript/blob/main/_autodocs/04-embeddings.md

Demonstrates how to generate embeddings for an array of strings.

```typescript
import Groq from 'groq-sdk';

const client = new Groq();

const response = await client.embeddings.create({
  input: [
    'Hello, how are you?',
    'This is a test.',
    'Another example text.',
  ],
  model: 'nomic-embed-text-v1_5',
});

response.data.forEach((embedding, i) => {
  console.log(`Text ${i} embedding:`, embedding.embedding);
});
```

--------------------------------

### Run Tests

Source: https://github.com/groq/groq-typescript/blob/main/CONTRIBUTING.md

Execute the project's test suite.

```sh
$ yarn run test
```

--------------------------------

### Stream toReadableStream() Example

Source: https://github.com/groq/groq-typescript/blob/main/_autodocs/10-api-promise-and-streaming.md

Converts a Groq stream into a native ReadableStream for platform-agnostic usage.

```typescript
import Groq from 'groq-sdk';

const client = new Groq();

const stream = await client.chat.completions.create({
  messages: [{ role: 'user', content: 'Hello' }],
  model: 'mixtral-8x7b-32768',
  stream: true,
});

const readable = stream.toReadableStream();
// Use with fetch Response, etc.
```

--------------------------------

### Handle File Uploads with Groq SDK

Source: https://github.com/groq/groq-typescript/blob/main/README.md

Demonstrates various methods for uploading files, including using Node.js `fs.ReadStream`, the web `File` API, a `fetch` Response, and the `toFile` helper function.

```ts
import fs from 'fs';
import Groq, { toFile } from 'groq-sdk';

const client = new Groq();

// If you have access to Node `fs` we recommend using `fs.createReadStream()`:
await client.audio.transcriptions.create({
  model: 'whisper-large-v3-turbo',
  file: fs.createReadStream('/path/to/file'),
});

// Or if you have the web `File` API you can pass a `File` instance:
await client.audio.transcriptions.create({
  model: 'whisper-large-v3-turbo',
  file: new File(['my bytes'], 'file'),
});

// You can also pass a `fetch` `Response`:
await client.audio.transcriptions.create({
  model: 'whisper-large-v3-turbo',
  file: await fetch('https://somesite/file'),
});

// Finally, if none of the above are convenient, you can use our `toFile` helper:
await client.audio.transcriptions.create({
  model: 'whisper-large-v3-turbo',
  file: await toFile(Buffer.from('my bytes'), 'file'),
});
await client.audio.transcriptions.create({
  model: 'whisper-large-v3-turbo',
  file: await toFile(new Uint8Array([0, 1, 2]), 'file'),
});
```

--------------------------------

### Single Text Embedding example

Source: https://github.com/groq/groq-typescript/blob/main/_autodocs/04-embeddings.md

Demonstrates how to generate an embedding for a single string input.

```typescript
import Groq from 'groq-sdk';

const client = new Groq();

const response = await client.embeddings.create({
  input: 'The quick brown fox jumped over the lazy dog',
  model: 'nomic-embed-text-v1_5',
});

console.log('Embedding:', response.data[0].embedding);
console.log('Tokens used:', response.usage.total_tokens);
```

--------------------------------

### Method: create

Source: https://github.com/groq/groq-typescript/blob/main/_autodocs/06-files-and-batches.md

Creates and executes a batch from an uploaded file of requests.

```typescript
create(
  body: BatchCreateParams,
  options?: RequestOptions
): APIPromise<BatchCreateResponse>
```

--------------------------------

### Environment Variables

Source: https://github.com/groq/groq-typescript/blob/main/_autodocs/00-index.md

Required and optional environment variables for configuring the client.

```bash
# Required
export GROQ_API_KEY=gsk_...

# Optional
export GROQ_BASE_URL=https://api.groq.com
export GROQ_LOG=debug
export GROQ_CUSTOM_HEADERS="X-Custom: value"
```

--------------------------------

### Configure Bun proxy

Source: https://github.com/groq/groq-typescript/blob/main/README.md

Configure proxy settings for Bun environments by specifying the proxy URL directly in `fetchOptions`.

```typescript
import Groq from 'groq-sdk';

const client = new Groq({
  fetchOptions: {
    proxy: 'http://localhost:8888',
  },
});
```

--------------------------------

### Method: list

Source: https://github.com/groq/groq-typescript/blob/main/_autodocs/06-files-and-batches.md

Lists all batches for the organization.

```typescript
list(options?: RequestOptions): APIPromise<BatchListResponse>
```

--------------------------------

### Cloning Groq Client with Options

Source: https://github.com/groq/groq-typescript/blob/main/_autodocs/09-configuration.md

Create new client instances by merging options from an existing client. Properties from the original client are preserved unless explicitly overridden.

```typescript
const client = new Groq({
  apiKey: process.env['GROQ_API_KEY'],
  timeout: 30000,
});

// Create variant with different timeout
const slowClient = client.withOptions({
  timeout: 120000,
  maxRetries: 5,
});

// Preserve original configuration
const fastClient = client.withOptions({
  timeout: 5000,
});
```

--------------------------------

### ClientOptions Interface

Source: https://github.com/groq/groq-typescript/blob/main/_autodocs/08-types.md

Configuration options used when instantiating the Groq client.

```typescript
interface ClientOptions {
  apiKey?: string | undefined;
  baseURL?: string | null | undefined;
  timeout?: number | undefined;
  fetchOptions?: MergedRequestInit | undefined;
  fetch?: Fetch | undefined;
  maxRetries?: number | undefined;
  defaultHeaders?: HeadersLike | undefined;
  defaultQuery?: Record<string, string | undefined> | undefined;
  dangerouslyAllowBrowser?: boolean | undefined;
  logLevel?: LogLevel | undefined;
  logger?: Logger | undefined;
}
```

--------------------------------

### Batch Processing with Embeddings example

Source: https://github.com/groq/groq-typescript/blob/main/_autodocs/04-embeddings.md

Demonstrates a wrapper function for processing batches of text and extracting vectors.

```typescript
import Groq from 'groq-sdk';

const client = new Groq();

async function embedTexts(texts: string[]) {
  const response = await client.embeddings.create({
    input: texts,
    model: 'nomic-embed-text-v1_5',
  });

  return response.data.map(embedding => ({
    index: embedding.index,
    vector: embedding.embedding,
  }));
}

const texts = ['Apple', 'Banana', 'Cherry'];
const embeddings = await embedTexts(texts);

embeddings.forEach(emb => {
  console.log(`Index ${emb.index}:`, emb.vector.slice(0, 5), '...');
});
```

--------------------------------

### Importing Resource Classes in Groq SDK

Source: https://github.com/groq/groq-typescript/blob/main/MIGRATION.md

Previously, resource classes like Completions could be imported directly from the root. Now, they must be referenced as static class properties or imported from their specific files.

```typescript
// Before
const { Completions } = require('groq-sdk');

// After
const { Groq } = require('groq-sdk');
Groq.Completions; // or import directly from groq-sdk/resources/completions
```

--------------------------------

### Files.list method signature

Source: https://github.com/groq/groq-typescript/blob/main/_autodocs/06-files-and-batches.md

Returns a list of files.

```typescript
list(options?: RequestOptions): APIPromise<FileListResponse>
```

--------------------------------

### TypeScript Configuration for Node.js

Source: https://github.com/groq/groq-typescript/blob/main/MIGRATION.md

Configure `tsconfig.json` with `target` and ensure `@types/node` is installed in `package.json` for Node.js environments.

```jsonc
{
  "target": "ES2018" // note: we recommend ES2020 or higher
}
```

```json
{
  "devDependencies": {
    "@types/node": ">= 20"
  }
}
```

--------------------------------

### Link Local Repository with pnpm

Source: https://github.com/groq/groq-typescript/blob/main/CONTRIBUTING.md

Clone the repository and use `pnpm link` to link it globally to your local package for development.

```sh
# With pnpm
$ pnpm link --global
$ cd ../my-package
$ pnpm link --global groq-sdk
```

--------------------------------

### create(body: BatchCreateParams, options?: RequestOptions)

Source: https://github.com/groq/groq-typescript/blob/main/_autodocs/06-files-and-batches.md

Creates and executes a batch from an uploaded file of requests.

```APIDOC
## create(body: BatchCreateParams, options?: RequestOptions)

### Description
Creates and executes a batch from an uploaded file of requests.

### Parameters
- **body** (BatchCreateParams) - Required - Request body
  - **input_file_id** (string) - Required - ID of the uploaded .jsonl file
  - **endpoint** (string) - Required - API endpoint for batch requests
  - **completion_window** (string) - Required - Time frame for processing ('24h')
  - **metadata** (Record<string, unknown>) - Optional - Custom metadata for the batch
- **options** (RequestOptions) - Optional - Optional request configuration

### Returns
APIPromise<BatchCreateResponse> - Batch information

### Throws
- BadRequestError (400) - Invalid input file or parameters
- AuthenticationError (401) - Invalid API key
- RateLimitError (429) - Rate limit exceeded
- InternalServerError (>=500) - Server error
```

--------------------------------

### Configure Node.js proxy with undici

Source: https://github.com/groq/groq-typescript/blob/main/README.md

Set up a proxy for Node.js environments by providing an `undici.ProxyAgent` instance within `fetchOptions`.

```typescript
import Groq from 'groq-sdk';
import * as undici from 'undici';

const proxyAgent = new undici.ProxyAgent('http://localhost:8888');
const client = new Groq({
  fetchOptions: {
    dispatcher: proxyAgent,
  },
});
```

--------------------------------

### Configure Per-Request Timeout

Source: https://github.com/groq/groq-typescript/blob/main/README.md

Override the default timeout for a specific request. This example sets the timeout to 5 seconds.

```typescript
await client.chat.completions.create({ messages: [{ role: 'system', content: 'You are a helpful assistant.' }, { role: 'user', content: 'Explain the importance of low latency LLMs' }], model: 'openai/gpt-oss-20b' }, {
  timeout: 5 * 1000,
});
```

--------------------------------

### Calling an Undocumented Endpoint

Source: https://github.com/groq/groq-typescript/blob/main/_autodocs/12-advanced-usage.md

Example of using the client.post method to send a request to an experimental path with body and query parameters.

```typescript
import Groq from 'groq-sdk';

const client = new Groq();

// Call an undocumented endpoint
const result = await client.post('/some/experimental/path', {
  body: { some_prop: 'foo' },
  query: { version: 'beta' },
});
```

--------------------------------

### Browser Environment Configuration

Source: https://github.com/groq/groq-typescript/blob/main/_autodocs/09-configuration.md

Enable the Groq client in web browsers by setting dangerouslyAllowBrowser to true. This is disabled by default to prevent credential exposure.

```typescript
const client = new Groq({
  apiKey: process.env['GROQ_API_KEY'],
  dangerouslyAllowBrowser: true, // ⚠️ Security risk
});
```

--------------------------------

### Configure Default Timeout

Source: https://github.com/groq/groq-typescript/blob/main/README.md

Set the default request timeout for all requests. The default is 1 minute. This example sets it to 20 seconds.

```typescript
const client = new Groq({
  timeout: 20 * 1000, // 20 seconds (default is 1 minute)
});
```

--------------------------------

### TypeScript Configuration for Cloudflare Workers

Source: https://github.com/groq/groq-typescript/blob/main/MIGRATION.md

Configure `tsconfig.json` with `target`, `lib`, and `types`, and ensure `@cloudflare/workers-types` is installed in `package.json` for Cloudflare Workers.

```jsonc
{
  "target": "ES2018", // note: we recommend ES2020 or higher
  "lib": ["ES2020"], // <- needed by @cloudflare/workers-types
  "types": ["@cloudflare/workers-types"]
}
```

```json
{
  "devDependencies": {
    "@cloudflare/workers-types": ">= 0.20221111.0"
  }
}
```

--------------------------------

### Files.create method signature

Source: https://github.com/groq/groq-typescript/blob/main/_autodocs/06-files-and-batches.md

Uploads a file for use with the Batch API. The file must be in .jsonl format and under 100 MB.

```typescript
create(
  body: FileCreateParams,
  options?: RequestOptions
): APIPromise<FileCreateResponse>
```

--------------------------------

### Accessing data and metadata with withResponse()

Source: https://github.com/groq/groq-typescript/blob/main/_autodocs/10-api-promise-and-streaming.md

Example of using withResponse to retrieve both the parsed completion data and the raw HTTP response metadata simultaneously.

```typescript
import Groq from 'groq-sdk';

const client = new Groq();

const { data: completion, response: raw } = await client.chat.completions
  .create({
    messages: [{ role: 'user', content: 'Hello' }],
    model: 'mixtral-8x7b-32768',
  })
  .withResponse();

console.log('Completion:', completion.choices[0].message.content);
console.log('Status:', raw.status);
console.log('X-RateLimit-Limit:', raw.headers.get('X-RateLimit-Limit'));
```

--------------------------------

### Create File

Source: https://github.com/groq/groq-typescript/blob/main/api.md

Uploads a file to be used with Groq API. The file can be of any type, but it will be converted to a `jsonl` format before being processed.

```APIDOC
## POST /openai/v1/files

### Description
Uploads a file to be used with Groq API.

### Method
POST

### Endpoint
/openai/v1/files

### Request Body
- **file** (binary) - Required - The file to upload.
- **purpose** (string) - Required - The purpose of the uploaded file. Use `fine-tune` for fine-tuning models.

### Response
#### Success Response (200)
- **id** (string) - The ID of the uploaded file.
- **object** (string) - The object type, which is `file`.
- **bytes** (integer) - The size of the file in bytes.
- **created_at** (integer) - The Unix timestamp of when the file was created.
- **filename** (string) - The name of the file.
- **purpose** (string) - The purpose of the file.
```

--------------------------------

### Access Raw Response and Parsed Data

Source: https://github.com/groq/groq-typescript/blob/main/README.md

Use `.withResponse()` to get both the raw `Response` object and the parsed data. This method consumes the response body.

```typescript
const { data: chatCompletion, response: raw } = await client.chat.completions
  .create({
    messages: [
      { role: 'system', content: 'You are a helpful assistant.' },
      { role: 'user', content: 'Explain the importance of low latency LLMs' },
    ],
    model: 'openai/gpt-oss-20b',
  })
  .withResponse();
console.log(raw.headers.get('X-My-Header'));
console.log(chatCompletion.id);
```

--------------------------------

### List Files

Source: https://github.com/groq/groq-typescript/blob/main/api.md

Returns a list of all files uploaded to your account.

```APIDOC
## GET /openai/v1/files

### Description
Returns a list of all files uploaded to your account.

### Method
GET

### Endpoint
/openai/v1/files

### Response
#### Success Response (200)
- **data** (array) - A list of file objects.
- **object** (string) - The object type, which is `list`.
```

--------------------------------

### Cancel Requests with AbortSignal

Source: https://github.com/groq/groq-typescript/blob/main/_autodocs/12-advanced-usage.md

Demonstrates how to use AbortController and AbortSignal to cancel a chat completion request. The example includes a timeout mechanism and error handling for Groq.APIUserAbortError.

```typescript
import Groq from 'groq-sdk';

const client = new Groq();

const controller = new AbortController();

// Cancel after 5 seconds
const timeout = setTimeout(() => controller.abort(), 5000);

try {
  const completion = await client.chat.completions.create(
    {
      messages: [{ role: 'user', content: 'Hello' }],
      model: 'mixtral-8x7b-32768',
    },
    {
      signal: controller.signal,
    }
  );
} catch (error) {
  if (error instanceof Groq.APIUserAbortError) {
    console.log('Request was cancelled');
  }
} finally {
  clearTimeout(timeout);
}
```

--------------------------------

### Custom header access pattern

Source: https://github.com/groq/groq-typescript/blob/main/_autodocs/12-advanced-usage.md

An example of wrapping the withResponse() method to extract specific metrics like request IDs and rate limits into a custom object.

```typescript
import Groq from 'groq-sdk';

const client = new Groq();

async function requestWithMetrics(prompt: string) {
  const { data, response } = await client.chat.completions
    .create({
      messages: [{ role: 'user', content: prompt }],
      model: 'mixtral-8x7b-32768',
    })
    .withResponse();

  return {
    content: data.choices[0].message.content,
    metadata: {
      requestId: response.headers.get('X-Groq-Request-Id'),
      rateLimit: parseInt(response.headers.get('X-RateLimit-Remaining-Requests') || '0'),
      processingTime: response.headers.get('X-Processing-Time'),
    },
  };
}

const result = await requestWithMetrics('Hello!');
```

--------------------------------

### Configure fetch options for Groq client

Source: https://github.com/groq/groq-typescript/blob/main/README.md

Provide a `fetchOptions` object during client instantiation to set custom fetch options without replacing the fetch function itself.

```typescript
import Groq from 'groq-sdk';

const client = new Groq({
  fetchOptions: {
    // `RequestInit` options
  },
});
```

--------------------------------

### Implement Multi-Round Conversation

Source: https://github.com/groq/groq-typescript/blob/main/_autodocs/12-advanced-usage.md

This example shows how to maintain conversation history by appending assistant responses to a message array, allowing the model to maintain context across multiple turns.

```typescript
import Groq from 'groq-sdk';

const client = new Groq();

async function multiTurnConversation() {
  const messages: Groq.Chat.ChatCompletionMessageParam[] = [
    {
      role: 'system',
      content: 'You are a helpful mathematics tutor.',
    },
  ];
  
  const questions = [
    'What is 2 + 2?',
    'Why is that?',
    'Can you give me a harder problem?',
  ];
  
  for (const question of questions) {
    messages.push({
      role: 'user',
      content: question,
    });
    
    const completion = await client.chat.completions.create({
      messages: messages,
      model: 'mixtral-8x7b-32768',
    });
    
    const assistantMessage = completion.choices[0]?.message;
    if (assistantMessage) {
      messages.push(assistantMessage);
      console.log(`Q: ${question}`);
      console.log(`A: ${assistantMessage.content}\n`);
    }
  }
}

await multiTurnConversation();
```
Tokens:
Raw