Groq Typescript (groq/groq-typescript)

Groq Typescript

https://github.com/groq/groq-typescript
Admin
A TypeScript/JavaScript library for convenient access to the Groq REST API, enabling server-side...

Tokens:18,649
Snippets:81
Trust Score:9
Update:4 months ago
Show doc for...
Context Summary (auto-generated)
Raw
# Groq Node SDK

The Groq Node SDK is an official TypeScript/JavaScript library providing convenient access to the Groq REST API from server-side applications. Built using the Stainless code generation framework, this SDK enables developers to leverage Groq's ultra-fast language models for chat completions, embeddings, audio transcription, and other AI capabilities. The library is designed for Node.js, Deno, Bun, Cloudflare Workers, Vercel Edge Runtime, and other JavaScript runtime environments.

The SDK provides a comprehensive set of features including automatic retries with exponential backoff, configurable timeouts, streaming support for real-time responses, and full TypeScript type definitions. It supports various authentication methods, proxy configuration, and custom fetch implementations. The library follows semantic versioning conventions and offers production-ready error handling with specific error types for different HTTP status codes. With built-in support for file uploads, multipart form data, and structured outputs using JSON schemas, the SDK simplifies integration with Groq's powerful AI infrastructure.

## API Reference

### Chat Completions

Generate AI-powered chat responses using Groq's language models with support for streaming, function calling, and tool use.

```javascript
const Groq = require('groq-sdk');

const client = new Groq({
  apiKey: process.env.GROQ_API_KEY
});

async function createChatCompletion() {
  try {
    const chatCompletion = await client.chat.completions.create({
      messages: [
        {
          role: 'system',
          content: 'You are a helpful assistant that explains technical concepts clearly.'
        },
        {
          role: 'user',
          content: 'Explain the importance of low latency LLMs'
        }
      ],
      model: 'mixtral-8x7b-32768',
      temperature: 0.7,
      max_tokens: 1024,
      top_p: 1,
      stop: null
    });

    console.log(chatCompletion.choices[0].message.content);
    console.log(`\nUsage: ${chatCompletion.usage.total_tokens} tokens`);
    console.log(`Model: ${chatCompletion.model}`);
    console.log(`Finish reason: ${chatCompletion.choices[0].finish_reason}`);
  } catch (error) {
    if (error instanceof Groq.APIError) {
      console.error(`API Error: ${error.status} - ${error.message}`);
    } else {
      throw error;
    }
  }
}

createChatCompletion();
```

### Streaming Chat Completions

Receive chat responses in real-time as tokens are generated, ideal for interactive applications and reducing perceived latency.

```javascript
const Groq = require('groq-sdk');

const client = new Groq({
  apiKey: process.env.GROQ_API_KEY
});

async function streamChatCompletion() {
  try {
    const stream = await client.chat.completions.create({
      messages: [
        {
          role: 'system',
          content: 'You are a helpful coding assistant.'
        },
        {
          role: 'user',
          content: 'Write a Python function to calculate fibonacci numbers'
        }
      ],
      model: 'llama-3.3-70b-versatile',
      temperature: 0.5,
      max_tokens: 1024,
      stream: true
    });

    process.stdout.write('Response: ');
    for await (const chunk of stream) {
      const content = chunk.choices[0]?.delta?.content || '';
      process.stdout.write(content);
    }
    process.stdout.write('\n');
  } catch (error) {
    if (error instanceof Groq.RateLimitError) {
      console.error('Rate limit exceeded. Please try again later.');
    } else if (error instanceof Groq.APIConnectionError) {
      console.error('Connection error. Check your network.');
    } else {
      throw error;
    }
  }
}

streamChatCompletion();
```

### Tool Calling and Function Execution

Enable models to call external functions and tools, supporting complex workflows and compound AI systems.

```javascript
const Groq = require('groq-sdk');

const client = new Groq();

async function chatWithTools() {
  const tools = [
    {
      type: 'function',
      function: {
        name: 'get_weather',
        description: 'Get the current weather in a given location',
        parameters: {
          type: 'object',
          properties: {
            location: {
              type: 'string',
              description: 'The city and state, e.g. San Francisco, CA'
            },
            unit: {
              type: 'string',
              enum: ['celsius', 'fahrenheit']
            }
          },
          required: ['location']
        }
      }
    },
    {
      type: 'browser_search'
    }
  ];

  try {
    const response = await client.chat.completions.create({
      messages: [
        {
          role: 'user',
          content: 'What is the weather like in San Francisco?'
        }
      ],
      model: 'llama-3.1-8b-instant',
      tools: tools,
      tool_choice: 'auto'
    });

    const toolCall = response.choices[0].message.tool_calls?.[0];
    if (toolCall && toolCall.function.name === 'get_weather') {
      const args = JSON.parse(toolCall.function.arguments);
      console.log(`Tool called: ${toolCall.function.name}`);
      console.log(`Arguments: ${JSON.stringify(args, null, 2)}`);

      // Simulate function execution
      const functionResponse = {
        temperature: 72,
        conditions: 'sunny',
        location: args.location
      };

      // Send function result back to model
      const finalResponse = await client.chat.completions.create({
        messages: [
          { role: 'user', content: 'What is the weather like in San Francisco?' },
          response.choices[0].message,
          {
            role: 'tool',
            tool_call_id: toolCall.id,
            content: JSON.stringify(functionResponse)
          }
        ],
        model: 'llama-3.1-8b-instant'
      });

      console.log(`\nFinal response: ${finalResponse.choices[0].message.content}`);
    }
  } catch (error) {
    if (error instanceof Groq.BadRequestError) {
      console.error(`Bad request: ${error.message}`);
    } else {
      throw error;
    }
  }
}

chatWithTools();
```

### Embeddings

Convert text into dense vector representations for semantic search, clustering, and similarity comparisons.

```javascript
const Groq = require('groq-sdk');

const client = new Groq({
  apiKey: process.env.GROQ_API_KEY
});

async function createEmbeddings() {
  try {
    const response = await client.embeddings.create({
      input: [
        'The quick brown fox jumped over the lazy dog',
        'Machine learning is a subset of artificial intelligence',
        'Groq provides ultra-fast inference for AI models'
      ],
      model: 'nomic-embed-text-v1_5',
      encoding_format: 'float'
    });

    console.log(`Generated ${response.data.length} embeddings`);
    console.log(`Model: ${response.model}`);
    console.log(`Total tokens: ${response.usage.total_tokens}`);

    response.data.forEach((embedding, index) => {
      console.log(`\nEmbedding ${index}:`);
      console.log(`  Index: ${embedding.index}`);
      console.log(`  Dimensions: ${embedding.embedding.length}`);
      console.log(`  First 5 values: ${embedding.embedding.slice(0, 5)}`);
    });

    // Calculate cosine similarity between first two embeddings
    const vec1 = response.data[0].embedding;
    const vec2 = response.data[1].embedding;
    const dotProduct = vec1.reduce((sum, val, i) => sum + val * vec2[i], 0);
    const mag1 = Math.sqrt(vec1.reduce((sum, val) => sum + val * val, 0));
    const mag2 = Math.sqrt(vec2.reduce((sum, val) => sum + val * val, 0));
    const similarity = dotProduct / (mag1 * mag2);
    console.log(`\nCosine similarity: ${similarity.toFixed(4)}`);
  } catch (error) {
    if (error instanceof Groq.UnprocessableEntityError) {
      console.error('Invalid input format');
    } else {
      throw error;
    }
  }
}

createEmbeddings();
```

### Audio Transcription

Transcribe audio files to text using Whisper models with support for multiple languages and timestamp granularities.

```javascript
const Groq = require('groq-sdk');
const fs = require('fs');

const client = new Groq({
  apiKey: process.env.GROQ_API_KEY
});

async function transcribeAudio() {
  try {
    const transcription = await client.audio.transcriptions.create({
      file: fs.createReadStream('/path/to/audio.mp3'),
      model: 'whisper-large-v3-turbo',
      language: 'en',
      response_format: 'verbose_json',
      temperature: 0.0,
      timestamp_granularities: ['word', 'segment'],
      prompt: 'This is a technical presentation about AI.'
    });

    console.log(`Transcription: ${transcription.text}`);
    console.log(`\nLanguage detected: ${transcription.language}`);
    console.log(`Duration: ${transcription.duration} seconds`);

    if (transcription.segments) {
      console.log('\nSegments:');
      transcription.segments.forEach((segment, index) => {
        console.log(`  [${segment.start}s - ${segment.end}s]: ${segment.text}`);
      });
    }

    if (transcription.words) {
      console.log(`\nTotal words: ${transcription.words.length}`);
      console.log('First 5 words with timestamps:');
      transcription.words.slice(0, 5).forEach(word => {
        console.log(`  ${word.word} (${word.start}s - ${word.end}s)`);
      });
    }
  } catch (error) {
    if (error instanceof Groq.AuthenticationError) {
      console.error('Invalid API key');
    } else if (error instanceof Groq.NotFoundError) {
      console.error('Audio file not found');
    } else {
      throw error;
    }
  }
}

transcribeAudio();
```

### Model Management

List available models, retrieve model details, and manage model configurations.

```javascript
const Groq = require('groq-sdk');

const client = new Groq({
  apiKey: process.env.GROQ_API_KEY
});

async function manageModels() {
  try {
    // List all available models
    const modelList = await client.models.list();
    console.log(`Available models: ${modelList.data.length}\n`);

    modelList.data.forEach(model => {
      console.log(`Model ID: ${model.id}`);
      console.log(`  Owner: ${model.owned_by}`);
      console.log(`  Created: ${new Date(model.created * 1000).toISOString()}`);
      console.log();
    });

    // Retrieve specific model details
    const modelId = 'llama-3.3-70b-versatile';
    const modelDetails = await client.models.retrieve(modelId);
    console.log(`\nDetailed info for ${modelDetails.id}:`);
    console.log(`  Object type: ${modelDetails.object}`);
    console.log(`  Owned by: ${modelDetails.owned_by}`);
    console.log(`  Created: ${new Date(modelDetails.created * 1000).toLocaleDateString()}`);
  } catch (error) {
    if (error instanceof Groq.NotFoundError) {
      console.error('Model not found');
    } else {
      throw error;
    }
  }
}

manageModels();
```

### Client Configuration and Error Handling

Configure SDK client with custom settings including timeouts, retries, proxies, and comprehensive error handling.

```javascript
const Groq = require('groq-sdk');
const { HttpsProxyAgent } = require('https-proxy-agent');

// Advanced client configuration
const client = new Groq({
  apiKey: process.env.GROQ_API_KEY,
  baseURL: 'https://api.groq.com',
  timeout: 30000, // 30 seconds
  maxRetries: 3,
  httpAgent: new HttpsProxyAgent(process.env.PROXY_URL),
  defaultHeaders: {
    'X-Custom-Header': 'value'
  },
  defaultQuery: {
    'api-version': '2024-01'
  }
});

async function robustAPICall() {
  try {
    const response = await client.chat.completions.create(
      {
        messages: [{ role: 'user', content: 'Hello!' }],
        model: 'llama-3.1-8b-instant',
        max_tokens: 100
      },
      {
        timeout: 10000, // Override timeout for this request
        maxRetries: 5,
        headers: { 'X-Request-ID': 'unique-id-123' }
      }
    );

    console.log(response.choices[0].message.content);

    // Access raw response headers
    const { data, response: rawResponse } = await client.chat.completions
      .create({
        messages: [{ role: 'user', content: 'Test' }],
        model: 'llama-3.1-8b-instant'
      })
      .withResponse();

    console.log(`\nRate limit remaining: ${rawResponse.headers.get('x-ratelimit-remaining')}`);
    console.log(`Request ID: ${rawResponse.headers.get('x-request-id')}`);
  } catch (error) {
    // Comprehensive error handling
    if (error instanceof Groq.APIError) {
      console.error(`API Error (${error.status}): ${error.message}`);
      console.error(`Error type: ${error.constructor.name}`);
      console.error(`Headers: ${JSON.stringify(error.headers)}`);

      // Handle specific error types
      switch (true) {
        case error instanceof Groq.RateLimitError:
          console.error('Rate limit exceeded - implement backoff');
          break;
        case error instanceof Groq.AuthenticationError:
          console.error('Authentication failed - check API key');
          break;
        case error instanceof Groq.InternalServerError:
          console.error('Server error - retry the request');
          break;
        case error instanceof Groq.APIConnectionError:
          console.error('Connection failed - check network');
          break;
        case error instanceof Groq.APIConnectionTimeoutError:
          console.error('Request timed out - increase timeout');
          break;
        default:
          console.error('Unexpected API error occurred');
      }
    } else {
      throw error;
    }
  }
}

robustAPICall();
```

### TypeScript Integration with Type Safety

Leverage full TypeScript support with type definitions for requests, responses, and advanced type inference.

```typescript
import Groq from 'groq-sdk';
import type {
  ChatCompletion,
  ChatCompletionCreateParams,
  ChatCompletionMessage,
  ChatCompletionMessageParam,
  ChatCompletionChunk
} from 'groq-sdk/resources/chat/completions';

const client = new Groq({
  apiKey: process.env.GROQ_API_KEY as string
});

async function typedChatCompletion(): Promise<void> {
  // Strongly typed request parameters
  const params: ChatCompletionCreateParams = {
    messages: [
      {
        role: 'system',
        content: 'You are a TypeScript expert.'
      },
      {
        role: 'user',
        content: 'Explain generics in TypeScript'
      }
    ],
    model: 'llama-3.3-70b-versatile',
    temperature: 0.7,
    max_tokens: 500,
    top_p: 0.9
  };

  try {
    // Strongly typed response
    const completion: ChatCompletion = await client.chat.completions.create(params);

    const message: ChatCompletionMessage = completion.choices[0].message;
    console.log(`Role: ${message.role}`);
    console.log(`Content: ${message.content}`);

    // Type-safe access to usage information
    if (completion.usage) {
      console.log(`Prompt tokens: ${completion.usage.prompt_tokens}`);
      console.log(`Completion tokens: ${completion.usage.completion_tokens}`);
      console.log(`Total tokens: ${completion.usage.total_tokens}`);
    }

    // Streaming with types
    const streamParams: ChatCompletionCreateParams = {
      ...params,
      stream: true
    };

    const stream = await client.chat.completions.create(streamParams);

    for await (const chunk of stream) {
      const delta = chunk.choices[0]?.delta;
      if (delta?.content) {
        process.stdout.write(delta.content);
      }
    }
  } catch (error) {
    if (error instanceof Groq.APIError) {
      console.error(`Error ${error.status}: ${error.message}`);
    } else {
      throw error;
    }
  }
}

typedChatCompletion();
```

### Structured Outputs with JSON Schema

Generate strictly formatted JSON responses using JSON Schema validation for reliable structured data extraction.

```javascript
const Groq = require('groq-sdk');

const client = new Groq();

async function structuredOutput() {
  const schema = {
    name: 'product_review',
    description: 'Extract structured product review information',
    schema: {
      type: 'object',
      properties: {
        product_name: {
          type: 'string',
          description: 'The name of the product'
        },
        rating: {
          type: 'number',
          minimum: 1,
          maximum: 5,
          description: 'Rating from 1 to 5'
        },
        pros: {
          type: 'array',
          items: { type: 'string' },
          description: 'List of positive aspects'
        },
        cons: {
          type: 'array',
          items: { type: 'string' },
          description: 'List of negative aspects'
        },
        recommendation: {
          type: 'boolean',
          description: 'Whether the reviewer recommends the product'
        }
      },
      required: ['product_name', 'rating', 'recommendation'],
      additionalProperties: false
    },
    strict: true
  };

  try {
    const response = await client.chat.completions.create({
      messages: [
        {
          role: 'system',
          content: 'Extract product review information in the specified format.'
        },
        {
          role: 'user',
          content: 'I bought the SuperWidget 3000 and it is amazing! The build quality is excellent and it is very fast. However, it is a bit expensive and the battery life could be better. Overall, I highly recommend it. 4.5 out of 5 stars.'
        }
      ],
      model: 'llama-3.3-70b-versatile',
      response_format: {
        type: 'json_schema',
        json_schema: schema
      },
      temperature: 0.3
    });

    const content = response.choices[0].message.content;
    const structuredData = JSON.parse(content);

    console.log('Structured Review Data:');
    console.log(JSON.stringify(structuredData, null, 2));
    console.log(`\nProduct: ${structuredData.product_name}`);
    console.log(`Rating: ${structuredData.rating}/5`);
    console.log(`Recommended: ${structuredData.recommendation ? 'Yes' : 'No'}`);
    console.log(`Pros: ${structuredData.pros.join(', ')}`);
    console.log(`Cons: ${structuredData.cons.join(', ')}`);
  } catch (error) {
    if (error instanceof Groq.BadRequestError) {
      console.error('Invalid schema or request format');
    } else {
      throw error;
    }
  }
}

structuredOutput();
```

## Summary

The Groq Node SDK serves as the primary interface for integrating Groq's ultra-fast language models into JavaScript and TypeScript applications. Its main use cases span conversational AI applications with streaming support, semantic search and document similarity using embeddings, audio transcription for accessibility and content processing, and complex AI workflows with tool calling and function execution. The SDK is particularly well-suited for production applications requiring low-latency responses, making it ideal for real-time chatbots, interactive assistants, and high-throughput AI services. The library also supports advanced scenarios like structured data extraction with JSON schemas, multi-model compound AI systems, and batch processing workflows.

Integration patterns include standalone Node.js services, serverless functions on platforms like Vercel and Cloudflare Workers, and microservices architectures where the SDK handles AI inference calls. Developers can leverage the SDK's built-in retry logic and connection pooling for reliability, use streaming responses to provide real-time feedback in user interfaces, and implement sophisticated error handling strategies with typed error classes. The SDK's TypeScript-first design ensures type safety throughout the development process, while its support for custom HTTP agents enables integration with corporate proxies and specialized networking requirements. Whether building simple chatbots or complex AI-powered applications with tool use and document retrieval, the Groq Node SDK provides a robust, type-safe, and developer-friendly foundation for leveraging Groq's inference infrastructure.