Sonilo JavaScript SDK

https://github.com/sonilo-ai/sonilo-js
Admin
The Sonilo JavaScript SDK is the official collection of TypeScript and JavaScript packages,...
Tokens:5,957
Snippets:69
Trust Score:7.5
License:MIT
Update:3 months ago
Show doc for...
Tokens:
Raw
### Install sonilo-video-kit

Source: https://github.com/sonilo-ai/sonilo-js/blob/main/packages/sonilo-video-kit/README.md

Install the package along with its required peer dependency.

```bash
npm install sonilo sonilo-video-kit
```

--------------------------------

### Install Sonilo package

Source: https://github.com/sonilo-ai/sonilo-js/blob/main/packages/sonilo/README.md

Use npm to install the Sonilo client library.

```bash
npm install sonilo
```

--------------------------------

### Install Sonilo Packages

Source: https://github.com/sonilo-ai/sonilo-js/blob/main/README.md

Commands to install the core SDK, video kit, and CLI globally.

```bash
npm install sonilo
```

```bash
npm install sonilo sonilo-video-kit
```

```bash
npm install -g sonilo-cli
```

--------------------------------

### Install Sonilo CLI

Source: https://github.com/sonilo-ai/sonilo-js/blob/main/packages/cli/README.md

Install the CLI globally or execute it directly using npx.

```bash
npm install -g sonilo-cli
```

```bash
npx sonilo-cli account
```

--------------------------------

### Generate Segmented Music

Source: https://github.com/sonilo-ai/sonilo-js/blob/main/packages/sonilo/README.md

Define specific composition segments with start times and prompts to shape the output.

```ts
await sonilo.textToMusic.generate({
  prompt: "epic trailer",
  duration: 60,
  segments: [
    { start: 0, prompt: "soft intro", label: "intro" },
    { start: 20, prompt: "building tension", label: "verse" },
    { start: 40, prompt: "full orchestra", label: "chorus" },
  ],
});
```

--------------------------------

### Initialize Sonilo Client

Source: https://github.com/sonilo-ai/sonilo-js/blob/main/packages/sonilo/README.md

Configure the client with an API key, base URL, and request timeout settings.

```ts
const client = new SoniloClient({
  apiKey: "sk_...", // defaults to SONILO_API_KEY
  baseUrl: "https://api.sonilo.com",
  timeout: 600_000, // milliseconds, default 600000 (10 minutes)
});
```

--------------------------------

### Configure Async Video-to-Music Options

Source: https://github.com/sonilo-ai/sonilo-js/blob/main/packages/sonilo/README.md

Configure ducking and output formats for async video-to-music tasks.

```typescript
const task = await client.videoToMusic.submit({
  video: "./my_video.mp4",
  preserveSpeech: true,
  outputFormat: "wav",
  // ducking is on by default in async — set `false` to disable
});
const result = await client.tasks.wait<MusicTaskResult>(task.task_id);
if (result.ducked) {
  await writeFile("ducked.wav", await download(result.ducked[0]!));
}
```

--------------------------------

### Configure Authentication

Source: https://github.com/sonilo-ai/sonilo-js/blob/main/packages/cli/README.md

Set the API key via environment variable or command-line flag.

```bash
export SONILO_API_KEY=sk-...
```

--------------------------------

### CLI Usage Commands

Source: https://github.com/sonilo-ai/sonilo-js/blob/main/packages/cli/README.md

Common commands for account management, usage reporting, and audio generation tasks.

```bash
# Plan limits and available services
sonilo account

# Usage summary (defaults to the last 30 days)
sonilo usage --days 7

# Generate music from a text prompt
sonilo text-to-music --prompt "warm lo-fi piano, rain in the background" --duration 30

# Generate music matched to a video
sonilo video-to-music --video clip.mp4 --prompt "tense, driving synths" --output score.wav --format wav

# Generate a sound effect from a text prompt
sonilo text-to-sfx --prompt "glass bottle shattering on concrete" --duration 3

# Generate a sound effect matched to a video
sonilo video-to-sfx --video clip.mp4 --output foley.wav

# Generate a combined music + SFX track for a video (async only)
sonilo video-to-sound --video clip.mp4 --music-prompt "tense strings" --sfx-prompt "footsteps, distant thunder" --output mix.wav

# Same, but muxed back into the video
sonilo video-to-video-sound --video clip.mp4 --sfx-prompt "footsteps" --output scored.mp4

# Check an async task
sonilo tasks get <task-id>
sonilo tasks wait <task-id> --poll-interval 2000 --timeout 120000
```

--------------------------------

### client.videoToVideoMusic.generate

Source: https://github.com/sonilo-ai/sonilo-js/blob/main/packages/sonilo/README.md

Generates a music soundtrack for a video and returns a re-hosted video with the audio muxed in.

```APIDOC
## client.videoToVideoMusic.generate

### Description
Generates music for a video file and returns a new video with the music muxed in. This is an asynchronous operation.

### Parameters
- **video** (string|File|Blob) - Required - The input video file path, URL, or object.
- **prompt** (string) - Required - Description of the desired music.
- **preserveSpeech** (boolean) - Optional - Whether to keep the original speech from the video.
```

--------------------------------

### Development Workflow Commands

Source: https://github.com/sonilo-ai/sonilo-js/blob/main/README.md

Standard commands for managing the monorepo workspace.

```bash
npm install          # install all workspaces
npm run build        # build every package (core first)
npm test             # build core, then test every package
```

--------------------------------

### sonilo.account.services()

Source: https://github.com/sonilo-ai/sonilo-js/blob/main/packages/sonilo/README.md

Retrieves a list of available services for the current account.

```APIDOC
## sonilo.account.services()

### Description
Retrieves the list of services associated with the current account.

### Signature
`await sonilo.account.services()`
```

--------------------------------

### client.videoToSfx.submit

Source: https://github.com/sonilo-ai/sonilo-js/blob/main/packages/sonilo/README.md

Submits a video-to-sfx task for asynchronous processing.

```APIDOC
## client.videoToSfx.submit(options)

### Description
Submits a video file and segment configuration to generate sound effects. Returns a task object containing a task_id for polling.

### Parameters
- **video** (string) - Required - Path to the video file (Node.js) or File/Blob (browser).
- **segments** (array) - Required - Array of objects containing start, end, and prompt for each segment.
- **audioFormat** (string) - Optional - The desired output format (e.g., 'wav').

### Returns
- **task_id** (string) - The unique identifier for the submitted task.
```

--------------------------------

### Release Management Commands

Source: https://github.com/sonilo-ai/sonilo-js/blob/main/README.md

Commands for managing package versions and releases using changesets.

```bash
npx changeset
```

--------------------------------

### Configure API Authentication

Source: https://github.com/sonilo-ai/sonilo-js/blob/main/packages/sonilo/README.md

Set the API key via environment variable or pass it directly to the client constructor.

```bash
export SONILO_API_KEY=sk_...
```

```typescript
const sonilo = new SoniloClient();                     // reads SONILO_API_KEY
// or pass it directly:
const sonilo = new SoniloClient({ apiKey: "sk_..." });
```

--------------------------------

### Generate Music from Text

Source: https://github.com/sonilo-ai/sonilo-js/blob/main/packages/sonilo/README.md

Generate music based on a text prompt using the Sonilo client.

```typescript
import { SoniloClient } from "sonilo";

const sonilo = new SoniloClient(); // reads SONILO_API_KEY

const track = await sonilo.textToMusic.generate({
  prompt: "cinematic orchestral score",
  duration: 60,
});
// track.audio is a Uint8Array of MP3 bytes
```

--------------------------------

### Generate Music from Video

Source: https://github.com/sonilo-ai/sonilo-js/blob/main/packages/sonilo/README.md

Generate music from a local file path or a hosted video URL.

```typescript
// Node: file path, browser: File/Blob from an <input type="file">
const track = await sonilo.videoToMusic.generate({
  video: "./my_video.mp4",
  prompt: "upbeat, energetic",
});

// Or point at a hosted video
await sonilo.videoToMusic.generate({ videoUrl: "https://example.com/clip.mp4" });
```

--------------------------------

### sonilo.account.usage()

Source: https://github.com/sonilo-ai/sonilo-js/blob/main/packages/sonilo/README.md

Retrieves usage statistics for the account over a specified number of days.

```APIDOC
## sonilo.account.usage()

### Description
Retrieves usage data for the account.

### Signature
`await sonilo.account.usage({ days: number })`

### Parameters
- **days** (number) - Required - The number of days to look back for usage statistics.
```

--------------------------------

### SoniloClient.videoToMusic.submit

Source: https://github.com/sonilo-ai/sonilo-js/blob/main/packages/sonilo/README.md

Submits an asynchronous task for video-to-music generation with advanced options.

```APIDOC
## SoniloClient.videoToMusic.submit

### Description
Submits a video-to-music task for asynchronous processing, supporting features like speech preservation, ducking, and custom output formats.

### Parameters
- **video** (string) - Required - Local file path to the video.
- **preserveSpeech** (boolean) - Optional - Whether to keep source vocals in the result.
- **ducking** (boolean) - Optional - Whether to duck music under source voice (default: true).
- **outputFormat** (string) - Optional - Output format, either 'm4a' or 'wav'.
```

--------------------------------

### Download individual audio stems

Source: https://github.com/sonilo-ai/sonilo-js/blob/main/packages/sonilo/README.md

Access individual music and sound effect stems from a mixed result to perform custom re-balancing.

```ts
await writeFile("music.m4a", await download(result.music));
await writeFile("sfx.wav", await download(result.sfx));
```

--------------------------------

### client.videoToSound.generate

Source: https://github.com/sonilo-ai/sonilo-js/blob/main/packages/sonilo/README.md

Generates a mixed soundtrack (music and SFX) for a video clip.

```APIDOC
## client.videoToSound.generate

### Description
Generates a music bed and sound effects for a video clip, returning them mixed into a single audio file.

### Parameters
- **videoUrl** (string) - Required - URL of the input video.
- **musicPrompt** (string) - Required - Description for the music.
- **sfxPrompt** (string) - Required - Description for the sound effects.
- **preserveSpeech** (boolean) - Optional - Whether to keep original speech.
- **ducking** (boolean) - Optional - Whether to dip music under speech (default: true).
```

--------------------------------

### client.textToSfx.generate

Source: https://github.com/sonilo-ai/sonilo-js/blob/main/packages/sonilo/README.md

Generates sound effects from a text prompt by automatically handling the submission and polling process.

```APIDOC
## client.textToSfx.generate(options)

### Description
Generates a sound effect based on a text prompt. This method wraps the submission and polling process into a single asynchronous call.

### Parameters
- **prompt** (string) - Required - The text description of the sound effect to generate.
- **duration** (number) - Optional - The desired duration of the sound effect in seconds.

### Returns
- **audio** (string) - A presigned URL to the generated audio file.
```

--------------------------------

### Generate and mix video soundtrack

Source: https://github.com/sonilo-ai/sonilo-js/blob/main/packages/sonilo-video-kit/README.md

Generate a soundtrack based on a video prompt and mix it into the video file.

```ts
import { generateMusicForVideo, mixWithVideo } from "sonilo-video-kit";

const track = await generateMusicForVideo("./clip.mp4", {
  prompt: "upbeat, energetic",
}); // uses SONILO_API_KEY

await mixWithVideo({
  video: "./clip.mp4",
  audio: track.audio,
  output: "./clip.scored.mp4",
});
```

--------------------------------

### client.tasks.get

Source: https://github.com/sonilo-ai/sonilo-js/blob/main/packages/sonilo/README.md

Fetches the current state of a task.

```APIDOC
## client.tasks.get(taskId)

### Description
Retrieves the current status and result of a task. Does not throw on failed tasks.

### Parameters
- **taskId** (string) - Required - The ID of the task to retrieve.
```

--------------------------------

### SoniloClient.textToMusic.generate

Source: https://github.com/sonilo-ai/sonilo-js/blob/main/packages/sonilo/README.md

Generates music from a text prompt synchronously.

```APIDOC
## SoniloClient.textToMusic.generate

### Description
Generates a music track based on a provided text prompt and duration.

### Parameters
- **prompt** (string) - Required - The description of the music to generate.
- **duration** (number) - Optional - The length of the track in seconds.

### Response
- **audio** (Uint8Array) - The generated MP3 bytes.
```

--------------------------------

### Generate video music and sound effects

Source: https://github.com/sonilo-ai/sonilo-js/blob/main/packages/sonilo/README.md

Use the SoniloClient to score music or add sound effects to a video file, returning a re-hosted video with audio muxed in.

```ts
import { SoniloClient, download } from "sonilo";
import { writeFile } from "node:fs/promises";

const client = new SoniloClient();

// Score music into the video (optionally keep the original speech)
const music = await client.videoToVideoMusic.generate({
  video: "./my_video.mp4", // Node path; File/Blob in the browser, or `videoUrl`
  prompt: "cinematic orchestral swell",
  preserveSpeech: true,
});
await writeFile("scored.mp4", await download(music.video!));

// Sound effects for the video, optionally per time segment
const sfx = await client.videoToVideoSfx.generate({
  video: "./my_video.mp4",
  segments: [{ start: 0, end: 2, prompt: "footsteps on gravel" }],
});
await writeFile("with_sfx.mp4", await download(sfx.video!));
```

--------------------------------

### client.videoToVideoSfx.generate

Source: https://github.com/sonilo-ai/sonilo-js/blob/main/packages/sonilo/README.md

Generates sound effects for specific time segments of a video.

```APIDOC
## client.videoToVideoSfx.generate

### Description
Generates sound effects for defined segments of a video and returns the video with the audio muxed in.

### Parameters
- **video** (string|File|Blob) - Required - The input video file path, URL, or object.
- **segments** (Array) - Required - List of objects containing { start, end, prompt } for specific sound effects.
```

--------------------------------

### Generate sound effects with automated polling

Source: https://github.com/sonilo-ai/sonilo-js/blob/main/packages/sonilo/README.md

Uses the generate() method to submit a prompt and automatically poll for the result. Requires importing SoniloClient and download from the sonilo package.

```ts
import { SoniloClient, download } from "sonilo";
import { writeFile } from "node:fs/promises";

const client = new SoniloClient();
const result = await client.textToSfx.generate({ prompt: "glass shattering", duration: 5 });
await writeFile("sfx.m4a", await download(result.audio));
```

--------------------------------

### SoniloClient.videoToMusic.generate

Source: https://github.com/sonilo-ai/sonilo-js/blob/main/packages/sonilo/README.md

Generates music from a video file or URL synchronously.

```APIDOC
## SoniloClient.videoToMusic.generate

### Description
Generates music based on a video input, either from a local file path or a hosted URL.

### Parameters
- **video** (string) - Optional - Local file path to the video.
- **videoUrl** (string) - Optional - URL to a hosted video.
- **prompt** (string) - Optional - Style description for the music.
```

--------------------------------

### Preserve Speech in Async Video-to-Music

Source: https://github.com/sonilo-ai/sonilo-js/blob/main/packages/sonilo/README.md

Use the async task API to preserve speech and download generated stems.

```typescript
import { SoniloClient, download } from "sonilo";
import type { MusicTaskResult } from "sonilo";
import { writeFile } from "node:fs/promises";

const client = new SoniloClient();
const task = await client.videoToMusic.submit({
  video: "./my_video.mp4",
  prompt: "upbeat, energetic",
  preserveSpeech: true,
});
const result = await client.tasks.wait<MusicTaskResult>(task.task_id);

// `audio` is always an array for async video-to-music (one entry per output
// stream); `vocals` and `mux` are only present when preserveSpeech is set.
await writeFile("mix.m4a", await download(result.audio[0]!));
await writeFile("vocals.m4a", await download(result.vocals!));
```

--------------------------------

### Duck music under speech with TypeScript

Source: https://github.com/sonilo-ai/sonilo-js/blob/main/packages/sonilo-video-kit/README.md

Use this function to automatically lower music volume during speech segments. The input video must be under 360 seconds and support stream-copying into the output container.

```ts
import { duckMusicUnderSpeech } from "sonilo-video-kit";

await duckMusicUnderSpeech({
  video: "./interview.mp4",
  audio: track.audio,
  output: "./interview.ducked.mp4",
});
```

--------------------------------

### Retrieve Account Services and Usage

Source: https://github.com/sonilo-ai/sonilo-js/blob/main/packages/sonilo/README.md

Fetches account service information and usage statistics for a specified number of days.

```ts
const services = await sonilo.account.services();
const usage = await sonilo.account.usage({ days: 7 });
```

--------------------------------

### Handle task cancellation in ducking operations

Source: https://github.com/sonilo-ai/sonilo-js/blob/main/packages/sonilo-video-kit/README.md

Demonstrates how to catch and identify user-initiated aborts during a ducking operation while acknowledging that the task may still be billed.

```ts
try {
  await duckMusicUnderSpeech({ video, audio, output, signal: controller.signal });
} catch (err) {
  if (err instanceof VideoKitError && (err.cause as Error | undefined)?.name === "AbortError") {
    return; // we aborted this ourselves
  }
  throw err; // note: the task is still running, and still billed — see the task id in the message
}
```

--------------------------------

### Generate mixed video sound

Source: https://github.com/sonilo-ai/sonilo-js/blob/main/packages/sonilo/README.md

Generate a combined music bed and sound effects track for a video clip in a single request.

```ts
import { SoniloClient, download } from "sonilo";
import { writeFile } from "node:fs/promises";

const client = new SoniloClient();

const result = await client.videoToSound.generate({
  videoUrl: "https://example.com/clip.mp4",
  musicPrompt: "uplifting orchestral score",
  sfxPrompt: "match the on-screen action",
});

await writeFile("soundtrack.wav", await download(result.output_url));
```

--------------------------------

### Stream Music Generation

Source: https://github.com/sonilo-ai/sonilo-js/blob/main/packages/sonilo/README.md

Consume audio chunks as they are generated using an asynchronous iterator.

```ts
import { SoniloClient, isAudioChunkEvent } from "sonilo";

for await (const event of sonilo.textToMusic.stream({ prompt: "lofi", duration: 30 })) {
  if (isAudioChunkEvent(event)) {
    // event.data is a Uint8Array — feed it to your player as it arrives
  }
}
```

--------------------------------

### SoniloClient.tasks.wait

Source: https://github.com/sonilo-ai/sonilo-js/blob/main/packages/sonilo/README.md

Polls for the completion of an asynchronous task.

```APIDOC
## SoniloClient.tasks.wait

### Description
Waits for an asynchronous task to complete and returns the result.

### Parameters
- **task_id** (string) - Required - The ID of the task to wait for.
```

--------------------------------

### client.tasks.wait

Source: https://github.com/sonilo-ai/sonilo-js/blob/main/packages/sonilo/README.md

Polls for the completion of an asynchronous task.

```APIDOC
## client.tasks.wait(taskId, options)

### Description
Waits for a task to complete by polling the server at specified intervals.

### Parameters
- **taskId** (string) - Required - The ID of the task to wait for.
- **pollInterval** (number) - Optional - Time in milliseconds between polls.
- **timeout** (number) - Optional - Maximum time in milliseconds to wait before throwing a TaskTimeoutError.
```

--------------------------------

### Manually control task submission and polling

Source: https://github.com/sonilo-ai/sonilo-js/blob/main/packages/sonilo/README.md

Submits a task using submit() and manually polls for completion using tasks.wait(). Useful for custom polling intervals or timeout configurations.

```ts
const task = await client.videoToSfx.submit({
  video: "clip.mp4", // Node.js path; pass File/Blob in the browser
  segments: [{ start: 0, end: 2.5, prompt: "footsteps on gravel" }],
  audioFormat: "wav",
});
const result = await client.tasks.wait(task.task_id, { pollInterval: 2000, timeout: 600000 });
```

--------------------------------

### duckMusicUnderSpeech

Source: https://github.com/sonilo-ai/sonilo-js/blob/main/packages/sonilo-video-kit/README.md

Automatically ducks music under speech in a video file using the Sonilo API. This function is billed based on the duration of the video's picture.

```APIDOC
## duckMusicUnderSpeech

### Description
Processes a video file to duck music under speech. The function uploads the extracted audio, processes it on the server, and muxes the result back with the original video picture.

### Signature
`await duckMusicUnderSpeech({ video, audio, output })`

### Parameters
- **video** (string) - Required - Path to the source video file.
- **audio** (string) - Required - Path to the audio track to be ducked.
- **output** (string) - Required - Path where the final ducked video will be saved.

### Requirements
- Video must contain an audio track and a picture.
- Maximum duration: 360 seconds.
- Output path must be writable and have a valid container extension (e.g., .mp4).
- Picture must be stream-copyable into the output container.

### Error Handling
- If the API call succeeds but local post-processing fails, the ducked audio is saved as `<output>.ducked.wav` to prevent loss of paid work.
- If a terminal failure occurs after submission, the task ID is provided to allow manual polling via `GET /v1/tasks/<task_id>`.
```
Tokens:
Raw