### Install @mediabunny/server Source: https://mediabunny.dev/guide/extensions/server Install both mediabunny and @mediabunny/server using npm. This library peer-depends on mediabunny. ```bash npm install mediabunny @mediabunny/server ``` -------------------------------- ### Install mediabunny and flac-encoder Source: https://mediabunny.dev/guide/extensions/flac-encoder Install the mediabunny library and the flac-encoder extension using npm. ```bash npm install mediabunny @mediabunny/flac-encoder ``` -------------------------------- ### Install Mediabunny with npm Source: https://mediabunny.dev/guide/installation Use npm to install the Mediabunny package. This is the recommended method for most projects. ```bash npm install mediabunny ``` -------------------------------- ### Install mediabunny and @mediabunny/prores Source: https://mediabunny.dev/guide/extensions/prores Install the Mediabunny core library and the ProRes extension using npm. This is the recommended method for integrating both packages into your project. ```bash npm install mediabunny @mediabunny/prores ``` -------------------------------- ### Initialize HLS Input and Get Tracks Source: https://mediabunny.dev/guide/quick-start Set up an Input object for HLS sources and retrieve all available tracks. This is the starting point for processing HLS content. ```typescript import { Input, UrlSource, HLS_FORMATS, desc } from 'mediabunny'; const input = new Input({ source: new UrlSource('https://example.com/master.m3u8'), formats: HLS_FORMATS, }); // Get all tracks const tracks = await input.getTracks(); ``` -------------------------------- ### Install Mediabunny with Yarn Source: https://mediabunny.dev/guide/installation Use Yarn to add the Mediabunny package to your project. Ensure Yarn is installed and configured. ```bash yarn add mediabunny ``` -------------------------------- ### Microphone Recording with NodeAV Source: https://mediabunny.dev/guide/extensions/server Example demonstrating how to record audio from the user's microphone using NodeAV's Device API and process it into Mediabunny AudioSamples, with timestamps offset to start at 0. ```typescript import { AudioSample } from 'mediabunny'; import { AvFrameAudioSampleResource } from '@mediabunny/server'; import { DeviceAPI, Decoder } from 'node-av'; await using mic = await DeviceAPI.openMicrophone(); const audioStream = mic.audio()!; using decoder = await Decoder.create(audioStream); let firstTimestamp: number | null = null; for await (const frame of decoder.frames(mic.packets(audioStream.index))) { if (!frame) { break; } const sample = new AudioSample(new AvFrameAudioSampleResource(frame)); if (firstTimestamp === null) { firstTimestamp = sample.timestamp; } // Offset timestamps so they start at 0 sample.setTimestamp(sample.timestamp - firstTimestamp); // Do something with the sample now, like passing it to an AudioSampleSource // ... } ``` -------------------------------- ### Install mediabunny and mp3-encoder Source: https://mediabunny.dev/guide/extensions/mp3-encoder Install the necessary packages using npm. This library peer-depends on Mediabunny. ```bash npm install mediabunny @mediabunny/mp3-encoder ``` -------------------------------- ### Example Video Decoder Configuration Source: https://mediabunny.dev/guide/reading-media-files An example of a VideoDecoderConfig for a 1080p Big Buck Bunny video, including codec, dimensions, and decoder configuration record bytes. ```json { "codec": "avc1.4d4029", "codedWidth": 1920, "codedHeight": 1080, "description": new Uint8Array([ // Bytes of the AVCDecoderConfigurationRecord 1, 77, 64, 41, 255, 225, 0, 22, 39, 77, 64, 41, 169, 24, 15, 0, 68, 252, 184, 3, 80, 16, 16, 27, 108, 43, 94, 247, 192, 64, 1, 0, 4, 40, 222, 9, 200, ]) } ``` -------------------------------- ### Retrieve the starting timestamp of media Source: https://mediabunny.dev/guide/reading-media-files Get the initial timestamp in seconds for the media file. This is useful as not all media begins at time zero. ```typescript await input.getFirstTimestamp(); // => 0.0 ``` -------------------------------- ### Install @mediabunny/ac3 with npm Source: https://mediabunny.dev/guide/extensions/ac3 Install the mediabunny core library and the ac3 extension package using npm. This library peer-depends on mediabunny. ```bash npm install mediabunny @mediabunny/ac3 ``` -------------------------------- ### Initialize Conversion with Video Resizing Source: https://mediabunny.dev/guide/converting-media-files Example of initializing a media conversion, specifying video dimensions and a fitting strategy to resize the video track to 720p. ```typescript const conversion = await Conversion.init({ input, output, video: { width: 1280, height: 720, fit: 'contain', }, }); ``` -------------------------------- ### Calculate live playback start time Source: https://mediabunny.dev/guide/reading-hls This code calculates a suitable start time for live playback by subtracting a buffer (defined by `fac` multiplied by the `refreshInterval`) from the current duration. This ensures playback can start without interruptions. ```typescript const currentDuration = await track.getDurationFromMetadata({ skipLiveWait: true, }); const refreshInterval = await track.getLiveRefreshInterval(); const fac = 2; const playbackStartTime = currentDuration! - fac * refreshInterval!; ``` -------------------------------- ### Install mediabunny and aac-encoder Source: https://mediabunny.dev/guide/extensions/aac-encoder Install both mediabunny and the aac-encoder package using npm. Alternatively, include them via script tags. ```bash npm install mediabunny @mediabunny/aac-encoder ``` ```html ``` -------------------------------- ### Install Mediabunny with pnpm Source: https://mediabunny.dev/guide/installation Use pnpm to install the Mediabunny package. pnpm is a fast, disk-space-efficient package manager. ```bash pnpm add mediabunny ``` -------------------------------- ### Start Output Writing Process Source: https://mediabunny.dev/guide/writing-media-files Initiates the media writing process and prepares the output to receive data. This call also prevents further track additions. ```typescript await output.start(); // Resolves once the output is ready to receive media data ``` -------------------------------- ### Install Mediabunny with Bun Source: https://mediabunny.dev/guide/installation Use Bun to add the Mediabunny package. Bun is a new JavaScript runtime, bundler, transpiler, and package manager. ```bash bun add mediabunny ``` -------------------------------- ### Example Audio Decoder Configuration Source: https://mediabunny.dev/guide/reading-media-files An example of an AudioDecoderConfig for an AAC audio track, including codec, channel count, sample rate, and audio specific configuration bytes. ```json { "codec": "mp4a.40.2", "numberOfChannels": 2, "sampleRate": 44100, "description": new Uint8Array([ // Bytes of the AudioSpecificConfig 17, 144, ]) } ``` -------------------------------- ### Creating Media Sources for Tracks Source: https://mediabunny.dev/guide/writing-media-files Example demonstrating how to create a video source from a canvas element and an audio source from a media stream, specifying codecs and bitrates. ```typescript import { CanvasSource, MediaStreamAudioTrackSource } from 'mediabunny'; // Assuming `canvasElement` exists const videoSource = new CanvasSource(canvasElement, { codec: 'avc', bitrate: 1e6, // 1 Mbps }); const stream = await navigator.mediaDevices.getUserMedia({ audio: true }); const audioStreamTrack = stream.getAudioTracks()[0]; const audioSource = new MediaStreamAudioTrackSource(audioStreamTrack, { codec: 'aac', bitrate: 128e3, // 128 kbps }); output.addVideoTrack(videoSource, { frameRate: 30 }); output.addAudioTrack(audioSource); ``` -------------------------------- ### Create VideoSampleSink Source: https://mediabunny.dev/guide/media-sinks Instantiate VideoSampleSink with a video track. Optionally configure the decoder, for example, to prefer software acceleration. ```typescript import { VideoSampleSink } from 'mediabunny'; const sink = new VideoSampleSink(videoTrack); ``` ```typescript const sink = new VideoSampleSink(videoTrack, { hardwareAcceleration: 'prefer-software', }); ``` -------------------------------- ### Trim Media File with Negative Start Offset Source: https://mediabunny.dev/guide/converting-media-files Use negative values for `start` to introduce a period of no media data (freeze frame or silence) at the beginning of the output file. ```typescript const conversion = await Conversion.init({ // ... trim: { start: -2, // Two seconds of no media data (freeze frame / silence) at the start }, // ... }); ``` -------------------------------- ### First Timestamp Source: https://mediabunny.dev/guide/reading-media-files Retrieve the track's start timestamp in seconds. This represents the start time of the first sample and can be used in conjunction with duration to understand the track's temporal bounds. ```APIDOC ## First Timestamp ### Description Retrieve the track's start timestamp in seconds. ### Method `track.getFirstTimestamp()` ### Response Example ```json { "example": "0.041666666666666664" } ``` ``` -------------------------------- ### Register Mediabunny Server Functionality Source: https://mediabunny.dev/guide/extensions/server Import and call registerMediabunnyServer() to enable the full Mediabunny feature set on the server. No further setup is required for basic usage. ```typescript import { registerMediabunnyServer } from '@mediabunny/server'; registerMediabunnyServer(); ``` -------------------------------- ### Calculate Average Loudness with AudioSampleSink Source: https://mediabunny.dev/guide/media-sinks Example of using AudioSampleSink to iterate over audio samples and calculate the average loudness using root mean square. ```typescript const sink = new AudioSampleSink(audioTrack); let sumOfSquares = 0; let totalSampleCount = 0; for await (const sample of sink.samples()) { const bytesNeeded = sample.allocationSize({ format: 'f32', planeIndex: 0 }); const floats = new Float32Array(bytesNeeded / 4); sample.copyTo(floats, { format: 'f32', planeIndex: 0 }); for (let i = 0; i < floats.length; i++) { sumOfSquares += floats[i] ** 2; } totalSampleCount += floats.length; } const averageLoudness = Math.sqrt(sumOfSquares / totalSampleCount); ``` -------------------------------- ### HLS Playlist with PROGRAM-DATE-TIME Source: https://mediabunny.dev/guide/writing-hls Example of an HLS media playlist generated with `#EXT-X-PROGRAM-DATE-TIME` tags, showing the mapping of segment times to specific dates and times. ```m3u8 #EXTM3U #EXT-X-VERSION:3 #EXT-X-PLAYLIST-TYPE:VOD #EXT-X-TARGETDURATION:2 #EXT-X-INDEPENDENT-SEGMENTS #EXTINF:2, #EXT-X-PROGRAM-DATE-TIME:2024-01-01T00:00:00.000Z segments-1-1.ts #EXTINF:2, #EXT-X-PROGRAM-DATE-TIME:2024-01-01T00:00:02.000Z segments-1-2.ts ... ``` -------------------------------- ### Unix Timestamped Media Example Source: https://mediabunny.dev/guide/reading-hls Demonstrates accessing timestamps from HLS media playlists that include Unix timestamps. Mediabunny shifts the timeline to use Unix timestamps. ```m3u8 #EXTM3U #EXT-X-TARGETDURATION:10 #EXTINF:10, #EXT-X-PROGRAM-DATE-TIME:2024-01-01T00:00:00Z segment-1.ts #EXTINF:10, #EXT-X-PROGRAM-DATE-TIME:2024-01-01T00:00:10Z segment-2.ts #EXT-X-ENDLIST ``` ```typescript await track.getFirstTimestamp(); // => 1704067200 (Unix timestamp for 2024-01-01T00:00:00Z) await track.computeDuration(); // => 1704067220 (Unix timestamp for 2024-01-01T00:00:20Z) const sink = new EncodedPacketSink(track); (await sink.getPacket(1704067210))!.timestamp; // => 1704067210 (Unix timestamp for 2024-01-01T00:00:10Z) ``` -------------------------------- ### Get Track First Timestamp Source: https://mediabunny.dev/guide/reading-media-files Retrieves the start timestamp of the first sample in seconds. This value can be positive, negative, or zero, indicating when the track's samples begin relative to the composition's start time. ```typescript await track.getFirstTimestamp(); // => 0.041666666666666664 ``` -------------------------------- ### Initialize and Execute a Media Conversion Source: https://mediabunny.dev/guide/converting-media-files Use `Conversion.init` to set up the conversion with input and output configurations, then `execute` to perform the conversion. Always check `isValid` before executing, as it indicates if any tracks were discarded. ```typescript import { Input, Output, WebMOutputFormat, BufferTarget, Conversion, } from 'mediabunny'; const input = new Input({ ... }); const output = new Output({ format: new WebMOutputFormat(), target: new BufferTarget(), }); const conversion = await Conversion.init({ input, output }); if (!conversion.isValid) { // Conversion is invalid and cannot be executed without error. // This field gives reasons for why tracks were discarded: conversion.discardedTracks; // => DiscardedTrack[] return; } await conversion.execute(); // output.target.buffer contains the final file ``` -------------------------------- ### VideoSampleSink Initialization Source: https://mediabunny.dev/guide/media-sinks Demonstrates how to create a VideoSampleSink, optionally configuring hardware acceleration. ```APIDOC ## VideoSampleSink Initialization ### Description Instantiate a `VideoSampleSink` to extract decoded video samples. You can optionally specify hardware acceleration preferences. ### Method `new VideoSampleSink(videoTrack, options?)` ### Parameters #### Path Parameters - **videoTrack** (InputVideoTrack) - Required - The video track to process. - **options** (object) - Optional - Configuration for the sink. - **hardwareAcceleration** (string) - Optional - Specifies hardware acceleration preference ('prefer-software', 'prefer-hardware', 'no-preference'). ### Request Example ```ts import { VideoSampleSink } from 'mediabunny'; const sink = new VideoSampleSink(videoTrack); // With options: const sinkWithOptions = new VideoSampleSink(videoTrack, { hardwareAcceleration: 'prefer-software' }); ``` ``` -------------------------------- ### AudioSample Constructor Source: https://mediabunny.dev/guide/packets-and-samples Demonstrates various ways to create an AudioSample instance, including from AudioData, raw data, and AudioBuffer. ```APIDOC ## `AudioSample` Constructor ### Description Audio samples can be constructed either from an `AudioData` instance, an initialization object, or an `AudioBuffer`. ### Usage ```ts import { AudioSample } from 'mediabunny'; // From AudioData: const sample = new AudioSample(audioData); // From raw data: const sample = new AudioSample({ data: new Float32Array([...]), format: 'f32-planar', // Audio sample format numberOfChannels: 2, sampleRate: 44100, // in Hz timestamp: 0, // in seconds }); // From AudioBuffer: const timestamp = 0; // in seconds const samples = AudioSample.fromAudioBuffer(audioBuffer, timestamp); // => Returns multiple AudioSamples if the AudioBuffer is very long ``` ### Supported Formats * `'u8'` * `'u8-planar'` * `'s16'` * `'s16-planar'` * `'s32'` * `'s32-planar'` * `'f32'` * `'f32-planar'` ### Custom Resource Constructor For advanced use cases, you can back an `AudioSample` with your own implementation of [`AudioSampleResource`](../api/AudioSampleResource). ```ts import { AudioSample, AudioSampleResource } from 'mediabunny'; class MyResource extends AudioSampleResource { // Implement getFormat(), getSampleRate(), getNumberOfFrames(), // getNumberOfChannels(), getTimestamp(), getDataPlane(), and close(). } const sample = new AudioSample(new MyResource()); ``` ``` -------------------------------- ### Trim Media File from Start Time to End Source: https://mediabunny.dev/guide/converting-media-files If only `start` is provided, the clip will run until the end of the input file. Ensure timestamps are not shifted by using `trim: { start: 0 }` if the input has an offset. ```typescript const conversion = await Conversion.init({ // ... trim: { start: 10, }, // ... }); ``` -------------------------------- ### VideoSample Constructors Source: https://mediabunny.dev/guide/packets-and-samples Demonstrates how to create VideoSample instances using different constructors: from an image source, from raw pixel data, or from a custom resource. ```APIDOC ## Image source constructor This constructor creates a `VideoSample` from a `CanvasImageSource`: ```ts import { VideoSample } from 'mediabunny'; // Creates a sample from a canvas element const sample = new VideoSample(canvas, { timestamp: 3, // in seconds duration: 1/24, // in seconds }); // Creates a sample from an image element, with some added rotation const sample = new VideoSample(imageElement, { timestamp: 5, // in seconds rotation: 90, // in degrees clockwise }); // Creates a sample from a VideoFrame (timestamp will be copied) const sample = new VideoSample(videoFrame); ``` ## Raw constructor This constructor creates a `VideoSample` from raw pixel data given in an `ArrayBuffer`: ```ts import { VideoSample } from 'mediabunny'; // Creates a sample from pixel data in the RGBX format const sample = new VideoSample(buffer, { format: 'RGBX', codedWidth: 1280, codedHeight: 720, timestamp: 0, }); // Creates a sample from pixel data in the YUV 4:2:0 format const sample = new VideoSample(buffer, { format: 'I420', codedWidth: 1280, codedHeight: 720, timestamp: 0, }); ``` ## Custom resource constructor For advanced use cases (custom decoders, GPU-backed frames, etc.), you can back a `VideoSample` with your own implementation of [`VideoSampleResource`](../api/VideoSampleResource): ```ts import { VideoSample, VideoSampleResource } from 'mediabunny'; class MyResource extends VideoSampleResource { // Implement getFormat(), getCodedWidth(), getCodedHeight(), // getSquarePixelWidth(), getSquarePixelHeight(), getColorSpace(), // getDataPlanes(), toRgbSample(), and close(). } const sample = new VideoSample(new MyResource(), { timestamp: 0, }); ``` ``` -------------------------------- ### Create Input with All Formats Source: https://mediabunny.dev/guide/input-formats Instantiate a new Input object, specifying ALL_FORMATS to support the widest range of media types. Be aware this can significantly increase bundle size due to demuxer inclusion. ```typescript import { Input, ALL_FORMATS } from 'mediabunny'; const input = new Input({ formats: ALL_FORMATS, // ... }); ``` -------------------------------- ### Create a RangedSource with slice Source: https://mediabunny.dev/guide/reading-media-files Derive a `RangedSource` from an existing source to represent a sub-section. `slice(start, end)` or `slice(start)` can be used. ```typescript // A source over only the first 1024 bytes: const sliced = source.slice(0, 1024); ``` ```typescript // A source that starts 8192 bytes into the original source: const sliced2 = source.slice(8192); ``` -------------------------------- ### Combine Codec Support with Output Format Source: https://mediabunny.dev/guide/supported-formats-and-codecs This example demonstrates how to find the best video codec that is supported by both the encoder and a specific output format (MP4 in this case). It first retrieves supported codecs from the output format and then finds the first encodable one. ```typescript import { Mp4OutputFormat, getFirstEncodableVideoCodec, } from 'mediabunny'; const outputFormat = new Mp4OutputFormat(); const containableVideoCodecs = outputFormat.getSupportedVideoCodecs(); const bestVideoCodec = await getFirstEncodableVideoCodec(containableVideoCodecs); ``` -------------------------------- ### Create an MP4 Output in Memory Source: https://mediabunny.dev/guide/writing-media-files Instantiate an `Output` object to begin creating an MP4 media file. This example configures the output to use the MP4 format and store the resulting file in memory using `BufferTarget`. ```typescript import { Output, Mp4OutputFormat, BufferTarget } from 'mediabunny'; // In this example, we'll be creating an MP4 file in memory: const output = new Output({ format: new Mp4OutputFormat(), target: new BufferTarget(), }); ``` -------------------------------- ### HLS Master Playlist Example 1 Source: https://mediabunny.dev/guide/reading-hls This is an example of an HLS master playlist where video and audio tracks are implicitly paired by their order or stream information. ```m3u8 #EXTM3U #EXT-X-STREAM-INF:CODECS="avc1.64001f,mp4a.40.2",RESOLUTION=1280x720 video-and-audio-1.m3u8 #EXT-X-STREAM-INF:CODECS="avc1.64001f,mp4a.40.2",RESOLUTION=640x480 video-and-audio-2.m3u8 ``` -------------------------------- ### Create QuickTime (.mov) Output Source: https://mediabunny.dev/guide/output-formats Use MovOutputFormat to create QuickTime files. Options are inherited from IsobmffOutputFormatOptions. ```typescript import { Output, MovOutputFormat } from 'mediabunny'; const output = new Output({ format: new MovOutputFormat(options), // ... }); ``` -------------------------------- ### Convert input file to FLAC format Source: https://mediabunny.dev/guide/extensions/flac-encoder Example demonstrating how to convert an input file to a FLAC file using Mediabunny and the registered FLAC encoder. It checks for native support first and then performs the conversion, outputting to a buffer. ```typescript import { Input, ALL_FORMATS, BlobSource, Output, BufferTarget, FlacOutputFormat, canEncodeAudio, Conversion, } from 'mediabunny'; import { registerFlacEncoder } from '@mediabunny/flac-encoder'; if (!(await canEncodeAudio('flac'))) { registerFlacEncoder(); } const input = new Input({ source: new BlobSource(file), // From a file picker, for example formats: ALL_FORMATS, }); const output = new Output({ format: new FlacOutputFormat(), target: new BufferTarget(), }); const conversion = await Conversion.init({ input, output, }); await conversion.execute(); output.target.buffer; // => ArrayBuffer containing the FLAC file ``` -------------------------------- ### Trim Media File by Start and End Times Source: https://mediabunny.dev/guide/converting-media-files Use the `trim` property to extract a specific section of the input file. The output duration will be the difference between `end` and `start` times. ```typescript const conversion = await Conversion.init({ input, output, trim: { start: 10, end: 25, }, }); ``` -------------------------------- ### Create Input with All Formats Source: https://mediabunny.dev/guide/reading-media-files Initialize an Input instance to read from a user-provided File object, supporting all media formats. The file is only read when data is requested. ```typescript import { Input, ALL_FORMATS, BlobSource } from 'mediabunny'; const input = new Input({ formats: ALL_FORMATS, source: new BlobSource(file), }); ``` -------------------------------- ### Get current duration of a live stream without waiting Source: https://mediabunny.dev/guide/reading-hls Use the `skipLiveWait: true` option to get the duration of the live stream up to the current point, without waiting for the stream to end. ```typescript // Get the duration up until the current point in the live stream const currentDuration = await track.getDurationFromMetadata({ skipLiveWait: true }); ``` -------------------------------- ### Configure WAVE Output Source: https://mediabunny.dev/guide/output-formats Use WavOutputFormat to create WAVE (.wav) files. Options control RF64 file creation for large files, metadata format (RIFF INFO LIST or ID3), and a callback for header writing. ```typescript import { Output, WavOutputFormat } from 'mediabunny'; const output = new Output({ format: new WavOutputFormat(options), // ... }); ``` ```typescript type WavOutputFormatOptions = { large?: boolean; metadataFormat?: 'info' | 'id3'; onHeader?: (data: Uint8Array, position: number) => unknown; }; ``` -------------------------------- ### Create Input with Specific Formats Source: https://mediabunny.dev/guide/reading-media-files Initialize an Input instance to read from a specific source, supporting only MP3 and WAVE formats to optimize bundle size. This reduces the included parsers. ```typescript import { Input, MP3, WAVE } from 'mediabunny'; const input = new Input({ formats: [MP3, WAVE], // .... }); ``` -------------------------------- ### Single Sample Retrieval Source: https://mediabunny.dev/guide/media-sinks Shows how to get a specific video sample by its timestamp. ```APIDOC ## Single Sample Retrieval ### Description Retrieve the video sample that was presented at or before a given timestamp. Returns `null` if no such sample exists. ### Method `getSample(timestamp)` ### Parameters #### Path Parameters - **timestamp** (number) - Required - The target timestamp in seconds. Use `Infinity` to get the last sample. ### Request Example ```ts // Get sample at 5 seconds const sampleAt5 = await sink.getSample(5); // Get the first sample const firstSample = await sink.getSample(await videoTrack.getFirstTimestamp()); // Get the last sample const lastSample = await sink.getSample(Infinity); ``` ### Response #### Success Response - **VideoSample | null** - The video sample at the specified timestamp, or null. ``` -------------------------------- ### Get Decoder Configuration Source: https://mediabunny.dev/guide/reading-hls Obtain the decoder configuration for a track. This operation requires the first segment. ```typescript const decoderConfig = await track.getDecoderConfig(); ``` -------------------------------- ### Get Duration from Track Metadata Source: https://mediabunny.dev/guide/reading-hls Retrieve the duration from track metadata. This operation only loads the media playlist. ```typescript const duration = await track.getDurationFromMetadata(); ``` -------------------------------- ### Register Mediabunny Server with Hardware Context Source: https://mediabunny.dev/guide/extensions/server Optionally, configure server-side Mediabunny with specific hardware rendering options by passing an options object to registerMediabunnyServer. This example shows how to specify a VAAPI hardware context. ```typescript import { registerMediabunnyServer } from '@mediabunny/server'; import * as NodeAv from 'node-av'; registerMediabunnyServer({ // Use a specific hardware rendering device: hardwareContext: NodeAv.HardwareContext.create( NodeAv.AV_HWDEVICE_TYPE_VAAPI, '/dev/dri/renderD128', ), }); ``` -------------------------------- ### Get Lists of Decodable Codecs Source: https://mediabunny.dev/guide/supported-formats-and-codecs Retrieve lists of codecs that can be decoded, optionally filtered by specific configurations. ```APIDOC ## Get Lists of Decodable Codecs ### `getDecodableCodecs()` Gets a list of all codecs that can be decoded. - **Returns** - `Promise` - An array of supported media codecs. ### `getDecodableVideoCodecs(codecs?: string[], config?: VideoDecoderConfig)` Gets a list of decodable video codecs, optionally filtered by a list of codecs and specific configurations. - **Parameters** - `codecs` (string[], optional) - An array of video codec strings to check (e.g., `['avc', 'hevc', 'vp8']`). - `config` (VideoDecoderConfig, optional) - Configuration for video decoding (e.g., `{ codedWidth: 1920, codedHeight: 1080 }`). - **Returns** - `Promise` - An array of supported video codecs. ### `getDecodableAudioCodecs(codecs?: string[], config?: AudioDecoderConfig)` Gets a list of decodable audio codecs, optionally filtered by a list of codecs and specific configurations. - **Parameters** - `codecs` (string[], optional) - An array of audio codec strings to check (e.g., `['aac', 'opus']`). - `config` (AudioDecoderConfig, optional) - Configuration for audio decoding (e.g., `{ numberOfChannels: 2, sampleRate: 48000 }`). - **Returns** - `Promise` - An array of supported audio codecs. ``` -------------------------------- ### Initialize MP4 Output Source: https://mediabunny.dev/guide/output-formats Use this to create an MP4 output format. Ensure the Mp4OutputFormat class is imported. ```typescript import { Output, Mp4OutputFormat } from 'mediabunny'; const output = new Output({ format: new Mp4OutputFormat(options), // ... }); ``` -------------------------------- ### Trim Media File from Beginning to End Time Source: https://mediabunny.dev/guide/converting-media-files If only `end` is provided, the clip will start from the beginning of the input file. ```typescript const conversion = await Conversion.init({ // ... trim: { end: 25, }, // ... }); ``` -------------------------------- ### MP4 Output Format Options Source: https://mediabunny.dev/guide/output-formats Defines the available options for configuring MP4 output. Pay attention to the `fastStart` options for streaming and metadata placement. ```typescript type IsobmffOutputFormatOptions = { fastStart?: false | 'in-memory' | 'reserve' | 'fragmented'; minimumFragmentDuration?: number; metadataFormat?: 'mdir' | 'mdta' | 'udta' | 'auto'; onFtyp?: (data: Uint8Array, position: number) => unknown; onMoov?: (data: Uint8Array, position: number) => unknown; onMdat?: (data: Uint8Array, position: number) => unknown; onMoof?: (data: Uint8Array, position: number, timestamp: number) => unknown; }; ``` -------------------------------- ### Retrieving Next Packets Source: https://mediabunny.dev/guide/media-sinks Functions to get the subsequent packet in decode order, optionally filtering for key packets. ```APIDOC ## Retrieving Next Packets ### Description Retrieves the successor packet in decode order relative to a given packet. It can also retrieve the next key packet. ### Method `getNextPacket(packet: EncodedPacket, options?: { metadataOnly?: boolean, verifyKeyPackets?: boolean }): Promise` `getNextKeyPacket(packet: EncodedPacket, options?: { metadataOnly?: boolean, verifyKeyPackets?: boolean }): Promise` ### Parameters #### Path Parameters - **packet** (EncodedPacket) - Required - The reference packet. - **options** (object) - Optional - Configuration options. - **metadataOnly** (boolean) - If true, only metadata is retrieved. - **verifyKeyPackets** (boolean) - If true, ensures the returned packet is a key packet. ### Response #### Success Response - **EncodedPacket | null** - The next packet or null if there is no successor. ### Request Example ```ts // Assuming 'packet' is an existing EncodedPacket await sink.getNextPacket(packet); // Get the next key packet await sink.getNextKeyPacket(packet); // Get the next packet with key packet verification await sink.getNextPacket(packet, { verifyKeyPackets: true }); ``` ``` -------------------------------- ### Retrieve Last Encoded Packet Source: https://mediabunny.dev/guide/media-sinks Get the last encoded packet in presentation order by using Infinity as the timestamp. ```typescript await sink.getPacket(Infinity); // => EncodedPacket | null ``` -------------------------------- ### Convert audio to MP3 using Mediabunny Source: https://mediabunny.dev/guide/extensions/mp3-encoder An example demonstrating how to convert an input audio file to MP3 format using Mediabunny. It includes conditional registration of the MP3 encoder and specifies the output format and target. ```typescript import { Input, ALL_FORMATS, BlobSource, Output, BufferTarget, Mp3OutputFormat, canEncodeAudio, Conversion, } from 'mediabunny'; import { registerMp3Encoder } from '@mediabunny/mp3-encoder'; if (!(await canEncodeAudio('mp3'))) { // Only register the custom encoder if there's no native support registerMp3Encoder(); } const input = new Input({ source: new BlobSource(file), // From a file picker, for example formats: ALL_FORMATS, }); const output = new Output({ format: new Mp3OutputFormat(), target: new BufferTarget(), }); const conversion = await Conversion.init({ input, output, }); await conversion.execute(); output.target.buffer; // => ArrayBuffer containing the MP3 file ``` -------------------------------- ### Get Lists of Encodable Codecs Source: https://mediabunny.dev/guide/supported-formats-and-codecs Functions to retrieve lists of codecs that the browser can encode, optionally filtered by configuration. ```APIDOC ## Get Lists of Encodable Codecs In addition, you can use the following functions to check encodability for multiple codecs at once, getting back a list of supported codecs: ```ts import { getEncodableCodecs, getEncodableVideoCodecs, getEncodableAudioCodecs, getEncodableSubtitleCodecs, } from 'mediabunny'; getEncodableCodecs(); // => Promise getEncodableVideoCodecs(); // => Promise getEncodableAudioCodecs(); // => Promise getEncodableSubtitleCodecs(); // => Promise // These functions also accept optional configuration options. // Here, we check which of AVC, HEVC and VP8 can be encoded at 1920x1080 @10Mbps: getEncodableVideoCodecs( ['avc', 'hevc', 'vp8'], { width: 1920, height: 1080, bitrate: 1e7 }, ); // => Promise ``` ``` -------------------------------- ### Get Input Format Name Source: https://mediabunny.dev/guide/input-formats Retrieve the full written name of an input format. This is useful for display purposes. ```typescript inputFormat.name; ``` -------------------------------- ### Configure FLAC Output Source: https://mediabunny.dev/guide/output-formats Use this to create FLAC (.flac) files. The `options` object can configure `appendOnly` for live streaming or `onFrame` to process each FLAC frame. ```typescript import { Output, FlacOutputFormat } from 'mediabunny'; const output = new Output({ format: new FlacOutputFormat(options), // ... }); ``` ```typescript type FlacOutputFormatOptions = { appendOnly?: boolean; onFrame?: (data: Uint8Array, position: number) => unknown; }; ``` -------------------------------- ### Iterate Over a Packet Range Source: https://mediabunny.dev/guide/media-sinks Use the packets() iterator to iterate over a specific range of packets, from a start packet up to (but excluding) an end packet. ```typescript const start = await sink.getPacket(5); const end = await sink.getPacket(10, { metadataOnly: true }); for await (const packet of sink.packets(start, end)) { // ... } ``` -------------------------------- ### Node.js Media Compression Server with Mediabunny Source: https://mediabunny.dev/guide/extensions/server Set up a server to stream media uploads, process them with Mediabunny, and save the output to disk. This approach uses O(1) memory and handles backpressure automatically. ```typescript import { ALL_FORMATS, Conversion, FilePathTarget, Input, Mp4OutputFormat, Output, QUALITY_MEDIUM, ReadableStreamSource } from "mediabunny"; import { registerMediabunnyServer } from "@mediabunny/server"; import { Readable } from "node:stream"; import http from "node:http"; registerMediabunnyServer(); const server = http.createServer(async (req, res) => { // Read the request body as a stream const stream = Readable.toWeb(req) as ReadableStream; const input = new Input({ source: new ReadableStreamSource(stream), formats: ALL_FORMATS, }); // Stream the output directly to the disk, could also stream to S3 etc. const output = new Output({ format: new Mp4OutputFormat(), target: new FilePathTarget(`./converted-${crypto.randomUUID()}.mp4`), }); try { const conversion = await Conversion.init({ input, output, video: async track => ({ codec: 'avc', height: Math.min(720, await track.getDisplayHeight()), bitrate: QUALITY_MEDIUM, }), }); await conversion.execute(); res.statusCode = 204; res.end(); } catch (error) { res.statusCode = 500; res.end(); console.error("Error processing media:", error); } }); server.listen(3000); ``` -------------------------------- ### Inspect VideoSample Properties Source: https://mediabunny.dev/guide/packets-and-samples Access read-only properties of a VideoSample to get information about its format, dimensions, rotation, timing, and color space. ```typescript // The internal pixel format in which the frame is stored videoSample.format; // => VideoPixelFormat | null // Raw dimensions of the sample videoSample.codedWidth; // => number videoSample.codedHeight; // => number // Pixel aspect ratio-corrected dimensions of the sample videoSample.squarePixelWidth; videoSample.squarePixelHeight; // Transformed display dimensions of the sample (after rotation) videoSample.displayWidth; // => number videoSample.displayHeight; // => number // Rotation of the sample in degrees clockwise. The raw sample should be // rotated by this amount when it is presented. videoSample.rotation; // => 0 | 90 | 180 | 270 // The sample's pixel aspect ratio videoSample.pixelAspectRatio; // => { num: number, den: number } // Timing information videoSample.timestamp; // => Presentation timestamp in seconds videoSample.duration; // => Duration in seconds videoSample.microsecondTimestamp; // => Presentation timestamp in microseconds videoSample.microsecondDuration; // => Duration in microseconds // Color space of the sample videoSample.colorSpace; // => VideoColorSpace videoSample.visibleRect; // Rectangle // Encode options used when this sample is passed to an encoder videoSample.encodeOptions; // => VideoEncoderEncodeOptions (defaults to {}) ``` -------------------------------- ### Create New Media File to Buffer Source: https://mediabunny.dev/guide/quick-start Use this snippet to create a new media file in memory. It demonstrates adding video and audio tracks, setting metadata, and starting/finalizing the output process. ```typescript import { Output, BufferTarget, Mp4OutputFormat, CanvasSource, AudioBufferSource, QUALITY_HIGH, } from 'mediabunny'; // An Output represents a new media file const output = new Output({ format: new Mp4OutputFormat(), // The format of the file target: new BufferTarget(), // Where to write the file (here, to memory) }); // Example: add a video track driven by a canvas const videoSource = new CanvasSource(canvas, { codec: 'avc', bitrate: QUALITY_HIGH, }); output.addVideoTrack(videoSource); // Example: add an audio track driven by AudioBuffers const audioSource = new AudioBufferSource({ codec: 'aac', bitrate: QUALITY_HIGH, }); output.addAudioTrack(audioSource); // Set some metadata tags output.setMetadataTags({ title: 'My Movie', artist: 'Me', }); await output.start(); // Add some video frames for (let frame = 0; ...) { await videoSource.add(frame / 30, 1 / 30); } // Add some audio data await audioSource.add(audioBuffer1); await audioSource.add(audioBuffer2); await output.finalize(); const buffer = output.target.buffer; // ArrayBuffer containing the final MP4 file ``` -------------------------------- ### Retrieve First Encoded Packets Source: https://mediabunny.dev/guide/media-sinks Get the very first encoded packet in decode order. Optionally, retrieve the first key frame. ```typescript await sink.getFirstPacket(); // => EncodedPacket | null ``` ```typescript // The first packet is typically a key frame, but this is not required. // This method returns the first key frame: await sink.getFirstKeyPacket(); // => EncodedPacket | null ``` -------------------------------- ### Get First Encodable Codec Source: https://mediabunny.dev/guide/supported-formats-and-codecs Functions to find the best (first supported) codec from a given list, optionally with specific configurations. ```APIDOC ## Get First Encodable Codec If you simply want to find the best codec that the browser can encode, you can use these functions, which return the first codec supported by the browser: ```ts import { getFirstEncodableVideoCodec, getFirstEncodableAudioCodec, getFirstEncodableSubtitleCodec, } from 'mediabunny'; getFirstEncodableVideoCodec(['avc', 'vp9', 'av1']); // => Promise getFirstEncodableAudioCodec(['opus', 'aac']); // => Promise getFirstEncodableVideoCodec( ['avc', 'hevc', 'vp8'], { width: 1920, height: 1080, bitrate: 1e7 }, ); // => Promise ``` If none of the listed codecs is supported, `null` is returned. These functions are especially useful in conjunction with an [output format](./output-formats) to retrieve the best codec that is supported both by the encoder as well as the container format: ```ts import { Mp4OutputFormat, getFirstEncodableVideoCodec, } from 'mediabunny'; const outputFormat = new Mp4OutputFormat(); const containableVideoCodecs = outputFormat.getSupportedVideoCodecs(); const bestVideoCodec = await getFirstEncodableVideoCodec(containableVideoCodecs); ``` ``` -------------------------------- ### Convert NodeAV Frames to Mediabunny Samples Source: https://mediabunny.dev/guide/extensions/server Demonstrates how to create Mediabunny VideoSample and AudioSample instances directly from NodeAV Frame objects, avoiding data copying. Also shows how to convert Mediabunny Samples back to NodeAV Frames. ```typescript import { VideoSample, AudioSample } from 'mediabunny'; import { AvFrameVideoSampleResource, AvFrameAudioSampleResource, toAvFrame } from '@mediabunny/server'; // Frame -> VideoSample new VideoSample(new AvFrameVideoSampleResource(frame), { timestamp }); // Frame -> AudioSample new AudioSample(new AvFrameAudioSampleResource(frame)); // (uses the timestamp in the frame) // VideoSample -> Frame await toAvFrame(videoSample, frame); // AudioSample -> Frame await toAvFrame(audioSample, frame); ``` -------------------------------- ### Extract Specific Track Types Source: https://mediabunny.dev/guide/reading-media-files Get lists of video or audio tracks from the input file. Useful for isolating specific media streams. ```typescript await input.getVideoTracks(); // => InputVideoTrack[] ``` ```typescript await input.getAudioTracks(); // => InputAudioTrack[] ``` -------------------------------- ### Create and Add Audio Samples Source: https://mediabunny.dev/guide/media-sources Use AudioSampleSource to encode audio samples and add them to an output. Ensure to close the audio sample if it's no longer needed. ```typescript import { AudioSampleSource } from 'mediabunny'; const sampleSource = new AudioSampleSource({ codec: 'aac', bitrate: 128e3, }); await sampleSource.add(audioSample); audioSample.close(); // If it's not needed anymore ``` -------------------------------- ### Get Video Sample Allocation Size Source: https://mediabunny.dev/guide/packets-and-samples Determine the required buffer size in bytes to hold the video sample's pixel data. ```typescript const bytesNeeded = videoSample.allocationSize(); // => number ``` -------------------------------- ### Initialize FilePathSource Source: https://mediabunny.dev/guide/reading-media-files Create a `FilePathSource` instance to read data directly from a local file path. This requires a server-side environment. ```typescript import { FilePathSource } from 'mediabunny'; const source = new FilePathSource('/home/david/Downloads/bigbuckbunny.mp4'); ``` -------------------------------- ### Read Video Frames and Audio Chunks Source: https://mediabunny.dev/guide/quick-start This snippet shows how to initialize an Input object, retrieve primary video and audio tracks, check their decodability, and extract specific samples or iterate over a range of samples. It includes examples for drawing video frames to a canvas and converting audio samples to AudioBuffers. ```typescript import { Input, ALL_FORMATS, BlobSource, VideoSampleSink, AudioSampleSink, } from 'mediabunny'; const input = new Input({ formats: ALL_FORMATS, source: new BlobSource(file), }); // Read video frames const videoTrack = await input.getPrimaryVideoTrack(); if (videoTrack) { const decodable = await videoTrack.canDecode(); if (decodable) { const sink = new VideoSampleSink(videoTrack); // Get the video frame at timestamp 5s const videoSample = await sink.getSample(5); videoSample.timestamp; // in seconds videoSample.duration; // in seconds // Draw the frame to a canvas videoSample.draw(ctx, 0, 0); // Loop over all frames in the first 30s of video for await (const sample of sink.samples(0, 30)) { // ... } } } // Read audio chunks const audioTrack = await input.getPrimaryAudioTrack(); if (audioTrack) { const decodable = await audioTrack.canDecode(); if (decodable) { const sink = new AudioSampleSink(audioTrack); // Get audio chunk at timestamp 5s; a short chunk of audio const audioSample = await sink.getSample(5); audioSample.timestamp; // in seconds audioSample.duration; // in seconds audioSample.numberOfFrames; // Convert to AudioBuffer for use with the Web Audio API const audioBuffer = audioSample.toAudioBuffer(); // Loop over all samples in the first 30s of audio for await (const sample of sink.samples(0, 30)) { // ... } } } ``` -------------------------------- ### Get the concrete format of a media file Source: https://mediabunny.dev/guide/reading-media-files Retrieve the specific input format of the media file, such as `Mp4InputFormat`. This helps in understanding the file's structure. ```typescript await input.getFormat(); // => Mp4InputFormat ``` -------------------------------- ### Draw Video Sample to Canvas with Fit Options Source: https://mediabunny.dev/guide/packets-and-samples Draws the video sample to fill a canvas, applying a specified fitting algorithm ('fill', 'contain', 'cover'). Rotation and cropping can also be configured. ```typescript drawWithFit( context: CanvasRenderingContext2D | OffscreenCanvasRenderingContext2D, options: { fit: 'fill' | 'contain' | 'cover'; rotation?: Rotation; // Overrides the sample's rotation crop?: CropRectangle; }, ): void; ``` -------------------------------- ### Retrieve Next Encoded Packet Source: https://mediabunny.dev/guide/media-sinks Get the successor packet in decode order relative to a given packet. Optionally, retrieve the next key packet. ```typescript await sink.getNextPacket(packet); // => EncodedPacket | null ``` ```typescript // Or jump straight to the next packet with type 'key': await sink.getNextKeyPacket(packet); // => EncodedPacket | null ``` -------------------------------- ### Get number of pairable tracks for Playlist #1 Source: https://mediabunny.dev/guide/reading-hls For Playlist #1, this code demonstrates that a video track can only be paired with one specific audio track. ```typescript const videoTrack = (await input.getVideoTracks())[0]; (await videoTrack.getPairableTracks()).length; // => 1 ``` -------------------------------- ### Initialize HLS Input with UrlSource Source: https://mediabunny.dev/guide/reading-hls Use UrlSource with HLS_FORMATS to initialize an Input for reading HLS playlists. This requires importing Input, UrlSource, and HLS_FORMATS from 'mediabunny'. ```typescript import { Input, UrlSource, HLS_FORMATS } from 'mediabunny'; const input = new Input({ source: new UrlSource('https://example.com/master.m3u8'), formats: HLS_FORMATS, // HLS_FORMATS includes HLS as well as the commonly-used segment formats }); ``` -------------------------------- ### Get All HLS Tracks Source: https://mediabunny.dev/guide/reading-hls Retrieves all tracks from an HLS input. Mediabunny automatically deduplicates tracks across variants and efficiently fetches only the master playlist. ```typescript const tracks = await input.getTracks(); ```