### Check decodability with canDecode() Source: https://github.com/vanilagy/mediabunny/blob/main/docs/guide/reading-media-files.md Returns boolean indicating whether the browser can decode the track. Takes custom decoders into account. ```ts await track.canDecode(); // => boolean ``` -------------------------------- ### AudioSampleSink with first sample Source: https://github.com/vanilagy/mediabunny/blob/main/dev/demux.html Creates an AudioSampleSink and iterates samples, logging the first one. The code is commented out. ```javascript /* const sink = new Mediabunny.AudioSampleSink(await input.getPrimaryAudioTrack()); for await (const sample of sink.samples()) { console.log(sample) break; } */ ``` -------------------------------- ### Example video decoder configuration Source: https://github.com/vanilagy/mediabunny/blob/main/docs/guide/reading-media-files.md Example VideoDecoderConfig for a 1080p Big Buck Bunny video, including the AVCDecoderConfigurationRecord bytes. ```ts { codec: 'avc1.4d4029', codedWidth: 1920, codedHeight: 1080, description: new Uint8Array([ // Bytes of the AVCDecoderConfigurationRecord 1, 77, 64, 41, 255, 225, 0, 22, 39, 77, 64, 41, 169, 24, 15, 0, 68, 252, 184, 3, 80, 16, 16, 27, 108, 43, 94, 247, 192, 64, 1, 0, 4, 40, 222, 9, 200, ]), } ``` -------------------------------- ### canDecode Source: https://github.com/vanilagy/mediabunny/blob/main/docs/guide/reading-media-files.md Checks whether the track's codec can be decoded by the user's browser, taking custom decoders into account. ```APIDOC ## canDecode ### Description Checks whether the track's codec can be decoded by the user's browser. ### Method async ### Endpoint track.canDecode() ### Parameters None ### Response #### Success Response - **boolean** - True if the codec can be decoded. ### Request Example ```ts await track.canDecode(); ``` ### Response Example ```ts // => boolean ``` ``` -------------------------------- ### getPrimaryAudioTrack Source: https://github.com/vanilagy/mediabunny/blob/main/docs/guide/reading-media-files.md Retrieves the primary audio track from the input file. Returns an InputAudioTrack or null if none exists. ```APIDOC ## getPrimaryAudioTrack ### Description Retrieves the primary audio track from the input file. ### Method async ### Endpoint input.getPrimaryAudioTrack() ### Parameters None ### Response #### Success Response - **InputAudioTrack | null** - The primary audio track, or null if none exists. ### Request Example ```ts await input.getPrimaryAudioTrack(); ``` ### Response Example ```ts // => InputAudioTrack | null ``` ``` -------------------------------- ### Query tracks with InputTrackQuery and helpers Source: https://github.com/vanilagy/mediabunny/blob/main/docs/guide/reading-media-files.md Uses optional InputTrackQuery parameter for filtering and sorting. Helpers asc, desc, prefer express sorting logic. Examples show sorting by resolution/bitrate, filtering by language, and preferring English tracks. ```ts import { desc, prefer } from 'mediabunny'; // Get the highest-resolution video track: await input.getPrimaryVideoTrack({ sortBy: async track => [ desc(await track.getDisplayWidth()), desc(await track.getBitrate()), // If resolution matches, prefer the highest bitrate ], }); // Get only English audio tracks: await input.getAudioTracks({ filter: async track => await track.getLanguageCode() === 'eng', }); // Get an English track but only if one exists: await input.getAudioTracks({ sortBy: async track => prefer(await track.getLanguageCode() === 'eng'), }); ``` -------------------------------- ### getTracks Source: https://github.com/vanilagy/mediabunny/blob/main/docs/guide/reading-media-files.md Retrieves the list of all media tracks in the input file. Returns an array of InputTrack objects. ```APIDOC ## getTracks ### Description Retrieves the list of all media tracks in the input file. ### Method async ### Endpoint input.getTracks() ### Parameters None ### Response #### Success Response - **InputTrack[]** - Array of all tracks in the input file. ### Request Example ```ts await input.getTracks(); ``` ### Response Example ```ts // => InputTrack[] ``` ``` -------------------------------- ### getAudioTracks Source: https://github.com/vanilagy/mediabunny/blob/main/docs/guide/reading-media-files.md Retrieves the list of all audio tracks in the input file. Returns an array of InputAudioTrack objects. ```APIDOC ## getAudioTracks ### Description Retrieves the list of all audio tracks in the input file. ### Method async ### Endpoint input.getAudioTracks() ### Parameters None ### Response #### Success Response - **InputAudioTrack[]** - Array of all audio tracks in the input file. ### Request Example ```ts await input.getAudioTracks(); ``` ### Response Example ```ts // => InputAudioTrack[] ``` ``` -------------------------------- ### Extract all tracks with getTracks() Source: https://github.com/vanilagy/mediabunny/blob/main/docs/guide/reading-media-files.md Returns an array of all media tracks in the input file. Use this to access the full track list. ```ts await input.getTracks(); // => InputTrack[] ``` -------------------------------- ### audioTrack.getSampleRate() Source: https://github.com/vanilagy/mediabunny/blob/main/docs/guide/reading-media-files.md Retrieves the audio sample rate in hertz for the audio track. This returns a Promise that resolves to a number. ```APIDOC ## audioTrack.getSampleRate() ### Description Retrieves the audio sample rate in hertz for the audio track. This returns a Promise that resolves to a number. ### Method GET ### Endpoint audioTrack.getSampleRate() ### Response #### Success Response - **result** (number) - The sample rate in hertz ### Request Example ```ts await audioTrack.getSampleRate(); // => number ``` ``` -------------------------------- ### InputTrackQuery Source: https://github.com/vanilagy/mediabunny/blob/main/docs/guide/reading-media-files.md Optional parameter for filtering and sorting tracks when retrieving them. Accepts filter and sortBy functions. ```APIDOC ## InputTrackQuery ### Description Optional parameter for filtering and sorting tracks when retrieving them. ### Parameters #### Query Parameters - **filter** (function) - Optional - Async function that returns a boolean indicating whether to include the track. - **sortBy** (function) - Optional - Async function that returns a sort key or array of sort keys for ordering. ### Example ```ts import { desc, prefer } from 'mediabunny'; // Get the highest-resolution video track: await input.getPrimaryVideoTrack({ sortBy: async track => [ desc(await track.getDisplayWidth()), desc(await track.getBitrate()), ], }); // Get only English audio tracks: await input.getAudioTracks({ filter: async track => await track.getLanguageCode() === 'eng', }); // Get an English track but only if one exists: await input.getAudioTracks({ sortBy: async track => prefer(await track.getLanguageCode() === 'eng'), }); ``` ``` -------------------------------- ### InputTrack Properties Source: https://github.com/vanilagy/mediabunny/blob/main/docs/guide/reading-media-files.md Common metadata properties available on InputTrack objects. ```APIDOC ## InputTrack Properties ### Description Common metadata properties available on InputTrack objects. ### Properties - **id** (number) - Unique ID for this track in the input file (1-based index). - **number** (number) - 1-based index among tracks of the same type. - **type** ('video' | 'audio' | 'subtitle') - Track type. ### Methods - **isVideoTrack()** (boolean) - Returns true if track is video. - **isAudioTrack()** (boolean) - Returns true if track is audio. - **getLanguageCode()** (async, string) - ISO 639-2/T language code, 'und' if unknown. - **getName()** (async, string | null) - User-defined name. - **getDisposition()** (async, TrackDisposition) - Intended usage information. - **getBitrate()** (async, number | null) - Peak bitrate if exposed. - **getAverageBitrate()** (async, number | null) - Average bitrate if exposed. - **hasOnlyKeyPackets()** (async, boolean) - Whether all packets are key packets. - **isLive()** (async, boolean) - Whether track is live. - **getLiveRefreshInterval()** (async, number | null) - Interval in seconds for live data. ### Example ```ts track.id; // => number track.number; // => number track.type; // => 'video' | 'audio' | 'subtitle' track.isVideoTrack(); // => boolean track.isAudioTrack(); // => boolean await track.getLanguageCode(); // => string await track.getName(); // => string | null await track.getDisposition(); // TrackDisposition await track.getBitrate(); // => number | null await track.getAverageBitrate(); // => number | null await track.hasOnlyKeyPackets(); // => boolean await track.isLive(); // => boolean await track.getLiveRefreshInterval(); // => number | null ``` ``` -------------------------------- ### UrlSource Source: https://github.com/vanilagy/mediabunny/blob/main/docs/guide/reading-media-files.md Creates an input source that fetches data from a remote URL, useful for reading media files over the network. It supports options for customizing requests, retry logic, caching, and fetch behavior. ```APIDOC ## UrlSource ### Description Creates an input source that fetches data from a remote URL, useful for reading media files over the network. It supports options for customizing requests, retry logic, caching, and fetch behavior. ### Constructor `new UrlSource(url: string | URL | Request, options?: UrlSourceOptions)` ### Parameters #### url (string | URL | Request) - Required The URL to fetch data from. #### options (UrlSourceOptions) - Optional - **requestInit** (RequestInit) - Optional - Custom request initialization, similar to the Fetch API. All fields are respected except `signal`, which is overridden by Mediabunny. - **getRetryDelay** ((previousAttempts: number, error: unknown, url: string | URL | Request) => number | null) - Optional - Controls retry logic. Should return the time to wait in seconds before retrying, or `null` to prevent further retries. Defaults to infinite exponential backoff capped at 16 seconds, with no retries on suspected CORS errors. - **maxCacheSize** (number) - Optional - Maximum number of bytes the cache is allowed to hold in memory. Defaults to 64 MiB. - **parallelism** (number) - Optional - Maximum number of parallel requests to use for fetching. Defaults to 2. - **fetchFn** (typeof fetch) - Optional - Custom fetch function, useful for polyfill reasons (e.g., React Native). - **handleUnhandledError** ((error: unknown) => unknown) - Optional - Handles errors that occur while no read is pending. ### Example ```ts import { UrlSource } from 'mediabunny'; const source = new UrlSource('https://example.com/bigbuckbunny.mp4'); ``` ### Example with options ```ts const source = new UrlSource('https://example.com/bigbuckbunny.mp4', { requestInit: { headers: { 'X-Custom-Header': 'my-value', }, }, }); ``` ### Example with retry logic ```ts const source = new UrlSource('https://example.com/bigbuckbunny.mp4', { getRetryDelay: (previousAttempts) => Math.min(2 ** previousAttempts, 16), }); ``` ### Example with custom fetch ```ts import { UrlSource } from 'mediabunny'; import { fetch } from 'expo/fetch'; const source = new UrlSource('https://example.com/bigbuckbunny.mp4', { fetchFn: (input, init) => { if (typeof input !== 'string') { // Expo requires string URLs throw new Error('Expected a string URL.'); } return fetch(input, init); }, }); ``` ### Notes - If using in the browser with a different origin, ensure CORS is properly configured. - The `signal` field in `requestInit` or a `Request` is overridden by Mediabunny. To cancel ongoing requests, dispose of the input. ``` -------------------------------- ### Mediabunny custom video player with canvas and audio Source: https://github.com/vanilagy/mediabunny/blob/main/dev/player.html Builds a video player that reads a file from an input, decodes video and audio tracks, renders video to a canvas, and plays audio via Web Audio. Handles play/pause, seeking, and time display. Requires a file input, canvas, and player UI elements in the HTML. ```javascript const fileInput = document.querySelector('input[type="file"]'); const currentTimeElement = document.querySelector('#currentTime'); const durationElement = document.querySelector('#duration'); const playerDiv = document.querySelector('#player'); const playButton = playerDiv.querySelector('button'); const rangeInput = playerDiv.querySelector('input[type="range"]'); const file = await new Promise(resolve => { fileInput.addEventListener('change', () => { resolve(fileInput.files[0]); }); }); fileInput.style.display = 'none'; const audioContext = new AudioContext(); const source = new Mediabunny.BlobSource(file); const input = new Mediabunny.Input({ formats: Mediabunny.ALL_FORMATS, source }); console.log(await input.getMimeType()); let videoTrack = await input.getPrimaryVideoTrack(); let audioTrack = await input.getPrimaryAudioTrack(); if (!(await videoTrack?.canDecode())) { videoTrack = null; } if (!(await audioTrack?.canDecode())) { audioTrack = null; } const canvas = document.querySelector('canvas'); const context = canvas.getContext('2d'); let videoRotation = 0; if (!videoTrack) { canvas.style.display = 'none'; } else { canvas.width = videoTrack.displayWidth; canvas.height = videoTrack.displayHeight; } const totalDuration = await input.computeDuration(); durationElement.textContent = formatSeconds(totalDuration); playerDiv.style.display = 'flex'; const videoSink = videoTrack && new Mediabunny.CanvasSink(videoTrack, { poolSize: 2 }); const audioSink = audioTrack && new Mediabunny.AudioBufferSink(audioTrack); let startTime = null; let playing = false; let playbackTimeAtStart = 0; let videoFrameIterator = null; let audioBufferIterator = null; let currentFrame = null; let nextFrame = null; let seeking = false; let seekId = 0; const queuedAudioNodes = new Set(); function getPlaybackTime() { if (playing) { return audioContext.currentTime - startTime + playbackTimeAtStart; } else { return playbackTimeAtStart; } } function play() { audioBufferIterator?.return(); audioBufferIterator = audioSink?.buffers(getPlaybackTime()); startTime = audioContext.currentTime; playing = true; runAudioIterator(); playButton.textContent = 'Pause'; } function pause() { playbackTimeAtStart = getPlaybackTime(); playing = false; audioBufferIterator?.return(); audioBufferIterator = null; for (const node of queuedAudioNodes) { node.stop(); } playButton.textContent = 'Play'; } function togglePlay() { if (seeking) { return; } if (playing) { pause(); } else { play(); } } async function seek() { if (!videoTrack) { return; } seeking = true; seekId++; await videoFrameIterator?.return(); videoFrameIterator = videoSink.canvases(getPlaybackTime()); const newCurrentFrame = (await videoFrameIterator.next()).value; const newNextFrame = (await videoFrameIterator.next()).value; currentFrame = newCurrentFrame; nextFrame = newNextFrame; if (currentFrame) { context.drawImage(currentFrame.canvas, 0, 0); } seeking = false; } await seek(); async function render() { const playbackTime = getPlaybackTime(); if (playbackTime >= totalDuration) { pause(); playbackTimeAtStart = totalDuration; } if (currentFrame) { while (nextFrame && nextFrame.timestamp <= playbackTime && !seeking) { currentFrame = nextFrame; context.drawImage(currentFrame.canvas, 0, 0); const currentSeekId = seekId; const newNextFrame = (await videoFrameIterator.next()).value; if (currentSeekId === seekId) { nextFrame = newNextFrame; } } } if (!movingRangeInput) { rangeInput.value = playbackTime / totalDuration; currentTimeElement.textContent = formatSeconds(playbackTime); } requestAnimationFrame(render); } async function runAudioIterator() { if (!audioTrack) { return; } for await (let { buffer, timestamp } of audioBufferIterator) { const node = audioContext.createBufferSource(); node.buffer = buffer; node.connect(audioContext.destination); const startTimestamp = startTime + timestamp - playbackTimeAtStart; if (startTimestamp >= audioContext.currentTime) { node.start(startTimestamp); } else { node.start(audioContext.currentTime, audioContext.currentTime - startTimestamp); } queuedAudioNodes.add(node); node.onended = () => { queuedAudioNodes.delete(node); }; if (timestamp - getPlaybackTime() >= 1) { await new Promise(resolve => { const id = setInterval(() => { if (timestamp - getPlaybackTime() < 1) { clearInterval(id); resolve(); } }, 100); }); } } } function formatSeconds(seconds) { seconds = Math.round(seconds * 1000) / 1000; // Round to milliseconds const minutes = Math.floor(seconds / 60); const remainingSeconds = Math.floor(seconds % 60); return `${minutes}:${remainingSeconds.toString().padStart(2, '0')}.${Math.floor(1000 * seconds % 1000).toString().padStart(3, '0')}`; } playButton.addEventListener('click', togglePlay); window.addEventListener('keydown', (e) => { if (e.code === 'Space') { togglePlay(); e.preventDefault(); } }); let movingRangeInput = false; rangeInput.addEventListener('input', () => { movingRangeInput = true; const time = Number(rangeInput.value) * ``` -------------------------------- ### Output.state Source: https://github.com/vanilagy/mediabunny/blob/main/docs/guide/writing-media-files.md Property to check the current state of the output. Returns one of 'pending', 'started', 'canceled', 'finalizing', or 'finalized'. ```APIDOC ## Output.state ### Description Property to check the current state of the output. Returns one of 'pending', 'started', 'canceled', 'finalizing', or 'finalized'. ### Method Property access ### Endpoint output.state ### Response #### Success Response - **state** (string) - Current state: 'pending' | 'started' | 'canceled' | 'finalizing' | 'finalized' ### Request Example ```ts output.state; // => 'pending' | 'started' | 'canceled' | 'finalizing' | 'finalized' ``` ``` -------------------------------- ### Create Video with Canvas and Audio using MediaBunny Source: https://github.com/vanilagy/mediabunny/blob/main/dev/mux.html This snippet shows how to use MediaBunny to create a video. It draws random colored rectangles on a canvas, adds them as frames to a video source, optionally decodes an audio file and adds it to an audio source, then finalizes and downloads the output. Note that the audio source and subtitle source lines are commented out. ```javascript //subtitleSource.add(simpleWebvttFile); //subtitleSource.close(); const p = document.createElement('p'); document.body.append(p); for (let i = 0; i < 100; i++) { context.fillStyle = ['red', 'green', 'blue', 'yellow'][i % 4]; context.fillRect(canvas.width * Math.random(), canvas.height * Math.random(), canvas.width * Math.random(), canvas.height * Math.random()); await videoSource.add(i / 10 - 1, 1 / 10); p.textContent = i; } let audioContext = new AudioContext(); let audioBuffer = await audioContext.decodeAudioData(await (await fetch('./CantinaBand60.wav')).arrayBuffer()); let length = 10; let slicedAudioBuffer = sliceAudioBuffer(audioBuffer, length, audioContext); //await audioSource.add(slicedAudioBuffer); await output.finalize(); console.log(target); download(new Blob([target.buffer]), 'test' + format.fileExtension); ``` -------------------------------- ### Backpressure promise from mediaSource.add() Source: https://github.com/vanilagy/mediabunny/blob/main/docs/guide/media-sources.md All media sources with an add method return a promise that resolves when ready for more data. Awaiting it propagates backpressure into your application logic. ```typescript mediaSource.add(...); // => Promise ``` -------------------------------- ### Create new media files with mediabunny Source: https://github.com/vanilagy/mediabunny/blob/main/README.md Create an Output with Mp4OutputFormat and BufferTarget, add a video track from a canvas, and finalize to get the MP4 buffer. ```js import { Output, Mp4OutputFormat, BufferTarget, CanvasSource, Quality } from 'mediabunny'; const output = new Output({ format: new Mp4OutputFormat(), target: new BufferTarget(), // Writing to memory }); // Add a video track backed by a canvas element const videoSource = new CanvasSource(canvas, { codec: 'avc', bitrate: new Quality('high'), }); output.addVideoTrack(videoSource); await output.start(); // Add frames... await output.finalize(); const buffer = output.target.buffer; // Final MP4 file ``` -------------------------------- ### Create UrlSource from URL Source: https://github.com/vanilagy/mediabunny/blob/main/docs/guide/reading-media-files.md Fetches data from a remote URL. Requires CORS configuration when used in the browser with a different origin. ```ts import { UrlSource } from 'mediabunny'; const source = new UrlSource('https://example.com/bigbuckbunny.mp4'); ``` -------------------------------- ### Read file metadata with mediabunny Source: https://github.com/vanilagy/mediabunny/blob/main/README.md Create an Input from a BlobSource, compute duration, get primary video/audio tracks, and retrieve metadata tags. ```js import { Input, ALL_FORMATS, BlobSource } from 'mediabunny'; // Reading from disk const input = new Input({ source: new BlobSource(file), formats: ALL_FORMATS, }); const duration = await input.computeDuration(); // in seconds const videoTrack = await input.getPrimaryVideoTrack(); const audioTrack = await input.getPrimaryAudioTrack(); const displayWidth = await videoTrack.getDisplayWidth(); const displayHeight = await videoTrack.getDisplayHeight(); const rotation = await videoTrack.getRotation(); const sampleRate = await audioTrack.getSampleRate(); const numberOfChannels = await audioTrack.getNumberOfChannels(); const { title, artist, album } = await input.getMetadataTags(); ``` -------------------------------- ### Convert files with mediabunny Source: https://github.com/vanilagy/mediabunny/blob/main/README.md Convert a file to WebM using Conversion.init and execute. ```js import { Input, Output, Conversion, ALL_FORMATS, BlobSource, WebMOutputFormat } from 'mediabunny'; const input = new Input({ source: new BlobSource(file), formats: ALL_FORMATS, }); const output = new Output({ format: new WebMOutputFormat(), // Convert to WebM target: new BufferTarget(), }); const conversion = await Conversion.init({ input, output }); await conversion.execute(); ``` -------------------------------- ### Mp4OutputFormat Source: https://github.com/vanilagy/mediabunny/blob/main/docs/guide/output-formats.md Creates an MP4 output format for media files. Configure options like fastStart, metadataFormat, and callbacks for box writing events. ```APIDOC ## Mp4OutputFormat ### Description Creates an MP4 output format for media files. Configure options like fastStart, metadataFormat, and callbacks for box writing events. ### Constructor `new Mp4OutputFormat(options)` ### Parameters #### Options - **fastStart** (`false | 'in-memory' | 'reserve' | 'fragmented'`) - Optional - Controls the placement of metadata in the file. Defaults to `'in-memory'` when using `BufferTarget` and `false` otherwise. - **minimumFragmentDuration** (`number`) - Optional - Only relevant when `fastStart` is `'fragmented'`. Sets the minimum duration in seconds a fragment must have to be finalized and written to the file. Defaults to 1 second. - **metadataFormat** (`'mdir' | 'mdta' | 'udta' | 'auto'`) - Optional - The metadata format to use for writing metadata tags. Defaults to `'auto'`. - **onFtyp** (`(data: Uint8Array, position: number) => unknown`) - Optional - Called once the ftyp (File Type) box has been written. - **onMoov** (`(data: Uint8Array, position: number) => unknown`) - Optional - Called once the moov (Movie) box has been written. - **onMdat** (`(data: Uint8Array, position: number) => unknown`) - Optional - Called for each finalized mdat (Media Data) box. - **onMoof** (`(data: Uint8Array, position: number, timestamp: number) => unknown`) - Optional - Called for each finalized moof (Movie Fragment) box, with the fragment's start timestamp in seconds. ### Example ```ts import { Output, Mp4OutputFormat } from 'mediabunny'; const output = new Output({ format: new Mp4OutputFormat(options), // ... }); ``` ``` -------------------------------- ### Check native support before registering Source: https://github.com/vanilagy/mediabunny/blob/main/docs/guide/extensions/mp3-encoder.md Use canEncodeAudio('mp3') to check for native browser support. Only register the custom encoder if there is no native support. ```ts import { canEncodeAudio } from 'mediabunny'; import { registerMp3Encoder } from '@mediabunny/mp3-encoder'; if (!(await canEncodeAudio('mp3'))) { registerMp3Encoder(); } ``` -------------------------------- ### audioTrack.getNumberOfChannels() Source: https://github.com/vanilagy/mediabunny/blob/main/docs/guide/reading-media-files.md Retrieves the number of audio channels for the audio track. This returns a Promise that resolves to a number. ```APIDOC ## audioTrack.getNumberOfChannels() ### Description Retrieves the number of audio channels for the audio track. This returns a Promise that resolves to a number. ### Method GET ### Endpoint audioTrack.getNumberOfChannels() ### Response #### Success Response - **result** (number) - The number of audio channels ### Request Example ```ts await audioTrack.getNumberOfChannels(); // => number ``` ``` -------------------------------- ### track.computeDurationOrNull() Source: https://github.com/vanilagy/mediabunny/blob/main/docs/guide/reading-media-files.md Computes the track's duration in seconds, returning null if the track has no packets. Use this to distinguish between a track with zero duration and one with no packets. ```APIDOC ## track.computeDurationOrNull() ### Description Computes the track's duration in seconds, returning null if the track has no packets. Use this to distinguish between a track with zero duration and one with no packets. ### Method computeDurationOrNull ### Endpoint await track.computeDurationOrNull() ### Response #### Success Response - **duration** (number | null) - The track duration in seconds, or null if no packets. ### Request Example ```ts await track.computeDurationOrNull(); // => number | null ``` ``` -------------------------------- ### Install Mediabunny with bun Source: https://github.com/vanilagy/mediabunny/blob/main/docs/guide/installation.md Install Mediabunny using bun. Requires a JavaScript environment that can run ECMAScript 2021 or later. ```bash bun add mediabunny ``` -------------------------------- ### Install with npm Source: https://github.com/vanilagy/mediabunny/blob/main/docs/guide/extensions/aac-encoder.md Install both mediabunny and @mediabunny/aac-encoder using npm. ```bash npm install mediabunny @mediabunny/aac-encoder ``` -------------------------------- ### Create OutputTrackGroup instances Source: https://github.com/vanilagy/mediabunny/blob/main/docs/guide/writing-media-files.md Creates two track groups for controlling output track pairability. Track groups are relevant for many-track formats like HLS. ```ts import { OutputTrackGroup } from 'mediabunny'; const groupA = new OutputTrackGroup(); const groupB = new OutputTrackGroup(); ``` -------------------------------- ### Upload media compression server example Source: https://github.com/vanilagy/mediabunny/blob/main/docs/guide/extensions/server.md Sets up a Node.js HTTP server that compresses uploaded media to MP4 with H.264 (AVC) codec, scaling height to at most 720p. Streams the request body directly to Mediabunny and writes the output to a file, using O(1) memory due to pipelining and backpressure. Responds with 204 on success and 500 on error. ```ts import { ALL_FORMATS, Conversion, FilePathTarget, Input, Mp4OutputFormat, Output, Quality, ReadableStreamSource } from "mediabunny"; import { registerMediabunnyServer } from "@mediabunny/server"; import { Readable } from "node:stream"; import http from "node:http"; registerMediabunnyServer(); const server = http.createServer(async (req, res) => { // Read the request body as a stream const stream = Readable.toWeb(req) as ReadableStream; const input = new Input({ source: new ReadableStreamSource(stream), formats: ALL_FORMATS, }); // Stream the output directly to the disk, could also stream to S3 etc. const output = new Output({ format: new Mp4OutputFormat(), target: new FilePathTarget(`./converted-${crypto.randomUUID()}.mp4`), }); try { const conversion = await Conversion.init({ input, output, video: async track => ({ codec: 'avc', height: Math.min(720, await track.getDisplayHeight()), quality: new Quality('medium'), }), }); await conversion.execute(); res.statusCode = 204; res.end(); } catch (error) { res.statusCode = 500; res.end(); console.error("Error processing media:", error); } }); server.listen(3000); ``` -------------------------------- ### Decode AC3 and read audio samples Source: https://github.com/vanilagy/mediabunny/blob/main/dev/demux.html Registers the AC3 decoder, creates a file input, and reads audio samples from the selected file. Logs timestamp, duration, and sample data as Float32Array. ```javascript MediabunnyAc3.registerAc3Decoder(); const fileInput = document.createElement('input'); fileInput.type = 'file'; document.body.append(fileInput); fileInput.addEventListener('change', async () => { const file = fileInput.files[0]; const input = new Mediabunny.Input({ formats: Mediabunny.ALL_FORMATS, source: new Mediabunny.BlobSource(file), }); const audioTrack = await input.getPrimaryAudioTrack(); const sink = new Mediabunny.AudioSampleSink(audioTrack); for await (const sample of sink.samples()) { const thing = new Float32Array(new ArrayBuffer(sample.allocationSize({ format: 'f32-planar', planeIndex: 0 }))); sample.copyTo(thing, { format: 'f32-planar', planeIndex: 0 }); console.log(sample.timestamp, sample.duration, thing); sample.close() } /* console.log(await audioTrack.getPreRollDuration()); const iter = await sink.samples(10 - 0 / 48000); //await iter.next(); const sample = (await iter.next()).value; console.log(sample); const buf = new Float32Array(new ArrayBuffer(sample.allocationSize({ format: 'f32-planar', planeIndex: 0 }))); sample.copyTo(buf, { format: 'f32-planar', planeIndex: 0 }); console.log(buf) */ /* const track = await input.getPrimaryVideoTrack(); if (!track) { throw new Error('The synthetic file has no video track.'); } for await (const packet of new Mediabunny.EncodedPacketSink(track).packets()) { console.log(packet.timestamp) } */ /* const track = await input.getPrimaryVideoTrack(); const sink = new Mediabunny.EncodedPacketSink(track); const packet = await sink.getFirstPacket(); console.log(packet); */ /* for await (const packet of packetSink.packets()) { //console.log(packet); break; } const sink = new Mediabunny.AudioSampleSink(track); let count = 0; for await (const sample of sink.samples()) { const buf = new Float32Array(new ArrayBuffer(sample.allocationSize({ format: 'f32-planar', planeIndex: 0 }))); sample.copyTo(buf, { format: 'f32-planar', planeIndex: 0 }); let max = Math.max(...buf.map(x => Math.abs(x))); console.log(sample, buf, max) if (++count === 3) { break; } } */ /* const track = await input.getPrimaryAudioTrack(); const sink = new Mediabunny.EncodedPacketSink(track); for await (const packet of sink.packets()) { console.log(packet); } const output = new Mediabunny.Output({ format: new Mediabunny.Mp4OutputFormat(), target: new Mediabunny.BufferTarget(), }); const conversion = await Mediabunny.Conversion.init({ input, output }); await conversion.execute(); return; // Download it now const blob = new Blob([output.target.buffer]); const url = URL.createObjectURL(blob); const a = document.createElement('a'); a.href = url; a.download = file.name.replace(/\.\w+$/, '.mp4'); a.click(); URL.revokeObjectURL(url); */ /* const track = await input.getPrimaryAudioTrack(); const sink = new Mediabunny.EncodedPacketSink(track); for await (const packet of sink.packets()) { console.log(packet); } console.log("Done") */ /* const input = new Mediabunny.Input({ source: new Mediabunny.UrlSource('https://storage.googleapis.com/shaka-demo-assets/angel-one-widevine-hls/hls.m3u8'), formats: Mediabunny.ALL_FORMATS, }); const videoTrack = await input.getPrimaryVideoTrack(); const sink = new Mediabunny.EncodedPacketSink(videoTrack); console.log(await sink.getFirstPacket()); */ /* return const tracks = await manifest.getTracks(); console.log(tracks); const [track] = tracks; window.kekw = () => { console.log(track.getDurationFromMetadata()) }; return; const ugh = new Mediabunny.EncodedPacketSink(audioTrack); for await (const packet of ugh.packets()) { console.log(packet); } //console.log(await manifest.getTracks()); return; const videoTrack = await manifest.getPrimaryVideoTrack(); await videoTrack.hydrate(); console.log("Doing...") console.log(await videoTrack.getDurationFromMetadata({skipLiveWait: true})); return; const sink = new Mediabunny.EncodedPacketSink(videoTrack); const firstTimestamp = await videoTrack.getFirstTimestamp(); console.log(await videoTrack.computeDuration({ skipLiveWait: true })); manifest.dispose() */ /* let last = -Infinity; for await (const packet of sink.packets()) { const rel = packet.timestamp - firstTimestamp; console.log(rel, rel - last); last = rel; if (rel >= 10) { break; } } */ /* console.log("here") const lastPacket = await sink.getPacket(Infinity, { skipLiveWait: true }); const other = await sink.getPacket(lastPacket.timestamp - 5); for await (const packet of sink.packets(other)) { console.log(packet.timestamp); } console.log("Done") */ /* const tracks = await manifest.getTracks(); //await input.hydrateAllTracks(); console.log(tracks, tracks.map(x => [x.getFirstTimestamp(), x.computeDuration()])); return; */ /* for await (using sample of sink.samples()) { console.log(sample.timestamp); } */ /* const manifest = new Mediabunny.ManifestInput({ entryPath: 'https://storage.googleapis.com/shaka-demo-assets/angel-one-hls/hls.m3u8' ?? 'https://playertest.longtailvideo.com/adaptive/vod-with-mp3/manifest.m3u8' ?? ``` -------------------------------- ### track.getFirstTimestampOrNull() Source: https://github.com/vanilagy/mediabunny/blob/main/docs/guide/reading-media-files.md Retrieves the track's start timestamp in seconds, returning null if the track has no packets. Use this to distinguish between a track with a zero timestamp and one with no packets. ```APIDOC ## track.getFirstTimestampOrNull() ### Description Retrieves the track's start timestamp in seconds, returning null if the track has no packets. Use this to distinguish between a track with a zero timestamp and one with no packets. ### Method getFirstTimestampOrNull ### Endpoint await track.getFirstTimestampOrNull() ### Response #### Success Response - **timestamp** (number | null) - The start timestamp in seconds, or null if no packets. ### Request Example ```ts await track.getFirstTimestampOrNull(); // => number | null ``` ``` -------------------------------- ### Convert input file to FLAC using Mediabunny Source: https://github.com/vanilagy/mediabunny/blob/main/docs/guide/extensions/flac-encoder.md Complete example converting an input file to a FLAC file. It registers the encoder if needed, creates an Input from a BlobSource, an Output with FlacOutputFormat and BufferTarget, then runs the conversion and accesses the resulting ArrayBuffer via `output.target.buffer`. ```ts import { Input, ALL_FORMATS, BlobSource, Output, BufferTarget, FlacOutputFormat, canEncodeAudio, Conversion, } from 'mediabunny'; import { registerFlacEncoder } from '@mediabunny/flac-encoder'; if (!(await canEncodeAudio('flac'))) { registerFlacEncoder(); } const input = new Input({ source: new BlobSource(file), // From a file picker, for example formats: ALL_FORMATS, }); const output = new Output({ format: new FlacOutputFormat(), target: new BufferTarget(), }); const conversion = await Conversion.init({ input, output, }); await conversion.execute(); output.target.buffer; // => ArrayBuffer containing the FLAC file ``` -------------------------------- ### Single-file segments Source: https://github.com/vanilagy/mediabunny/blob/main/docs/guide/writing-hls.md Package all media data into one monolithic segment file per playlist. When used with fragmented MP4, the resulting file is a standalone valid fMP4 file. ```ts new HlsOutputFormat({ // ... singleFilePerPlaylist: true, }); ``` -------------------------------- ### input.getMimeType() Source: https://github.com/vanilagy/mediabunny/blob/main/docs/guide/reading-media-files.md Retrieves the full MIME type of the file, including track codecs. Returns a string like 'video/mp4; codecs="avc1.42c032, mp4a.40.2"'. ```APIDOC ## getMimeType() ### Description Retrieves the full MIME type of the file, including track codecs. ### Method getMimeType() ### Endpoint input.getMimeType() ### Response #### Success Response - **string** - The full MIME type, e.g., `'video/mp4; codecs="avc1.42c032, mp4a.40.2"'`. ### Request Example ```ts await input.getMimeType(); // => 'video/mp4; codecs="avc1.42c032, mp4a.40.2"' ``` ``` -------------------------------- ### Include @mediabunny/flac-encoder via script tag Source: https://github.com/vanilagy/mediabunny/blob/main/docs/guide/extensions/flac-encoder.md Include Mediabunny and the FLAC encoder via script tags. This exposes the global objects `Mediabunny` and `MediabunnyFlacEncoder`; use `mediabunny-flac-encoder.d.ts` for type definitions. ```html ``` -------------------------------- ### CustomSource Source: https://github.com/vanilagy/mediabunny/blob/main/docs/guide/reading-media-files.md A general-purpose input source for reading data from anywhere. It allows custom read and size retrieval logic, with optional caching, prefetching, and error handling. ```APIDOC ## CustomSource ### Description A general-purpose input source for reading data from anywhere. It allows custom read and size retrieval logic, with optional caching, prefetching, and error handling. ### Constructor `new CustomSource(options: CustomSourceOptions)` ### Parameters #### Options - **getSize** (function) - Required - Called when the size of the entire file is requested. Must return or resolve to the size in bytes. This function is guaranteed to be called before `read`. - **read** (function) - Required - Called when data is requested. Must return or resolve to the bytes from the specified byte range, or a stream that yields these bytes. You are guaranteed that `0 <= start < end <= fileSize`. - **dispose** (function) - Optional - Called when the `Input` driven by this source is disposed. - **maxCacheSize** (number) - Optional - The maximum number of bytes the cache is allowed to hold in memory. Defaults to 8 MiB. - **prefetchProfile** (string) - Optional - Specifies the prefetch profile that the reader should use with this source. A prefetch profile specifies the pattern with which bytes outside of the requested range are preloaded to reduce latency for future reads. - `'none'` (default): No prefetching; only the data needed in the moment is requested. - `'fileSystem'`: File system-optimized prefetching: a small amount of data is prefetched bidirectionally, aligned with page boundaries. - `'network'`: Network-optimized prefetching, or more generally, prefetching optimized for any high-latency environment: tries to minimize the amount of read calls and aggressively prefetches data when sequential access patterns are detected. - **handleUnhandledError** (function) - Optional - Handles errors that occur while no read is pending. By default, these become unhandled promise rejections. ### Type Definition ```ts type CustomSourceOptions = { getSize: () => MaybePromise; read: (start: number, end: number) => MaybePromise>; dispose?: () => unknown; maxCacheSize?: number; prefetchProfile?: 'none' | 'fileSystem' | 'network'; handleUnhandledError?: (error: unknown) => unknown; }; type MaybePromise = T | Promise; ``` ### Example ```ts import { CustomSource } from 'mediabunny'; import { open } from 'node:fs/promises'; const fileHandle = await open('bigbuckbunny.mp4', 'r'); const source = new CustomSource({ read: async (start, end) => { const buffer = Buffer.alloc(end - start); await fileHandle.read(buffer, 0, end - start, start); return buffer; }, getSize: async () => { const { size } = await fileHandle.stat(); return size; }, }); ``` ### Notes - `CustomSource` was previously known as `StreamSource` and is still available under that alias, but usage of `StreamSource` is deprecated. ``` -------------------------------- ### Include via script tags Source: https://github.com/vanilagy/mediabunny/blob/main/docs/guide/extensions/mp3-encoder.md Directly include the library and encoder using script tags. This exposes the global objects Mediabunny and MediabunnyMp3Encoder. Use mediabunny-mp3-encoder.d.ts for types. ```html ``` -------------------------------- ### BlobSource constructor Source: https://github.com/vanilagy/mediabunny/blob/main/docs/guide/reading-media-files.md Creates a BlobSource backed by a Blob or File object, ideal for reading data from disk in the browser. Accepts optional options for cache size and error handling. ```APIDOC ## BlobSource constructor ### Description Creates a BlobSource backed by a Blob or File object, ideal for reading data from disk in the browser. Accepts optional options for cache size and error handling. ### Method constructor ### Signature `new BlobSource(blob, options?)` ### Parameters - `blob` (Blob | File) - Required - The Blob or File object to use as the source. - `options` (BlobSourceOptions) - Optional - Additional options. - `maxCacheSize` (number) - Optional - The maximum number of bytes the cache is allowed to hold in memory. Defaults to 8 MiB. - `handleUnhandledError` (function) - Optional - Handles errors that occur while no read is pending. ### Response Returns a new `BlobSource` instance. ### Example ```ts import { BlobSource } from 'mediabunny'; fileInput.addEventListener('change', (event) => { const file = event.target.files[0]; const source = new BlobSource(file); }); ``` ``` -------------------------------- ### Read file metadata with Mediabunny Input Source: https://github.com/vanilagy/mediabunny/blob/main/docs/guide/quick-start.md Use this snippet to read file metadata from a File instance using Mediabunny. It creates an Input with ALL_FORMATS and a BlobSource, then computes duration, retrieves all tracks, extracts video and audio metadata, and reads metadata tags. The code assumes a File instance and demonstrates methods like computeDuration, getPrimaryVideoTrack, getDisplayWidth, getDisplayHeight, getRotation, computeFrameRateMetrics, getPrimaryAudioTrack, getNumberOfChannels, getSampleRate, and getMetadataTags. ```ts import { Input, ALL_FORMATS, BlobSource } from 'mediabunny'; const input = new Input({ formats: ALL_FORMATS, // Supporting all file formats source: new BlobSource(file), // Assuming a File instance }); const duration = await input.computeDuration(); // in seconds const allTracks = await input.getTracks(); // List of all tracks // Extract video metadata const videoTrack = await input.getPrimaryVideoTrack(); if (videoTrack) { await videoTrack.getDisplayWidth(); // in pixels await videoTrack.getDisplayHeight(); // in pixels await videoTrack.getRotation(); // in degrees clockwise // Estimate frame rate (FPS) const frameRateMetrics = await videoTrack.computeFrameRateMetrics(); const frameRate = frameRateMetrics.bestGuessFrameRate; } // Extract audio metadata const audioTrack = await input.getPrimaryAudioTrack(); if (audioTrack) { await audioTrack.getNumberOfChannels(); await audioTrack.getSampleRate(); // in Hz } // Extract metadata tags const tags = await input.getMetadataTags(); tags.title; // Title tags.date; // Release date tags.images[0]; // Cover art tags.raw['TBPM']; // Custom tags // ... ``` -------------------------------- ### Create PathedTarget with HLS output Source: https://github.com/vanilagy/mediabunny/blob/main/docs/guide/writing-media-files.md Use PathedTarget to write multi-file outputs like HLS. The root path is 'master.m3u8' and the callback maps each requested file path to a FilePathTarget under '/output/'. ```typescript import { Output, PathedTarget, FilePathTarget, HlsOutputFormat } from 'mediabunny'; const output = new Output({ format: new HlsOutputFormat({ /* ... */ }), target: new PathedTarget( 'master.m3u8', ({ path, isRoot }) => new FilePathTarget(`/output/${path}`), ), }); ``` -------------------------------- ### Check if track is live Source: https://github.com/vanilagy/mediabunny/blob/main/docs/guide/media-sinks.md Check whether a track is live, meaning new media data is still being produced. Live tracks are treated like any other track, but reading from the unavailable end waits until data becomes available or the stream ends. ```ts await track.isLive(); ```