### Installation and Server Headers Source: https://context7.com/diffusionstudio/core-docs/llms.txt Instructions for installing the Diffusion Studio Core package and configuring server headers required for `SharedArrayBuffer`. ```bash npm install @diffusionstudio/core ``` ```javascript // vite.config.js export default { server: { headers: { 'Cross-Origin-Opener-Policy': 'same-origin', 'Cross-Origin-Embedder-Policy': 'credentialless', }, }, }; ``` ```javascript // next.config.js const nextConfig = { async headers() { return [ { source: '/:path*', headers: [ { key: 'Cross-Origin-Opener-Policy', value: 'same-origin' }, { key: 'Cross-Origin-Embedder-Policy', value: 'credentialless' }, ], }, ]; }, }; module.exports = nextConfig; ``` -------------------------------- ### Run Development Server Source: https://github.com/diffusionstudio/core-docs/blob/main/README.md Start the development server to view the project locally. Access it at http://localhost:3000. ```bash npm run dev ``` ```bash yarn dev ``` ```bash pnpm dev ``` ```bash bun dev ``` -------------------------------- ### Install Diffusion Studio Core Source: https://github.com/diffusionstudio/core-docs/blob/main/src/content/index.mdx Install the Diffusion Studio Core package using npm. ```bash npm install @diffusionstudio/core ``` -------------------------------- ### Complete Workflow: Save and Restore Project Source: https://github.com/diffusionstudio/core-docs/blob/main/src/content/checkpoints.mdx A full example demonstrating how to create a composition, serialize its sources, create a checkpoint, save them to local storage, and later restore the composition. ```typescript import * as core from '@diffusionstudio/core'; // Step 1: Create and configure the composition const sources = await Promise.all([ core.Source.from('https://example.com/image.png'), core.Source.from('https://example.com/video.mp4'), ]); const composition = new core.Composition({ width: 1920, height: 1080 }); const layer0 = await composition.add(new core.Layer()); const layer1 = await composition.add(new core.Layer()); await layer0.add(new core.ImageClip(sources[0], { height: '100%' })); await layer1.add(new core.VideoClip(sources[1], { range: [0, 10] })); // Step 2: Serialize sources and create checkpoint const assets = core.serializeSources(sources); const checkpoint = await composition.createCheckpoint(); // Step 3: Save to storage (e.g., localStorage, database, etc.) localStorage.setItem('project-checkpoint', JSON.stringify(checkpoint)); localStorage.setItem('project-assets', JSON.stringify(assets)); // Step 4: Later, restore from storage const savedCheckpoint = JSON.parse(localStorage.getItem('project-checkpoint')); const savedAssets = JSON.parse(localStorage.getItem('project-assets')); const restoredComposition = new core.Composition(); await restoredComposition.restoreCheckpoint(savedCheckpoint, savedAssets); ``` -------------------------------- ### Loading Media Assets with Source Source: https://context7.com/diffusionstudio/core-docs/llms.txt Demonstrates various methods to load media assets like videos, audio, and captions using the `Source` class. Supports loading from URLs, File inputs, Blobs, and FileSystemFileHandles. Includes examples of accessing source properties and generating thumbnails or waveforms. ```typescript import * as core from '@diffusionstudio/core'; // From a URL const videoSource = await core.Source.from( 'https://example.com/video.mp4', { mimeType: 'video/mp4' }, // optional MIME hint for faster init ); // From a File input const file = document.querySelector('input[type="file"]')!.files![0]; const fileSource = await core.Source.from(file); // From a Blob const blob = new Blob([/* data */], { type: 'video/mp4' }); const blobSource = await core.Source.from(blob); // From FileSystemFileHandle (File System Access API) const [fileHandle] = await window.showOpenFilePicker(); const handleSource = await core.Source.from(fileHandle); // VideoSource properties and thumbnails videoSource.fps; // 30 videoSource.width; // 1920 videoSource.height; // 1080 videoSource.bitrate; // 1000000 for (const thumb of videoSource.thumbnailsInRange({ start: 0, end: 20, count: 10, height: 100 })) { // thumb contains image data for drawing preview thumbnails } // AudioSource: waveform samples and silence detection const audioSource = await core.Source.from('https://example.com/audio.mp3'); audioSource.duration; // 10 (seconds) audioSource.sampleRate; // 44100 audioSource.numberOfChannels; // 2 for (const sample of audioSource.samplesInRange({ start: 0, end: 20 })) { // draw waveform with sample data } const silences = await audioSource.silences(); // e.g. [{ start: 5.2, end: 6.8 }, ...] // CaptionSource: transcript processing const captionSource = await core.Source.from('/captions.json'); const wpm = captionSource.computeWpm(); const groups = captionSource.groupBy({ count: 2 }); const { text, blob: srtBlob } = captionSource.toSrt(); captionSource.optimize(); // Share one source across multiple clips const src = await core.Source.from('https://example.com/video.mp4'); const clip1 = new core.VideoClip(src, { range: [0, 5] }); const clip2 = new core.VideoClip(src, { range: [10, 15] }); const clip3 = new core.VideoClip(src, { range: [20, 25] }); ``` -------------------------------- ### Install Node Packages Source: https://github.com/diffusionstudio/core-docs/blob/main/README.md Install project dependencies using your preferred Node Package Manager (pnpm, yarn, or npm). ```bash pnpm install ``` ```bash yarn install ``` ```bash npm install ``` -------------------------------- ### Mount Composition Player Source: https://github.com/diffusionstudio/core-docs/blob/main/src/content/composition.mdx Mount the Composition to a DOM element to visualize it. This example centers the player and scales it to fit the container. ```typescript const container = document.getElementById('player-container') as HTMLDivElement; const player = document.getElementById('player') as HTMLDivElement; composition.mount(player); const scale = Math.min( container.clientWidth / composition.width, container.clientHeight / composition.height, ); player.style.width = `${composition.width}px`; player.style.height = `${composition.height}px`; player.style.transform = `scale(${scale})`; player.style.transformOrigin = 'center'; ``` ```html
``` ```css #player-container { display: flex; justify-content: center; align-items: center; } ``` -------------------------------- ### Configure Layer in Sequential Mode Source: https://github.com/diffusionstudio/core-docs/blob/main/src/content/layers.mdx Create a Layer that operates in sequential mode, ensuring each new clip starts immediately after the previous one ends. To disable, set the mode back to 'DEFAULT'. ```typescript const layer = new core.Layer({ mode: 'SEQUENTIAL', }); // undo sequential mode layer.mode = 'DEFAULT'; ``` -------------------------------- ### Get Image Dimensions with ImageSource Source: https://github.com/diffusionstudio/core-docs/blob/main/src/content/source.mdx Use `Source.from` to create an ImageSource and access its width and height properties in pixels. Ensure the URL points to a valid image. ```typescript const source = await core.Source.from('https://example.com/image.jpg'); // Image dimensions source.width; // 1920 (pixels) source.height; // 1080 (pixels) ``` -------------------------------- ### Manipulate VideoClip Delay and Range Source: https://github.com/diffusionstudio/core-docs/blob/main/src/content/clips/video.mdx Adjust the video clip's start time by setting a negative delay and trim its duration by specifying a start and end time range. ```typescript video.delay = -4; // move the clip to the left by 4 seconds video.range = [4, 8]; // trims the clip from start to end seconds ``` -------------------------------- ### Source Creation and Usage Source: https://context7.com/diffusionstudio/core-docs/llms.txt Demonstrates how to create Source objects from various inputs like URLs, Files, Blobs, and FileSystemFileHandles. It also shows how to access properties and generate thumbnails for VideoSource and AudioSource, and process captions for CaptionSource. ```APIDOC ## Source `Source` loads, caches, and prepares media assets for clips. A single source can be shared across multiple clips for efficient memory use. Sources are typed: `VideoSource`, `AudioSource`, `ImageSource`, `CaptionSource`. ```typescript import * as core from '@diffusionstudio/core'; // From a URL const videoSource = await core.Source.from( 'https://example.com/video.mp4', { mimeType: 'video/mp4' }, // optional MIME hint for faster init ); // From a File input const file = document.querySelector('input[type="file"]')!.files![0]; const fileSource = await core.Source.from(file); // From a Blob const blob = new Blob([/* data */], { type: 'video/mp4' }); const blobSource = await core.Source.from(blob); // From FileSystemFileHandle (File System Access API) const [fileHandle] = await window.showOpenFilePicker(); const handleSource = await core.Source.from(fileHandle); // VideoSource properties and thumbnails videoSource.fps; // 30 videoSource.width; // 1920 videoSource.height; // 1080 videoSource.bitrate; // 1000000 for (const thumb of videoSource.thumbnailsInRange({ start: 0, end: 20, count: 10, height: 100 })) { // thumb contains image data for drawing preview thumbnails } // AudioSource: waveform samples and silence detection const audioSource = await core.Source.from('https://example.com/audio.mp3'); audioSource.duration; // 10 (seconds) audioSource.sampleRate; // 44100 audioSource.numberOfChannels; // 2 for (const sample of audioSource.samplesInRange({ start: 0, end: 20 })) { // draw waveform with sample data } const silences = await audioSource.silences(); // e.g. [{ start: 5.2, end: 6.8 }, ...] // CaptionSource: transcript processing const captionSource = await core.Source.from('/captions.json'); const wpm = captionSource.computeWpm(); const groups = captionSource.groupBy({ count: 2 }); const { text, blob: srtBlob } = captionSource.toSrt(); captionSource.optimize(); // Share one source across multiple clips const src = await core.Source.from('https://example.com/video.mp4'); const clip1 = new core.VideoClip(src, { range: [0, 5] }); const clip2 = new core.VideoClip(src, { range: [10, 15] }); const clip3 = new core.VideoClip(src, { range: [20, 25] }); ``` ``` -------------------------------- ### Get Playback Position Source: https://github.com/diffusionstudio/core-docs/blob/main/src/content/composition.mdx Calculate the current playback position as a fraction of the total duration. Both currentTime and duration are in seconds. ```typescript const position = composition.currentTime / composition.duration; ``` -------------------------------- ### VideoClip Creation and Configuration Source: https://context7.com/diffusionstudio/core-docs/llms.txt Details how to create and configure VideoClip objects. This includes specifying the source, position, dimensions, and trimming the video using a range. It also shows how to set the delay for timeline placement. ```APIDOC ## VideoClip `VideoClip` renders a video asset onto the canvas. It supports trimming (`range`), timeline placement (`delay`), and all common visual properties. ```typescript import * as core from '@diffusionstudio/core'; const composition = new core.Composition(); const source = await core.Source.from( 'https://diffusion-studio-public.s3.eu-central-1.amazonaws.com/videos/big_buck_bunny_1080p_30fps.mp4' ); const video = new core.VideoClip(source, { position: 'center', // center on canvas height: '100%', // fill full height, maintain aspect ratio }); // Trim to seconds [4, 8] of the source, shift left on timeline by 4s video.range = [4, 8]; video.delay = -4; const layer = await composition.add(new core.Layer()); await layer.add(video); ``` ``` -------------------------------- ### Set ImageClip Delay Source: https://github.com/diffusionstudio/core-docs/blob/main/src/content/clips/image.mdx Adjust the start time of an ImageClip by modifying its 'delay' property. This moves the clip along the timeline. ```typescript image.delay = 0; ``` -------------------------------- ### Manage Loaded Fonts Source: https://github.com/diffusionstudio/core-docs/blob/main/src/content/clips/text.mdx Provides functions to get a JSON serializable array of all currently loaded fonts and to restore them later. ```typescript const loadedFonts = await core.getLoadedFonts(); // JSON serializable array ``` ```typescript await core.restoreFonts(loadedFonts); ``` -------------------------------- ### Create an Audio Clip Source: https://github.com/diffusionstudio/core-docs/blob/main/src/content/clips/audio.mdx Fetches an MP3 file and creates an initial audio clip. The MP3 file contains 16 seconds of audio. ```typescript import * as core from '@diffusionstudio/core'; const source = await core.Source.from('https://diffusion-studio-public.s3.eu-central-1.amazonaws.com/audio/piano.mp3'); const layer = await composition.add(new core.Layer()); const clip0 = await layer.add(new core.AudioClip(source)); ``` -------------------------------- ### Create Source with MIME Type Hint Source: https://github.com/diffusionstudio/core-docs/blob/main/src/content/source.mdx Provide a MIME type hint during source creation for faster initialization, especially when loading from URLs. ```typescript const source = await core.Source.from('https://example.com/video.mp4', { mimeType: 'video/mp4', }); ``` -------------------------------- ### Rendering Video Content with VideoClip Source: https://context7.com/diffusionstudio/core-docs/llms.txt Shows how to create and configure a `VideoClip` to render a video asset. Covers setting position, dimensions, trimming the source range, and adjusting timeline delay. Demonstrates adding the video clip to a layer within a composition. ```typescript import * as core from '@diffusionstudio/core'; const composition = new core.Composition(); const source = await core.Source.from( 'https://diffusion-studio-public.s3.eu-central-1.amazonaws.com/videos/big_buck_bunny_1080p_30fps.mp4' ); const video = new core.VideoClip(source, { position: 'center', // center on canvas height: '100%', // fill full height, maintain aspect ratio }); // Trim to seconds [4, 8] of the source, shift left on timeline by 4s video.range = [4, 8]; video.delay = -4; const layer = await composition.add(new core.Layer()); await layer.add(video); ``` -------------------------------- ### Group Words in Caption Source Source: https://github.com/diffusionstudio/core-docs/blob/main/src/content/source.mdx Group words together within a caption source, for example, to display multiple words as a single caption unit. ```typescript // Group words together (useful for displaying multiple words per caption) const groups = source.groupBy({ count: 2 }); // Groups words into pairs ``` -------------------------------- ### Alternative Rendering Methods Source: https://github.com/diffusionstudio/core-docs/blob/main/src/content/rendering.mdx Export the video as a Blob, download it with a specified filename, or stream it to a callback function for custom handling. ```typescript const blob = await encoder.render(); // Returns the video as a Blob. core.assert(blob.type == 'success'); const blob = res.data; await encoder.render('myVideo.mp4'); // Downloads the result with a specified name. await encoder.render((data: Uint8Array, position: number) => { console.log(data, position) }); // Streams the video to a callback. ``` -------------------------------- ### Split an AudioClip Source: https://github.com/diffusionstudio/core-docs/blob/main/src/content/clips/audio.mdx Splits an existing audio clip into two. The original clip is shortened to end at the split time, and a new clip is created starting after the split time. ```typescript const clip1 = await clip0.split(8); ``` -------------------------------- ### Create Video Source from FileSystemFileHandle Source: https://github.com/diffusionstudio/core-docs/blob/main/src/content/source.mdx Load a video asset using a FileSystemFileHandle, typically obtained from the File System Access API. ```typescript const fileHandle = await window.showOpenFilePicker(); const source = await core.Source.from(fileHandle[0]); ``` -------------------------------- ### Create VideoClip with Position and Height Source: https://github.com/diffusionstudio/core-docs/blob/main/src/content/clips/video.mdx Instantiate a VideoClip from a VideoSource. Accepts blobs or file handles. Sets position to center and stretches to full height. ```typescript const video = new core.VideoClip(source, { // also accepts blobs or file handles position: 'center', // ensures the clip is centered height: '100%', // stretches the clip to the full height }); ``` -------------------------------- ### Create and Control a Composition Source: https://context7.com/diffusionstudio/core-docs/llms.txt Create a new Composition, mount it to the DOM for interactive preview, and control playback. Ensure the server is configured with the necessary HTTP headers for SharedArrayBuffer. ```typescript import * as core from '@diffusionstudio/core'; // Create a 1920x1080 composition (defaults) const composition = new core.Composition({ width: 1920, height: 1080, background: '#000000', playbackEndBehavior: 'loop', licenseKey: 'your-license-key', // optional }); // Mount to a DOM element for interactive preview const player = document.getElementById('player') as HTMLDivElement; const container = document.getElementById('player-container') as HTMLDivElement; composition.mount(player); const scale = Math.min( container.clientWidth / composition.width, container.clientHeight / composition.height, ); player.style.width = `${composition.width}px`; player.style.height = `${composition.height}px`; player.style.transform = `scale(${scale})`; player.style.transformOrigin = 'center'; // Playback controls await composition.play(); await composition.pause(); await composition.seek(30); // seek to 30 seconds await composition.seek('10s'); // string time format // Read playback position as fraction (0–1) const progress = composition.currentTime / composition.duration; // Capture a screenshot at the current frame const dataUrl = await composition.screenshot(); // Unmount before attaching to another element composition.unmount(); ``` -------------------------------- ### Create Video Source from URL Source: https://github.com/diffusionstudio/core-docs/blob/main/src/content/source.mdx Load a video asset from a remote URL. Ensure the URL points to a valid video file. ```typescript const source = await core.Source.from('https://example.com/video.mp4'); ``` -------------------------------- ### Composition Object Usage Source: https://context7.com/diffusionstudio/core-docs/llms.txt Demonstrates how to create, mount, control playback, and unmount a `Composition` object in Diffusion Studio Core. ```typescript import * as core from '@diffusionstudio/core'; // Create a 1920x1080 composition (defaults) const composition = new core.Composition({ width: 1920, height: 1080, background: '#000000', playbackEndBehavior: 'loop', licenseKey: 'your-license-key', // optional }); // Mount to a DOM element for interactive preview const player = document.getElementById('player') as HTMLDivElement; const container = document.getElementById('player-container') as HTMLDivElement; composition.mount(player); const scale = Math.min( container.clientWidth / composition.width, container.clientHeight / composition.height, ); player.style.width = `${composition.width}px`; player.style.height = `${composition.height}px`; player.style.transform = `scale(${scale})`; player.style.transformOrigin = 'center'; // Playback controls await composition.play(); await composition.pause(); await composition.seek(30); // seek to 30 seconds await composition.seek('10s'); // string time format // Read playback position as fraction (0–1) const progress = composition.currentTime / composition.duration; // Capture a screenshot at the current frame const dataUrl = await composition.screenshot(); // Unmount before attaching to another element composition.unmount(); ``` -------------------------------- ### Rendering CaptionClips from Transcripts Source: https://context7.com/diffusionstudio/core-docs/llms.txt Explains how to create CaptionClips using remote or in-memory transcript sources. Demonstrates applying presets and trimming the clip like an audio clip. ```typescript import * as core from '@diffusionstudio/core'; const composition = new core.Composition(); // Load transcript from remote JSON const source = await core.Source.from('/captions.json'); // Expected JSON shape: // [{ text: string, words: [{ text: string, start: number, end: number }] }] // Or from an in-memory object const transcript: core.Transcript = [/* your transcript data */]; const blob = new Blob([JSON.stringify(transcript)]); const file = new File([blob], 'transcript.json', { type: 'application/json' }); const inMemorySource = await core.Source.from(file); // Create clip with a preset const clip = new core.CaptionClip(source, { preset: new core.PaperCaptionPreset(), // Other available presets: ClassicCaptionPreset, CascadeCaptionPreset, // GuineaCaptionPreset, SolarCaptionPreset, WhisperCaptionPreset }); const layer = await composition.add(new core.Layer()); await layer.add(clip); // Trim like an audio clip clip.range = [0.5, 4]; ``` -------------------------------- ### Create an ImageClip Source: https://github.com/diffusionstudio/core-docs/blob/main/src/content/clips/image.mdx Instantiate an ImageClip by providing a source URL and specifying its delay and duration. Ensure the source is correctly typed as ImageSource. ```typescript import * as core from '@diffusionstudio/core'; const url = 'https://diffusion-studio-public.s3.eu-central-1.amazonaws.com/images/lenna.png'; const source = await core.Source.from(url); const image = new core.ImageClip(source, { delay: 20, duration: 100 }); ``` -------------------------------- ### Create Composition Checkpoint Source: https://github.com/diffusionstudio/core-docs/blob/main/src/content/checkpoints.mdx Create a checkpoint to capture the current state of your composition. Ensure all sources are properly configured before creating the checkpoint. ```typescript import * as core from '@diffusionstudio/core'; // Create and configure your composition const sources = await Promise.all([ core.Source.from('/image1.png'), core.Source.from('/video1.mp4'), ]); const composition = new core.Composition(); const layer0 = await composition.add(new core.Layer()); const layer1 = await composition.add(new core.Layer()); await layer0.add(new core.ImageClip(sources[0], { height: '100%' })); await layer1.add(new core.VideoClip(sources[1], { range: [0, 10] })); // Create a checkpoint of the current state const checkpoint = await composition.createCheckpoint(); ``` -------------------------------- ### Create Video Source from File Input Source: https://github.com/diffusionstudio/core-docs/blob/main/src/content/source.mdx Load a video asset from a file selected by the user via an HTML file input element. ```typescript const fileInput = document.querySelector('input[type="file"]').files[0]; const source = await core.Source.from(fileInput); ``` -------------------------------- ### Create and Apply RectangleMask Source: https://github.com/diffusionstudio/core-docs/blob/main/src/content/masks.mdx Demonstrates creating a RectangleMask with rounded corners and applying it to a VideoClip. Ensure the core library is imported. ```typescript import * as core from '@diffusionstudio/core'; const composition = new core.Composition(); // Create a RectangleMask const mask = new core.RectangleMask({ x: 480, y: 0, width: composition.width - 960, height: composition.height, radius: 100, }); // Load a video source const source = await core.Source.from( 'https://example.com/video.mp4' ); // Apply the mask to a video clip const videoClip = new core.VideoClip(source, { position: 'center', mask, height: '100%', }); const layer = await composition.add(new core.Layer()); await layer.add(videoClip); ``` -------------------------------- ### Managing Clips with Layer Source: https://context7.com/diffusionstudio/core-docs/llms.txt Illustrates how to use `Layer` to organize `Clip` objects within a `Composition`. Covers adding layers, setting render order, and managing clips in sequential or default modes. Demonstrates removing clips and layers. ```typescript import * as core from '@diffusionstudio/core'; const composition = new core.Composition(); // Create layers and add to composition const videoLayer = await composition.add(new core.Layer()); const imageLayer = await composition.add(new core.Layer()); const textLayer = await composition.add(new core.Layer()); // textLayer is at index 0: rendered on top // Reorder: bring videoLayer to the top videoLayer.index = 0; // Or use named positions: videoLayer.index = 'top'; videoLayer.index = 'bottom'; // Sequential mode: each new clip starts where the previous one ended const seqLayer = await composition.add(new core.Layer({ mode: 'SEQUENTIAL' })); await seqLayer.add(new core.TextClip({ text: 'First', duration: 3 })); await seqLayer.add(new core.TextClip({ text: 'Second', duration: 3 })); // Second clip automatically starts at t=3s // Undo sequential mode seqLayer.mode = 'DEFAULT'; // Remove a clip from a layer await layer.remove(clip); // Remove the layer from the composition composition.remove(layer); ``` -------------------------------- ### Default Encoder Configuration Source: https://github.com/diffusionstudio/core-docs/blob/main/src/content/rendering.mdx View the default settings for video and audio encoding. These can be overridden during encoder instantiation. ```typescript { video: { enabled: true, // true by default codec: 'avc', bitrate: 10e6, fps: 30, resolution: 1, }, audio: { enabled: true, sampleRate: 48000, numberOfChannels: 2, bitrate: 128e3, codec: 'aac', }, } ``` -------------------------------- ### Create Audio Source from Asset Object Source: https://github.com/diffusionstudio/core-docs/blob/main/src/content/source.mdx Create an audio source directly from an Asset object, specifying input and MIME type. ```typescript const source = await core.Source.fromAsset({ input: new File([], 'my-audio.MP3', { type: 'audio/mpeg' }), mimeType: 'audio/mpeg', }); ``` -------------------------------- ### Load Video Source from URL Source: https://github.com/diffusionstudio/core-docs/blob/main/src/content/clips/video.mdx Create a VideoSource from a URL. This is efficient for sharing resources between multiple VideoClips. ```typescript const source = await core.Source.from('https://diffusion-studio-public.s3.eu-central-1.amazonaws.com/videos/big_buck_bunny_1080p_30fps.mp4'); ``` -------------------------------- ### Clone Diffusion Studio Core Project Source: https://github.com/diffusionstudio/core-docs/blob/main/README.md Use this command to clone the project repository from GitHub. ```bash git clone git@github.com:diffusionstudio/core-docs.git ``` -------------------------------- ### Create Rectangle, Ellipse, and Polygon Clips Source: https://context7.com/diffusionstudio/core-docs/llms.txt Demonstrates the creation of various shape clips with customizable properties like fill, strokes, and effects. Ensure the '@diffusionstudio/core' library is imported. ```typescript import * as core from '@diffusionstudio/core'; const composition = new core.Composition(); const layer = await composition.add(new core.Layer()); // Rectangle with rounded corners await layer.add(new core.RectangleClip({ position: 'center', keepAspectRatio: true, width: 300, height: 200, fill: '#FF0000', radius: 10, strokes: [{ width: 2, color: '#000000' }], })); // Ellipse with drop-shadow effect await layer.add(new core.EllipseClip({ fill: '#FF0000', blendMode: 'screen', height: 800, width: 300, anchorX: 0, // origin point 0–1 or pixel value anchorY: 0, rotation: 45, effects: [{ type: 'drop-shadow', value: { offsetX: -9, offsetY: 9, blur: 3, color: '#000000' }, }], })); // Polygon (hexagon) await layer.add(new core.PolygonClip({ x: '50%', y: '50%', fill: '#FFFF00', sides: 6, // 3=triangle, 5=pentagon, 6=hexagon, 8=octagon width: 200, height: 200, rotation: 30, })); ``` -------------------------------- ### Configure and Use Encoder for Rendering Source: https://context7.com/diffusionstudio/core-docs/llms.txt Configure the Encoder with custom video and audio settings for rendering. Supports rendering to File System Access API, Blob, filename, or a callback. ```typescript import * as core from '@diffusionstudio/core'; const composition = new core.Composition(); // ... add layers and clips ... // Create encoder with custom settings const encoder = new core.Encoder(composition, { video: { codec: 'avc', // H.264 bitrate: 10e6, // 10 Mbps fps: 60, resolution: 2, // 2× → 4K output from a 1080p composition }, audio: { codec: 'aac', sampleRate: 48000, numberOfChannels: 2, bitrate: 128e3, }, }); // Track progress encoder.onProgress = ({ progress, total }) => { console.log(Math.round(progress * 100 / total) + '%'); }; // Export via File System Access API (streams to disk — supports files larger than RAM) const fileHandle = await window.showSaveFilePicker({ suggestedName: 'output.mp4', types: [{ description: 'Video File', accept: { 'video/mp4': ['.mp4'] } }], }); await encoder.render(fileHandle); // Fallback: render to Blob const res = await encoder.render(); core.assert(res.type == 'success'); const blob = res.data; // Download with filename await encoder.render('myVideo.mp4'); // Stream to callback await encoder.render((data: Uint8Array, position: number) => { console.log('chunk at byte', position, 'length', data.length); }); // Cancel an in-progress export encoder.cancel(); // Polyfill showSaveFilePicker for unsupported browsers if (!('showSaveFilePicker' in window)) { Object.assign(window, { showSaveFilePicker: async () => 'myVideo.mp4' }); } ``` -------------------------------- ### Initialize Composition in Diffusion Studio Core Source: https://github.com/diffusionstudio/core-docs/blob/main/src/content/index.mdx Import and initialize a new Composition object from the Diffusion Studio Core library. This is a fundamental step for creating media compositions. ```typescript import * as core from '@diffusionstudio/core'; const composition = new core.Composition(); ``` -------------------------------- ### Rendering TextClips with Advanced Styling Source: https://context7.com/diffusionstudio/core-docs/llms.txt Illustrates creating TextClips with various styling options including custom fonts, multi-segment styles, strokes, shadows, and glow effects. Supports loading local and web fonts. ```typescript import * as core from '@diffusionstudio/core'; const composition = new core.Composition(); // Load web fonts from the built-in font library const thin = await core.loadFont({ family: 'Geologica', weight: '300', size: 19 }); const bold = await core.loadFont({ family: 'Geologica', weight: '700', size: 24 }); // Load a custom web font by URL const roboto = await core.loadFont({ source: 'https://fonts.gstatic.com/s/roboto/v32/KFOlCnqEu92Fr1MmSU5fBBc4AMP6lQ.woff2', weight: '400', family: 'Roboto', }); // Load a local font (Chrome only) const localFonts = await core.getLocalFonts(); const localFont = await core.loadFont(localFonts[0].variants[0]); // Serialize / restore fonts (for checkpoints) const loadedFonts = await core.getLoadedFonts(); await core.restoreFonts(loadedFonts); await composition.add( new core.TextClip({ text: "The quick\nbrown fox jumps over the lazy dog.", align: 'center', baseline: 'middle', font: bold, maxWidth: '40%', leading: 1.3, x: '50%', y: '50%', glow: { intensity: 1, color: '#000000', radius: 10, opacity: 100 }, strokes: [{ width: 5, color: '#000000', lineJoin: 'round', lineCap: 'round', miterLimit: 4 }], shadows: [ { offsetX: 4, offsetY: 5, blur: 20, color: '#000000', opacity: 100 }, { offsetX: -8, offsetY: -4, blur: 20, color: '#FFFFFF', opacity: 90 }, ], // Override styles for characters 0–9 (e.g., "The quick\n") styles: [{ start: 0, stop: 9, style: { font: thin, color: '#0000FF', casing: 'upper' }, }], }) ); ``` -------------------------------- ### Displaying ImageClips Source: https://context7.com/diffusionstudio/core-docs/llms.txt Shows how to create and manipulate ImageClips, including setting delay, duration, and splitting clips. Use trim to adjust the display range. ```typescript import * as core from '@diffusionstudio/core'; const composition = new core.Composition(); const source = await core.Source.from( 'https://diffusion-studio-public.s3.eu-central-1.amazonaws.com/images/lenna.png' ); // Place image starting at 20s for 100 seconds const image = new core.ImageClip(source, { delay: 20, duration: 100 }); // Move to start of timeline image.delay = 0; // Equivalent using trim(delay, duration): image.trim(0, 100); const layer = await composition.add(new core.Layer()); await layer.add(image); // Split at 50 seconds into two clips const imageNew = image.split(50); ``` -------------------------------- ### Configure Next.js for Diffusion Studio Core Headers Source: https://github.com/diffusionstudio/core-docs/blob/main/src/content/index.mdx Configure Next.js to dynamically set the Cross-Origin-Opener-Policy and Cross-Origin-Embedder-Policy headers for Diffusion Studio Core. ```javascript /** @type {import('next').NextConfig} */ const nextConfig = { async headers() { return [ { source: '/:path*', headers: [ { key: 'Cross-Origin-Opener-Policy', value: 'same-origin', }, { key: 'Cross-Origin-Embedder-Policy', value: 'credentialless', }, ], }, ]; }, }; module.exports = nextConfig; ``` -------------------------------- ### Create Encoder Instance Source: https://github.com/diffusionstudio/core-docs/blob/main/src/content/rendering.mdx Instantiate an Encoder using a Composition object. This encoder will handle the video processing and export. ```typescript const encoder = new core.Encoder(composition); ``` -------------------------------- ### Layer Management Source: https://context7.com/diffusionstudio/core-docs/llms.txt Explains how to create and manage Layers within a Composition. This includes adding layers, reordering them, and controlling their rendering order. It also covers sequential clip addition and removing clips or layers. ```APIDOC ## Layer `Layer` is a chronological container for clips. Clips within a layer cannot overlap in time. The layer order in the composition determines render order (index 0 = top / rendered last). ```typescript import * as core from '@diffusionstudio/core'; const composition = new core.Composition(); // Create layers and add to composition const videoLayer = await composition.add(new core.Layer()); const imageLayer = await composition.add(new core.Layer()); const textLayer = await composition.add(new core.Layer()); // textLayer is at index 0: rendered on top // Reorder: bring videoLayer to the top videoLayer.index = 0; // Or use named positions: videoLayer.index = 'top'; videoLayer.index = 'bottom'; // Sequential mode: each new clip starts where the previous one ended const seqLayer = await composition.add(new core.Layer({ mode: 'SEQUENTIAL' })); await seqLayer.add(new core.TextClip({ text: 'First', duration: 3 })); await seqLayer.add(new core.TextClip({ text: 'Second', duration: 3 })); // Second clip automatically starts at t=3s // Undo sequential mode seqLayer.mode = 'DEFAULT'; // Remove a clip from a layer await layer.remove(clip); // Remove the layer from the composition composition.remove(layer); ``` ``` -------------------------------- ### Share Sources Between Clips Source: https://github.com/diffusionstudio/core-docs/blob/main/src/content/source.mdx Demonstrates how to create multiple clips from a single source, enabling efficient asset reuse and caching. ```typescript const source = await core.Source.from('https://example.com/video.mp4'); // Create multiple clips from the same source const clip1 = new core.VideoClip(source, { range: [0, 5] }); const clip2 = new core.VideoClip(source, { range: [10, 15] }); const clip3 = new core.VideoClip(source, { range: [20, 25] }); // All three clips share the same cached source data ``` -------------------------------- ### Configure Encoder for 4K 60 FPS Source: https://github.com/diffusionstudio/core-docs/blob/main/src/content/rendering.mdx Customize the encoder settings to output a 4K video at 60 frames per second, overriding the default resolution and FPS. ```typescript const encoder = new core.Encoder(composition, { video: { fps: 60, resolution: 2, }, }); ``` -------------------------------- ### Implement Rectangle and Ellipse Masks Source: https://context7.com/diffusionstudio/core-docs/llms.txt Demonstrates how to use RectangleMask and EllipseMask to restrict the visible area of clips. Requires '@diffusionstudio/core' import and a source for video clips. ```typescript import * as core from '@diffusionstudio/core'; const composition = new core.Composition(); const source = await core.Source.from('https://example.com/video.mp4'); // RectangleMask with rounded corners const rectMask = new core.RectangleMask({ x: 480, y: 0, width: composition.width - 960, height: composition.height, radius: 100, }); const videoClip = new core.VideoClip(source, { position: 'center', height: '100%', mask: rectMask, }); const layer = await composition.add(new core.Layer()); await layer.add(videoClip); // Circular EllipseMask crop const circularMask = new core.EllipseMask({ x: composition.width / 2, y: composition.height / 2, width: 800, height: 800, }); // Picture-in-picture const pipMask = new core.RectangleMask({ x: composition.width - 400, y: composition.height - 300, width: 360, height: 240, radius: 10, }); const pipVideo = new core.VideoClip(source, { position: 'center', mask: pipMask, width: 400, height: 300, }); ``` -------------------------------- ### Create Caption Clip with Custom Preset Source: https://github.com/diffusionstudio/core-docs/blob/main/src/content/clips/caption.mdx Instantiate a CaptionClip using a specific appearance preset, such as PaperCaptionPreset. This allows for visual customization of the subtitles. ```typescript const clip = new core.CaptionClip(source, { preset: new core.PaperCaptionPreset(), }); ``` -------------------------------- ### Style Text Clips with Advanced Properties Source: https://github.com/diffusionstudio/core-docs/blob/main/src/content/clips/text.mdx Demonstrates creating a TextClip with custom text, alignment, font, and advanced styling properties like glow, strokes, and shadows. Use the `styles` property to override specific sections of the text. ```typescript import * as core from '@diffusionstudio/core'; const thin = await core.loadFont({ family: 'Geologica', weight: '300', size: 19 }); const bold = await core.loadFont({ family: 'Geologica', weight: '700', size: 24 }); await composition.add( new core.TextClip({ text: "The quick\nbrown fox jumps over the lazy dog.", align: 'center', baseline: 'middle', font: bold, maxWidth: '40%', leading: 1.3, glow: { intensity: 1, color: '#000000', radius: 10, opacity: 100, }, strokes: [{ width: 5, color: '#000000', lineJoin: 'round', lineCap: 'round', miterLimit: 4, }], shadows: [ { offsetX: 4, offsetY: 5, blur: 20, color: '#000000', opacity: 100, }, { offsetX: -8, offsetY: -4, blur: 20, color: '#FFFFFF', opacity: 90, }, ], x: '50%', y: '50%', styles: [{ start: 0, stop: 9, style: { font: thin, color: '#0000FF', casing: 'upper', } }] }) ); ``` -------------------------------- ### Manipulating AudioClips Source: https://context7.com/diffusionstudio/core-docs/llms.txt Demonstrates adding, splitting, trimming, and adjusting volume for AudioClips. Includes manual and sequential approaches, and automatic silence removal. ```typescript import * as core from '@diffusionstudio/core'; const composition = new core.Composition(); const source = await core.Source.from( 'https://diffusion-studio-public.s3.eu-central-1.amazonaws.com/audio/piano.mp3' ); // Manual approach const layer = await composition.add(new core.Layer()); const clip0 = await layer.add(new core.AudioClip(source)); // Split at 8 seconds: clip0 covers [0,8), clip1 covers [8, end) const clip1 = await clip0.split(8); clip0.range = [0.5, 4]; // trim source range in seconds clip0.delay = -0.5; // shift on timeline clip1.range = [14, 16]; clip1.delay = clip0.end - 14; // align with end of clip0 clip1.volume = 0.5; // 50% volume // Automated approach with sequential layer const seqLayer = await composition.add(new core.Layer({ mode: 'SEQUENTIAL' })); await seqLayer.add(new core.AudioClip(source)); await seqLayer.clips[0].split(8); seqLayer.clips[0].range = [0.5, 4]; seqLayer.clips[1].range = [14, 16]; // Remove silences automatically await clip0.removeSilences(); ``` -------------------------------- ### Configure Express for Diffusion Studio Core Headers Source: https://github.com/diffusionstudio/core-docs/blob/main/src/content/index.mdx Configure a custom Express server to set the Cross-Origin-Opener-Policy and Cross-Origin-Embedder-Policy headers for Diffusion Studio Core. ```javascript const express = require('express'); const app = express(); app.use((req, res, next) => { res.setHeader('Cross-Origin-Opener-Policy', 'same-origin'); res.setHeader('Cross-Origin-Embedder-Policy', 'credentialless'); next(); }); // ... rest of your app ``` -------------------------------- ### Render Video using showSaveFilePicker Source: https://github.com/diffusionstudio/core-docs/blob/main/src/content/rendering.mdx Initiate video rendering and export directly to a file using the browser's `showSaveFilePicker` API. This method is RAM-efficient for large videos. ```typescript const fileHandle = await window.showSaveFilePicker({ suggestedName: 'untitled_video.mp4', types: [ { description: 'Video File', accept: { 'video/mp4': ['.mp4'] }, }, ], }); await encoder.render(fileHandle); ``` -------------------------------- ### Animate Clip Properties with Keyframes Source: https://context7.com/diffusionstudio/core-docs/llms.txt Shows how to animate properties like position, size, and color using keyframes. Supports configurable easing functions and extrapolation modes. Requires '@diffusionstudio/core' import. ```typescript import * as core from '@diffusionstudio/core'; const composition = new core.Composition(); const layer = await composition.add(new core.Layer()); await layer.add(new core.RectangleClip({ animations: [ { key: 'x', easing: 'ease-in-out', extrapolate: 'clamp', // values outside range are clamped frames: [ { time: 2.5, value: 960 }, // at 2.5s, x=960 { time: 4.0, value: 50 }, // at 4.0s, x=50 ], }, { key: 'width', easing: 'ease-out', frames: [ { time: 0, value: 0 }, { time: 1.5, value: 1000 }, { time: 4.0, value: 60 }, ], }, { key: 'height', easing: 'ease-in', frames: [ { time: 0, value: 0 }, { time: 1.5, value: 700 }, { time: 4.0, value: 40 }, ], }, { key: 'fill', // linear interpolation by default frames: [ { time: 0, value: '#FF0000' }, // red { time: 4, value: '#00FF00' }, // green ], }, ], })); ``` -------------------------------- ### Create and Apply EllipseMask Source: https://github.com/diffusionstudio/core-docs/blob/main/src/content/masks.mdx Demonstrates creating a circular EllipseMask centered within the composition and applying it to a VideoClip. The width and height parameters define the diameter. ```typescript // Create a circular mask const circularMask = new core.EllipseMask({ x: composition.width / 2, y: composition.height / 2, width: 800, height: 800, }); // Apply to a video clip const videoClip = new core.VideoClip(source, { position: 'center', mask: circularMask, height: '100%', }); ``` -------------------------------- ### Create a PolygonClip Source: https://github.com/diffusionstudio/core-docs/blob/main/src/content/clips/shapes.mdx PolygonClip generates regular polygons with a configurable number of sides. Useful for triangles, hexagons, etc. Supports positioning, sizing, and rotation. ```typescript const polygon = new core.PolygonClip({ x: '50%', y: '50%', fill: '#FFFF00', sides: 6, // Creates a hexagon width: 200, height: 200, rotation: 30, // Optional rotation }); ``` -------------------------------- ### Apply Transitions Between Clips Source: https://context7.com/diffusionstudio/core-docs/llms.txt Illustrates how to apply transitions between sequential clips within a layer. The transition is defined on the first clip and automatically overlaps with the next. Ensure '@diffusionstudio/core' is imported. ```typescript import * as core from '@diffusionstudio/core'; const composition = new core.Composition(); const layer = await composition.add(new core.Layer({ mode: 'SEQUENTIAL' })); await layer.add( new core.ImageClip('/image1.png', { height: '100%', duration: 3, transition: { duration: 1, type: 'dissolve', // fade between clips }, }) ); await layer.add( new core.ImageClip('/image2.png', { height: '100%', duration: 3, }) ); // Supported transition types: // 'dissolve' – smooth crossfade // 'slide-from-right' // 'slide-from-left' // 'fade-to-black' // 'fade-to-white' ``` -------------------------------- ### Create a New Layer Source: https://github.com/diffusionstudio/core-docs/blob/main/src/content/layers.mdx Instantiate a new Layer object. It's recommended to add layers to the composition before adding clips to them. ```typescript import * as core from '@diffusionstudio/core'; const layer = new core.Layer(); ```