### Install Development Dependencies Source: https://github.com/stability-ai/stable-virtual-camera/blob/main/docs/INSTALL.md Installs optional dependencies for development, including pre-commit hooks. ```bash # [OPTIONAL] Install seva dependencies for development. pip install -e ".[dev]" pre-commit install ``` -------------------------------- ### Install Demo Dependencies Source: https://github.com/stability-ai/stable-virtual-camera/blob/main/docs/INSTALL.md Installs dependencies needed for the command-line and Gradio demos, including pycolmap and dust3r. ```bash # Initialize and update submodules for demo. git submodule update --init --recursive # Install pycolmap dependencies for cli and gradio demo (our model is not dependent on it). echo "Installing pycolmap (for both cli and gradio demo)..." pip install git+https://github.com/jensenz-sai/pycolmap@543266bc316df2fe407b3a33d454b310b1641042 # Install dust3r dependencies for gradio demo (our model is not dependent on it). echo "Installing dust3r dependencies (only for gradio demo)..." pushd third_party/dust3r pip install -r requirements.txt popd ``` -------------------------------- ### Install Model Dependencies Source: https://github.com/stability-ai/stable-virtual-camera/blob/main/docs/INSTALL.md Installs the core dependencies required for the Stable Virtual Camera model using pip. ```bash # Install seva model dependencies. pip install -e . ``` -------------------------------- ### Example img2img run on dl3d140-165f5af8bfe32f70595a1c9393a6e442acf7af019998275144f605b89a306557 Source: https://github.com/stability-ai/stable-virtual-camera/blob/main/docs/CLI_USAGE.md This is a concrete example of running the img2img command on a specific dataset instance ('dl3d140-165f5af8bfe32f70595a1c9393a6e442acf7af019998275144f605b89a306557') with 3 input views, demonstrating practical application of the parameters. ```bash python demo.py \ --data_path /path/to/assets_demo_cli/ \ --data_items dl3d140-165f5af8bfe32f70595a1c9393a6e442acf7af019998275144f605b89a306557 \ --num_inputs 3 \ --video_save_fps 10 ``` -------------------------------- ### Install PyTorch Nightly for Speed-up Source: https://github.com/stability-ai/stable-virtual-camera/blob/main/docs/INSTALL.md Installs the PyTorch nightly version to leverage torch.compile for faster JIT compilation, potentially doubling sampling speed. Users should adjust the CUDA version in the command to match their system. ```bash # [OPTIONAL] Install the torch nightly version for faster JIT via. torch.compile (speed up sampling by 2x in our testing). # Please adjust to your own cuda version. For example, if you have cuda 11.8, use the following command. pip install --pre torch --index-url https://download.pytorch.org/whl/nightly/cu118 ``` -------------------------------- ### Run CLI Demo Source: https://github.com/stability-ai/stable-virtual-camera/blob/main/docs/CLI_USAGE.md Executes the CLI demo script with specified data path and additional arguments. This is the primary command to start the demo. ```bash python demo.py --data_path [additional arguments] ``` -------------------------------- ### Install Stable Virtual Camera Source: https://github.com/stability-ai/stable-virtual-camera/blob/main/README.md Clones the repository and installs the Stable Virtual Camera package. Requires Python 3.10+ and PyTorch 2.6.0+. ```bash git clone --recursive https://github.com/Stability-AI/stable-virtual-camera cd stable-virtual-camera pip install -e . ``` -------------------------------- ### Run img2img for single-view conditioning with camera scale sweep Source: https://github.com/stability-ai/stable-virtual-camera/blob/main/docs/CLI_USAGE.md This command illustrates the setup for single-view conditioning tests, where a range of camera scales are applied to find optimal parameters. It also shows how to adjust the 'cfg' parameter for datasets like RealEstate10K to improve results. ```bash python demo.py \ --data_path \ --num_inputs

\ --video_save_fps 10 \ --camera_scale \ --cfg 6.0 ``` -------------------------------- ### Data Conversion Script Example Source: https://github.com/stability-ai/stable-virtual-camera/blob/main/benchmark/README.md Provides an example script for converting existing academic datasets into the 'reconfusion' format, which is compatible with ReconfusionParser. This script is useful for preparing data for the Stable Virtual Camera project. ```Python import os def convert_to_reconfusion(dataset_path, output_path): # Placeholder for conversion logic print(f"Converting dataset from {dataset_path} to {output_path}") # Example: Copying files and creating necessary metadata pass # Example usage: # convert_to_reconfusion('path/to/academic/dataset', 'path/to/reconfusion/format') ``` -------------------------------- ### Run img2img for semi-dense-view regime with extended T Source: https://github.com/stability-ai/stable-virtual-camera/blob/main/docs/CLI_USAGE.md This example configures the img2img process for semi-dense-view scenarios on datasets like DL3DV-140 and Tanks and Temples. It involves extending the 'T' parameter to accommodate all input and target views in a single forward pass. ```bash python demo.py \ --data_path \ --num_inputs 32 \ --video_save_fps 10 \ --T 90 ``` -------------------------------- ### Run img2img with trajectory prior and nearest-gt chunking Source: https://github.com/stability-ai/stable-virtual-camera/blob/main/docs/CLI_USAGE.md This example shows how to leverage trajectory priors for improved 3D consistency, specifically using the 'orbit' prior for CO3D dataset with the 'nearest-gt' chunking strategy. This is beneficial when trajectory information is available. ```bash python demo.py \ --data_path \ --num_inputs

\ --video_save_fps 10 \ --use_traj_prior True \ --traj_prior orbit \ --chunking_strategy nearest-gt ``` -------------------------------- ### Run CLI Demo Source: https://github.com/stability-ai/stable-virtual-camera/blob/main/README.md Execute the command-line interface demo for the Stable Virtual Camera. This command allows for passing data paths and additional arguments for fine-grained control over the model, suitable for advanced users and researchers. ```bash python demo.py --data_path [additional arguments] ``` -------------------------------- ### Run Gradio Demo Source: https://github.com/stability-ai/stable-virtual-camera/blob/main/README.md Launches the Gradio demo for Stable Virtual Camera, providing a GUI interface for users. ```python python demo_gr.py ``` -------------------------------- ### Run Gradio Demo Locally Source: https://github.com/stability-ai/stable-virtual-camera/blob/main/docs/GR_USAGE.md This command initiates the Gradio demo application locally, allowing users to interact with the Stable Virtual Camera project. ```bash python demo_gr.py ``` -------------------------------- ### Run img2img with specified data path and number of inputs Source: https://github.com/stability-ai/stable-virtual-camera/blob/main/docs/CLI_USAGE.md This command demonstrates the basic usage of the img2img functionality. It requires a data path and optionally the number of inputs if multiple train_test_split_*.json files are present. Additional parameters can be set for trajectory priors, chunking strategies, and camera scales depending on the dataset and desired output quality. ```bash python demo.py \ --data_path \ --num_inputs

\ --video_save_fps 10 ``` -------------------------------- ### Hugging Face Authentication Source: https://github.com/stability-ai/stable-virtual-camera/blob/main/README.md Logs into Hugging Face CLI to authenticate and download model weights. This is necessary for the Stable Virtual Camera to function. ```bash # This will prompt you to enter your Hugging Face credentials. huggingface-cli login ``` -------------------------------- ### Run img2img for ViewCrafter split with extended T and L_short Source: https://github.com/stability-ai/stable-virtual-camera/blob/main/docs/CLI_USAGE.md This command is tailored for the ViewCrafter split, optimizing for original image resolutions by extending 'T' and setting 'L_short'. This approach is beneficial for datasets like RealEastate10K, CO3D, and Tanks and Temples. ```bash python demo.py \ --data_path \ --num_inputs

\ --video_save_fps 10 \ --T 25 \ --L_short 576 ``` -------------------------------- ### Image Preprocessing and Postprocessing Source: https://github.com/stability-ai/stable-virtual-camera/blob/main/benchmark/README.md Details the image preprocessing steps applied to model inputs and postprocessing steps applied to model outputs for evaluation and benchmarking. These operations ensure data consistency and prepare samples for metric computation. ```General Image Preprocessing: resize the shortest side to 576 (--L_short 576), center crop Image Postprocessing: center crop to 576 ``` -------------------------------- ### Benchmark Dataset Details Source: https://github.com/stability-ai/stable-virtual-camera/blob/main/benchmark/README.md This snippet outlines the structure and content of the benchmark dataset used for evaluating NVS models. It details various datasets, their splits, file paths, content descriptions, and image processing steps. ```Markdown # :bar_chart: Benchmark We provide in this release (`benchmark.zip`) with the following 17 entries as a benchmark to evaluate NVS models. We hope this will help standardize the evaluation of NVS models and facilitate fair comparison between different methods. ``` -------------------------------- ### Run img2vid for ViewCrafter split with extended T and L_short Source: https://github.com/stability-ai/stable-virtual-camera/blob/main/docs/CLI_USAGE.md Similar to img2img, this command configures img2vid for the ViewCrafter split by extending 'T' and setting 'L_short' to match original image resolutions, ensuring better performance on datasets like RealEastate10K, CO3D, and Tanks and Temples. ```bash python demo.py \ --data_path \ --task img2vid \ --replace_or_include_input True \ --num_inputs

\ --use_traj_prior True \ --chunk_strategy interp \ --T 25 \ --L_short 576 ``` -------------------------------- ### Cite Stable Virtual Camera Source: https://github.com/stability-ai/stable-virtual-camera/blob/main/README.md BibTeX entry for citing the Stable Virtual Camera project, which utilizes diffusion models for generative view synthesis. This citation is useful for academic and research purposes. ```bibtex @article{zhou2025stable, title={Stable Virtual Camera: Generative View Synthesis with Diffusion Models}, author={Jensen (Jinghao) Zhou and Hang Gao and Vikram Voleti and Aaryaman Vasishta and Chun-Han Yao and Mark Boss and Philip Torr and Christian Rupprecht and Varun Jampani }, journal={arXiv preprint arXiv:2503.14489}, year={2025} } ``` -------------------------------- ### Run img2trajvid in sparse-view regime with specific cfg and chunking Source: https://github.com/stability-ai/stable-virtual-camera/blob/main/docs/CLI_USAGE.md This command demonstrates the img2trajvid task in a sparse-view regime (P<=8). It specifies 'cfg' values for multiple passes and uses 'interp-gt' as the chunking strategy, with an option for 'gt-nearest' for larger numbers of input frames. ```bash python demo.py \ --data_path \ --task img2trajvid \ --num_inputs

\ --cfg 3.0,2.0 \ --use_traj_prior True \ --chunk_strategy interp-gt ``` -------------------------------- ### Data Format for img2trajvid_s-prob Source: https://github.com/stability-ai/stable-virtual-camera/blob/main/docs/CLI_USAGE.md Specifies the directory structure for the 'img2trajvid_s-prob' task, which requires a single input image. ```bash / ├── scene_1.png ├── scene_2.png └── scene_3.png ``` -------------------------------- ### Generate Trajectory Video from Images (Bash) Source: https://github.com/stability-ai/stable-virtual-camera/blob/main/docs/CLI_USAGE.md This bash command executes the `demo.py` script to generate a trajectory video from input images. It requires specifying the data path, task as 'img2trajvid', the number of input views, and configuration parameters like chunking strategy and sequence length. ```bash python demo.py \ --data_path \ --task img2trajvid \ --num_inputs

\ --cfg 3.0 \ --L_short 576 \ --use_traj_prior True \ --chunk_strategy interp ``` -------------------------------- ### Run img2vid with input view replacement and interpolation chunking Source: https://github.com/stability-ai/stable-virtual-camera/blob/main/docs/CLI_USAGE.md This command utilizes the img2vid task, which requires replacing or including input views to form a complete trajectory. It uses the 'interp' chunking strategy by default and can be configured with trajectory priors and extended 'T' values for specific datasets. ```bash python demo.py \ --data_path \ --task img2vid \ --replace_or_include_input True \ --num_inputs

\ --use_traj_prior True \ --chunk_strategy interp ``` -------------------------------- ### Data Format for Other Tasks Source: https://github.com/stability-ai/stable-virtual-camera/blob/main/docs/CLI_USAGE.md Defines the directory structure for tasks other than 'img2trajvid_s-prob'. It includes scene subdirectories with images, transforms.json, and train_test_split files. ```bash / ├── scene_1/ ├── train_test_split_1.json # for single-view regime ├── train_test_split_6.json # for sparse-veiw regime ├── train_test_split_32.json # for semi-dense-view regime ├── transforms.json └── images/ ├── image_0.png ├── image_1.png ├── ... └── image_1000.png ├── scene_2 └── scene_3 ``` -------------------------------- ### Run img2trajvid_s-prob with orbit prior and adjusted parameters Source: https://github.com/stability-ai/stable-virtual-camera/blob/main/docs/CLI_USAGE.md This command showcases the img2trajvid_s-prob task, utilizing an 'orbit' trajectory prior. It includes adjustments for 'cfg', 'guider', 'num_targets', and 'L_short', along with options to tune 'camera_scale' for panning motions. ```bash python demo.py \ --data_path \ --task img2trajvid_s-prob \ --replace_or_include_input True \ --traj_prior orbit \ --cfg 4.0,2.0 \ --guider 1,2 \ --num_targets 111 \ --L_short 576 \ --use_traj_prior True \ --chunk_strategy interp ``` -------------------------------- ### Configure Procedural Sampling Source: https://github.com/stability-ai/stable-virtual-camera/blob/main/docs/CLI_USAGE.md Enables procedural sampling by setting specific flags. This is recommended for controlling the sampling process. ```bash python demo.py --use_traj_prior True --chunk_strategy ``` -------------------------------- ### Set Custom Resolution Source: https://github.com/stability-ai/stable-virtual-camera/blob/main/docs/CLI_USAGE.md Overrides default image preprocessing to set a custom width and height for input and output images. Alternatively, setting `--L_short` maintains the aspect ratio. ```bash python demo.py --W --H ``` ```bash python demo.py --L_short 576 ``` === COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.

Dataset Split Path Content Image Preprocessing Image Postprocessing
OmniObject3D S (SV3D), O (Ours) omniobject3d train_test_split_*.json center crop to 576 \
GSO S (SV3D), O (Ours) gso train_test_split_*.json center crop to 576 \
RealEstate10K D (4DiM) re10k-4dim train_test_split_*.json center crop to 576 resize to 256
R (ReconFusion) re10k train_test_split_*.json center crop to 576 \
P (pixelSplat) re10k-pixelsplat train_test_split_*.json center crop to 576 resize to 256
V (ViewCrafter) re10k-viewcrafter images/*.png,transforms.json,train_test_split_*.json resize the shortest side to 576 (--L_short 576) center crop
LLFF R (ReconFusion) llff train_test_split_*.json center crop to 576 \
DTU R (ReconFusion) dtu train_test_split_*.json center crop to 576 \
CO3D R (ReconFusion) co3d train_test_split_*.json center crop to 576 \
V (ViewCrafter) co3d-viewcrafter images/*.png,transforms.json,train_test_split_*.json resize the shortest side to 576 (--L_short 576) center crop
WildRGB-D Oₑ (Ours, easy) wildgbd/easy train_test_split_*.json center crop to 576 \
Oₕ (Ours, hard) wildgbd/hard train_test_split_*.json center crop to 576 \
Mip-NeRF360 R (ReconFusion) mipnerf360 train_test_split_*.json center crop to 576 \
DL3DV-140