### Install Project Dependencies Source: https://github.com/sudo-ai-3d/zero123plus/blob/main/README.md Installs all necessary dependencies for the project from the requirements.txt file. This is a prerequisite for running the local demo. ```bash pip install -r requirements.txt ``` -------------------------------- ### Run Streamlit Demo Locally Source: https://github.com/sudo-ai-3d/zero123plus/blob/main/README.md Launches the Streamlit-based interactive demo application. Ensure dependencies are installed first. ```bash streamlit run app.py ``` -------------------------------- ### Run Gradio Demo Locally Source: https://github.com/sudo-ai-3d/zero123plus/blob/main/README.md Launches the Gradio-based interactive demo application. Ensure dependencies are installed first. ```python python gradio_app.py ``` -------------------------------- ### Generate Multi-View Images with Zero123++ Source: https://github.com/sudo-ai-3d/zero123plus/blob/main/README.md Use this pipeline to generate multi-view images from a single input image. Ensure you have torch, diffusers, and transformers installed. The recommended diffusers version is 0.20.2. This example requires approximately 5GB of VRAM. Input images should be square with a resolution of 320x320 or higher. ```python import torch import requests from PIL import Image from diffusers import DiffusionPipeline, EulerAncestralDiscreteScheduler # Load the pipeline pipeline = DiffusionPipeline.from_pretrained( "sudo-ai/zero123plus-v1.1", custom_pipeline="sudo-ai/zero123plus-pipeline", torch_dtype=torch.float16 ) # Feel free to tune the scheduler! # `timestep_spacing` parameter is not supported in older versions of `diffusers` # so there may be performance degradations # We recommend using `diffusers==0.20.2` pipeline.scheduler = EulerAncestralDiscreteScheduler.from_config( pipeline.scheduler.config, timestep_spacing='trailing' ) pipeline.to('cuda:0') # Download an example image. cond = Image.open(requests.get("https://d.skis.ltd/nrp/sample-data/lysol.png", stream=True).raw) # Run the pipeline! result = pipeline(cond, num_inference_steps=75).images[0] # for general real and synthetic images of general objects # usually it is enough to have around 28 inference steps # for images with delicate details like faces (real or anime) # you may need 75-100 steps for the details to construct result.show() result.save("output.png") ``` -------------------------------- ### Remove Background with rembg Source: https://github.com/sudo-ai-3d/zero123plus/blob/main/README.md After generating an image, you can optionally remove the gray background using the `rembg` library. Install it using `pip install rembg`. ```python # !pip install rembg import rembg result = rembg.remove(result) result.show() ``` -------------------------------- ### Matting Postprocess Example Source: https://github.com/sudo-ai-3d/zero123plus/blob/main/README.md This script shows how to use the generated normal images for matting postprocessing. It helps obtain a more accurate mask compared to SAM-based approaches. ```python import torch from diffusers import Zero123plusPipeline pipe = Zero123plusPipeline.from_pretrained( "sudo-ai/zero123plus", torch_dtype=torch.float16 ) pipe.load_lora_weights("sudo-ai/zero123plus-lora-normal", weight_name="zero123plus-lora-normal.safetensors") pipe.to("cuda") # Load your image init_image = load_image("burger.png").convert("RGB") # Generate normal image normal_image = pipe( init_image, num_inference_steps=25, guidance_scale=3.0, controlnet_conditioning_scale=0.8, camera_params={ "intrinsics": [1000, 1000, 512, 512], "extrinsics": [0, 0, 0, 0, 0, 0, 1], }, output_type="pil", ).images[0] # Use the normal image to generate a mask # This part would involve a separate matting model or function # For demonstration, let's assume we have a function `generate_mask_from_normals` # mask = generate_mask_from_normals(normal_image) # The actual matting postprocess logic would go here. # This example snippet focuses on the setup and normal generation part. # For a complete matting example, refer to the full script in the repository. print("Normal image generated. Matting postprocess logic to be implemented.") ``` -------------------------------- ### Normal Generator Example Source: https://github.com/sudo-ai-3d/zero123plus/blob/main/README.md This script demonstrates the usage of the normal generator ControlNet. It can be used to generate view-space normal images, which can then be used to obtain more accurate masks. ```python import torch from diffusers import Zero123plusPipeline pipe = Zero123plusPipeline.from_pretrained( "sudo-ai/zero123plus", torch_dtype=torch.float16 ) pipe.load_lora_weights("sudo-ai/zero123plus-lora-normal", weight_name="zero123plus-lora-normal.safetensors") pipe.to("cuda") # Load your image init_image = load_image("burger.png").convert("RGB") # Generate normal image # The normal generator is a ControlNet, so we pass it to the pipeline # The controlnet_conditioning_scale controls the strength of the controlnet # The camera_params are the intrinsics and extrinsics of the camera # The output_type can be "pil" or "np" (numpy array) output = pipe( init_image, num_inference_steps=25, guidance_scale=3.0, controlnet_conditioning_scale=0.8, camera_params={ "intrinsics": [1000, 1000, 512, 512], "extrinsics": [0, 0, 0, 0, 0, 0, 1], }, output_type="pil", ).images[0] output.save("burger-normal.png") ``` -------------------------------- ### Generate Images with Depth ControlNet Source: https://github.com/sudo-ai-3d/zero123plus/blob/main/README.md This example demonstrates how to use the depth ControlNet with Zero123++ for image generation. It requires approximately 5.7GB of VRAM. Ensure you have the necessary models loaded and the conditioning scale set appropriately. ```python import torch import requests from PIL import Image from diffusers import DiffusionPipeline, EulerAncestralDiscreteScheduler, ControlNetModel # Load the pipeline pipeline = DiffusionPipeline.from_pretrained( "sudo-ai/zero123plus-v1.1", custom_pipeline="sudo-ai/zero123plus-pipeline", torch_dtype=torch.float16 ) pipeline.add_controlnet(ControlNetModel.from_pretrained( "sudo-ai/controlnet-zp11-depth-v1", torch_dtype=torch.float16 ), conditioning_scale=0.75) # Feel free to tune the scheduler pipeline.scheduler = EulerAncestralDiscreteScheduler.from_config( pipeline.scheduler.config, timestep_spacing='trailing' ) pipeline.to('cuda:0') # Run the pipeline cond = Image.open(requests.get("https://d.skis.ltd/nrp/sample-data/0_cond.png", stream=True).raw) depth = Image.open(requests.get("https://d.skis.ltd/nrp/sample-data/0_depth.png", stream=True).raw) result = pipeline(cond, depth_image=depth, num_inference_steps=36).images[0] result.show() result.save("output.png") ``` -------------------------------- ### Zero123++ Citation Source: https://github.com/sudo-ai-3d/zero123plus/blob/main/README.md BibTeX entry for citing the Zero123++ research paper. Include this in academic works that utilize the model. ```bibtex @misc{shi2023zero123plus, title={Zero123++: a Single Image to Consistent Multi-view Diffusion Base Model}, author={Ruoxi Shi and Hansheng Chen and Zhuoyang Zhang and Minghua Liu and Chao Xu and Xinyue Wei and Linghao Chen and Chong Zeng and Hao Su}, year={2023}, eprint={2310.15110}, archivePrefix={arXiv}, primaryClass={cs.CV} } ``` === COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.