### Clone SadTalker repository and setup Python environment

Source: https://github.com/opentalker/sadtalker/blob/main/README.md

Bash commands to clone the SadTalker repository and create a Python virtual environment with required dependencies. This is the foundational setup step for Linux/Unix systems before installing additional requirements.

```bash
git clone https://github.com/OpenTalker/SadTalker.git

cd SadTalker 

conda create -n sadtalker python=3.8

conda activate sadtalker

pip install torch==1.12.1+cu113 torchvision==0.13.1+cu113 torchaudio==0.12.1 --extra-index-url https://download.pytorch.org/whl/cu113

conda install ffmpeg

pip install -r requirements.txt
```

--------------------------------

### Running Local Gradio Demo with SadTalker

Source: https://github.com/opentalker/sadtalker/blob/main/README.md

This snippet starts a local Gradio instance for SadTalker, similar to the Hugging Face demo, allowing interactive audio-driven talking face animation. Requires manual installation of TTS library via pip install tts for advanced features; dependencies include pre-trained models in checkpoints folder. Inputs are audio and source images/videos; outputs are generated talking head videos; limitations include potential need for GPU and automatic requirements installation on Windows via bat file or Linux/Mac via sh script.

```bash
python app_sadtalker.py
```

```bash
webui.bat
```

```bash
bash webui.sh
```

--------------------------------

### Launch SadTalker Web UI with Bash Scripts

Source: https://context7.com/opentalker/sadtalker/llms.txt

These Bash commands launch the SadTalker web UI, with different scripts for Linux/Mac and Windows that handle automatic dependency installation. Requires the scripts and a compatible Python environment (3.8-3.11). Input is the script execution; it checks Python version, installs necessary components if missing, and launches the Gradio interface. No direct output, but starts a web server. Limitations include platform-specific scripts and potential need for manual intervention if PyTorch installation fails.

```bash
# Linux/Mac: Launch web UI with automatic dependency installation
bash webui.sh

# Windows: Launch web UI
webui.bat

# Both scripts will:
# 1. Check Python version (3.8-3.11 recommended)
# 2. Install PyTorch if not present
# 3. Install requirements
# 4. Launch Gradio interface
```

--------------------------------

### Setup Python Environment and Install Dependencies

Source: https://github.com/opentalker/sadtalker/blob/main/quick_demo.ipynb

Installs Python 3.8, updates alternatives, installs required system packages, clones the SadTalker repository, and installs Python dependencies including PyTorch with CUDA support.

```python
!update-alternatives --install /usr/local/bin/python3 python3 /usr/bin/python3.8 2
!update-alternatives --install /usr/local/bin/python3 python3 /usr/bin/python3.9 1
!sudo apt install python3.8

!sudo apt-get install python3.8-distutils

!python --version

!apt-get update

!apt install software-properties-common

!sudo dpkg --remove --force-remove-reinstreq python3-pip python3-setuptools python3-wheel

!apt-get install python3-pip

print('Git clone project and install requirements...')
!git clone https://github.com/Winfredy/SadTalker &> /dev/null
%cd SadTalker
!export PYTHONPATH=/content/SadTalker:$PYTHONPATH
!python3.8 -m pip install torch==1.12.1+cu113 torchvision==0.13.1+cu113 torchaudio==0.12.1 --extra-index-url https://download.pytorch.org/whl/cu113
!apt update
!apt install ffmpeg &> /dev/null
!python3.8 -m pip install -r requirements.txt
```

--------------------------------

### Download SadTalker pre-trained models

Source: https://github.com/opentalker/sadtalker/blob/main/README.md

Bash script execution to automatically download all required pre-trained models for SadTalker. This includes mapping networks and face rendering checkpoints necessary for video generation.

```bash
bash scripts/download_models.sh
```

--------------------------------

### Install SadTalker Dependencies with Bash

Source: https://context7.com/opentalker/sadtalker/llms.txt

This Bash script installs the necessary dependencies for SadTalker, including cloning the repository, setting up a Conda environment, installing PyTorch and FFmpeg, and downloading model checkpoints. It requires Conda and Git to be installed on the system. Input is none, but it assumes a Linux/Mac or Windows setup; output is a configured environment for running SadTalker. Limitations include manual download option if automatic script fails and specific Python version requirements.

```bash
# Clone repository
git clone https://github.com/OpenTalker/SadTalker.git
cd SadTalker

# Create conda environment
conda create -n sadtalker python=3.8
conda activate sadtalker

# Install PyTorch
pip install torch==1.12.1+cu113 torchvision==0.13.1+cu113 torchaudio==0.12.1 \
    --extra-index-url https://download.pytorch.org/whl/cu113

# Install FFmpeg
conda install ffmpeg

# Install requirements
pip install -r requirements.txt

# Optional: Install TTS for Gradio demo
pip install TTS

# Download model checkpoints
bash scripts/download_models.sh

# Or manually download from:
# - Google Drive: https://drive.google.com/file/d/1gwWh45pF7aelNP_P78uDJL8Sycep-K7j
# - GitHub Releases: https://github.com/OpenTalker/SadTalker/releases
```

--------------------------------

### Deploy with Cog Predictor API (Python)

Source: https://context7.com/opentalker/sadtalker/llms.txt

This code demonstrates how to initialize a `Predictor` object, which is required for deploying SadTalker as a containerized prediction service using Replicate's Cog.  The `setup()` method needs to be called.

```Python
from predict import Predictor
from pathlib import Path

# Initialize predictor
predictor = Predictor()
predictor.setup()
```

--------------------------------

### CLI Animation of Portrait Image in SadTalker

Source: https://github.com/opentalker/sadtalker/blob/main/README.md

This command animates a portrait image or video using audio input via SadTalker's inference script, saving results with timestamps. Depends on pre-trained models like ExpNet and Wav2Lip; requires source image/video and audio file as inputs. Outputs MP4 videos in results directory; supports enhancer like gfpgan for quality improvement, but may require GPU for efficiency.

```bash
python inference.py --driven_audio <audio.wav> \
                    --source_image <video.mp4 or picture.png> \
                    --enhancer gfpgan
```

--------------------------------

### CLI Full Body Video Generation in SadTalker

Source: https://github.com/opentalker/sadtalker/blob/main/README.md

Generates natural full-body videos from audio and source using --still and --preprocess full options for extended animation. Relies on same model checkpoints as portrait mode; inputs include audio, source, and optional result directory. Outputs enhanced MP4 videos; enhancer gfpgan improves quality, but processing time increases with full body mode and potential limitations on input resolution.

```bash
python inference.py --driven_audio <audio.wav> \
                    --source_image <video.mp4 or picture.png> \
                    --result_dir <a file to store results> \
                    --still \
                    --preprocess full \
                    --enhancer gfpgan
```

--------------------------------

### Select Source Image for Animation

Source: https://github.com/opentalker/sadtalker/blob/main/quick_demo.ipynb

Provides a widget interface to select a source image from the examples folder. Displays the selected image and allows users to choose which image to animate.

```python
# borrow from makeittalk
import ipywidgets as widgets
import glob
import matplotlib.pyplot as plt
print("Choose the image name to animate: (saved in folder 'examples/')")
img_list = glob.glob1('examples/source_image', '*.png')
img_list.sort()
img_list = [item.split('.')[0] for item in img_list]
default_head_name = widgets.Dropdown(options=img_list, value='full3')
def on_change(change):
    if change['type'] == 'change' and change['name'] == 'value':
        plt.imshow(plt.imread('examples/source_image/{}.png'.format(default_head_name.value)))
        plt.axis('off')
        plt.show()
default_head_name.observe(on_change)
display(default_head_name)
plt.imshow(plt.imread('examples/source_image/{}.png'.format(default_head_name.value)))
plt.axis('off')
plt.show()
```

--------------------------------

### Distributed Training Launch for SadTalker - Model Parallel and Partial FC

Source: https://github.com/opentalker/sadtalker/blob/main/src/face3d/models/arcface_torch/docs/speed_benchmark.md

These commands launch distributed training using PyTorch's torch.distributed.launch on 8 GPUs for testing performance with 3 million synthetic identities, mixed precision, ResNet50 backbone, and batch size 1024. The first uses model parallel via configs/3millions; the second uses Partial FC 0.1 via configs/3millions_pfc. No external dependencies beyond PyTorch and the project setup; inputs are config paths, outputs are training logs with speed, loss, and learning rate metrics. Limitations include requirement for multi-GPU setup and synthetic data generation.

```shell
# Model Parallel
python -m torch.distributed.launch --nproc_per_node=8 --nnodes=1 --node_rank=0 --master_addr="127.0.0.1" --master_port=1234 train.py configs/3millions
```

```shell
# Partial FC 0.1
python -m torch.distributed.launch --nproc_per_node=8 --nnodes=1 --node_rank=0 --master_addr="127.0.0.1" --master_port=1234 train.py configs/3millions_pfc
```

--------------------------------

### Launch Gradio Web Interface (Python)

Source: https://context7.com/opentalker/sadtalker/llms.txt

This code snippet explains how to launch the interactive SadTalker web UI using Gradio. It imports the `sadtalker_demo` function, initializes the demo with specified paths, queues the requests, and launches the interface.

```Python
from app_sadtalker import sadtalker_demo

# Launch Gradio interface
demo = sadtalker_demo(
    checkpoint_path='checkpoints',
    config_path='src/config'
)

demo.queue()
demo.launch(
    share=False,
    server_name='0.0.0.0',
    server_port=7860
)

# Access at http://localhost:7860
# Interface provides:
# - Image upload
# - Audio upload or TTS generation
# - Pose style selection (0-46)
# - Face model resolution (256/512)
# - Preprocess mode selection
# - Still mode toggle
# - Batch size configuration
# - GFPGAN face enhancement option
```

--------------------------------

### Generate Talking Head Video Using SadTalker Python API (Python)

Source: https://context7.com/opentalker/sadtalker/llms.txt

Instantiate the SadTalker class with checkpoint and configuration paths, then call the test method to produce a talking‑head video. Provides fine‑grained control over preprocessing, pose style, batch size, and optional enhancer or blink usage. Returns the path to the generated video file.

```python
from src.gradio_demo import SadTalker

# Initialize with checkpoint and config paths
sad_talker = SadTalker(
    checkpoint_path='checkpoints',
    config_path='src/config',
    lazy_load=True
)

# Generate talking head video
result_video = sad_talker.test(
    source_image='portrait.png',
    driven_audio='speech.wav',
    preprocess='crop',
    still_mode=False,
    use_enhancer=True,
    batch_size=2,
    size=256,
    pose_style=0,
    exp_scale=1.0,
    use_blink=True,
    result_dir='./results/'
)
print(f"Generated video saved at: {result_video}")

# Full body animation example
result = sad_talker.test(
    source_image='full_body_portrait.jpg',
    driven_audio='audio.wav',
    preprocess='full',
    still_mode=True,
    use_enhancer=True,
    size=512,
    exp_scale=1.0
)

# Using reference video for pose and blink
result = sad_talker.test(
    source_image='photo.png',
    driven_audio='voice.wav',
    use_ref_video=True,
    ref_video='reference.mp4',
    ref_info='pose+blink',
    batch_size=4
)
```

--------------------------------

### Animate Face from Coefficients (Python)

Source: https://context7.com/opentalker/sadtalker/llms.txt

This code snippet showcases animating a face from motion coefficients and a source image using `AnimateFromCoeff`. It sets up paths, initializes the animator, prepares rendering data using `get_facerender_data`, and generates the final video output.  Options for enhancers and preprocess settings are included.

```Python
from src.facerender.animate import AnimateFromCoeff
from src.generate_facerender_batch import get_facerender_data
import torch

device = "cuda" if torch.cuda.is_available() else "cpu"

sadtalker_paths = {
    'checkpoint': 'checkpoints/SadTalker_V0.0.2_256.safetensors',
    'use_safetensor': True,
    'free_view_checkpoint': 'checkpoints/facevid2vid.pth',
    'path_of_lm3d': 'checkpoints/BFM_Fitting/similarity_Lm3D_all.mat',
    'dir_of_BFM_fitting': 'checkpoints/BFM_Fitting'
}

# Initialize animator
animator = AnimateFromCoeff(sadtalker_paths, device)

# Prepare rendering data
data = get_facerender_data(
    coeff_path='results/portrait##audio.mat',
    crop_pic_path='results/first_frame_dir/portrait.png',
    first_coeff_path='results/first_frame_dir/portrait.mat',
    audio_path='audio.wav',
    batch_size=2,
    input_yaw_list=None,
    input_pitch_list=None,
    input_roll_list=None,
    expression_scale=1.0,
    still_mode=False,
    preprocess='crop',
    size=256
)

# Generate final video
result_path = animator.generate(
    data=data,
    save_dir='./results',
    pic_path='portrait.jpg',
    crop_info=crop_info,
    enhancer='gfpgan',
    background_enhancer='realesrgan',
    preprocess='crop',
    img_size=256
)
print(f"Video generated at: {result_path}")
```

--------------------------------

### Process Batch Data (Python)

Source: https://context7.com/opentalker/sadtalker/llms.txt

This code demonstrates preparing audio features and coefficients for batch neural network processing. It utilizes the `get_data` function to generate data with various configurations, including eye blinking, idle mode, and using a reference video for blinks.

```Python
from src.generate_batch import get_data
import torch

device = "cuda" if torch.cuda.is_available() else "cpu"

# Generate batch data with eye blinking
batch = get_data(
    first_coeff_path='results/portrait.mat',
    audio_path='speech.wav',
    device=device,
    ref_eyeblink_coeff_path=None,
    still=False,
    idlemode=False,
    use_blink=True
)

# Returns dictionary with:
# - indiv_mels: mel spectrogram features (batch_size, num_frames, 1, 80, 16)
# - ref: reference 3DMM coefficients (batch_size, 1, 70)
# - num_frames: total number of frames
# - ratio_gt: eye blink ratio per frame
# - audio_name: name of audio file
# - pic_name: name of image file

# Idle mode for silent video
idle_batch = get_data(
    first_coeff_path='portrait.mat',
    audio_path='dummy.wav',
    device=device,
    ref_eyeblink_coeff_path=None,
    still=True,
    idlemode=True,
    length_of_audio=5.0,  # 5 seconds
    use_blink=True
)

# Using reference video for eye blinks
batch_with_ref = get_data(
    first_coeff_path='portrait.mat',
    audio_path='audio.wav',
    device=device,
    ref_eyeblink_coeff_path='ref_video.mat',
    still=False,
    use_blink=False  # disable random blinks, use reference
)
```

--------------------------------

### Download Pre-trained Models

Source: https://github.com/opentalker/sadtalker/blob/main/quick_demo.ipynb

Downloads the pre-trained models required for SadTalker to generate animations. This script removes any existing checkpoints and downloads fresh model files.

```python
print('Download pre-trained models...')
!rm -rf checkpoints
!bash scripts/download_models.sh
```

--------------------------------

### Display Generated Animation

Source: https://github.com/opentalker/sadtalker/blob/main/quick_demo.ipynb

Visualizes the generated animation by reading the output MP4 file, encoding it in base64, and displaying it in an HTML video player within the notebook.

```python
# visualize code from makeittalk
from IPython.display import HTML
from base64 import b64encode
import os, sys

# get the last from results

results = sorted(os.listdir('./results/'))

mp4_name = glob.glob('./results/*.mp4')[0]

mp4 = open('{}'.format(mp4_name),'rb').read()
data_url = "data:video/mp4;base64," + b64encode(mp4).decode()

print('Display animation: {}'.format(mp4_name), file=sys.stderr)
display(HTML("""
  <video width=256 controls>
        <source src="%s" type="video/mp4">
  </video>
  """ % data_url))
```

--------------------------------

### Generate Talking Head Video with Python API

Source: https://context7.com/opentalker/sadtalker/llms.txt

This Python code demonstrates how to use the SadTalker predictor to generate talking head animations from a source image and driven audio. It requires the SadTalker library dependencies, including PyTorch and model checkpoints, with inputs being image and audio paths, and optional references for eyeblink and pose. Outputs a Path to the generated MP4 video. Limitations include dependency on pre-installed models and GPU for optimal performance.

```python
# Run prediction
output = predictor.predict(
    source_image=Path("portrait.jpg"),
    driven_audio=Path("speech.wav"),
    enhancer="gfpgan",
    preprocess="full",
    ref_eyeblink=None,
    ref_pose=None,
    still=True
)

# Returns Path to generated MP4 video
print(f"Output video: {output}")

# With reference videos
output = predictor.predict(
    source_image=Path("photo.png"),
    driven_audio=Path("audio.mp3"),
    enhancer="RestoreFormer",
    preprocess="crop",
    ref_eyeblink=Path("blink_ref.mp4"),
    ref_pose=Path("pose_ref.mp4"),
    still=False
)
```

--------------------------------

### Generate Talking Face Video via Command-Line Interface (Bash)

Source: https://context7.com/opentalker/sadtalker/llms.txt

Run the provided inference script from the terminal with various options to animate a portrait using an audio file. Supports full‑body animation, still mode, custom preprocessing, and optional GFPGAN enhancement. Outputs the generated video to the specified results directory.

```bash
# Basic usage: animate a portrait with audio
python inference.py --driven_audio examples/driven_audio/bus_chinese.wav \
                    --source_image examples/source_image/full_body_1.png \
                    --enhancer gfpgan \
                    --result_dir ./results

# Full body animation with still mode
python inference.py --driven_audio audio.wav \
                    --source_image portrait.png \
                    --still \
                    --preprocess full \
                    --enhancer gfpgan \
                    --size 512 \
                    --expression_scale 1.2

# Advanced control with reference video for pose
python inference.py --driven_audio speech.wav \
                    --source_image face.jpg \
                    --ref_pose reference_video.mp4 \
                    --pose_style 5 \
                    --batch_size 4 \
                    --preprocess crop

# Custom head pose angles
python inference.py --driven_audio audio.wav \
                    --source_image image.png \
                    --input_yaw 10 20 15 \
                    --input_pitch 5 -5 0 \
                    --input_roll 0 0 0
```

--------------------------------

### Convert Audio to 3D Facial Motion Coefficients with Audio2Coeff (Python)

Source: https://context7.com/opentalker/sadtalker/llms.txt

Leverages the Audio2Coeff module to transform an audio track into 3D facial motion coefficients (expression and pose). Requires configuration YAML files, a model checkpoint, and optional reference blink or pose data. The generated coefficients are saved to a directory for later rendering.

```python
from src.test_audio2coeff import Audio2Coeff
from src.generate_batch import get_data
import torch

device = "cuda" if torch.cuda.is_available() else "cpu"

# Initialize audio-to-coefficient converter
sadtalker_paths = {
    'audio2pose_yaml_path': 'src/config/auido2pose.yaml',
    'audio2exp_yaml_path': 'src/config/auido2exp.yaml',
    'checkpoint': 'checkpoints/SadTalker_V0.0.2_256.safetensors',
    'use_safetensor': True
}

audio_to_coeff = Audio2Coeff(sadtalker_paths, device)

# Prepare batch data from first frame and audio
batch = get_data(
    first_coeff_path='results/first_frame_dir/portrait.mat',
    audio_path='audio.wav',
    device=device,
    ref_eyeblink_coeff_path=None,
    still=False
)

# Generate motion coefficients
coeff_path = audio_to_coeff.generate(
    batch=batch,
    coeff_save_dir='./results',
    pose_style=0,
    ref_pose_coeff_path=None
)
print(f"Coefficients saved to: {coeff_path}")
```

--------------------------------

### Generate Talking Face Animation

Source: https://github.com/opentalker/sadtalker/blob/main/quick_demo.ipynb

Runs the inference script to generate a talking face animation using the selected source image and a predefined audio file. Uses GFPGAN for result enhancement.

```python
# selected audio from exmaple/driven_audio
img = 'examples/source_image/{}.png'.format(default_head_name.value)
print(img)
!python3.8 inference.py --driven_audio ./examples/driven_audio/RD_Radio31_000.wav \
           --source_image {img} \
           --result_dir ./results --still --preprocess full --enhancer gfpgan
```

--------------------------------

### Crop and Extract 3D Face Coefficients (Python)

Source: https://context7.com/opentalker/sadtalker/llms.txt

This code snippet demonstrates how to extract 3D face model coefficients from input images or videos using the `CropAndExtract` class.  It loads necessary paths, initializes the preprocessor, and generates coefficients for both a single image and a reference video. Different preprocessing modes (crop, extcrop, resize, full, extfull) are available.

```Python
from src.utils.preprocess import CropAndExtract
import torch

device = "cuda" if torch.cuda.is_available() else "cpu"

sadtalker_paths = {
    'checkpoint': 'checkpoints/SadTalker_V0.0.2_256.safetensors',
    'use_safetensor': True,
    'dir_of_BFM_fitting': 'checkpoints/BFM_Fitting'
}

# Initialize preprocessor
preprocessor = CropAndExtract(sadtalker_paths, device)

# Extract 3DMM coefficients from image
coeff_path, crop_pic_path, crop_info = preprocessor.generate(
    input_path='portrait.jpg',
    save_dir='./results/first_frame_dir',
    crop_or_resize='crop',
    source_image_flag=True,
    pic_size=256
)

# Process video for reference pose
ref_coeff_path, _, _ = preprocessor.generate(
    input_path='reference_video.mp4',
    save_dir='./results/ref_video',
    crop_or_resize='full',
    source_image_flag=False,
    pic_size=512
)

# Different preprocessing modes
# crop: crops face region
# extcrop: extended crop with more context
# resize: resize entire image
# full: full image/body animation
# extfull: extended full mode
```

--------------------------------

### Inference with PyTorch

Source: https://github.com/opentalker/sadtalker/blob/main/src/face3d/models/arcface_torch/docs/eval.md

Runs inference using a PyTorch model. Specifies the model weight and network architecture.

```shell
python inference.py --weight ms1mv3_arcface_r50/backbone.pth --network r50
```

--------------------------------

### Check CUDA GPU Availability

Source: https://github.com/opentalker/sadtalker/blob/main/quick_demo.ipynb

Verifies that CUDA is available and displays GPU memory information. This is essential for running the deep learning models that power SadTalker's animation capabilities.

```python
!nvidia-smi --query-gpu=name,memory.total,memory.free --format=csv,noheader
```

--------------------------------

### Eval IJBC With Pytorch

Source: https://github.com/opentalker/sadtalker/blob/main/src/face3d/models/arcface_torch/docs/eval.md

Evaluates the IJBC dataset using a PyTorch model. Supports multi-GPU, specifies model prefix, image path, result directory, batch size, job name, target, and network architecture.

```shell
CUDA_VISIBLE_DEVICES=0,1 python eval_ijbc.py \
--model-prefix ms1mv3_arcface_r50/backbone.pth \
--image-path IJB_release/IJBC \
--result-dir ms1mv3_arcface_r50 \
--batch-size 128 \
--job ms1mv3_arcface_r50 \
--target IJBC \
--network iresnet50
```

--------------------------------

### Eval IJBC With Onnx

Source: https://github.com/opentalker/sadtalker/blob/main/src/face3d/models/arcface_torch/docs/eval.md

Evaluates the IJBC dataset using an ONNX model. Requires CUDA and specifies model root, image path, and result directory.

```shell
CUDA_VISIBLE_DEVICES=0 python onnx_ijbc.py --model-root ms1mv3_arcface_r50 --image-path IJB_release/IJBC --result-dir ms1mv3_arcface_r50
```

--------------------------------

### Configure CUDA Memory Allocation for PyTorch (Windows/Linux)

Source: https://github.com/opentalker/sadtalker/blob/main/docs/FAQ.md

Sets the PYTORCH_CUDA_ALLOC_CONF environment variable to manage CUDA memory allocation, helping to resolve 'CUDA out of memory' errors. This is useful when running inference with large models or high-resolution inputs. The variable `max_split_size_mb` controls the maximum size of memory splits, and setting it to 128MB can help prevent fragmentation.

```windows
set PYTORCH_CUDA_ALLOC_CONF=max_split_size_mb:128 
python inference.py ...
```

```linux
export PYTORCH_CUDA_ALLOC_CONF=max_split_size_mb:128 
python inference.py ...
```

=== COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.