### Installation and Setup Source: https://context7.com/kunato/transnetv2pt/llms.txt Instructions for installing TransNet V2 PyTorch and its dependencies, ensuring FFmpeg is available, and performing a quick verification. ```APIDOC ## Installation and Setup ### Description Install the package and its dependencies for immediate use in video processing pipelines. ### Method Not applicable (Shell commands) ### Endpoint Not applicable (Shell commands) ### Parameters None ### Request Example ```bash # Install from source pip install torch>=1.7 ffmpeg-python pillow pip install -e . # Or install dependencies manually pip install torch torchvision pip install ffmpeg-python pip install pillow # Verify FFmpeg is available ffmpeg -version # Quick test python -c "from transnetv2pt import predict_video; print('TransNet V2 ready')" ``` ### Response #### Success Response (200) - **Output**: Confirmation messages indicating successful installation and readiness of TransNet V2. #### Response Example ```text TransNet V2 ready ``` ``` -------------------------------- ### Installation and Setup for TransNet V2 PyTorch Source: https://context7.com/kunato/transnetv2pt/llms.txt Provides instructions for installing TransNet V2 PyTorch from source and its dependencies, including PyTorch, FFmpeg, and Pillow. It also includes commands to verify FFmpeg installation and perform a quick test of the library's readiness. ```bash # Install from source pip install torch>=1.7 ffmpeg-python pillow pip install -e . # Or install dependencies manually pip install torch torchvision pip install ffmpeg-python pip install pillow # Verify FFmpeg is available ffmpeg -version # Quick test python -c "from transnetv2pt import predict_video; print('TransNet V2 ready')" ``` -------------------------------- ### Predict Video Scenes with TransNetV2PT (Python) Source: https://github.com/kunato/transnetv2pt/blob/master/README.md This snippet demonstrates how to use the predict_video function from the transnetv2pt library to analyze a video file and extract scene information. It requires the 'transnetv2pt' library to be installed and a video file as input. ```python from transnetv2pt import predict_video scenes = predict_video('video.mp4') ``` -------------------------------- ### Initialize and Run TransNetV2 Model Source: https://context7.com/kunato/transnetv2pt/llms.txt Shows how to instantiate the TransNetV2 neural network, load pre-trained weights, and perform inference on a video tensor. This is useful for custom pipelines requiring direct access to transition probabilities. ```python import torch from transnetv2pt.transnetv2_pytorch import TransNetV2 import numpy as np # Initialize model with default configuration model = TransNetV2(F=16, L=3, S=2, D=1024, use_many_hot_targets=True, use_frame_similarity=True, use_color_histograms=True, dropout_rate=0.5) # Load pre-trained weights state_dict = torch.load('transnetv2pt/transnetv2-pytorch-weights.pth') model.load_state_dict(state_dict) model.eval() # Prepare input tensor video_frames = np.random.randint(0, 255, (1, 100, 27, 48, 3), dtype=np.uint8) video_tensor = torch.from_numpy(video_frames) # Run inference with torch.no_grad(): single_frame_pred, all_frame_pred = model(video_tensor) probabilities = torch.sigmoid(single_frame_pred).cpu().numpy() ``` -------------------------------- ### Predict Video Scene Boundaries Source: https://context7.com/kunato/transnetv2pt/llms.txt Demonstrates how to use the high-level predict_video function to detect scene transitions from either a video file path or a pre-processed numpy array of frames. It also shows how to convert resulting frame indices into timestamps based on frame rate. ```python from transnetv2pt import predict_video import numpy as np # Basic usage with video file path scenes = predict_video('video.mp4') # Using pre-processed frames (must be resized to 48x27 RGB) frames = np.random.randint(0, 255, (500, 27, 48, 3), dtype=np.uint8) scenes = predict_video(frames) # Converting frame numbers to timestamps (assuming 30 fps) fps = 30.0 for start, end in scenes: start_time = start / fps end_time = end / fps print(f"Scene: {start_time:.2f}s - {end_time:.2f}s") ``` -------------------------------- ### Convert Probabilities to Scene Boundaries Source: https://context7.com/kunato/transnetv2pt/llms.txt Explains how to use the predictions_to_scenes utility to transform raw transition probability arrays into discrete scene segments. It demonstrates the impact of adjusting the detection threshold on the final output. ```python from transnetv2pt.inference import predictions_to_scenes import numpy as np predictions = np.array([0.1, 0.05, 0.02, 0.03, 0.8, 0.9, 0.1, 0.05, 0.02, 0.01, 0.85, 0.92, 0.15, 0.03, 0.02, 0.01, 0.02, 0.03]) # Convert to scene boundaries with default threshold (0.5) scenes = predictions_to_scenes(predictions, threshold=0.5) # Using a higher threshold for fewer, more confident detections scenes_high_thresh = predictions_to_scenes(predictions, threshold=0.8) ``` -------------------------------- ### Raw Prediction Functionality in TransNet V2 PyTorch Source: https://context7.com/kunato/transnetv2pt/llms.txt Demonstrates the usage of the `predict_raw` function for low-level prediction, providing access to single-frame and all-frame prediction outputs. This is useful for custom post-processing and analysis of transition probabilities. It requires PyTorch and NumPy, and expects video frames as input. ```python import torch import numpy as np from transnetv2pt.inference import predict_raw, model # Prepare video frames (must be 27x48 RGB) video = np.random.randint(0, 255, (200, 27, 48, 3), dtype=np.uint8) # Run raw prediction device = torch.device('cuda:0' if torch.cuda.is_available() else 'cpu') num_frames, single_frame_pred, all_frames_pred = predict_raw(model, video, device) print(f"Total frames processed: {num_frames}") print(f"Single-frame predictions shape: {single_frame_pred.shape}") print(f"All-frames predictions shape: {all_frames_pred.shape}") # Analyze transition probabilities high_prob_frames = np.where(single_frame_pred > 0.5)[0] print(f"Frames with high transition probability: {high_prob_frames}") # Compare single vs all-frame predictions for smoothing analysis for frame_idx in high_prob_frames[:5]: print(f"Frame {frame_idx}: single={single_frame_pred[frame_idx]:.3f}, " f"all={all_frames_pred[frame_idx]:.3f}") ``` -------------------------------- ### Convert Predictions to Scenes Source: https://context7.com/kunato/transnetv2pt/llms.txt The `predictions_to_scenes` utility function converts raw transition probability predictions into discrete scene boundary segments based on a specified threshold. ```APIDOC ## predictions_to_scenes ### Description Converts raw transition probability predictions into discrete scene boundary segments using a configurable threshold. ### Method `predictions_to_scenes(predictions, threshold)` ### Parameters #### Path Parameters - **predictions** (np.ndarray) - Required - Array of transition probabilities per frame. - **threshold** (float) - Optional - The probability threshold to consider a scene transition. Defaults to 0.5. ### Request Example ```python from transnetv2pt.inference import predictions_to_scenes import numpy as np predictions = np.array([ 0.1, 0.05, 0.02, 0.03, 0.8, 0.9, 0.1, 0.05, 0.02, 0.01, 0.85, 0.92, 0.15, 0.03, 0.02, 0.01, 0.02, 0.03 ]) # Convert to scene boundaries with default threshold (0.5) scenes = predictions_to_scenes(predictions, threshold=0.5) # Output: [[0, 4], [6, 10], [12, 18]] # Using a higher threshold scenes_high_thresh = predictions_to_scenes(predictions, threshold=0.8) # Using a lower threshold scenes_low_thresh = predictions_to_scenes(predictions, threshold=0.3) ``` ### Response #### Success Response (200) - **scenes** (list) - A list of lists, where each inner list is a `[start_frame, end_frame]` pair representing a detected scene. #### Response Example ```json [[0, 4], [6, 10], [12, 18]] ``` ``` -------------------------------- ### Predict Video Scenes Source: https://context7.com/kunato/transnetv2pt/llms.txt The `predict_video` function is the primary entry point for detecting scene boundaries in a video. It can process either a video file path or a NumPy array of pre-processed video frames. ```APIDOC ## predict_video ### Description Detects scene boundaries in a video file or a numpy array of frames. Returns an array of scene boundaries as start/end frame pairs. ### Method `predict_video(video_source)` ### Parameters #### Path Parameters - **video_source** (str or np.ndarray) - Required - Path to the video file or a numpy array of video frames (shape: [num_frames, 27, 48, 3], dtype: np.uint8). ### Request Example ```python from transnetv2pt import predict_video import numpy as np # Basic usage with video file path scenes = predict_video('video.mp4') # Returns: array([[0, 150], [151, 320], [321, 489]], dtype=int32) # Using pre-processed frames (must be resized to 48x27 RGB) frames = np.random.randint(0, 255, (500, 27, 48, 3), dtype=np.uint8) scenes = predict_video(frames) ``` ### Response #### Success Response (200) - **scenes** (np.ndarray) - An array where each row is a `[start_frame, end_frame]` pair representing a detected scene. #### Response Example ```json [[0, 150], [151, 320], [321, 489]] ``` ``` -------------------------------- ### predict_raw - Low-level Prediction Function Source: https://context7.com/kunato/transnetv2pt/llms.txt Provides access to single-frame and all-frame prediction outputs for custom post-processing or analysis of transition probabilities. This function is useful for advanced users who need direct control over prediction results. ```APIDOC ## predict_raw ### Description Low-level prediction function that provides access to both single-frame and all-frame prediction outputs, useful for custom post-processing or analysis of transition probabilities. ### Method Not applicable (Python function) ### Endpoint Not applicable (Python function) ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body None ### Request Example ```python import torch import numpy as np from transnetv2pt.inference import predict_raw, model # Prepare video frames (must be 27x48 RGB) video = np.random.randint(0, 255, (200, 27, 48, 3), dtype=np.uint8) # Run raw prediction device = torch.device('cuda:0' if torch.cuda.is_available() else 'cpu') num_frames, single_frame_pred, all_frames_pred = predict_raw(model, video, device) print(f"Total frames processed: {num_frames}") print(f"Single-frame predictions shape: {single_frame_pred.shape}") print(f"All-frames predictions shape: {all_frames_pred.shape}") # Analyze transition probabilities high_prob_frames = np.where(single_frame_pred > 0.5)[0] print(f"Frames with high transition probability: {high_prob_frames}") # Compare single vs all-frame predictions for smoothing analysis for frame_idx in high_prob_frames[:5]: print(f"Frame {frame_idx}: single={single_frame_pred[frame_idx]:.3f}, " f"all={all_frames_pred[frame_idx]:.3f}") ``` ### Response #### Success Response (200) - **num_frames** (int) - Total number of frames processed. - **single_frame_pred** (numpy.ndarray) - Array of prediction scores for each frame, representing single-frame analysis. - **all_frames_pred** (numpy.ndarray) - Array of prediction scores for each frame, representing analysis considering all frames (potentially smoothed). #### Response Example ```json { "num_frames": 200, "single_frame_pred": [0.1, 0.2, 0.6, 0.7, ...], "all_frames_pred": [0.15, 0.25, 0.55, 0.65, ...] } ``` ``` -------------------------------- ### TransNetV2 Model Class Source: https://context7.com/kunato/transnetv2pt/llms.txt The `TransNetV2` class represents the core neural network model for scene detection. It allows for customization of various architectural parameters and loading of pre-trained weights. ```APIDOC ## TransNetV2 Model Class ### Description Initializes and uses the TransNet V2 neural network model. Processes batched video tensors to output scene transition probabilities. ### Method `TransNetV2(F, L, S, D, use_many_hot_targets, use_frame_similarity, use_color_histograms, dropout_rate)` ### Parameters #### Constructor Parameters - **F** (int) - Optional - Base filter count. Defaults to 16. - **L** (int) - Optional - Number of SDDCNN layers. Defaults to 3. - **S** (int) - Optional - Stacked blocks per layer. Defaults to 2. - **D** (int) - Optional - Dense layer dimension. Defaults to 1024. - **use_many_hot_targets** (bool) - Optional - Enable many-hot predictions. Defaults to True. - **use_frame_similarity** (bool) - Optional - Enable frame similarity features. Defaults to True. - **use_color_histograms** (bool) - Optional - Enable color histogram features. Defaults to True. - **dropout_rate** (float) - Optional - Dropout rate for training. Defaults to 0.5. ### Request Example ```python import torch from transnetv2pt.transnetv2_pytorch import TransNetV2 import numpy as np # Initialize model with default configuration model = TransNetV2( F=16, L=3, S=2, D=1024, use_many_hot_targets=True, use_frame_similarity=True, use_color_histograms=True, dropout_rate=0.5 ) # Load pre-trained weights # state_dict = torch.load('transnetv2pt/transnetv2-pytorch-weights.pth') # model.load_state_dict(state_dict) model.eval() # Prepare input tensor video_frames = np.random.randint(0, 255, (1, 100, 27, 48, 3), dtype=np.uint8) video_tensor = torch.from_numpy(video_frames) # Move to GPU if available device = torch.device('cuda:0' if torch.cuda.is_available() else 'cpu') model.to(device) video_tensor = video_tensor.to(device) # Run inference with torch.no_grad(): single_frame_pred, all_frame_pred = model(video_tensor) probabilities = torch.sigmoid(single_frame_pred).cpu().numpy() ``` ### Response #### Success Response (200) - **single_frame_pred** (torch.Tensor) - Per-frame transition probability tensor with shape [batch, frames, 1]. - **all_frame_pred** (dict) - Dictionary containing smoothed predictions, e.g., `all_frame_pred["many_hot"]` with shape [batch, frames, 1]. #### Response Example ```json { "single_frame_pred": "", "all_frame_pred": { "many_hot": "" } } ``` ``` === COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.