### Setup Python Environment and Install Dependencies Source: https://github.com/awslabs/aws-media-replay-engine/blob/main/README.md Sets up a Python virtual environment named 'venv' and activates it. It then installs all required Python packages listed in the 'requirements.txt' file, preparing the environment for subsequent build and deployment steps. ```python python3 -m venv venv source venv/bin/activate pip install -r requirements.txt ``` -------------------------------- ### Install FFmpeg on EC2 Source: https://github.com/awslabs/aws-media-replay-engine/blob/main/samples/source/mre-video-ingestion-samples/HLS_Harvester/runtime/chalicelib/user_data.txt Installs the latest static build of FFmpeg on an Amazon Linux EC2 instance. It downloads, extracts, and sets up symlinks for ffmpeg and ffprobe in the system's PATH. ```bash #!/bin/bash -xe echo '===START OF USER DATA===' exec > >(tee /var/log/user-data.log|logger -t user-data -s 2>/dev/console) 2>&1 echo 'Updating all the presently installed packages' sudo yum update -y echo 'Installing ffmpeg from the latest release build' cd /usr/local/bin mkdir ffmpeg && cd ffmpeg wget https://johnvansickle.com/ffmpeg/releases/ffmpeg-release-amd64-static.tar.xz tar -xf ffmpeg-release-amd64-static.tar.xz rm -rf ffmpeg-release-amd64-static.tar.xz for dir in ffmpeg*/ do if [ -d $dir ];then cp -a /usr/local/bin/ffmpeg/$dir/. /usr/local/bin/ffmpeg/; rm -rf $dir; break fi done echo 'Creating symlinks for ffmpeg and ffprobe' ln -s /usr/local/bin/ffmpeg/ffmpeg /usr/bin/ffmpeg ln -s /usr/local/bin/ffmpeg/ffprobe /usr/bin/ffprobe ``` -------------------------------- ### Install incron on EC2 Source: https://github.com/awslabs/aws-media-replay-engine/blob/main/samples/source/mre-video-ingestion-samples/HLS_Harvester/runtime/chalicelib/user_data.txt Installs the incron daemon on the EC2 instance, which allows for file system event-driven execution of commands. It also removes the /etc/incron.allow file to permit all users to use incron. ```bash echo 'Installing incron via the epel-release package' sudo amazon-linux-extras install epel sudo yum install -y incron sudo rm -f /etc/incron.allow ``` -------------------------------- ### Run Frontend Locally Source: https://github.com/awslabs/aws-media-replay-engine/blob/main/README.md Installs frontend dependencies and starts the development server for the MRE frontend. This allows for local testing and UI experimentation on localhost:3000, requiring Node.js version 20.10.0. ```npm npm i --legacy-peer-deps npm run start ``` -------------------------------- ### Install Python Dependencies for Media Replay Source: https://github.com/awslabs/aws-media-replay-engine/blob/main/samples/source/mre-video-ingestion-samples/HLS_Harvester/runtime/chalicelib/user_data.txt Installs essential Python libraries required for the media replay functionality. This includes AWS SDK (boto3), FFmpeg wrapper (ffmpeg-python), and HTTP request libraries (requests, requests-aws4auth). ```shell echo 'Installing boto3, ffmpeg-python, and requests libraries required by the ffmpeg_hls_streamer_poller python script via pip' sudo -u ec2-user pip3 install --upgrade boto3 sudo -u ec2-user pip3 install --upgrade ffmpeg-python sudo -u ec2-user pip3 install --upgrade requests sudo -u ec2-user pip3 install --upgrade requests-aws4auth ``` -------------------------------- ### Profile Configuration File Example Source: https://github.com/awslabs/aws-media-replay-engine/blob/main/samples/README.md Structure for a profile configuration file, defining how MRE should use specific plugins. This file is placed in the profile samples directory. ```json { "ProfileName": "MyCustomProfile", "Description": "A profile using specific plugins.", "Plugins": [ { "PluginName": "DetectAudioPeaks", "Configuration": { "threshold": 0.8 } }, { "PluginName": "LabelBasic" } ] } ``` -------------------------------- ### Generative AI Search API Request Examples Source: https://github.com/awslabs/aws-media-replay-engine/blob/main/docs/guides/MRE-Developer-Guide-GenAI.md Provides example JSON payloads for requests made to the Generative AI Search API, demonstrating different query types and parameters such as SessionId, Query, Program, and Event. ```json { "SessionId": "session-123", "Query": "Find scenes with celebrations after goals, lasting more than 10 seconds", "Program": "soccer-match", "Event": "championship-final" } ``` ```json { "SessionId": "session-456", "Query": "What happened between minute 15 and 30?", "Program": "basketball-game" } ``` -------------------------------- ### Configure and Start Incron Service Source: https://github.com/awslabs/aws-media-replay-engine/blob/main/samples/source/mre-video-ingestion-samples/HLS_Harvester/runtime/chalicelib/user_data.txt This shell script configures the `incrond` service to monitor the HLS output directory for new file creations. It sets up a rule to execute a specified script upon detecting new files, then starts the `incrond` service. ```shell echo 'Adding an entry in incrontab to capture IN_CREATE events in /home/ec2-user/ffmpeg_hls_output directory' echo '/home/ec2-user/ffmpeg_hls_output IN_CREATE /home/ec2-user/send_to_s3.sh $# $@ %%DESTINATION_S3_BUCKET%%' | sudo -u ec2-user incrontab - echo 'Starting the incrond service' sudo systemctl start incrond ``` -------------------------------- ### List All Prompts using curl Source: https://github.com/awslabs/aws-media-replay-engine/blob/main/docs/guides/MRE-Developer-Guide-PromptCatalog.md Shows how to retrieve all available prompt templates, with options to include disabled prompts. This uses a GET request to the /prompt/all endpoint. ```bash # List only enabled prompts curl -X GET "https://your-mre-endpoint/prompt/all" # Include disabled prompts curl -X GET "https://your-mre-endpoint/prompt/all?include_disabled=true" ``` ```json [ { "Name": "DescribeScenePrompt", "Description": "Analyzes video frames to describe the scene", "ContentGroups": ["Sports", "Football"], "Template": "Analyze these video frames and provide a detailed description...", "Version": "v1", "Created": "2025-07-03T20:15:30.123Z", "Latest": 1, "Enabled": true }, { "Name": "SummarizeHighlightPrompt", "Description": "Creates a summary of a highlight clip", "ContentGroups": ["Sports", "Basketball"], "Template": "Summarize this highlight clip in an exciting way...", "Version": "v2", "Created": "2025-07-02T18:45:12.456Z", "Latest": 2, "Enabled": true } ] ``` -------------------------------- ### Install ffmpeg and ffmpeg-python Source: https://github.com/awslabs/aws-media-replay-engine/blob/main/samples/source/mre-model-samples/Models/AudioSpectrumClassificationModel/AudioSpectrumClassification.ipynb Installs the ffmpeg-python library and ffmpeg itself using conda and pip. Includes commands for resolving potential library issues like missing shared objects. ```shell #!pip install ffmpeg-python #!conda update ffmpeg !conda install -c conda-forge ffmpeg-python -y #Success !conda update ffmpeg -y # Needed for libopenh264.so lib missing issue ``` -------------------------------- ### Plugin Lambda Function Example Source: https://github.com/awslabs/aws-media-replay-engine/blob/main/samples/README.md Illustrative Python code for a plugin's Lambda function. It shows how to handle input events and return results within the MRE framework. ```python # Example: source/mre-plugin-samples/Plugins/DetectAudioPeaks/.py def lambda_handler(event, context): # Process input event and perform plugin logic # Example: Analyze audio peaks result = { "message": "Audio peaks detected successfully" } return result ``` -------------------------------- ### Template Profile Configuration File Source: https://github.com/awslabs/aws-media-replay-engine/blob/main/samples/README.md Reference to a template file for creating new profile configurations. This file guides users on structuring their profile definitions. ```plaintext source/mre-profile-samples/template_profile_config.json.template ``` -------------------------------- ### Get Segment State for Optimization Source: https://github.com/awslabs/aws-media-replay-engine/blob/main/docs/guides/MRE-Developer-Guide-Optimizer.md Retrieves the state of segments available for optimization from the MRE dataplane. It fetches segment data and output from dependent plugins, which is then used to optimize segment start and end times. ```Python segments = mre_dataplane.get_segment_state_for_optimization(search_window_sec=optimization_search_window_sec) for segment in segments: segment_to_optimize = segment['Segment'] # Non-optimized Segment object dependent_detectors_data = segment['DependentDetectorsOutput'] # Output from DependentPlugins of the Optimizer plugin # TODO: Use the dependent detectors data to optimize the segment start and/or end ``` -------------------------------- ### MRE Deployment Setup Commands Source: https://github.com/awslabs/aws-media-replay-engine/blob/main/samples/README.md Sets up environment variables for deploying Media Replay Engine sample applications using AWS CDK. It defines the AWS region and the directory for deployment and source files. ```shell REGION=[specify the region where MRE is deployed in a format like us-east-1] deployment_dir=$(pwd)/deployment source_dir=$(pwd)/source ``` -------------------------------- ### Get Prompt by Name using curl Source: https://github.com/awslabs/aws-media-replay-engine/blob/main/docs/guides/MRE-Developer-Guide-PromptCatalog.md Retrieves the latest version of a specific prompt by its name using a GET request to the /prompt/{name} endpoint. ```bash curl -X GET "https://your-mre-endpoint/prompt/DescribeScenePrompt" ``` ```json { "Name": "DescribeScenePrompt", "Description": "Analyzes video frames to describe the scene", "ContentGroups": ["Sports", "Football"], "Template": "Analyze these video frames and provide a detailed description...", "Version": "v1", "Created": "2025-07-03T20:15:30.123Z", "Latest": 1, "Enabled": true } ``` -------------------------------- ### Replay Data Export Schema Example Source: https://github.com/awslabs/aws-media-replay-engine/blob/main/docs/guides/MRE-Developer-Guide-Export.md An example of the JSON formatted data that is exported when requesting a replay export. It details event information, replay configuration, and segment data. ```json { "Event": { "Name": "", "Id": "adc70e33-bf21-4b30-be61-588f3c9a6e20", "Program": "", "Start": "2021-09-12T20:01:14Z", "AudioTracksFound": [ 1, 2 ], "AllOutputAttributes": { "Labeler": [ "MatchPoint", "Ace", "DoubleFaults", "GamePoint", "SetPoint", "BreakPoint" ] }, "ProgramId": "CUS-e100019790c0ch20010", "Profile": { "Name": "", "ChunkSize": 20, "MaxSegmentLengthSeconds": 120, "ProcessingFrameRate": 5, "Classifier": { "Name": "TennisSegmentation", "Configuration": { "start_seq": "[['near','far'],['topview','far']]", "padding_seconds": "1", "end_seq": "[['far','near'],['far','topview']]" }, "DependentPlugins": [ { "Name": "TennisSceneClassification", "Configuration": { "minimum_confidence": "30" }, "DependentFor": [ "TennisSegmentation" ], "Level": 1, "SupportedMediaType": "Video" } ] }, "Labeler": { "Name": "TennisScoreExtraction", "Configuration": {}, "DependentPlugins": [ { "Name": "TennisScoreBoxDetection", "Configuration": { "Minimum-Confidence": "0.6" }, "DependentFor": [ "TennisScoreExtraction" ], "Level": 1, "SupportedMediaType": "Video" } ] } }, "SourceVideoMetadata": {} }, "Replay": { "Duration": 5.0, "EqualDistribution": "N", "Id": "2eae3581-5057-4274-a5c2-1d6cc4e10dab", "AudioTrack": 1.0, "FeaturesSelected": [ { "FeatureName": "MatchPoint", "FeatureValue": "True", "Weight": 100.0 }, { "FeatureName": "Ace", "FeatureValue": "True", "Weight": 55.0 }, { "FeatureName": "DoubleFaults", "FeatureValue": "True", "Weight": 45.0 }, { "FeatureName": "GamePoint", "FeatureValue": "True", "Weight": 75.0 }, { "FeatureName": "SetPoint", "FeatureValue": "True", "Weight": 85.0 }, { "FeatureName": "BreakPoint", "FeatureValue": "True", "Weight": 55.0 } ], "ReplayFormat": "Mp4", "Resolutions": [ "16:9 (1920 x 1080)", "9:16 (608 x 1080)" ], "Catchup": "Y", "Mp4Location": { "16:9": { "ReplayClips": [ "S3 Location of mp4 replay clip" ] }, "9:16": { "ReplayClips": [ "S3 Location of mp4 replay clip" ] } }, "Mp4ThumbnailLocation": { "16:9": { "ReplayThumbnails": [ "S3 Location of mp4 Thumbnail" ] }, "9:16": { "ReplayThumbnails": [ "S3 Location of mp4 Thumbnail" ] } } }, "Segments": [ { "Start": 1121.6, "End": 1158.4, "OptoStart": {}, "OptoEnd": {}, "OptimizedClipLocation": {}, "OriginalClipLocation": { "1": "S3 Location of mp4 clip", "2": "S3 Location of mp4 clip" }, "OriginalThumbnailLocation": "S3 location of Thumbnail", "Feedback": [], "FeaturesFound": [ { "AttribName": "DoubleFaults", "PluginName": "TennisScoreExtraction", "AttribValue": "True", "MultiplierChosen": 5.0, "Weight": 45.0, "Name": "TennisScoreExtraction - DoubleFaults - True" }, { "AttribName": "GamePoint", "PluginName": "TennisScoreExtraction", "AttribValue": "True", "MultiplierChosen": 8.0, "Weight": 75.0, "Name": "TennisScoreExtraction - GamePoint - True" } ] }, { "Start": 1467.2, "End": 1498.8, "OptoStart": {}, "OptoEnd": {}, "OptimizedClipLocation": {}, "OriginalClipLocation": { "1": "S3 Location of mp4 clip", "2": "S3 Location of mp4 clip" }, "OriginalThumbnailLocation": "S3 location of Thumbnail", "Feedback": [], "FeaturesFound": [ { "AttribName": "DoubleFaults", "PluginName": "TennisScoreExtraction", "AttribValue": "True", "MultiplierChosen": 5.0, "Weight": 45.0, "Name": "TennisScoreExtraction - DoubleFaults - True" }, { "AttribName": "GamePoint", "PluginName": "TennisScoreExtraction", "AttribValue": "True", "MultiplierChosen": 8.0, "Weight": 75.0, "Name": "TennisScoreExtraction - GamePoint - True" } ] } ] } ``` -------------------------------- ### Deploy plugin-samples Application Source: https://github.com/awslabs/aws-media-replay-engine/blob/main/samples/README.md Command to deploy the plugin-samples application to AWS. It requires the deployment directory and region, with an optional AWS profile for authentication. ```bash cd $deployment_dir ./build-and-deploy.sh --app plugin-samples --region $REGION [--profile ] ``` -------------------------------- ### Initialize Rekognition Client and Dependencies Source: https://github.com/awslabs/aws-media-replay-engine/blob/main/samples/source/mre-model-samples/Models/SceneClassificationModel/RekognitionCLTraining.ipynb Installs necessary Boto3 libraries and imports required modules for interacting with AWS services, specifically Amazon Rekognition. It sets up a logger and initializes the Rekognition client. ```python !pip install botocore --upgrade !pip install boto3 --upgrade import boto3 import argparse import logging import time import json from botocore.exceptions import ClientError logger = logging.getLogger(__name__) rek_client=boto3.client('rekognition') ``` -------------------------------- ### Starter Lambda Function for MRE Plugin Source: https://github.com/awslabs/aws-media-replay-engine/blob/main/docs/guides/MRE-Developer-Guide-Plugins.md A comprehensive starter Lambda handler function for an MRE plugin. It demonstrates initializing helpers, fetching segment data, processing segments using a custom function, saving results, updating plugin status, and returning the final output object. Includes robust error handling. ```python from MediaReplayEnginePluginHelper import OutputHelper from MediaReplayEnginePluginHelper import Status from MediaReplayEnginePluginHelper import DataPlane def your_function_to_format_a_label(segments): results = [] result = {} #loop through all the segments (0 to many) for this chunk for segment in segments: #your logic to format a string to be the Label for this segment result['Label'] = 'TBD' results.append(result) return results def lambda_handler(event, context): print (event) results = [] mre_dataplane = DataPlane(event) # 'event' is the input event payload passed to Lambda mre_outputhelper = OutputHelper(event) try: jsonResults = mre_dataplane.get_segment_state_for_labeling() print('jsonResults=',jsonResults) results = your_function_to_format_a_label(jsonResults) print('results=',results) # Add the results of the plugin to the payload (required if the plugin status is "complete"; Optional if the plugin has any errors) mre_outputhelper.add_results_to_output(results) # Persist plugin results for later use mre_dataplane.save_plugin_results(results) # Update the processing status of the plugin (required) mre_outputhelper.update_plugin_status(Status.PLUGIN_COMPLETE) # Returns expected payload built by MRE helper library return mre_outputhelper.get_output_object() except Exception as e: print(e) # Update the processing status of the plugin (required) mre_outputhelper.update_plugin_status(Status.PLUGIN_ERROR) # Re-raise the exception to MRE processing where it will be handled raise ``` -------------------------------- ### Template Dockerfile for Plugins Source: https://github.com/awslabs/aws-media-replay-engine/blob/main/samples/README.md Reference to a template Dockerfile for plugins that have dependencies exceeding Lambda deployment package size limits. It aids in building custom deployment packages. ```plaintext source/mre-plugin-samples/Plugins/Dockerfile.template ``` -------------------------------- ### AWS Media Replay Engine Optimizer Plugin Example (Python) Source: https://github.com/awslabs/aws-media-replay-engine/blob/main/docs/guides/MRE-Developer-Guide-Optimizer.md This Python code implements a general-purpose optimizer plugin for the AWS Media Replay Engine. It allows for extending segment lengths based on dependent feature plugin results, such as scene change detection or speech analysis. The plugin helps avoid issues like flicker frames or interruptions in speech by adjusting segment start or end times within a defined search window. ```python import json import boto3 from botocore.exceptions import ClientError from MediaReplayEnginePluginHelper import OutputHelper from MediaReplayEnginePluginHelper import PluginHelper from MediaReplayEnginePluginHelper import Status from MediaReplayEnginePluginHelper import DataPlane def optimize_segment(segment_mark, segment_type, detectors, config, max_search_window_sec): opto_segment_mark = segment_mark status = False result = {} print (f'Segment {segment_type} at:', str(segment_mark)) print('Dependent Plugin(s) config:', config) #loop through all dependent detectors for detector in detectors: print('Dependent detector:', detector) detector_name = detector['DependentDetector'] bias = config[detector_name]['bias'] #'safe range', 'unsafe range' print('Detector bias: ' + bias) if segment_type in detector: #loop through all the data for the respective detector for detection in detector[segment_type]: print('Detection starting at ' + str(detection['Start']) + ' ending at ' + str(detection['End'])) if bias == 'safe range': if detection['Start'] <= segment_mark <= detection['End']: print('already in safe range') if not status: status = True else: # not in a good range, attempt to move it if status: print("already optimized. exiting the loop") break print('segment_mark: ' + str(segment_mark) + ' detection_start: ' + str(detection['Start'])) if segment_type == 'Start': if abs(detection['Start'] - segment_mark) > max_search_window_sec: opto_segment_mark = segment_mark - max_search_window_sec print('adjusted but limited by max window') status = True else: opto_segment_mark = detection['Start'] print('adjusted to start edge of range') status = True else: #end processing if abs(detection['End'] - segment_mark) > max_search_window_sec: opto_segment_mark = segment_mark + max_search_window_sec print('adjusted but limited by max window') status = True else: opto_segment_mark = detection['End'] print('adjusted to end edge of range') status = True print("Opto segment mark:", opto_segment_mark) elif bias == 'unsafe range': if detection['Start'] <= segment_mark <= detection['End']: # not in a good range, attempt to move it if status: print("already optimized. exiting the loop") break if segment_type == 'Start': if abs(detection['Start'] - segment_mark) > max_search_window_sec: opto_segment_mark = segment_mark - max_search_window_sec print('adjusted but limited by max window') status = True else: opto_segment_mark = detection['Start'] print('adjusted to start edge of range') status = True else: #end processing if abs(detection['End'] - segment_mark) > max_search_window_sec: ``` -------------------------------- ### Build Frontend Application Source: https://github.com/awslabs/aws-media-replay-engine/blob/main/README.md Installs frontend dependencies using npm, specifically handling potential peer dependency issues with '--legacy-peer-deps'. It then builds the frontend application, generating optimized assets for deployment. ```npm npm i --legacy-peer-deps && npm run build ``` -------------------------------- ### Offset Example: Offset = 1 Source: https://github.com/awslabs/aws-media-replay-engine/blob/main/samples/source/mre-plugin-samples/Plugins/SegmentBySceneChangeWithOffset/README.md Illustrates how an offset of 1 affects the detection of start and end sequences. The pointer moves one step after a match to find the subsequent pattern, impacting the precise frame marked as the segment boundary. ```json start_seq = { "offset": 1, "patter": [["scene1", "scene2"]] } end_seq = { "offset": 1, "patter": [["scene2", "scene1"]] } # Example de-dupped sequence: # frame 100 >> scene1 # frame 120 >> scene2 # frame 150 >> scene1 # With offset 1, start_seq matches at frame 100 (scene1 -> scene2). # The pointer moves to frame 120. end_seq matches at frame 120 (scene2 -> scene1). # Frame 150 is marked as end_seq. ``` -------------------------------- ### Initialize Amplify Deployment Source: https://github.com/awslabs/aws-media-replay-engine/blob/main/README.md Runs a Python script to initialize or update the Amplify deployment configuration. This command is typically executed from the 'cdk' directory within the frontend source code. ```python python3 init-amplify.py $REGION Update ``` -------------------------------- ### Plugin Configuration List Source: https://github.com/awslabs/aws-media-replay-engine/blob/main/samples/README.md JSON structure defining the list of plugins to be deployed by the plugin-samples application. This list is used by the CDK to identify and register plugins. ```json { "Plugins": [ "DetectAudioPeaks", "DetectViaPrompt", "SegmentByKeyMoment", "LabelBasic", "OptimizePassThrough", "SegmentPassThrough100", "LabelPassThrough", "DetectPassThrough100", "DetectSpeech", "DetectSentiment", "DetectCelebrities", "DetectSceneLabels", "SegmentNews", "LabelNews" ] } ``` -------------------------------- ### GET /export/data/replay/{id}/event/{event}/program/{program} API Source: https://github.com/awslabs/aws-media-replay-engine/blob/main/docs/guides/MRE-Developer-Guide-Export.md This API endpoint is used to request a replay specific export. The provided link offers detailed specifications for this control plane API. ```APIDOC APIDOC: GET /export/data/replay/{id}/event/{event}/program/{program} - Description: API to request a replay specific export. - Link: https://htmlpreview.github.io/?https://github.com/awslabs/aws-media-replay-engine/blob/main/docs/source/output/api/controlplane-replay.html#get-replay-export-data ``` -------------------------------- ### Create Rekognition Project Source: https://github.com/awslabs/aws-media-replay-engine/blob/main/samples/source/mre-model-samples/Models/SceneClassificationModel/RekognitionCLTraining.ipynb Initializes a new project within AWS Rekognition Custom Labels. This is the first step before creating model versions. ```python project_name = 'MRE-workshop-project' DATA_BUCKET = 'S3_BUCEKT_NAME' MANIFEST = 'dataset/output.manifest' response=rek_client.create_project(ProjectName=project_name) project_arn = response['ProjectArn'] #dataset_arn=create_dataset(rek_client, project_arn, 'TRAIN', DATA_BUCKET, MANIFEST) ``` -------------------------------- ### Translate Chunk Runtime to Absolute Time Source: https://github.com/awslabs/aws-media-replay-engine/blob/main/docs/guides/MRE-Developer-Guide-Featurer.md Explains how to convert a timecode relative to the start of a video chunk (HLS segment) into an absolute timestamp. This is useful for accurate temporal analysis and event logging. ```python from MediaReplayEnginePluginHelper import OutputHelper mre_outputhelper = OutputHelper(event) # event is the input event payload passed to Lambda mre_outputhelper.get_segment_absolute_time(1.3) # Assuming a chunk_size of 20 secs, for the first chunk, this will output 1.3 and for the second chunk, this will output 21.3 and so on. ``` -------------------------------- ### List Prompts by Content Group using curl Source: https://github.com/awslabs/aws-media-replay-engine/blob/main/docs/guides/MRE-Developer-Guide-PromptCatalog.md Demonstrates filtering prompts by a specific content group using a GET request to the /prompt/contentgroup/{group}/all endpoint, with an option to include disabled prompts. ```bash # List only enabled prompts for a content group curl -X GET "https://your-mre-endpoint/prompt/contentgroup/Sports/all" # Include disabled prompts for a content group curl -X GET "https://your-mre-endpoint/prompt/contentgroup/Football/all?include_disabled=true" ``` ```json [ { "Name": "DescribeScenePrompt", "Description": "Analyzes video frames to describe the scene", "ContentGroups": ["Sports", "Football"], "Template": "Analyze these video frames and provide a detailed description...", "Version": "v1", "Created": "2025-07-03T20:15:30.123Z", "Latest": 1, "Enabled": true } ] ``` -------------------------------- ### MRE Event Payload Structure Source: https://github.com/awslabs/aws-media-replay-engine/blob/main/docs/guides/MRE-Developer-Guide-Events.md Example JSON structure for an event payload, illustrating the use of context variables like TimeOffset, BroadcastId, and EventData. This payload is used when creating or updating events via the MRE API. ```JSON { "TimeOffset": 12, "BroadcastId": 12345, "EventData": "New Data" } ``` -------------------------------- ### Use Prompt Templates with MRE Plugin Helper Source: https://github.com/awslabs/aws-media-replay-engine/blob/main/docs/guides/MRE-Developer-Guide-PromptCatalog.md Demonstrates how to use the MediaReplayEnginePluginHelper library in Python to retrieve prompt templates. This allows plugins to easily access and utilize prompt content for generative AI applications. ```python from MediaReplayEnginePluginHelper import ControlPlane def lambda_handler(event, context): # Initialize the control plane helper mre_controlplane = ControlPlane() # Get a prompt template by name prompt = mre_controlplane.get_prompt_template(name="DescribeScenePrompt") # Use the template in your generative AI application prompt_text = prompt["Template"] # Invoke Model and process the response # ... ``` -------------------------------- ### Event Data Export Schema Source: https://github.com/awslabs/aws-media-replay-engine/blob/main/docs/guides/MRE-Developer-Guide-Export.md Example JSON structure for exported event data. This schema includes details about the event itself, audio tracks, output attributes, profile configurations (including classifiers and labelers), and segment information with clip locations. ```JSON { "Event": { "Name": "", "Id": "adc70e33-bf21-4b30-be61-588f3c9a6e20", "Program": "", "Start": "", "AudioTracksFound": [ 1, 2 ], "AllOutputAttributes": { "Labeler": [ "MatchPoint", "Ace", "DoubleFaults", "GamePoint", "SetPoint", "BreakPoint" ] }, "ProgramId": "", "Profile": { "Name": "", "ChunkSize": 20, "MaxSegmentLengthSeconds": 120, "ProcessingFrameRate": 5, "Classifier": { "Name": "TennisSegmentation", "Configuration": { "start_seq": "[['near','far'],['topview','far']]", "padding_seconds": "1", "end_seq": "[['far','near'],['far','topview']]" }, "DependentPlugins": [ { "Name": "TennisSceneClassification", "Configuration": { "minimum_confidence": "30" }, "DependentFor": [ "TennisSegmentation" ], "Level": 1, "SupportedMediaType": "Video" } ] }, "Labeler": { "Name": "TennisScoreExtraction", "Configuration": {}, "DependentPlugins": [ { "Name": "TennisScoreBoxDetection", "Configuration": { "Minimum-Confidence": "0.6" }, "DependentFor": [ "TennisScoreExtraction" ], "Level": 1, "SupportedMediaType": "Video" } ] } }, "SourceVideoMetadata": {} }, "Segments": [ { "Start": 222.2, "End": 224.4, "OptoStart": {}, "OptoEnd": {}, "OptimizedClipLocation": {}, "OriginalClipLocation": { "1": "S3 Location of mp4 clip", "2": "S3 Location of mp4 clip" }, "OriginalThumbnailLocation": "S3 location of Thumbnail", "FeaturesFound": [], "Feedback": [] }, { "Start": 227, "End": 235.4, "OptoStart": {}, "OptoEnd": {}, "OptimizedClipLocation": {}, "OriginalClipLocation": { "1": "S3 Location of mp4 clip", "2": "S3 Location of mp4 clip" }, "OriginalThumbnailLocation": "S3 location of Thumbnail", "FeaturesFound": [], "Feedback": [] }, { "Start": 236.4, "End": 238.6, "OptoStart": {}, "OptoEnd": {}, "OptimizedClipLocation": {}, "OriginalClipLocation": { "1": "S3 Location of mp4 clip", "2": "S3 Location of mp4 clip" }, "OriginalThumbnailLocation": "S3 location of Thumbnail", "FeaturesFound": [], "Feedback": [] } ] } ``` -------------------------------- ### Configure HLS Harvester Sample Source: https://github.com/awslabs/aws-media-replay-engine/blob/main/samples/README.md Copies the HLS harvester configuration template and requires manual editing of variables within the `harvester_config.py` file before deployment. ```python cd $source_dir/mre-video-ingestion-samples/HLS_Harvester cp harvester_config_template.py harvester_config.py ``` -------------------------------- ### Start Media Replay Poller Script Source: https://github.com/awslabs/aws-media-replay-engine/blob/main/samples/source/mre-video-ingestion-samples/HLS_Harvester/runtime/chalicelib/user_data.txt Launches the main Python script responsible for polling the SQS queue and processing media replay tasks. The script is run in the background using `nohup` and its output is redirected to a log file for monitoring. ```shell echo 'Starting the ffmpeg_hls_streamer_poller python script' sudo -u ec2-user nohup python3 -u /home/ec2-user/ffmpeg_hls_streamer_poller.py &> /home/ec2-user/ffmpeg_hls_streamer_poller.log & ``` -------------------------------- ### Initialize Rekognition Boto3 Client Source: https://github.com/awslabs/aws-media-replay-engine/blob/main/samples/source/mre-model-samples/Models/ScoreBoxDetectionModel/RekognitionCLTraining.ipynb Installs and upgrades the necessary Boto3 libraries and initializes the Amazon Rekognition client. This client is used to interact with the Rekognition API for dataset and model operations. ```python !pip install botocore --upgrade !pip install boto3 --upgrade import boto3 import argparse import logging import time import json from botocore.exceptions import ClientError logger = logging.getLogger(__name__) rek_client=boto3.client('rekognition') ``` -------------------------------- ### Python Bedrock Client Configuration Source: https://github.com/awslabs/aws-media-replay-engine/blob/main/docs/guides/MRE-Developer-Guide-GenAI.md Demonstrates setting up a Python client for Amazon Bedrock Runtime with custom retry logic. This is essential for robust integration with foundation models, ensuring stability during API calls. ```python import os import cv2 import base64 import json import boto3 from botocore.config import Config from MediaReplayEnginePluginHelper import OutputHelper from MediaReplayEnginePluginHelper import Status from MediaReplayEnginePluginHelper import DataPlane from MediaReplayEnginePluginHelper import ControlPlane # Configure Bedrock client with retry logic config = Config(retries={"max_attempts": 100, "mode": "standard"}) bedrock = boto3.client("bedrock-runtime", config=config) ``` -------------------------------- ### Sample Featurer Plugin Payload Event Source: https://github.com/awslabs/aws-media-replay-engine/blob/main/docs/guides/MRE-Developer-Guide-Featurer.md An example JSON payload structure representing the input event passed to a Featurer plugin. It includes details about the plugin's configuration, the processing profile, input media source, and event metadata. ```json { "Plugin": { "Name": "TennisSceneClassification", "Class": "Featurer", "ExecutionType": "SyncModel", "ModelEndpoint": "arn:aws:rekognition:us-east-1:99999999999999:project/Tennis-Segmentation/version/Tennis-Segmentation.2021-10-26T16.39.56/1635284396508", "DependentPlugins": [], "Configuration": { "minimum_confidence": "30" }, "OutputAttributes": { "Label": { "Description": "output" } } }, "Profile": { "Name": "TennisProfile", "ChunkSize": 20, "MaxSegmentLengthSeconds": 120, "ProcessingFrameRate": 5, "Classifier": { "Name": "Tennis-Segmentation", "Configuration": { "start_seq": "[['near','far'],['topview','far']]", "padding_seconds": "1", "end_seq": "[['far','near'],['far','topview']]" }, "DependentPlugins": [{ "Name": "TennisSceneClassification", "Configuration": { "minimum_confidence": "30" }, "DependentFor": ["Tennis-Segmentation"], "Level": 1, "SupportedMediaType": "Video" }] }, "Labeler": { "Name": "Tennis-Score-Extraction", "Configuration": {}, "DependentPlugins": [{ "Name": "Tennis-Score-Box-Detection", "Configuration": { "Minimum-Confidence": "0.6" }, "DependentFor": ["Tennis-Score-Extraction"], "Level": 1, "SupportedMediaType": "Video" }] } }, "Input": { "ExecutionId": "350c6153-33b7-4ec2-9062-7cad5a574c88", "Media": { "S3Bucket": "aws-mre-controlplane-medialivedestinationbucket13-99999999", "S3Key": "9999999/TennisProfile/a_tennis_match_00007.ts" }, "Metadata": { "HLSSegment": { "StartTime": 120.12, "StartTimeUtc": "00:02:00.120", "StartPtsTime": 122.42, "StartPtsTimeUtc": "00:02:02.420", "Duration": 20.02, "FrameRate": 60 } } }, "Event": { "Name": "SampleTennisMatchEvent", "Program": "Some Big Tennis Tournament", "Start": "2021-11-03T16:32:57Z", "AudioTracks": [1] } } ``` -------------------------------- ### Template Plugin Configuration File Source: https://github.com/awslabs/aws-media-replay-engine/blob/main/samples/README.md Reference to a template file for creating new plugin configurations. This file provides the structure and required fields for registering a plugin with MRE. ```plaintext source/mre-plugin-samples/Plugins/template_plugin_config.json.template ``` -------------------------------- ### Generative AI Search API Request (Bash) Source: https://github.com/awslabs/aws-media-replay-engine/blob/main/docs/guides/MRE-Developer-Guide-GenAI.md Shows an example of how to make a POST request to the MRE Generative AI Search API using curl. It specifies the endpoint, content type, and a JSON payload containing session ID, query, program, event, and the model ID. ```bash curl -X POST "https://your-mre-genai-search-endpoint/" \ -H "Content-Type: application/json" \ -d '{ "SessionId": "unique-session-id", "Query": "Show me all the goals scored in the first half", "Program": "soccer-match-2024", "Event": "world-cup-final", "ModelId": "amazon.nova-pro-v1:0" }' ``` -------------------------------- ### Package Models and Upload to S3 Source: https://github.com/awslabs/aws-media-replay-engine/blob/main/samples/source/mre-model-samples/Models/PoseKeypointsExtractionModel/PoseKeypointsExtractionModel.ipynb This section provides shell commands to package the downloaded model files into a tarball and upload it to an S3 bucket. This packaged model is then used to create a SageMaker model. ```bash cd ~/.mxnet/models tar -cvzf model.tar.gz simple_pose_resnet18_v1b-f63d42ac.params yolo3_mobilenet1.0_coco-66dbbae6.params ``` ```awscli aws s3 cp model.tar.gz s3://{S3_BUCKET_NAME}/{MODEL_FILE_KEY_PREFIX}/model.tar.gz ```