### POST getRecommendations Endpoint Example Source: https://github.com/aws-samples/amazon-personalize-samples/blob/master/next_steps/operations/streaming_events/README.md Example body and endpoint for retrieving recommendations. Ensure you include the API key in the request header. ```json { "userId":"12345" } ``` ```plaintext https://XXXXXX.execute-api.us-east-1.amazonaws.com/dev2/recommendations ``` -------------------------------- ### Build the SAM Project Source: https://github.com/aws-samples/amazon-personalize-samples/blob/master/next_steps/operations/streaming_events/README.md Build your Serverless Application Model (SAM) project. Refer to the AWS SAM CLI installation guide for details. ```bash sam build ``` -------------------------------- ### Get recommendations with promotions Source: https://github.com/aws-samples/amazon-personalize-samples/blob/master/next_steps/core_use_cases/filters/promotions/Promotions.ipynb Retrieve recommendations for a user, incorporating promotions to boost specific items. This example uses a promotion to include a percentage of items from the 'halloween' category. ```python get_recommendations_response = personalize_runtime.get_recommendations( recommenderArn = recommended_for_you_arn, userId = test_user_id, numResults = 20, promotions = [{ "name" : "halloween_promotion", "percentPromotedItems" : 20, "filterArn": filter_arn, "filterValues": { "CATEGORY" : "\"halloween\"" } }] ) # Build a new dataframe for the recommendations item_list = get_recommendations_response['itemList'] recommendation_id_list = [] recommendation_description_list = [] recommendation_category_list = [] for item in item_list: description = get_item_by_id(item['itemId'], items_df) recommendation_description_list.append(description) recommendation_id_list.append(item['itemId']) recommendation_category_list.append(get_category_by_id(item['itemId'], items_df)) user_recommendations_df = pd.DataFrame(recommendation_id_list, columns = ["ID"]) user_recommendations_df["description"] = recommendation_description_list user_recommendations_df["category level 2"] = recommendation_category_list pd.options.display.max_rows =20 display(user_recommendations_df) ``` -------------------------------- ### Deploy the SAM Project (Guided) Source: https://github.com/aws-samples/amazon-personalize-samples/blob/master/next_steps/operations/streaming_events/README.md Deploy your project using SAM's guided deployment. You will be prompted to provide necessary parameters, including an email address for notifications. ```bash sam deploy --guided ``` -------------------------------- ### Import Libraries and Check Versions Source: https://github.com/aws-samples/amazon-personalize-samples/blob/master/next_steps/generative_ai/personalized_marketing_campaign/personalized_marketing_campaign_10032023_1600_github.ipynb Imports necessary libraries such as boto3, botocore, transformers, and langchain. It also prints the installed versions of boto3, botocore, and transformers to verify the setup. ```python import boto3 import botocore import transformers # Get the Boto3 version boto3_version = boto3.__version__ # Get the Botocore version botocore_version = botocore.__version__ # Get the Transformers version transformers_version = transformers.__version__ # Print the Boto3 version print("Current Boto3 Version:", boto3_version) # Print the Botocore version print("Current Botocore Version:", botocore_version) # Print the Transformers version print("Transformers Version:", transformers_version) ``` -------------------------------- ### Setup Personalize Clients Source: https://github.com/aws-samples/amazon-personalize-samples/blob/master/getting_started/notebooks/2.View_Campaign_And_Interactions.ipynb Initializes boto3 clients for Amazon Personalize, Personalize Runtime, and Personalize Events. ```python # Setup and Config # Recommendations from Event data personalize = boto3.client('personalize') personalize_runtime = boto3.client('personalize-runtime') # Establish a connection to Personalize's Event Streaming personalize_events = boto3.client(service_name='personalize-events') ``` -------------------------------- ### Import Libraries and Install tqdm Source: https://github.com/aws-samples/amazon-personalize-samples/blob/master/next_steps/data_science/offline_performance_evaluation/personalize_temporal_holdout.ipynb Imports necessary libraries for data manipulation, AWS interaction, and progress tracking. Installs the 'tqdm' library for progress bars. ```python import boto3, os import json import numpy as np import pandas as pd import time from botocore.exceptions import ClientError !pip install tqdm from tqdm import tqdm_notebook from metrics import mean_reciprocal_rank, ndcg_at_k, precision_at_k ``` -------------------------------- ### Test Get Recommended Products Tool Source: https://github.com/aws-samples/amazon-personalize-samples/blob/master/next_steps/generative_ai/personalized_recommender_agent/02_Recommender-Agent_Build-Agent-ConverseAPI.ipynb Demonstrates how to call the `get_recommended_products` function with a sample user ID and shows the expected output format. ```python get_recommended_products('1234') ``` -------------------------------- ### Include Filter Expression Example Source: https://github.com/aws-samples/amazon-personalize-samples/blob/master/next_steps/operations/filter_rotator/README.md This example demonstrates how to include items based on interaction event types. It uses a SQL-like syntax to filter recommendations. ```sql INCLUDE ItemID WHERE Interactions.event_type IN ('watched','favorited') ``` -------------------------------- ### Install Faker Library Source: https://github.com/aws-samples/amazon-personalize-samples/blob/master/next_steps/core_use_cases/automatic_context/notebooks/1.Building_Personalize_Campaign.ipynb Install the Faker library, which is used for generating synthetic data for the campaign. ```python !pip install Faker ``` -------------------------------- ### Test Get Product Details Tool Source: https://github.com/aws-samples/amazon-personalize-samples/blob/master/next_steps/generative_ai/personalized_recommender_agent/02_Recommender-Agent_Build-Agent-ConverseAPI.ipynb Demonstrates how to call the `get_product_details` function with a sample product ID and shows the expected output format. ```python get_product_details(['dff3f75f-838d-4b4d-9c7c-cee7ae2aac10']) ``` -------------------------------- ### POST Event Endpoint Example Source: https://github.com/aws-samples/amazon-personalize-samples/blob/master/next_steps/operations/streaming_events/README.md Example body and endpoint for streaming interaction data. The API key must be included in the request header. ```json { "Event":{ "itemId": "ITEMID", "eventValue": EVENT-VALUE, "CONTEXT": "VALUE" //optional }, "SessionId": "SESSION-ID-IDENTIFIER", "EventType": "YOUR-EVENT-TYPE", "UserId": "USERID" } ``` ```plaintext https://XXXXXX.execute-api.us-east-1.amazonaws.com/dev2/history ``` -------------------------------- ### Get Initial User Recommendations Source: https://github.com/aws-samples/amazon-personalize-samples/blob/master/next_steps/workshops/POC_in_a_box/05_Interacting_with_Campaigns_and_Filters.ipynb Fetches the initial set of recommendations for a given user before any real-time events are applied. This serves as a baseline for comparison. ```python # First pick a user user_id = user # Get recommendations for the user get_recommendations_response = personalize_runtime.get_recommendations( campaignArn = userpersonalization_campaign_arn, userId = str(user_id), ) # Build a new dataframe for the recommendations item_list = get_recommendations_response['itemList'] recommendation_list = [] for item in item_list: artist = get_movie_by_id(item['itemId']) recommendation_list.append(artist) user_recommendations_df = pd.DataFrame(recommendation_list, columns = [user_id]) user_recommendations_df ``` -------------------------------- ### Create a Campaign Source: https://github.com/aws-samples/amazon-personalize-samples/blob/master/getting_started/notebooks/1.Building_Your_First_Campaign.ipynb Deploy a trained solution version as a campaign to serve real-time recommendations. Configure minimum provisioned throughput and item exploration settings. ```python create_campaign_response = personalize.create_campaign( name = "personalize-demo-camp", solutionVersionArn = solution_version_arn, minProvisionedTPS = 1, campaignConfig = { "itemExplorationConfig": { "explorationWeight": "0.5" } } ) campaign_arn = create_campaign_response['campaignArn'] print(json.dumps(create_campaign_response, indent=2)) ``` -------------------------------- ### Get Item Description by ID Source: https://github.com/aws-samples/amazon-personalize-samples/blob/master/getting_started/notebooks_managed_domains/Building_Your_First_Recommender_Ecommerce.ipynb Retrieves the description for a specific item using its ID and the loaded items DataFrame. This is a direct usage example of the `get_item_by_id` helper function. ```python # use a random valid id for a quick sanity check, modify the line of code bellow to a valid id in your dataset get_item_by_id("c72257d4-430b-4eb7-9de3-28396e593381", items_df) ``` -------------------------------- ### Define CloudWatch Query Timeframe Source: https://github.com/aws-samples/amazon-personalize-samples/blob/master/next_steps/evaluation/measuring_impact_of_recommendations/Measure_Impact_of_Recommendations.ipynb Defines the start and end times for querying CloudWatch metrics in Unix timestamp format. This example queries data from the last 5 hours. ```python # Define the start and end_times in Unix format this is the timeframe we will query start_time = int(time.mktime((today + datetime.timedelta(hours = -5)).timetuple())) end_time = int(time.mktime(today.timetuple())) ``` -------------------------------- ### AWS and Personalize Client Setup Source: https://github.com/aws-samples/amazon-personalize-samples/blob/master/next_steps/operations/lambda_examples/Sending_Events_to_S3.ipynb Initializes AWS clients for Personalize, Personalize Runtime, and Personalize Events. Also configures S3 bucket name. ```python # Setup and Config # Recommendations from Event data personalize = boto3.client('personalize') personalize_runtime = boto3.client('personalize-runtime') # Campaign ARN Campaign_ARN = "arn:aws:personalize:us-east-1:059124553121:campaign/personalize-demo-camp" # Dataset Group Arn: datasetGroupArn = "arn:aws:personalize:us-east-1:059124553121:dataset-group/personalize-lambda-demo" # Establish a connection to Personalize's Event Streaming personalize_events = boto3.client(service_name='personalize-events') # S3 Configuration bucket = "personalizedemo9000chris" ``` -------------------------------- ### Create a Campaign Source: https://github.com/aws-samples/amazon-personalize-samples/blob/master/getting_started/notebooks/3.Best_Practices-Clientside.ipynb Creates a campaign to deploy a solution version for real-time recommendations. Specify the solution version ARN and desired throughput. ```python create_campaign_response = personalize.create_campaign( name = "DEMO-campaign", solutionVersionArn = solution_version_arn, minProvisionedTPS = 1 ) campaign_arn = create_campaign_response['campaignArn'] print(json.dumps(create_campaign_response, indent=2)) ``` -------------------------------- ### Delete Filter Match Template Example Source: https://github.com/aws-samples/amazon-personalize-samples/blob/master/next_steps/operations/filter_rotator/README.md This template matches filters with names starting with 'filter-include-recent-items-' and whose suffix (assumed to be a date) is older than one day. This allows for a transition period before deletion. ```text starts_with(filter.name,'filter-include-recent-items-') and int(end(filter.name,8)) < int(datetime_format(now - timedelta_days(1),'%Y%m%d')) ``` -------------------------------- ### Initialize S3 and Personalize Clients Source: https://github.com/aws-samples/amazon-personalize-samples/blob/master/next_steps/core_use_cases/trending_now/trending_now_example.ipynb Sets up the AWS SDK clients for S3, Personalize, and Personalize Events. Ensure your AWS credentials and region are configured. ```python import boto3 region = "us-east-1" s3 = boto3.client('s3') account_id = boto3.client('sts').get_caller_identity().get('Account') bucket_name = account_id + "-" + region + "-" + "personalize-trending-now" print('bucket_name:', bucket_name) try: if region == "us-east-1": s3.create_bucket(Bucket=bucket_name) else: s3.create_bucket( Bucket = bucket_name, CreateBucketConfiguration={'LocationConstraint': region} ) except s3.exceptions.BucketAlreadyOwnedByYou: print("Bucket already exists. Using bucket", bucket_name) ``` ```python # Configure the SDK to Personalize: personalize = boto3.Session(region_name=region).client('personalize') personalize_events = boto3.Session(region_name=region).client('personalize-events') personalize_runtime = boto3.Session(region_name=region).client('personalize-runtime') ``` -------------------------------- ### Install Python Packages for Generative AI Source: https://github.com/aws-samples/amazon-personalize-samples/blob/master/next_steps/generative_ai/personalized_marketing_campaign/personalized_marketing_campaign_10032023_1600_github.ipynb Installs or upgrades essential Python packages including pip, boto3, botocore, langchain, and transformers. Restart the kernel if prompted after installation. ```python %pip install --upgrade pip %pip install boto3 --upgrade %pip install botocore --upgrade ``` ```python %pip install langchain %pip install transformers ``` ```python %pip install langchain --upgrade %pip install transformers --upgrade ``` -------------------------------- ### Update pip and Install Botocore Source: https://github.com/aws-samples/amazon-personalize-samples/blob/master/next_steps/core_use_cases/updating_datasets/update-item-dataset-schema-example.ipynb Ensures a current version of botocore is installed, which is necessary for accessing newer Amazon Personalize features. This command upgrades pip and then installs or reinstalls botocore without its dependencies. ```python import sys !{sys.executable} -m pip install --upgrade pip !{sys.executable} -m pip install --upgrade --no-deps --force-reinstall botocore ``` -------------------------------- ### Install ipywidgets Source: https://github.com/aws-samples/amazon-personalize-samples/blob/master/next_steps/generative_ai/personalized_recommender_agent/02_Recommender-Agent_Build-Agent-ConverseAPI.ipynb Installs the ipywidgets library, which is necessary for creating interactive elements in Jupyter notebooks. ```python !pip install ipywidgets ``` -------------------------------- ### Initialize AWS Clients and SDK Source: https://github.com/aws-samples/amazon-personalize-samples/blob/master/next_steps/operations/ml_ops_ds_sdk/Personalize-Stepfunction-Workflow.ipynb Initializes Boto3 clients for Amazon Personalize and Personalize Runtime, along with necessary libraries for Step Functions. Sets up logging for the Step Functions SDK. ```python import boto3 import json import numpy as np import pandas as pd import time personalize = boto3.client('personalize') personalize_runtime = boto3.client('personalize-runtime') import stepfunctions import logging from stepfunctions.steps import * from stepfunctions.workflow import Workflow stepfunctions.set_stream_logger(level=logging.INFO) workflow_execution_role = "" # paste the StepFunctionsWorkflowExecutionRole ARN from above ``` -------------------------------- ### Configure Boto3 SDK Source: https://github.com/aws-samples/amazon-personalize-samples/blob/master/next_steps/workshops/POC_in_a_box/06_Clean_Up_Resources.ipynb Initializes the Boto3 clients for Personalize and Personalize Runtime. Ensure the AWS SDK is configured before running. ```python import boto3 import json ``` ```python # Configure the SDK to Personalize: personalize = boto3.client('personalize') personalize_runtime = boto3.client('personalize-runtime') ``` -------------------------------- ### Install Pillow Library Source: https://github.com/aws-samples/amazon-personalize-samples/blob/master/next_steps/generative_ai/personalized_marketing_campaign/personalized_marketing_campaign_10032023_1600_github.ipynb Installs the Pillow library, a fork of PIL, for image manipulation. Ensure version compatibility. ```python %pip install --quiet "pillow>=9.5,<10" ``` -------------------------------- ### Exclude Filter Expression Example Source: https://github.com/aws-samples/amazon-personalize-samples/blob/master/next_steps/operations/filter_rotator/README.md This example shows how to exclude items that are out of stock. It uses a SQL-like syntax to filter recommendations. ```sql EXCLUDE ItemID WHERE Items.out_of_stock IN ('yes') ``` -------------------------------- ### Apply Filters to Recommendations Source: https://context7.com/aws-samples/amazon-personalize-samples/llms.txt Demonstrates how to apply static, dynamic, and promotional filters when retrieving recommendations. Ensure correct quoting for string values in filterValues. ```python response = personalize_runtime.get_recommendations( campaignArn= campaign_arn, userId = "user_42", filterArn = filter_arn_static ) ``` ```python response_genre = personalize_runtime.get_recommendations( campaignArn = campaign_arn, userId = "user_42", filterArn = filter_arn_dynamic, filterValues = {"GENRE": '"Action"'} # double-quotes required for string values ) ``` ```python import time thirty_days_ago = int(time.time()) - 30 * 24 * 3600 response_promo = personalize_runtime.get_recommendations( campaignArn = campaign_arn, userId = "user_42", filterArn = filter_arn_promo, filterValues = {"MIN_TS": str(thirty_days_ago)}, promotionList= [{"filterArn": filter_arn_promo, "percentPromotedItems": 30}] ) ``` -------------------------------- ### Download Retail Demo Store Datasets Source: https://github.com/aws-samples/amazon-personalize-samples/blob/master/getting_started/notebooks_managed_domains/Building_Your_First_Recommender_Ecommerce.ipynb Downloads the 'interactions.csv' and 'items.csv' datasets from the Retail Demo Store project using wget. ```bash !wget https://code.retaildemostore.retail.aws.dev/csvs/interactions.csv !wget https://code.retaildemostore.retail.aws.dev/csvs/items.csv ``` -------------------------------- ### Upgrade Pip and Install Botocore Source: https://github.com/aws-samples/amazon-personalize-samples/blob/master/next_steps/generative_ai/personalized_recommender_agent/01_Recommender-Agent_Configure-Personalize-Resources.ipynb Ensures the latest version of pip and botocore are installed for optimal SDK performance and feature availability. ```bash import sys !{sys.executable} -m pip install --upgrade pip !{sys.executable} -m pip install --upgrade --no-deps --force-reinstall botocore ``` -------------------------------- ### Display Langchain Package Information Source: https://github.com/aws-samples/amazon-personalize-samples/blob/master/next_steps/generative_ai/personalized_marketing_campaign/personalized_marketing_campaign_10032023_1600_github.ipynb Shows the installed version and metadata for the langchain package. This is useful for confirming the installation and checking compatibility. ```python %pip show langchain ``` -------------------------------- ### Create Campaign with Item Exploration Configuration Source: https://github.com/aws-samples/amazon-personalize-samples/blob/master/next_steps/core_use_cases/user_personalization/user-personalization-with-exploration.ipynb Creates a campaign for the solution version, enabling item exploration with a specified exploration weight and item age cut-off. This encourages exploration of newer items. ```python create_campaign_response = personalize.create_campaign( name = prefix + suffix, solutionVersionArn = solution_version_arn, minProvisionedTPS = 1, campaignConfig = { "itemExplorationConfig": { "explorationWeight": "0.9", "explorationItemAgeCutOff": "7" } } ) campaign_arn = create_campaign_response['campaignArn'] print('campaign_arn:', campaign_arn) ``` -------------------------------- ### Initialize S3 Client Source: https://github.com/aws-samples/amazon-personalize-samples/blob/master/next_steps/core_use_cases/user_personalization/user-personalization-with-exploration.ipynb Sets up an S3 client using provided AWS credentials and region. This client is used for interacting with S3 buckets. ```python # Public s3 bucket owned by Personalize service which used to store the example dataset. personalize_s3_bucket = "personalize-cli-json-models" s3_client = boto3.Session(aws_access_key_id=accessKeyId, aws_secret_access_key=secretAccessKey, region_name=region_name).client('s3') ``` -------------------------------- ### Create Campaign for Recommendations Source: https://github.com/aws-samples/amazon-personalize-samples/blob/master/next_steps/core_use_cases/trending_now/trending_now_example.ipynb Deploys the trained solution version as a campaign to serve real-time recommendations. It sets the minimum provisioned transactions per second (TPS). ```python campaign_response = personalize.create_campaign( name="trending-now-campaign", solutionVersionArn=solution_version_arn, minProvisionedTPS=1, ) campaign_arn = campaign_response['campaignArn'] ``` -------------------------------- ### Get Recommendations from Campaign Source: https://github.com/aws-samples/amazon-personalize-samples/blob/master/next_steps/core_use_cases/trending_now/trending_now_example.ipynb Retrieves a list of recommended items from a trained Personalize campaign. Use this to get initial recommendations after training. ```python item_ids = personalize_runtime.get_recommendations(campaignArn=campaign_arn, numResults=25) ``` -------------------------------- ### Initialize Bedrock LLM with Anthropic Claude Instant Source: https://github.com/aws-samples/amazon-personalize-samples/blob/master/next_steps/generative_ai/personalized_marketing_campaign/personalized_marketing_campaign_10032023_1600_github.ipynb Configure and initialize the Bedrock LLM using the 'anthropic.claude-instant-v1' model. This model is chosen for its speed, cost-effectiveness, and capability in various text-based tasks. Ensure 'boto3_bedrock' client is available. ```python from langchain.llms.bedrock import Bedrock inference_modifier = {"max_tokens_to_sample":4096, "temperature":0.7, "top_k":250, "top_p":1, "stop_sequences": ["\n\nHuman"] } textgen_llm = Bedrock(model_id = "anthropic.claude-instant-v1", client = boto3_bedrock, model_kwargs = inference_modifier ) ``` -------------------------------- ### Print 'aws-sims' Solution Version Details Source: https://github.com/aws-samples/amazon-personalize-samples/blob/master/next_steps/core_use_cases/related_items/personalize_aws_similar_items_example.ipynb Outputs the details of the created 'aws-sims' solution version, including its ARN. This confirms the successful creation of the solution version. ```python sims_solution_version_arn = sims_solution_version_response['solutionVersionArn'] print(json.dumps(sims_solution_version_response, indent=2)) ``` -------------------------------- ### Install Step Functions SDK Source: https://github.com/aws-samples/amazon-personalize-samples/blob/master/next_steps/operations/ml_ops_ds_sdk/Personalize-Stepfunction-Workflow.ipynb Installs or upgrades the AWS Step Functions Data Science SDK. Ensure you are using the correct Python executable for your environment. ```python #import sys #!{sys.executable} -m pip install --upgrade stepfunctions ``` -------------------------------- ### Download Product Details YAML Source: https://github.com/aws-samples/amazon-personalize-samples/blob/master/next_steps/generative_ai/personalized_recommender_agent/02_Recommender-Agent_Build-Agent-ConverseAPI.ipynb Downloads the products.yaml file from a GitHub repository using wget. This file contains product details for the retail demo store. ```bash !wget https://raw.githubusercontent.com/aws-samples/retail-demo-store/master/src/products/data/products.yaml -O ./products.yaml ``` -------------------------------- ### Print Solution Version Details Source: https://github.com/aws-samples/amazon-personalize-samples/blob/master/next_steps/core_use_cases/metadata/item-text-unstructured-metadata.ipynb Prints the full response from creating a solution version, including the solution version ARN and metadata. Use `json.dumps` for pretty-printing. ```python user_personalization_with_solution_version_arn = user_personalization_solution_version_response['solutionVersionArn'] print(json.dumps(user_personalization_solution_version_response, indent=2)) ``` -------------------------------- ### Install and Upgrade Pip and Botocore Source: https://github.com/aws-samples/amazon-personalize-samples/blob/master/next_steps/core_use_cases/filters/promotions/Promotions.ipynb Ensures the latest versions of pip and botocore are installed for optimal SDK functionality. This is a prerequisite for using the AWS SDK with Amazon Personalize. ```python # Get the latest version of botocore to ensure we have the latest features in the SDK import sys !{sys.executable} -m pip install --upgrade pip !{sys.executable} -m pip install --upgrade --no-deps --force-reinstall botocore ``` -------------------------------- ### Install Boto3 for Amazon Bedrock Converse API Source: https://github.com/aws-samples/amazon-personalize-samples/blob/master/next_steps/generative_ai/personalized_recommender_agent/02_Recommender-Agent_Build-Agent-ConverseAPI.ipynb Installs or upgrades the Boto3 library to a version compatible with the Amazon Bedrock Converse API. Ensure version is greater than 1.34.123. ```python !pip install boto3==1.34.123 ``` -------------------------------- ### Build and Deploy the Application with SAM CLI Source: https://github.com/aws-samples/amazon-personalize-samples/blob/master/next_steps/operations/filter_rotator/README.md Use these commands to build the application's source code and then deploy it to your AWS account. The deploy command will prompt for configuration details. ```bash sam build --use-container --cached sam deploy --guided ``` -------------------------------- ### Import Libraries and Install Unzip Source: https://github.com/aws-samples/amazon-personalize-samples/blob/master/getting_started/notebooks/1.Building_Your_First_Campaign.ipynb Imports necessary Python libraries for AWS SDK, data manipulation, and time operations. Also installs the 'unzip' utility using conda. ```python # Imports import boto3 import json import numpy as np import pandas as pd import time import os !conda install -y -c conda-forge unzip ``` -------------------------------- ### Dynamic Filter Expression Example Source: https://github.com/aws-samples/amazon-personalize-samples/blob/master/next_steps/operations/filter_rotator/README.md This example illustrates a dynamic filter that includes items based on a genre specified at runtime. The filter expression uses a placeholder ($GENRES) for the dynamic value. ```sql INCLUDE ItemID WHERE Items.genre IN ($GENRES) ``` -------------------------------- ### Set Data Directory and Download MovieLens Dataset Source: https://github.com/aws-samples/amazon-personalize-samples/blob/master/next_steps/workshops/POC_in_a_box/completed/01_Validating_and_Importing_User_Item_Interaction_Data.ipynb Configures the data directory and conditionally downloads either the small or full MovieLens dataset, then unzips it. Use `USE_FULL_MOVIELENS = True` for the larger 25M dataset. ```python USE_FULL_MOVIELENS = False ``` ```python data_dir = "poc_data" !mkdir $data_dir if not USE_FULL_MOVIELENS: !cd $data_dir && wget http://files.grouplens.org/datasets/movielens/ml-latest-small.zip !cd $data_dir && unzip ml-latest-small.zip dataset_dir = data_dir + "/ml-latest-small/" else: !cd $data_dir && wget http://files.grouplens.org/datasets/movielens/ml-25m.zip !cd $data_dir && unzip ml-25m.zip dataset_dir = data_dir + "/ml-25m/" ``` -------------------------------- ### Get Personalized Ranking with a Rerank Campaign Source: https://github.com/aws-samples/amazon-personalize-samples/blob/master/next_steps/workshops/POC_in_a_box/completed/05_Interacting_with_Campaigns_and_Filters.ipynb Use the personalize_runtime client to get a reranked list of items for a given user and input list. This is useful for reordering a set of items based on personalization. ```python get_recommendations_response_rerank = personalize_runtime.get_personalized_ranking( campaignArn = rerank_campaign_arn, userId = user_id, inputList = rerank_item_list ) ``` -------------------------------- ### Import Libraries and Install Packages Source: https://github.com/aws-samples/amazon-personalize-samples/blob/master/next_steps/core_use_cases/objective_optimization/objective-optimization.ipynb Imports necessary Python libraries (boto3, pandas, numpy, etc.) and installs the 'unzip' package using conda. Sets pandas display options for chained assignments. ```python # Imports import boto3 import json import numpy as np import pandas as pd import time from botocore.exceptions import ClientError !conda install -y -c conda-forge unzip pd.options.mode.chained_assignment = None ``` -------------------------------- ### Initialize Bedrock Client and List Models Source: https://github.com/aws-samples/amazon-personalize-samples/blob/master/next_steps/generative_ai/personalized_marketing_campaign/personalized_marketing_campaign_10032023_1600_github.ipynb Initializes the AWS Bedrock client for the 'us-east-1' region and lists available foundation models. Requires AWS credentials and configuration. ```python import boto3 from PIL import Image bedrock_client = boto3.client('bedrock' , 'us-east-1', endpoint_url='https://bedrock.us-east-1.amazonaws.com') bedrock_client.list_foundation_models() ``` -------------------------------- ### Inference Job Output Example Source: https://github.com/aws-samples/amazon-personalize-samples/blob/master/next_steps/core_use_cases/updating_datasets/update-datasets-user-personalization-example.ipynb This is an example of the output from a batch inference job. It shows the job status progression and the final JSON output containing recommended items and their scores for specific user IDs. Errors, if any, are also detailed. ```json Inference Job Started on: 04:02:59 PM DatasetInferenceJob: CREATE PENDING DatasetInferenceJob: CREATE IN_PROGRESS DatasetInferenceJob: CREATE IN_PROGRESS DatasetInferenceJob: CREATE IN_PROGRESS DatasetInferenceJob: CREATE IN_PROGRESS DatasetInferenceJob: CREATE IN_PROGRESS DatasetInferenceJob: CREATE IN_PROGRESS DatasetInferenceJob: CREATE IN_PROGRESS DatasetInferenceJob: CREATE IN_PROGRESS DatasetInferenceJob: CREATE IN_PROGRESS DatasetInferenceJob: CREATE IN_PROGRESS DatasetInferenceJob: CREATE IN_PROGRESS DatasetInferenceJob: ACTIVE Inference Job Completed on: 04:16:01 PM {"input":{"userId":"4016"},"output":{"recommendedItems":["ccdf737c-c4fd-4c78-abd2-d5ef0428ef20","425cc876-3935-4e87-ad8d-77f42b0b6a75","6be08307-1ec0-44dc-b436-5d489a8010e8","2c1b34d6-0f3d-463d-be76-226cb87bdc6d","89c4eeb4-c146-4434-a9f1-6943b4b552dc","5a94b7d5-b210-44b3-9287-c8b0b5488a15","61b1ad14-4e70-4029-ba55-d17bbf4ab62b","8f8f015a-4166-4e9e-ac0b-6d980614ca5d","eecbee28-73a3-425d-84e8-516c326e399c","1daacea7-7d46-464a-8326-ed81951fecab"],"scores":[0.8449224,0.0186943,0.0148416,0.0100115,0.008885,0.0055925,0.0055068,0.0053057,0.0040184,0.0038556]},"error":null} {"input":{"userId":"2755"},"output":{"recommendedItems":["3630053e-3962-4549-bcce-402c3a980557","6d488475-1d67-4076-96b1-8e706709a847","b947ee58-a7e7-40bf-9926-42a445f3480f","90ccfbb9-4538-4951-af8d-4f728578b237","d537d92a-23fe-4673-a697-795652ff10c8","78080d05-b078-441f-b245-54b2a2dec872","61840d6a-6ba2-4ece-a644-6db6a3377b1c","2e95f6fc-6be7-46cf-9e50-8c35313c2768","b630250c-41f3-4f14-865c-c1dc12e448ac","d2d8147f-0f24-42c3-bcbe-a232bab7e94d"],"scores":[0.5338279,0.0922309,0.0577862,0.0209987,0.0206773,0.0148014,0.0141705,0.0129197,0.0120157,0.0077951]},"error":null} {"input":{"userId":"5866"},"output":{"recommendedItems":["5afced84-ed2d-4520-a06d-dcfeab382e52","6cc0deb8-4a56-4148-a2ab-677277522c80","e1146e90-3274-4ad6-a6a2-0170f0f8d597","575c0ac0-5494-4c64-a886-a9c0cf8b779a","24c62ad2-6977-4f69-be75-e37d897c1434","4496471c-b098-4915-9a1a-8b9e60043737","9c1a2048-7aac-4565-b836-d8d4f726322c","ccb407b1-7620-4303-8521-fea86c51f503","8cd7ffe0-a8a6-45b1-8d1f-bf731c9cd17b","aa564ee3-67ef-4428-8ad9-fe785a0fff63"],"scores":[0.5773515,0.2774695,0.1156833,0.0155479,0.0066222,0.0059881,3.773E-4,7.17E-5,6.78E-5,3.1E-5]},"error":null} {"input":{"userId":"2896"},"output":{"recommendedItems":["1dd4c2da-d174-43b1-8d40-fadc666c26c9","e99c24df-ebe9-429e-8c69-cd80132b87b3","e06f53cd-7776-41ce-9f7a-a88986192e24","75cb828e-ccc5-41ff-9bdd-9ac3dc7740aa","153b2374-36e3-466c-b08c-1078b839cd9b","2f2995da-4768-478a-a4ae-906b76d8c6fe","2a0a5c7b-ca68-4abf-9798-18ffb706832b","e8e48eb7-0b66-4087-b280-1c3a62804a5c","e5816ea5-3ce2-4b86-9530-9b221b357b43","17ab5081-8414-4cab-9003-033ec02b44da"],"scores":[0.6146742,0.0403633,0.0256617,0.0107649,0.0107413,0.0106864,0.0100017,0.007999,0.0077459,0.0065279]},"error":null} {"input":{"userId":"5023"},"output":{"recommendedItems":["c6dd0909-46f3-4cf9-a059-fbdff93198dd","441c2a65-4b68-4864-b014-04a9bd9fe08a","84d6c26d-9760-49d8-854b-0a22becd8241","9257351d-59f7-481a-86c4-30dea451afa2","635be5a7-3345-46f9-aa0d-419a6652b0f2","72ae72f3-e7f0-4f03-b8eb-12e78c77741d","d4cf35dd-b543-4b4f-9efb-c2de473c3fed","4994caee-f0b7-4ce8-a4df-d542ce1d9bda","0e3eb8f1-8f23-41fd-9f45-8e7747a5eb37","fe96a096-a0b6-4b20-a332-e11db6c0c7b0"],"scores":[0.3460568,0.2430023,0.1881302,0.025838,0.023981,0.0178926,0.0130894,0.0092487,0.0080664,0.0077484]},"error":null} CPU times: user 366 ms, sys: 32.5 ms, total: 399 ms Wall time: 13min 1s ``` -------------------------------- ### Build and Deploy ML Ops Pipeline Source: https://github.com/aws-samples/amazon-personalize-samples/blob/master/next_steps/operations/ml_ops/README.md Commands to clone the repository, build the SAM project, and deploy the Step Functions pipeline. Follow the guided deployment for configuration. ```bash git clone https://github.com/aws-samples/amazon-personalize-samples.git ``` ```bash cd next_steps/operations/ml_ops/personalize-step-functions ``` ```bash sam build ``` ```bash sam deploy --guided ``` -------------------------------- ### Execute Recommendation Workflow Source: https://github.com/aws-samples/amazon-personalize-samples/blob/master/next_steps/operations/ml_ops_ds_sdk/Personalize-Stepfunction-Workflow.ipynb Starts an execution of the 'Recommendation_Workflow4' using the `execute()` method. ```python recommendation_workflow_execution = recommendation_workflow.execute() ``` -------------------------------- ### Create and Train Solution Version Source: https://github.com/aws-samples/amazon-personalize-samples/blob/master/next_steps/core_use_cases/trending_now/trending_now_example.ipynb Creates a new version of the solution and initiates the training process. The `trainingMode` can be set to 'FULL' for complete retraining. ```python solution_version_response = personalize.create_solution_version(solutionArn=solution_arn, trainingMode="FULL") solution_version_arn = solution_version_response["solutionVersionArn"] print(json.dumps(solution_version_response, indent=4, default=str)) ``` -------------------------------- ### Create Solution Version Source: https://github.com/aws-samples/amazon-personalize-samples/blob/master/next_steps/generative_ai/personalized_marketing_campaign/airline_ticket_user_segmentation_09212023_github.ipynb Initiates the creation of a new version for a Personalize solution. ```python create_solution_version_response = personalize.create_solution_version( solutionArn = solution_arn ) solution_version_arn = create_solution_version_response['solutionVersionArn'] print(solution_version_arn) ``` -------------------------------- ### Display Boto3 Version Source: https://github.com/aws-samples/amazon-personalize-samples/blob/master/next_steps/generative_ai/personalized_recommender_agent/02_Recommender-Agent_Build-Agent-ConverseAPI.ipynb Prints the installed version of the Boto3 library to verify compatibility. ```python print(boto3.__version__) ``` -------------------------------- ### Create 'Top Picks for You' Recommender Source: https://github.com/aws-samples/amazon-personalize-samples/blob/master/next_steps/generative_ai/user_personalized_marketing_messaging_with_amazon_personalize_and_gen_ai/03_Train_Personalize_Model_02_Training.ipynb Creates a 'Top picks for you' recommender for the VIDEO_ON_DEMAND domain. Includes configuration to enable metadata with recommendations. Handles cases where the recommender already exists. ```python recommenderConfig = {"enableMetadataWithRecommendations": True} try: create_recommender_response = personalize.create_recommender( name = recommender_top_picks_for_you_name, recipeArn = 'arn:aws:personalize:::recipe/aws-vod-top-picks', datasetGroupArn = workshop_dataset_group_arn, recommenderConfig = {"enableMetadataWithRecommendations": True} ) workshop_recommender_top_picks_arn = create_recommender_response["recommenderArn"] print (json.dumps(create_recommender_response)) print ('\nCreating the Top Picks For You recommender with workshop_recommender_top_picks_arn = {}'.format(workshop_recommender_top_picks_arn)) except personalize.exceptions.ResourceAlreadyExistsException as e: workshop_recommender_top_picks_arn = 'arn:aws:personalize:'+region+':'+account_id+':recommender/'+recommender_top_picks_for_you_name print('The Top Picks For You recommender {} already exists.'.format(workshop_recommender_top_picks_arn)) print ('\nWe will be using the existing Top Picks For You recommender with workshop_recommender_top_picks_arn = {}'.format(workshop_recommender_top_picks_arn)) ``` -------------------------------- ### Get Item Description Source: https://github.com/aws-samples/amazon-personalize-samples/blob/master/next_steps/core_use_cases/related_items/personalize_aws_similar_items_example.ipynb Retrieves the description of an item given its ID from the items DataFrame. ```python def get_item_description(item_id): """ Takes in an ID, returns its brand """ return items_df.query('ITEM_ID=="{}"'.format(item_id))['DESCRIPTION'].item() ``` -------------------------------- ### Get Item Price Source: https://github.com/aws-samples/amazon-personalize-samples/blob/master/next_steps/core_use_cases/related_items/personalize_aws_similar_items_example.ipynb Retrieves the price of an item given its ID from the items DataFrame. ```python def get_item_price(item_id): """ Takes in an ID, returns its brand """ return items_df.query('ITEM_ID=="{}"'.format(item_id))['PRICE'].item() ``` -------------------------------- ### Create Campaign with Automatic Exploration Source: https://github.com/aws-samples/amazon-personalize-samples/blob/master/next_steps/core_use_cases/automatic_context/notebooks/1.Building_Personalize_Campaign.ipynb Create a campaign (a hosted solution version endpoint) and configure automatic item exploration. Set the minimum provisioned TPS and specify exploration parameters like explorationWeight. ```python create_campaign_response = personalize.create_campaign( name = "personalize-auto-context-demo-campaign", solutionVersionArn = solution_version_arn, minProvisionedTPS = 1, campaignConfig = { "itemExplorationConfig": { "explorationWeight": "0.5" } } ) campaign_arn = create_campaign_response['campaignArn'] print(json.dumps(create_campaign_response, indent=2)) ``` -------------------------------- ### Download Sample Datasets Source: https://github.com/aws-samples/amazon-personalize-samples/blob/master/next_steps/core_use_cases/user_personalization/user-personalization-with-exploration.ipynb Downloads sample interaction and item metadata datasets from a public S3 bucket to the local environment for use in the demo. ```python interaction_dataset_key = "sample-dataset/interactions-sample.csv" items_dataset_key = "sample-dataset/items-with-creation-timestamp-sample.csv" interactions_file = os.getcwd() + "/interaction_raw.csv" items_metadata_file = os.getcwd() + "/items_raw.csv" s3_client.download_file(personalize_s3_bucket, interaction_dataset_key, interactions_file) s3_client.download_file(personalize_s3_bucket, items_dataset_key, items_metadata_file) ``` -------------------------------- ### Get Item Brand Source: https://github.com/aws-samples/amazon-personalize-samples/blob/master/next_steps/core_use_cases/related_items/personalize_aws_similar_items_example.ipynb Retrieves the brand of an item given its ID from the items DataFrame. ```python def get_item_brand(item_id): """ Takes in an ID, returns its brand """ return items_df.query('ITEM_ID=="{}"'.format(item_id))['BRAND'].item() ``` -------------------------------- ### Render Campaign Workflow Graph (Commented Out) Source: https://github.com/aws-samples/amazon-personalize-samples/blob/master/next_steps/operations/ml_ops_ds_sdk/Personalize-Stepfunction-Workflow.ipynb This is a commented-out example for rendering the graph of a campaign workflow. ```python #Main_workflow_execution.render_graph() ``` -------------------------------- ### Create Similar Items Solution and Handle Existing Resources Source: https://github.com/aws-samples/amazon-personalize-samples/blob/master/next_steps/generative_ai/personalized_recommender_agent/01_Recommender-Agent_Configure-Personalize-Resources.ipynb Creates a Personalize solution for the Similar-Items recipe. If a solution with the same name already exists, it retrieves the existing solution ARN and its most recent solution version ARN. ```python similar_items_solution_version_arn = None try: create_solution_response = personalize.create_solution( name = "related-items-demo", datasetGroupArn = dataset_group_arn, recipeArn = similar_items_recipe_arn ) similar_items_solution_arn = create_solution_response['solutionArn'] print(json.dumps(create_solution_response, indent=2)) except personalize.exceptions.ResourceAlreadyExistsException: print('You aready created this solution, seemingly') paginator = personalize.get_paginator('list_solutions') for paginate_result in paginator.paginate(datasetGroupArn = dataset_group_arn): for solution in paginate_result['solutions']: if solution['name'] == 'related-items-demo': similar_items_solution_arn = solution['solutionArn'] print(f'Similar Items solution ARN = {similar_items_solution_arn}') response = personalize.list_solution_versions( solutionArn = similar_items_solution_arn, maxResults = 100 ) if len(response['solutionVersions']) > 0: similar_items_solution_version_arn = response['solutionVersions'][-1]['solutionVersionArn'] print(f'Will use most recent solution version for this solution: {similar_items_solution_version_arn}') break ``` -------------------------------- ### Get Movie Title by ID Source: https://github.com/aws-samples/amazon-personalize-samples/blob/master/next_steps/workshops/POC_in_a_box/05_Interacting_with_Campaigns_and_Filters.ipynb Retrieves the title of a movie from the loaded DataFrame using its ID. ```python movie_id_example = 589 title = items_df.loc[movie_id_example]['title'] print(title) ``` -------------------------------- ### Get Recommendation Output Source: https://github.com/aws-samples/amazon-personalize-samples/blob/master/next_steps/operations/ml_ops_ds_sdk/Personalize-Stepfunction-Workflow.ipynb Retrieves the output payload from the recommendation workflow execution, specifically the 'item_list'. ```python item_list = recommendation_workflow_execution.get_output()['Payload']['item_list'] ``` -------------------------------- ### Load Item Data and Sample User Source: https://github.com/aws-samples/amazon-personalize-samples/blob/master/next_steps/operations/ml_ops_ds_sdk/Personalize-Stepfunction-Workflow.ipynb Loads item metadata from a CSV file and samples a user, item, and rating from the dataset for recommendation generation. ```python items = pd.read_csv('./ml-100k/u.item', sep='|', usecols=[0,1], encoding='latin-1') items.columns = ['ITEM_ID', 'TITLE'] user_id, item_id, rating, timestamp = data.sample().values[0] user_id = int(user_id) item_id = int(item_id) print("user_id",user_id) print("items",items) item_title = items.loc[items['ITEM_ID'] == item_id].values[0][-1] print("USER: {}".format(user_id)) print("ITEM: {}".format(item_title)) print("ITEM ID: {}".format(item_id)) ``` -------------------------------- ### Get Random User ID Source: https://github.com/aws-samples/amazon-personalize-samples/blob/master/getting_started/notebooks/1.Building_Your_First_Campaign.ipynb Selects a random user ID from the dataset for generating recommendations. ```python # Getting a random user: user_id, item_id, _ = data.sample().values[0] print("USER: {}".format(user_id)) ``` -------------------------------- ### Create SQLite Database for Products Source: https://github.com/aws-samples/amazon-personalize-samples/blob/master/next_steps/generative_ai/personalized_recommender_agent/02_Recommender-Agent_Build-Agent-ConverseAPI.ipynb Creates a local SQLite database named 'products.db' and populates it with product data from a YAML file. It defines a 'products' table and inserts data, handling potential missing fields. ```python import sqlite3 import yaml # Read the YAML file and parse the data with open('products.yaml', 'r') as file: products = yaml.safe_load(file) # Create an SQLite database and table for storing the products conn = sqlite3.connect('products.db') cursor = conn.cursor() # Create the products table cursor.execute(''' CREATE TABLE IF NOT EXISTS products ( id TEXT PRIMARY KEY, category TEXT, current_stock INTEGER, description TEXT, gender_affinity TEXT, image TEXT, image_license TEXT, link TEXT, name TEXT, price REAL, style TEXT, where_visible TEXT, promoted BOOLEAN ) ''') # Insert the products into the SQLite table for product in products: # Prepare the product data with null values for missing fields product_data = ( product.get('id', None), product.get('category', None), product.get('current_stock', None), product.get('description', None), product.get('gender_affinity', None), product.get('image', None), product.get('image_license', None), product.get('link', None), product.get('name', None), product.get('price', None), product.get('style', None), product.get('where_visible', None), product.get('promoted', False) ) # Insert the product into the table cursor.execute(''' INSERT OR IGNORE INTO products ( id, category, current_stock, description, gender_affinity, image, image_license, link, name, price, style, where_visible, promoted ) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) ''', product_data) # Commit the changes and close the connection conn.commit() conn.close() ``` -------------------------------- ### Get Random Item ID Source: https://github.com/aws-samples/amazon-personalize-samples/blob/master/next_steps/core_use_cases/related_items/personalize_aws_similar_items_example.ipynb Selects a random item ID from the asin_interaction_count_df DataFrame for testing recommendations. ```python asin_interaction_count_df.sample()['asin'].item() ``` -------------------------------- ### Setup AWS Credentials Source: https://github.com/aws-samples/amazon-personalize-samples/blob/master/next_steps/core_use_cases/user_personalization/user-personalization-with-exploration.ipynb Configure your AWS access key ID, secret access key, and region for API access. Ensure these credentials have the necessary permissions. ```python accessKeyId = "" secretAccessKey = "" region_name = "" ``` -------------------------------- ### Get Item Interaction Count Source: https://github.com/aws-samples/amazon-personalize-samples/blob/master/next_steps/core_use_cases/related_items/personalize_aws_similar_items_example.ipynb Retrieves the interaction count for a given item ID from the asin_interaction_count_df DataFrame. ```python def get_item_count(item_id): return asin_interaction_count_df.query('asin=="{}"'.format(item_id))['count'].item() ``` -------------------------------- ### Create Solution Version for Similar Items Source: https://github.com/aws-samples/amazon-personalize-samples/blob/master/next_steps/generative_ai/personalized_recommender_agent/01_Recommender-Agent_Configure-Personalize-Resources.ipynb Creates a solution version for the similar items solution if one does not already exist. Prints the ARN upon successful creation. ```python if not similar_items_solution_version_arn: create_solution_version_response = personalize.create_solution_version( solutionArn = similar_items_solution_arn ) similar_items_solution_version_arn = create_solution_version_response['solutionVersionArn'] print(json.dumps(create_solution_version_response, indent=2)) else: print(f'Solution version {similar_items_solution_version_arn} already exists; not creating') ``` -------------------------------- ### Get Recommendations Source: https://github.com/aws-samples/amazon-personalize-samples/blob/master/getting_started/notebooks/1.Building_Your_First_Campaign.ipynb Retrieves personalized movie recommendations for a given user ID using the trained campaign. ```python get_recommendations_response = personalize_runtime.get_recommendations( campaignArn = campaign_arn, userId = str(user_id), ) # Update DF rendering pd.set_option('display.max_rows', 30) print("Recommendations for user: ", user_id) item_list = get_recommendations_response['itemList'] recommendation_list = [] for item in item_list: title = get_movie_title(item['itemId']) recommendation_list.append(title) recommendations_df = pd.DataFrame(recommendation_list, columns = ['OriginalRecs']) recommendations_df ``` -------------------------------- ### Initialize Personalize Clients Source: https://github.com/aws-samples/amazon-personalize-samples/blob/master/next_steps/data_science/offline_performance_evaluation/personalize_temporal_holdout.ipynb Initializes boto3 clients for Amazon Personalize and Personalize Runtime services. ```python personalize = boto3.client('personalize') personalize_runtime = boto3.client('personalize-runtime') ```