### Installation and Setup Source: https://github.com/autominda1/generative-ai/blob/main/gemini/agents/always-on-memory-agent/README.md Commands to clone the repository, install dependencies, and configure the required API key. ```bash git clone https://github.com/Shubhamsaboo/always-on-memory-agent.git cd always-on-memory-agent pip install -r requirements.txt ``` ```bash export GOOGLE_API_KEY="your-gemini-api-key" ``` -------------------------------- ### Install Dependencies and Start Server Source: https://github.com/autominda1/generative-ai/blob/main/gemini/multimodal-live-api/native-audio-websocket-demo-apps/plain-js-demo-app/README.md Install Python dependencies, authenticate with Google Cloud, and start the server which serves the UI and WebSocket proxy. Then open the application in your browser. ```bash pip3 install -r requirements.txt gcloud auth application-default login python3 server.py open http://localhost:8000 ``` -------------------------------- ### Run Backend Setup and Application Source: https://github.com/autominda1/generative-ai/blob/main/gemini/sample-apps/quickbot/image-background-changer-using-imagen3/README.md Execute the setup script and start the FastAPI server using Uvicorn. ```bash # from the backend/ directory python3 setup.py ``` ```bash # from the backend/ directory uvicorn main:app --reload --port 8080 ``` -------------------------------- ### Setup Project and Install Dependencies Source: https://github.com/autominda1/generative-ai/blob/main/genkit/postcard-generator/docs/README.md Run these commands to set up the project environment, initialize Terraform, and install Node.js dependencies. ```sh export PROJECT_ID="my-project-id" cd terraform terraform init && terraform apply -var="project_id=${PROJECT_ID}" cd .. npm install ``` -------------------------------- ### Quick Start Workflow Source: https://github.com/autominda1/generative-ai/blob/main/search/auto-rag-eval/README.md A complete sequence of commands to install dependencies, configure the environment, and generate a benchmark. ```bash # 1. Install requirements pip install -r requirements.txt # 2. Download required files from Google Cloud Storage gcloud storage cp gs://github-repo/search/auto-rag-eval/qa_profiles.json . # 3. Set up environment # Edit .env with your values: # - PROJECT_ID=your-gcp-project-id # - LOCATION=us-central1 # - DATA_STORE_ID=your-vertex-ai-search-datastore-id # 4. Authenticate with Google Cloud gcloud auth application-default login # 5. Generate benchmark python main.py --docs 2 --chunks 2 --clues 2 --profiles 2 # 6. (Optional) Transform benchmark for evaluation frameworks python transform_benchmark.py benchmark.json converted_benchmark.json ``` -------------------------------- ### Initialize Application Environment Source: https://github.com/autominda1/generative-ai/blob/main/gemini/function-calling/sql-talk-app/tutorial.md Executes the setup script to enable required APIs, install dependencies, and launch the application. ```bash bash setup.sh ``` -------------------------------- ### Initialize Frontend Environment Source: https://github.com/autominda1/generative-ai/blob/main/gemini/multimodal-live-api/native-audio-websocket-demo-apps/gaming-assistant-demo-app/README.md Commands to install Node.js dependencies and start the React development server. ```bash # Install Node modules npm install # Start development server npm run dev ``` -------------------------------- ### Run Setup Script Source: https://github.com/autominda1/generative-ai/blob/main/gemini/sample-apps/quickbot/conversational-app-multi-playbook/backend/README.md Executes the project's setup script. This command should be run after the virtual environment is activated and dependencies are installed. ```bash python3 setup.py ``` -------------------------------- ### Run the Backend Server Source: https://github.com/autominda1/generative-ai/blob/main/gemini/multimodal-live-api/project-livewire/README.md Install dependencies and start the Python WebSocket server locally. ```bash pip install -r requirements.txt python server.py # Backend runs on localhost:8081 ``` -------------------------------- ### Install Python Dependencies and Run Server Source: https://github.com/autominda1/generative-ai/blob/main/gemini/multimodal-live-api/native-audio-websocket-demo-apps/plain-js-python-sdk-demo-app/README.md Install necessary Python packages and start the FastAPI server. Ensure you have authenticated with Google Cloud. ```bash pip install -r requirements.txt gcloud auth application-default login python main.py ``` -------------------------------- ### Setup Python Environment and Dependencies Source: https://github.com/autominda1/generative-ai/blob/main/audio/speech/sample-apps/live-translator/README.md Initializes a virtual environment and installs required packages from requirements.txt. ```bash python3 -m venv gemini-streamlit source gemini-streamlit/bin/activate pip install -r requirements.txt ``` -------------------------------- ### Setup Environment for Colab Source: https://github.com/autominda1/generative-ai/blob/main/gemini/evaluation/synthetic-data-evals/run.ipynb Initializes the environment by cloning the repository and installing dependencies via uv. ```python IN_COLAB = False try: pass IN_COLAB = True !git clone https://github.com/ash0ts/generative-ai.git %cd generative-ai/gemini/evaluation/synthetic-data-evals !pip install -qqq uv !uv pip install --system --requirements pyproject.toml except: pass ``` -------------------------------- ### Run function locally Source: https://github.com/autominda1/generative-ai/blob/main/search/cloud-function/python/README.md Install dependencies and start the local development server using the Functions Framework. ```bash pip install -r requirements.txt pip install functions-framework functions-framework --target=vertex_ai_search ``` -------------------------------- ### Setup LLM EvalKit Environment Source: https://github.com/autominda1/generative-ai/blob/main/tools/llmevalkit/README.md Clone the repository, set up a Python virtual environment, install dependencies, and run the Streamlit application. Ensure to configure the .env file with your BUCKET_NAME and PROJECT_ID. ```bash git clone https://github.com/GoogleCloudPlatform/generative-ai.git cd generative-ai/tools/llmevalkit python -m venv venv source venv/bin/activate pip install -r requirements.txt streamlit run index.py ``` -------------------------------- ### Configure Installation Directory and Settings Source: https://github.com/autominda1/generative-ai/blob/main/gemini/sample-apps/photo-discovery/app/windows/CMakeLists.txt Sets up installation directories and ensures the 'install' step is default for Visual Studio builds. The installation prefix is set to the executable's directory for in-place running. ```cmake set(BUILD_BUNDLE_DIR "$") set(CMAKE_VS_INCLUDE_INSTALL_TO_DEFAULT_BUILD 1) if(CMAKE_INSTALL_PREFIX_INITIALIZED_TO_DEFAULT) set(CMAKE_INSTALL_PREFIX "${BUILD_BUNDLE_DIR}" CACHE PATH "..." FORCE) endif() set(INSTALL_BUNDLE_DATA_DIR "${CMAKE_INSTALL_PREFIX}/data") set(INSTALL_BUNDLE_LIB_DIR "${CMAKE_INSTALL_PREFIX}") ``` -------------------------------- ### Start Genkit Development Environment Source: https://github.com/autominda1/generative-ai/blob/main/genkit/postcard-generator/docs/README.md Use this command to start the Genkit development server. Navigate to localhost:4000 after starting. ```sh export PROJECT_ID="my-project-id" npx genkit@latest start ``` -------------------------------- ### Generate Developer Getting Started Guide with Gemini API (Python) Source: https://github.com/autominda1/generative-ai/blob/main/gemini/use-cases/code/analyze_codebase.ipynb Creates a getting started guide for developers using the Gemini API's streaming option. This method is suitable for generating longer content incrementally. ```python question = """ Provide a getting started guide to onboard new developers to the codebase. """ # Generate text using streaming method responses = client.models.generate_content_stream( model=MODEL_ID, contents=question, config=GenerateContentConfig( cached_content=cached_content.name, ), ) for response in responses: print(response.text, end="") ``` -------------------------------- ### Initialize a Multi-Turn Chat with System Instructions Source: https://github.com/autominda1/generative-ai/blob/main/sdk/intro_genai_sdk.ipynb Start a chat session with a specified model and configure system instructions to guide the AI's persona and capabilities. This example sets the AI as an expert software developer. ```python system_instruction = """ You are an expert software developer and a helpful coding assistant. You are able to generate high-quality code in any programming language. """ chat = client.chats.create( model=MODEL_ID, config=GenerateContentConfig( system_instruction=system_instruction, temperature=0.5, ), ) ``` -------------------------------- ### Run Backend Setup Script Source: https://github.com/autominda1/generative-ai/blob/main/gemini/sample-apps/quickbot/README.md Execute the setup script to perform initial configurations. ```bash # from the backend/ directory python3 setup.py ``` -------------------------------- ### Complete Client Configuration Example Source: https://github.com/autominda1/generative-ai/blob/main/sdk/retries/configure_retries.ipynb A full example of initializing the client and executing a model request with custom retry settings. ```python client = genai.Client( vertexai=True, project=PROJECT_ID, location=LOCATION, http_options=types.HttpOptions( retry_options=types.HttpRetryOptions( initial_delay=1.0, attempts=10, http_status_codes=[429, 500, 502, 503, 504], ), timeout=120 * 1000, ), ) # You can now use this client to make API calls with the custom retry policy. # For example: response = client.models.generate_content( model="gemini-3-flash-preview", contents="Tell me a story" ) display(Markdown(response.text)) ``` -------------------------------- ### Install Helm Source: https://github.com/autominda1/generative-ai/blob/main/gemini/sample-apps/genwealth/alternate-configs/alloydb-omni.md Download and execute the Helm installation script. ```bash curl -fsSL -o get_helm.sh https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 chmod 700 get_helm.sh ./get_helm.sh ``` -------------------------------- ### Example Invocation Source: https://github.com/autominda1/generative-ai/blob/main/search/vertexai-search-options/vertex_ai_search_website_summary.ipynb An example of how to invoke the `invoke` function with a search query to get a summarized answer from Vertex AI. ```python search_query = "What are the benefits of Vertex AI?" invoke(search_query) ``` -------------------------------- ### Clone Repository and Navigate to Project Source: https://github.com/autominda1/generative-ai/blob/main/gemini/agents/genai-experience-concierge/langgraph-demo/backend/README.md Clone the repository and navigate to the project root directory to begin setup. ```bash # Clone repository and navigate to project root directory git clone https://github.com/GoogleCloudPlatform/generative-ai.git cd generative-ai/gemini/agents/genai-experience-concierge ``` -------------------------------- ### Zero-Shot Prompting Example Source: https://github.com/autominda1/generative-ai/blob/main/gemini/prompts/intro_prompt_design.ipynb Use zero-shot prompting when no examples are needed to guide the LLM. This method is suitable for open-ended tasks. ```python prompt = """Decide whether a Tweet's sentiment is positive, neutral, or negative. Tweet: I loved the new YouTube video you made! Sentiment: """ response = client.models.generate_content(model=MODEL_ID, contents=prompt) display(Markdown(response.text)) ``` -------------------------------- ### Setup and Deployment Source: https://github.com/autominda1/generative-ai/blob/main/vision/use-cases/hey_llm/README.md Instructions for setting up the necessary tools and deploying the Google Apps Script for HEY_LLM and IMAGEN. ```APIDOC ## Prerequisites and Setup ### Install and set up clasp 1. Install clasp globally: `npm install -g @google/clasp` 2. Enable the Google Apps Script API: Visit [https://script.google.com/home/usersettings](https://script.google.com/home/usersettings) and enable the API. 3. Log in with clasp: Run `clasp login` and follow the authentication instructions. ### Prepare a Google Cloud project 1. Create or select a project in Google Cloud Console. 2. Navigate to "APIs & Services" > "OAuth consent screen" and configure it. ### Enable Google Drive API 1. Navigate to "APIs & Services" > "Library". 2. Search for and enable the "Google Drive API". ### Create a spreadsheet with HEY_LLM and IMAGEN 1. Navigate to the project directory. 2. Run the command: `TITLE="YOUR_SHEET_NAME" npm run create` ### Get Spreadsheet ID and Script ID - The Spreadsheet ID and Script ID are provided in the output of the `npm run create` command. - They can also be found in the `.clasp.json` file (`scriptId` and `parentId`). ``` -------------------------------- ### Initialize Backend Environment Source: https://github.com/autominda1/generative-ai/blob/main/gemini/multimodal-live-api/native-audio-websocket-demo-apps/gaming-assistant-demo-app/README.md Commands to install dependencies, authenticate with Google Cloud, and launch the proxy server. ```bash # Install dependencies pip install -r requirements.txt # Authenticate with Google Cloud gcloud auth application-default login # Start the proxy server python server.py ``` -------------------------------- ### Define One-Shot Example for Improved Reasoning Source: https://github.com/autominda1/generative-ai/blob/main/gemini/prompts/examples/chain_of_thought_react.ipynb Provides a complete example of the expected reasoning process to guide the LLM toward more accurate outputs. ```python one_shot_example = """ JSON: { "@type" : "dcat:Dataset", "description" : "The primary purpose of this system of records is to properly pay medical insurance benefits to or on behalf of entitled beneficiaries.", "title" : "Medicare Multi-Carrier Claims System", "accessLevel" : "restricted public", "dataQuality" : true, "identifier" : "b6ffafab-1cfd-42dd-b8cb-7a554efaefa7", "landingPage" : "http://www.cms.gov/Research-Statistics-Data-and-Systems/Computer-Data-and-Systems/Privacy/Systems-of-Records-Items/09-70-0501-MCS.html", "license" : "http://www.usa.gov/publicdomain/label/1.0/", "modified" : "2014-09-30", "rights" : "Contains personally identifiable information and is subject to the Privacy Act of 1974, as amended at 5 United States Code (U.S.C.) 552a. Requests should be directed to the appropriate System Manager, identified in the System of Records notice.", "primaryITInvestmentUII" : "009-000004256, 009-000004254", "systemOfRecords" : "09-70-0501", "contactPoint" : { "@type" : "vcard:Contact", "fn" : "Health Data Initiative", "hasEmail" : "mailto:Healthdata@hhs.gov" }, "bureauCode" : [ "009:38" ], "keyword" : [ "medicare", "part b", "claims" ], "programCode" : [ "009:078" ], "theme" : [ "Medicare" ], "publisher" : { "@type" : "org:Organization", "name" : "Centers for Medicare & Medicaid Services", "subOrganizationOf" : { "@type" : "org:Organization", "name" : "Department of Health & Human Services" } } } Answer: The 'rights' tag says 'Contains personally identifiable information' so pii is True. The 'modified' tag is '2014-09-30'. The current year is 2023, 2023 minus 2014 is 9, so the age is 9. To determine keywords I will look at all the fields that describe the dataset. Then I will take the most salient and distinctive aspects of the fields and make those keywords. Looking at all the fields, the ones that describe the dataset are "description" and "title". The "title" field is "Medicare Multi-Carrier Claims System". Good keywords from the "title" field are "medicare" and "claims". The "description" field is ""The primary purpose of this system of records is to properly pay medical insurance benefits to or on behalf of entitled beneficiaries." Good keywords from the "description" field are "medical insurance benefits". Good proposed keywords from both fields are "medicare", "claims", and "medical insurance benefits". Next inspect the "keyword" field to make sure the proposed keywords are not already included. The "keyword" field contains the keywords "medicare", "part b", and "claims". """ ``` -------------------------------- ### Environment Setup Source: https://github.com/autominda1/generative-ai/blob/main/gemini/sample-apps/llamaindex-rag/README.md Create a Conda environment and install dependencies using Poetry. ```bash conda create -n llamaindex-rag python=3.10 poetry install ``` -------------------------------- ### Initialize Backend Environment Source: https://github.com/autominda1/generative-ai/blob/main/gemini/sample-apps/quickbot/image-background-changer-using-imagen3/README.md Create a virtual environment and install required Python dependencies. ```bash # Check if you are already in an environment pip -V # If not, create and activate (for Linux/macOS) python3 -m venv .venv source .venv/bin/activate # Install requirements pip3 install -r requirements.txt ``` -------------------------------- ### Set Up Tutorial Folder Structure Source: https://github.com/autominda1/generative-ai/blob/main/embeddings/intro_embeddings_tuning.ipynb Creates a directory structure for the tutorial, including 'tutorial' and 'tutorial/data' subdirectories, using Python's `pathlib`. This organizes project files and data. ```python from pathlib import Path as path root_path = path.cwd() tutorial_path = root_path / "tutorial" data_path = tutorial_path / "data" data_path.mkdir(parents=True, exist_ok=True) ``` -------------------------------- ### One-Shot Prompting Example Source: https://github.com/autominda1/generative-ai/blob/main/gemini/prompts/intro_prompt_design.ipynb Employ one-shot prompting by providing a single example to guide the LLM's response format and style. This helps in achieving more predictable outputs. ```python prompt = """Decide whether a Tweet's sentiment is positive, neutral, or negative. Tweet: I loved the new YouTube video you made! Sentiment: positive Tweet: That was awful. Super boring 😠 Sentiment: """ response = client.models.generate_content(model=MODEL_ID, contents=prompt) display(Markdown(response.text)) ``` -------------------------------- ### Install Vertex AI SDK and Hugging Face packages Source: https://github.com/autominda1/generative-ai/blob/main/open-models/use-cases/guess_app.ipynb Install the google-cloud-aiplatform SDK and the huggingface_hub package to run this example. Gradio is also included for potential UI development. ```python %pip install --upgrade --user --quiet google-cloud-aiplatform huggingface_hub gradio ``` -------------------------------- ### Define Tutorial Directories Source: https://github.com/autominda1/generative-ai/blob/main/open-models/serving/vertex_ai_ollama_gemma2_rag_agent.ipynb Sets up local directories for tutorial assets, including build and model storage locations. Ensures the model directory exists. ```python from etils import epath TUTORIAL_DIR = epath.Path("ollama_on_vertex_ai_tutorial") BUILD_DIR = TUTORIAL_DIR / "build" MODELS_DIR = BUILD_DIR / "ollama_models" MODELS_DIR.mkdir(exist_ok=True, parents=True) ``` -------------------------------- ### Generate Memories with Custom Topics + Few-Shot Examples Source: https://github.com/autominda1/generative-ai/blob/main/agents/agent_engine/memory_bank/get_started_with_memory_bank_custom_topics.ipynb Use this to generate memories when you have specific topics and few-shot examples to guide the extraction process. Ensure the client and necessary names are configured. ```python print("\n" + "=" * 80) print("GENERATING MEMORIES WITH CUSTOM TOPICS + FEW-SHOT EXAMPLES") print("=" * 80 + "\n") print("ā³ Processing conversation...") # Generate memories with few-shot configuration fewshot_operation = client.agent_engines.memories.generate( name=custom_engine_name, vertex_session_source={"session": fewshot_session_name}, config={"wait_for_completion": True}, ) print("\nāœ… Memory generation complete!") ``` -------------------------------- ### Configure project and initialize SDK Source: https://github.com/autominda1/generative-ai/blob/main/gemini/evaluation/create_agent_and_run_evaluation.ipynb Sets up project IDs, locations, and GCS buckets, then initializes the Vertex AI SDK and client. ```python import os from urllib.parse import urlparse import vertexai from vertexai import Client from google.cloud import storage from google.genai import types as genai_types def get_config_value(initial_value: str, placeholder: str, env_var_name: str) -> str: """Gets a configuration value or environment variable if unspecified.""" if not initial_value or initial_value == placeholder: return os.environ.get(env_var_name) return initial_value def get_or_create_gcs_bucket(project_id: str, gcs_dest: str) -> str: """Retrieves GCS bucket or creates a default.""" gcs_dest = gcs_dest or f"{project_id}/agent-evaluation" storage_client = storage.Client(project=project_id) bucket = gcs_dest.replace("gs://", "").split("/")[0] if not storage_client.lookup_bucket(bucket): print(f"Creating bucket: {bucket}") storage_client.create_bucket(bucket) if not gcs_dest.startswith("gs://"): return f"gs://{gcs_dest}" return gcs_dest # Configuration # fmt: off PROJECT_ID = "" # @param {type: "string", placeholder: "[your-project-id]", isTemplate: true} PROJECT_ID = get_config_value(PROJECT_ID, "[your-project-id]", "GOOGLE_CLOUD_PROJECT") assert PROJECT_ID, "Please specify a valid project" LOCATION = "" # @param {type: "string", placeholder: "[us-central1]", isTemplate: true} LOCATION = get_config_value(LOCATION, None, "GOOGLE_CLOUD_REGION") assert LOCATION, "Please specify a valid location" GCS_DEST = "" # @param {type: "string", placeholder: "[your-gcs-bucket]", isTemplate: true} # fmt: on GCS_DEST = get_config_value( GCS_DEST, "[your-gcs-bucket]", "GOOGLE_CLOUD_STORAGE_BUCKET" ) GCS_DEST = get_or_create_gcs_bucket(PROJECT_ID, GCS_DEST) assert GCS_DEST, "Please specify a valid GCS destination" STAGING_BUCKET = f"gs://{urlparse(GCS_DEST).netloc}" # Initialize SDK vertexai.init(project=PROJECT_ID, location=LOCATION) client = Client( project=PROJECT_ID, location=LOCATION, http_options=genai_types.HttpOptions(api_version="v1beta1"), ) ``` -------------------------------- ### Start Frontend Server Source: https://github.com/autominda1/generative-ai/blob/main/gemini/multimodal-live-api/project-livewire/docs/local_setup.md Launch the frontend HTTP server to serve client files. ```bash cd ../client # Or navigate from the project root: cd project-livewire/client ``` ```bash python -m http.server 8000 ``` -------------------------------- ### Setup BigQuery Client and Dataset Source: https://github.com/autominda1/generative-ai/blob/main/gemini/use-cases/video-analysis/video_analysis_with_youtube_data_api_and_batch_prediction.ipynb Sets up the BigQuery client, defines dataset and table names, and creates the dataset if it does not exist. Assumes PROJECT_ID and LOCATION are defined. ```python # Create BQ client BQ_CLIENT = bigquery.Client(project=PROJECT_ID) # Function to run BQ query using created client and return results as data frame def get_bq_query_results_as_df(query_text): bq_results_table = BQ_CLIENT.query(query_text).to_dataframe() return bq_results_table # Names of BQ dataset and tables to be created/used in this notebook BQ_DATASET = "youtube_video_analysis" # @param {type:"string"} # fmt: off BATCH_PREDICTION_REQUESTS_TABLE = "video_analysis_batch_requests" # @param {type:"string"} BATCH_PREDICTION_RESULTS_TABLE = "video_analysis_batch_results" # @param {type:"string"} # fmt: on # Create BQ dataset if it doesn't already exist create_dataset_if_nec_query = f""" CREATE SCHEMA IF NOT EXISTS `{BQ_DATASET}` OPTIONS( location='{LOCATION}' ); """ get_bq_query_results_as_df(create_dataset_if_nec_query) ``` -------------------------------- ### Run Frontend Application Source: https://github.com/autominda1/generative-ai/blob/main/gemini/sample-apps/quickbot/image-background-changer-using-imagen3/README.md Start the frontend development server. ```bash npm start # Or, for many Angular projects: # ng serve ``` -------------------------------- ### Execute Backend Setup Script Source: https://github.com/autominda1/generative-ai/blob/main/gemini/sample-apps/quickbot/website-search-using-agent-builder/README.md Run the Python script to initialize backend resources like BigQuery tables. ```bash python3 backend/setup.py ``` -------------------------------- ### Define Custom Memory Topics and Examples Source: https://github.com/autominda1/generative-ai/blob/main/agents/agent_engine/memory_bank/get_started_with_memory_bank_on_adk.ipynb Configure custom memory topics with labels, descriptions, and examples for memory generation. This setup is useful for fine-tuning the AI's understanding of specific feedback or information. ```python from vertexai.types import MemoryBankCustomizationConfig as CustomizationConfig from vertexai.types import ( MemoryBankCustomizationConfigGenerateMemoriesExample as GenerateMemoriesExample, ) from vertexai.types import ( MemoryBankCustomizationConfigGenerateMemoriesExampleConversationSource as ConversationSource, ) from vertexai.types import ( MemoryBankCustomizationConfigGenerateMemoriesExampleConversationSourceEvent as ConversationSourceEvent, ) from vertexai.types import ( MemoryBankCustomizationConfigGenerateMemoriesExampleGeneratedMemory as ExampleGeneratedMemory, ) from vertexai.types import MemoryBankCustomizationConfigMemoryTopic as MemoryTopic from vertexai.types import ( MemoryBankCustomizationConfigMemoryTopicCustomMemoryTopic as CustomMemoryTopic, ) memory_topic = MemoryTopic( custom_memory_topic=CustomMemoryTopic( label="business_feedback", description="""Specific user feedback about their experience at the coffee shop. This includes opinions on drinks, food, pastries, ambiance, staff friendliness, service speed, cleanliness, and any suggestions for improvement.""", ) ) example = GenerateMemoriesExample( conversation_source=ConversationSource( events=[ ConversationSourceEvent( content=Content( role="model", parts=[ Part( text="Welcome back to The Daily Grind! We'd love to hear your feedback on your visit." ) ], ) ), ConversationSourceEvent( content=Content( role="user", parts=[ Part( text="Hey. The drip coffee was a bit lukewarm today, which was a bummer. Also, the music was way too loud, I could barely hear my friend." ) ], ) ), ] ), generated_memories=[ ExampleGeneratedMemory( fact="The user reported that the drip coffee was lukewarm." ), ExampleGeneratedMemory( fact="The user felt the music in the shop was too loud." ), ], ) noop_example = GenerateMemoriesExample( conversation_source=ConversationSource( events=[ ConversationSourceEvent( content=Content( role="model", parts=[ Part( text="Welcome back to The Daily Grind! We'd love to hear your feedback on your visit." ) ], ) ), ConversationSourceEvent( content=Content( role="user", parts=[Part(text="Thanks for the coffee!")] ) ), ] ), generated_memories=[], ) client.agent_engines.update( name=agent_engine.api_resource.name, config={ "context_spec": { "memory_bank_config": { "customization_configs": [ CustomizationConfig( memory_topics=[memory_topic], generate_memories_examples=[example, noop_example], ) ], } } }, ) ``` -------------------------------- ### Start Backend Application Source: https://github.com/autominda1/generative-ai/blob/main/gemini/sample-apps/quickbot/README.md Launch the backend server using uvicorn. ```bash # from the backend/ directory uvicorn main:app --reload --port 8080 ``` -------------------------------- ### Install Backend Dependencies Source: https://github.com/autominda1/generative-ai/blob/main/gemini/multimodal-live-api/project-livewire/docs/local_setup.md Set up a virtual environment and install required Python packages. ```bash python3 -m venv venv source venv/bin/activate # Linux/macOS # venv\Scripts\activate # Windows ``` ```bash pip install -r requirements.txt ``` -------------------------------- ### Initialize clients and imports Source: https://github.com/autominda1/generative-ai/blob/main/gemini/use-cases/multimodal-data-curation/quality-filtering.ipynb Import required libraries and initialize the storage and Gemini clients. ```python import io import json import math from PIL import Image import av import cv2 as cv from google import genai from google.cloud import storage from google.genai import types import numpy as np import torch from torch import nn from transformers import CLIPModel, CLIPProcessor storage_client = storage.Client(project=PROJECT_ID) gemini_client = genai.Client(vertexai=True, project=PROJECT_ID, location=REGION) ``` -------------------------------- ### Implementing few-shot prompting Source: https://github.com/autominda1/generative-ai/blob/main/gemini/prompts/examples/question_answering.ipynb Provides multiple examples of context-question-answer pairs to guide the model's response format. ```python prompt = """ Context: The term "artificial intelligence" was first coined by John McCarthy in 1956. Since then, AI has developed into a vast field with numerous applications, ranging from self-driving cars to virtual assistants like Siri and Alexa. Question: What is artificial intelligence? Answer: Artificial intelligence refers to the simulation of human intelligence in machines that are programmed to think and learn like humans. --- Context: The Wright brothers, Orville and Wilbur, were two American aviation pioneers who are credited with inventing and building the world's first successful airplane and making the first controlled, powered and sustained heavier-than-air human flight, on December 17, 1903. Question: Who were the Wright brothers? Answer: The Wright brothers were American aviation pioneers who invented and built the world's first successful airplane and made the first controlled, powered and sustained heavier-than-air human flight, on December 17, 1903. --- Context: The Mona Lisa is a 16th-century portrait painted by Leonardo da Vinci during the Italian Renaissance. It is one of the most famous paintings in the world, known for the enigmatic smile of the woman depicted in the painting. Question: Who painted the Mona Lisa? Answer: """ response = generation_model.generate_content(contents=prompt).text print(response) ``` -------------------------------- ### Perform Few-Shot Text Classification Source: https://github.com/autominda1/generative-ai/blob/main/gemini/prompts/examples/text_classification.ipynb Classifies text by providing labeled examples within the prompt to guide the model. ```python prompt = """ What is the topic for a given news headline? \n - business \n - entertainment \n - health \n - sports \n - technology \n\n Text: Pixel 7 Pro Expert Hands On Review. \n The answer is: technology \n Text: Quit smoking? \n The answer is: health \n Text: Birdies or bogeys? Top 5 tips to hit under par \n The answer is: sports \n Text: Relief from local minimum-wage hike looking more remote \n The answer is: business \n Text: You won't guess who just arrived in Bari, Italy for the movie premiere. \n The answer is: """ response = generation_model.generate_content( contents=prompt, generation_config=generation_config ).text print(response) ``` -------------------------------- ### Configure Environment Variables Source: https://github.com/autominda1/generative-ai/blob/main/search/auto-rag-eval/README.md Initializes the .env file from the provided example and sets required configuration values. ```bash # Copy the example file cp env_example .env # Edit with your values PROJECT_ID=your-gcp-project-id LOCATION=us-central1 DATA_STORE_ID=your-data-store-id ``` -------------------------------- ### Implement Few-Shot Prompting Source: https://github.com/autominda1/generative-ai/blob/main/gemini/prompts/examples/question_answering.ipynb Provides examples within the prompt to guide the model toward a specific output format or behavior. ```python prompt = """Q: Who is the current President of France?\n A: Emmanuel Macron \n\n Q: Who invented the telephone? \n A: Alexander Graham Bell \n\n Q: Who wrote the novel "1984"? A: George Orwell Q: Who discovered penicillin? A: """ generation_config = GenerationConfig(temperature=0.1, max_output_tokens=20) response = generation_model.generate_content( contents=prompt, generation_config=generation_config ).text print(response) ``` -------------------------------- ### Define System Instruction Template Source: https://github.com/autominda1/generative-ai/blob/main/agents/gemini_data_analytics/intro_gemini_data_analytics_sdk.ipynb Example YAML template for defining system instructions to guide the agent's response strategy. ```python ############################################################################### ####### Define prompt / system instruction for the question. ####### ####### It's optional and can guide the agent's response strategy. ####### ############################################################################### ############################################################################### ##### Example template for system instruction for a sales analyst agent: ##### ############################################################################### # """ # - system_instruction: >- # You are an expert sales analyst and understand how to answer questions about # the sales data for a fictitious e-commence store # - tables: # - table: # - name: bigquery-public-data.thelook_ecommerce.orders # - description: orders for The Look fictitious e-commerce store. # - synonyms: sales ``` -------------------------------- ### Configure the Backend Environment Source: https://github.com/autominda1/generative-ai/blob/main/gemini/multimodal-live-api/project-livewire/README.md Prepare the server environment variables by copying the example file and editing it with required API keys. ```bash cd server cp .env.example .env nano .env # Edit with your API keys & Function URLs # --> See server/README.md for detailed .env options <-- ``` -------------------------------- ### Setup and Compile Vertex AI Pipeline Source: https://github.com/autominda1/generative-ai/blob/main/gemini/tuning/genai-mlops-tune-and-eval/README.md Set up a Python virtual environment, install pipeline dependencies, and compile the pipeline definition from Python to JSON. ```sh python3 -m venv venv source venv/bin/activate pip install -r requirements.txt kfp dsl compile --py pipeline.py --output pipeline.json ``` -------------------------------- ### Configure Backend Environment Source: https://github.com/autominda1/generative-ai/blob/main/gemini/multimodal-live-api/project-livewire/docs/local_setup.md Navigate to the server directory and prepare the environment configuration file. ```bash cd server ``` ```bash cp .env.example .env ``` ```bash nano .env ``` -------------------------------- ### Streaming Transcription Setup Source: https://context7.com/autominda1/generative-ai/llms.txt This function generates streaming recognition requests, starting with configuration and then sending audio chunks. It's suitable for real-time transcription. ```python def create_streaming_requests(audio_file_path: str): # First request: configuration yield cloud_speech.StreamingRecognizeRequest( recognizer=recognizer, streaming_config=cloud_speech.StreamingRecognitionConfig( config=config, ), ) # Subsequent requests: audio chunks with open(audio_file_path, "rb") as f: while chunk := f.read(3200): yield cloud_speech.StreamingRecognizeRequest(audio=chunk) responses = client.streaming_recognize(requests=create_streaming_requests("audio.mp3")) for response in responses: for result in response.results: print(result.alternatives[0].transcript) ``` -------------------------------- ### Configure Project ID and Account Source: https://github.com/autominda1/generative-ai/blob/main/gemini/sample-apps/image-bash-jam/README.md Copy the environment configuration file, update it with your project ID and email, and then run the setup script to configure authentication and download resources. ```bash cp .envrc.dist .envrc vim .envrc # Change PROJECT_ID and ACCOUNT with your project and email. ./01-setup.sh # sets up authentication, and includes `make images` to download resources locally. ``` -------------------------------- ### Generate Content with a Model Source: https://github.com/autominda1/generative-ai/blob/main/gemini/multimodal-live-api/real_time_rag_retail_gemini_2_0.ipynb Send a query to a generative model to get a response. This example shows an initial query that the model may not answer correctly without grounding. ```python query = "What is the price of a basic tune-up at Cymbal Bikes?" response = client.models.generate_content( model=MODEL_ID, contents=query, ) display(Markdown(response.text)) ``` -------------------------------- ### Start Backend Server Source: https://github.com/autominda1/generative-ai/blob/main/gemini/multimodal-live-api/project-livewire/docs/local_setup.md Launch the backend websocket server from the server directory. ```bash python server.py ``` -------------------------------- ### Test Geocoding Function Source: https://github.com/autominda1/generative-ai/blob/main/gemini/agent-engine/tutorial_google_maps_agent.ipynb Example usage of the `geocode_address` function to convert a location name into latitude and longitude coordinates. This helps verify the function's setup and API key. ```python # Test geocoding geocode_address("Tempelhof Airport Field in Berlin, Germany") ``` -------------------------------- ### Set YouTube Data API Key Source: https://github.com/autominda1/generative-ai/blob/main/gemini/use-cases/video-analysis/video_analysis_with_youtube_data_api_and_batch_prediction.ipynb Stores your YouTube Data API key. Obtain this key by following the instructions in the YouTube Data API v3 setup guide. ```python # Enter your YouTube data API key here # fmt: off YOUTUBE_DATA_API_KEY = "[your-youtube-data-api-key]" # @param {type:"string"} # fmt: on ``` -------------------------------- ### Python Function Calling Example Source: https://github.com/autominda1/generative-ai/blob/main/gemini/multimodal-live-api/intro_multimodal_live_api_genai_sdk.ipynb Use function calling to describe functions to the model. The model responds with a function name and arguments to call. All functions must be declared at the start of the session. ```python def get_current_weather(location: str) -> str: """Example method. Returns the current weather. Args: location: The city and state, e.g. San Francisco, CA """ weather_map: dict[str, str] = { "Boston, MA": "snowing", "San Francisco, CA": "foggy", "Seattle, WA": "raining", "Austin, TX": "hot", "Chicago, IL": "windy", } return weather_map.get(location, "unknown") config = LiveConnectConfig( response_modalities=["audio"], tools=[get_current_weather], ) async with client.aio.live.connect( model=MODEL_ID, config=config, ) as session: text_input = "Get the current weather in Boston." display(Markdown(f"**Input:** {text_input}")) await session.send_client_content( turns=Content(role="user", parts=[Part(text=text_input)]) ) async for message in session.receive(): if message.tool_call: for function_call in message.tool_call.function_calls: display(Markdown(f"**FunctionCall >** {str(function_call)}")) ``` -------------------------------- ### Execute MCP Server Examples Source: https://github.com/autominda1/generative-ai/blob/main/gemini/mcp/develop_mcp_with_gemini_and_adk.ipynb Examples of configuring and running agents with different MCP server implementations. ```python ct_server_params = StdioServerParameters( command="python", args=["./server/cocktail2.py"], ) ``` ```python events_async = await run_adk_agent( ct_server_params, "Please get cocktail margarita id and then full detail of cocktail margarita", ) ``` ```python bq_server_params = StdioServerParameters( command="python", # Make sure to update to the full absolute path to your server file args=["./server/bq.py"], ) ``` ```python await run_adk_agent( bq_server_params, "Please list my BigQuery tables, project id is 'dw-genai-dev', location is 'us'", ) ``` ```python med_server_params = StdioServerParameters( command="python", # Make sure to update to the full absolute path to your server file args=["./server/med.py"], ) ``` ```python await run_adk_agent(med_server_params, "Please explain flu in detail.") ``` ```python nih_server_params = StdioServerParameters( command="python", # Make sure to update to the full absolute path to your server file args=["./server/nih.py"], ) ``` ```python await run_adk_agent(nih_server_params, "Please tell me icd-10 code for pneumonia") ``` ```python ct_server_params = StdioServerParameters( command="python", args=["./server/cocktail2.py"], ) ``` ```python await run_adk_agent( ct_server_params, "Please get cocktail margarita id and then full detail of cocktail margarita", ) ``` -------------------------------- ### Get Video Embeddings (With Segmentation) Source: https://github.com/autominda1/generative-ai/blob/main/workshops/qa-ops/intro_Vertex_AI_embeddings.ipynb Generates video embeddings for a specified segment of a video, with options for start time, end time, and interval. Video embedding only supports the 1408 dimension. ```python # Video embeddings with 1408 dimension and Video Segment. # Video embedding only supports 1408 result = get_image_video_text_embeddings( video_path="gs://github-repo/embeddings/getting_started_embeddings/Google's newest and most capable AI Gemini.mp4", video_segment_config=VideoSegmentConfig( start_offset_sec=0, end_offset_sec=120, interval_sec=60 ), debug=True, ) # printing first 10 vectors only ``` -------------------------------- ### Set Up Image Variables and Display Source: https://github.com/autominda1/generative-ai/blob/main/vision/getting-started/veo2_reference_to_video.ipynb Assign downloaded image file paths to variables and display them using a helper function. Ensure the `show_images` function is available in your environment. ```python first_image = "flowers.png" # @param {type: 'string'} second_image = "suitcase.png" # @param {type: 'string'} third_image = "woman.jpg" # @param {type: 'string'} show_images([first_image, second_image, third_image]) ``` -------------------------------- ### Get Video Part Metadata Source: https://github.com/autominda1/generative-ai/blob/main/gemini/use-cases/video-analysis/multimodal_video_transcription.ipynb Constructs metadata for a video part, including start and end offsets and frames per second (FPS). Offsets are formatted as strings representing seconds. ```python def get_video_part_metadata( video_segment: VideoSegment | None = None, fps: float | None = None, ) -> VideoMetadata: def offset_as_str(offset: timedelta) -> str: return f"{offset.total_seconds()}s" if video_segment: start_offset = offset_as_str(video_segment.start) end_offset = offset_as_str(video_segment.end) else: start_offset = None end_offset = None return VideoMetadata(start_offset=start_offset, end_offset=end_offset, fps=fps) ``` -------------------------------- ### Start Development Server Source: https://github.com/autominda1/generative-ai/blob/main/genkit/postcard-generator/docs/demo.md Initializes the Next.js development environment for the application. ```sh npm run dev ``` -------------------------------- ### Chain of Thought Prompting - Example 2 (with intermediate steps) Source: https://github.com/autominda1/generative-ai/blob/main/gemini/prompts/examples/chain_of_thought_react.ipynb Shows how rewriting a prompt to include intermediate reasoning steps can guide the LLM towards a more structured and potentially correct answer. ```python question = """ Q: Roger has 5 tennis balls. He buys 2 more cans of tennis balls. Each can has 3 tennis balls. How many tennis balls does he have now? A: Roger started with 5 balls. 2 cans of 3 tennis balls each is 6 tennis balls. 5 + 6 = 11. The answer is 11. Q: The cafeteria had 23 apples. If they used 20 to make lunch and bought 6 more, how many apples do they have? A: """ print(llm.invoke(question)) ``` -------------------------------- ### Prepare Directories and Download Models Source: https://github.com/autominda1/generative-ai/blob/main/open-models/serving/get_started_with_vllm_lora_serving_on_vertex_ai.ipynb Creates local directories for storing models and adapters, then downloads the base Gemma 2 2B Instruct model and its SQL LoRA adapter from Hugging Face using `snapshot_download`. It ensures that symlinks are not used for the downloaded files. ```python from pathlib import Path as p from huggingface_hub import interpreter_login from huggingface_hub import get_token, snapshot_download from google.cloud import aiplatform import json base_model_id = "google/gemma-2-2b-it" sql_adapter_id = "google-cloud-partnership/gemma-2-2b-it-lora-sql" magicoder_adapter_id = "google-cloud-partnership/gemma-2-2b-it-lora-magicoder" models_dir = "./models" adapters_dir = "./adapters" p(models_dir).mkdir(exist_ok=True, parents=True) p(adapters_dir).mkdir(exist_ok=True, parents=True) # Download base model base_model_path = snapshot_download( repo_id=base_model_id, token=get_token(), local_dir=f"{models_dir}/gemma-2-2b-it", local_dir_use_symlinks=False ) # Download SQL LoRA adapter sql_adapter_path = snapshot_download( repo_id=sql_adapter_id, token=get_token(), local_dir=f"{adapters_dir}/sql-lora", local_dir_use_symlinks=False ) ``` -------------------------------- ### Generate Animated GIF with Pillow Source: https://github.com/autominda1/generative-ai/blob/main/gemini/use-cases/education/ai_quick_build_experience_backend.ipynb This code generates an animated GIF by blending start and end images to create a fade transition, then saves it. Ensure Pillow is installed and image URLs are valid. ```python # Load the start and end images for the GIF start_image_url = image_urls[winning_player_number - 1] end_image_url = "favourite_generated_image.png" start_image = PIL.Image.open(start_image_url).convert("RGB") end_image = PIL.Image.open(end_image_url).convert("RGB") # Initialize a list to store the GIF frames gif_frames = [] # Define the number of steps for the fade transition gif_steps = 3 # Generate the frames for the fade-in for i in range(gif_steps + 1): gif_frame = PIL.Image.blend(start_image, end_image, i / gif_steps) gif_frames.append(gif_frame) # Add the end frame again so that it pauses at the generated image. gif_frames.append(end_image) # Add the frames for the fade-out (reverse of fade-in) gif_frames += gif_frames[::-1] # Save the frames as a GIF start_image.save( "generated.gif", format="GIF", append_images=gif_frames, save_all=True, duration=300, loop=0, ) ``` -------------------------------- ### Generate Git Changelog with Gemini Source: https://github.com/autominda1/generative-ai/blob/main/gemini/use-cases/code/analyze_codebase.ipynb Use this Python snippet to get Git diffs and prompt Gemini to summarize important changes for changelog generation. Ensure the 'gemini-python' library is installed and configured. ```python diff_text = get_git_diff(repo_dir) question = f""" Given the below git diff output, Summarize the important changes made. ```diff {diff_text} ``` """ responses = client.models.generate_content_stream( model=MODEL_ID, contents=question, config=GenerateContentConfig( cached_content=cached_content.name, ), ) for response in responses: print(response.text, end="") ``` -------------------------------- ### Initialize Clients and Libraries Source: https://github.com/autominda1/generative-ai/blob/main/gemini/use-cases/multimodal-data-curation/semantic-deduplication.ipynb Imports necessary modules and initializes Vertex AI, BigQuery, and Storage clients. ```python from concurrent.futures import ThreadPoolExecutor, as_completed import io import threading from PIL import Image import av from google.cloud import aiplatform, bigquery, storage import pandas as pd import pandas_gbq import torch from torch import nn from tqdm import tqdm from transformers import AutoModel, AutoProcessor import vertexai from vertexai.vision_models import MultiModalEmbeddingModel, Video vertexai.init(project=PROJECT_ID, location=LOCATION) aiplatform.init(project=PROJECT_ID, location=LOCATION) bq_client = bigquery.Client() storage_client = storage.Client() ``` -------------------------------- ### Get Google Cloud Access Token Source: https://github.com/autominda1/generative-ai/blob/main/gemini/multimodal-live-api/websocket-demo-app/README.md Update gcloud components, install the beta component, set your project ID, and print your access token. This token is required to authenticate with the Gemini API. ```sh gcloud components update gcloud components install beta gcloud config set project YOUR-PROJECT-ID gcloud auth print-access-token ``` -------------------------------- ### Initialize Vertex AI SDK Source: https://github.com/autominda1/generative-ai/blob/main/gemini/agent-engine/evaluating_crewai_agent_engine_customized_template.ipynb Sets up the Vertex AI SDK with project information, location, and a Cloud Storage bucket for staging. It also initializes an experiment for tracking evaluations. ```python # Use the environment variable if the user doesn't provide Project ID. import os import vertexai PROJECT_ID = "[your-project-id]" # @param {type: "string", placeholder: "[your-project-id]", isTemplate: true} if not PROJECT_ID or PROJECT_ID == "[your-project-id]": PROJECT_ID = str(os.environ.get("GOOGLE_CLOUD_PROJECT")) LOCATION = os.environ.get("GOOGLE_CLOUD_REGION", "us-central1") BUCKET_NAME = "[your-bucket-name]" # @param {type: "string", placeholder: "[your-bucket-name]", isTemplate: true} if not BUCKET_NAME or BUCKET_NAME == "[your-bucket-name]": BUCKET_NAME = f"{PROJECT_ID}-bucket" BUCKET_URI = f"gs://{BUCKET_NAME}" ! gsutil mb -p $PROJECT_ID -l $LOCATION $BUCKET_URI EXPERIMENT_NAME = "evaluate-re-agent" # @param {type:"string"} vertexai.init( project=PROJECT_ID, location=LOCATION, staging_bucket=BUCKET_URI, experiment=EXPERIMENT_NAME, ) ``` -------------------------------- ### Define tutorial variables Source: https://github.com/autominda1/generative-ai/blob/main/embeddings/intro_embeddings_tuning.ipynb Sets configuration variables for dataset paths, tuning parameters, and machine specifications. ```python # Dataset PROCESSOR_ID = f"preprocess-docs-llm-{ID}" LOCATION = REGION.split("-")[0] RAW_DATA_URI = "gs://github-repo/embeddings/get_started_with_embedding_tuning" PROCESSED_DATA_URI = f"{BUCKET_URI}/data/processed" PREPARED_DATA_URI = f"{BUCKET_URI}/data/prepared" PROCESSED_DATA_OCR_URI = f"{BUCKET_URI}/data/processed/ocr" PROCESSED_DATA_TUNING_URI = f"{BUCKET_URI}/data/processed/tuning" # Tuning PIPELINE_ROOT = f"{BUCKET_URI}/pipelines" BATCH_SIZE = 32 # @param {type:"integer"} TRAINING_ACCELERATOR_TYPE = "NVIDIA_TESLA_T4" # @param {type:"string"} TRAINING_MACHINE_TYPE = "n1-standard-16" # @param {type:"string"} ``` -------------------------------- ### Development Setup and Docker Run Source: https://github.com/autominda1/generative-ai/blob/main/partner-models/claude/computer-use-demo/README.md This script configures the development environment and installs dependencies. The subsequent Docker command runs the application with local code mounted for development and necessary ports exposed. ```bash ./setup.sh # configure venv, install development dependencies, and install pre-commit hooks ``` ```bash docker build . -t computer-use-demo:local # manually build the docker image (optional) ``` ```bash export ANTHROPIC_API_KEY=%your_api_key% ``` ```bash docker run \ -e ANTHROPIC_API_KEY=$ANTHROPIC_API_KEY \ -v $(pwd)/computer_use_demo:/home/computeruse/computer_use_demo/ `# mount local python module for development` \ -v $HOME/.anthropic:/home/computeruse/.anthropic \ -p 5900:5900 \ -p 8501:8501 \ -p 6080:6080 \ -p 8080:8080 \ -it computer-use-demo:local # can also use ghcr.io/anthropics/anthropic-quickstarts:computer-use-demo-latest ```