### ChatGPT Vision Prompt Template Source: https://context7.com/quinny1187/gridgpt/llms.txt Template for prompting ChatGPT Vision with gridded images. Specify the cell size and the object of interest. This prompt guides the model to identify object locations using the alphanumeric grid. ```text I have given you a picture with a grid on top of it. There is a grid line every {cell_size} pixels for both the x and y axis. They form a grid in which one cell of that grid we will call a "grid cell". Every grid cell has a unique identifier in faint text inside the cell. You can recognize this identifier because it is an alphanumeric number ranging from 000 to 999. I will ask you to locate a specific object in this image underneath the grid. Can you now give me a grid cell or a group of grid cells that lies on top of the {object_of_interest}? ``` -------------------------------- ### Convert Grid Cell to Pixel Coordinates Source: https://context7.com/quinny1187/gridgpt/llms.txt Calculates the center pixel coordinates of a grid cell based on cell size and image dimensions. Includes examples for coordinate calculation and GUI automation using pyautogui. ```python def grid_cell_to_coordinates(cell_id, cell_size=100, image_width=1920): """ Convert a grid cell identifier to pixel coordinates. Args: cell_id: Grid cell identifier (e.g., "042", "A05") cell_size: Size of grid cells used when generating the image image_width: Width of the original image in pixels Returns: Tuple of (center_x, center_y) pixel coordinates """ num_x_cells = image_width // cell_size # Parse alphanumeric cell ID if cell_id[0].isalpha(): # Alphabetic format (A00-Z99) letter_value = (ord(cell_id[0]) - 65) * 100 index = 1000 + letter_value + int(cell_id[1:]) else: # Numeric format (000-999) index = int(cell_id) # Calculate grid position cell_x = index % num_x_cells cell_y = index // num_x_cells # Return center coordinates of the cell center_x = (cell_x * cell_size) + (cell_size // 2) center_y = (cell_y * cell_size) + (cell_size // 2) return (center_x, center_y) # Example: Convert ChatGPT's response to clickable coordinates cell_id = "042" # Response from ChatGPT Vision x, y = grid_cell_to_coordinates(cell_id, cell_size=100, image_width=1920) print(f"Click coordinates: ({x}, {y})") # Output: Click coordinates: (250, 450) # Example: GUI automation with pyautogui import pyautogui coordinates = grid_cell_to_coordinates("042", cell_size=100, image_width=1920) pyautogui.click(coordinates[0], coordinates[1]) ``` -------------------------------- ### Complete Workflow with OpenAI API Source: https://context7.com/quinny1187/gridgpt/llms.txt Demonstrates the end-to-end process of using GridGPT with the OpenAI API. This includes generating a gridded image, encoding it for the API, and sending a prompt with the image to ChatGPT Vision. ```python import openai from PIL import Image import base64 # Step 1: Generate gridded image add_alpha_numeric_grid_to_image( base_image_path='screenshot.jpg', output_path='gridded_screenshot.jpg', cell_size=100 ) # Step 2: Encode image for API with open('gridded_screenshot.jpg', 'rb') as f: image_data = base64.b64encode(f.read()).decode('utf-8') # Step 3: Send to ChatGPT Vision with the prompt prompt = """I have given you a picture with a grid on top of it. There is a grid line every 100 pixels for both the x and y axis. They form a grid in which one cell of that grid we will call a "grid cell". Every grid cell has a unique identifier in faint text inside the cell. You can recognize this identifier because it is an alphanumeric number ranging from 000 to 999. I will ask you to locate a specific object in this image underneath the grid. Can you now give me a grid cell or a group of grid cells that lies on top of the submit button?""" response = openai.ChatCompletion.create( model="gpt-4-vision-preview", messages=[ { "role": "user", "content": [ {"type": "text", "text": prompt}, {"type": "image_url", "image_url": {"url": f"data:image/jpeg;base64,{image_data}"}} ] } ] ) ``` -------------------------------- ### ChatGPT Vision Prompt Template Source: https://context7.com/quinny1187/gridgpt/llms.txt A template for constructing prompts to send gridded images to the ChatGPT Vision API, specifying the grid's properties and the object of interest. ```APIDOC ## ChatGPT Vision Prompt Template ### Description Use this prompt template when sending gridded images to ChatGPT Vision API. Replace the placeholder values with your specific cell size and target object. ### Method Text Prompt ### Endpoint N/A (Used with OpenAI API) ### Parameters None (Placeholders within the template) ### Request Example ```text I have given you a picture with a grid on top of it. There is a grid line every {cell_size} pixels for both the x and y axis. They form a grid in which one cell of that grid we will call a "grid cell". Every grid cell has a unique identifier in faint text inside the cell. You can recognize this identifier because it is an alphanumeric number ranging from 000 to 999. I will ask you to locate a specific object in this image underneath the grid. Can you now give me a grid cell or a group of grid cells that lies on top of the {object_of_interest}? ``` ### Response #### Success Response (200) N/A (Response depends on the ChatGPT Vision API) #### Response Example ```python # Example: Complete workflow with OpenAI API import openai from PIL import Image import base64 # Step 1: Generate gridded image add_alpha_numeric_grid_to_image( base_image_path='screenshot.jpg', output_path='gridded_screenshot.jpg', cell_size=100 ) # Step 2: Encode image for API with open('gridded_screenshot.jpg', 'rb') as f: image_data = base64.b64encode(f.read()).decode('utf-8') # Step 3: Send to ChatGPT Vision with the prompt prompt = """I have given you a picture with a grid on top of it. There is a grid line every 100 pixels for both the x and y axis. They form a grid in which one cell of that grid we will call a "grid cell". Every grid cell has a unique identifier in faint text inside the cell. You can recognize this identifier because it is an alphanumeric number ranging from 000 to 999. I will ask you to locate a specific object in this image underneath the grid. Can you now give me a grid cell or a group of grid cells that lies on top of the submit button?""" response = openai.ChatCompletion.create( model="gpt-4-vision-preview", messages=[ { "role": "user", "content": [ {"type": "text", "text": prompt}, {"type": "image_url", "image_url": {"url": f"data:image/jpeg;base64,{image_data}"}} ] } ] ) ``` ``` -------------------------------- ### add_alpha_numeric_grid_to_image Function Source: https://context7.com/quinny1187/gridgpt/llms.txt This function takes an input image and generates a new image with a numbered grid overlay. Each grid cell receives a unique alphanumeric identifier displayed with semi-transparent text, allowing ChatGPT Vision to reference specific locations while still seeing the underlying image content. ```APIDOC ## add_alpha_numeric_grid_to_image ### Description Adds an alphanumeric grid overlay to an image for ChatGPT Vision coordinate detection. ### Method Python Function ### Endpoint N/A (Python function) ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body None ### Parameters - **base_image_path** (str) - Required - Path to the input image file - **output_path** (str) - Required - Path where the gridded image will be saved - **cell_size** (int) - Optional - Size of each grid cell in pixels (default: 100) - **grid_color** (str) - Optional - Color of the grid lines (default: 'black') - **text_opacity** (int) - Optional - Opacity of cell labels (0-255) (default: 128) - **text_offset_x** (int) - Optional - Horizontal offset for label positioning (default: 5) - **label_background_color** (tuple) - Optional - RGBA background for labels (default: (255, 255, 255, 128)) - **font_size** (int) - Optional - Font size for cell identifiers (default: 20) ### Request Example ```python from PIL import Image, ImageDraw, ImageFont def add_alpha_numeric_grid_to_image( base_image_path, output_path, cell_size=100, grid_color='black', text_opacity=128, text_offset_x=5, label_background_color=(255, 255, 255, 128), font_size=20 ): """Adds an alphanumeric grid overlay to an image for ChatGPT Vision coordinate detection.""" pass # Example: Process a 4K screenshot for GUI automation add_alpha_numeric_grid_to_image( base_image_path='screenshot.jpg', output_path='screenshot_with_grid.jpg', cell_size=100 # Recommended for 4K images ) # Example: Process a 1080p image with smaller cells for precision add_alpha_numeric_grid_to_image( base_image_path='desktop_capture.png', output_path='desktop_with_grid.jpg', cell_size=50, # Recommended for 1080p images text_opacity=100, # Slightly more transparent labels grid_color='white' # White grid for dark backgrounds ) # Example: High-visibility configuration for complex images add_alpha_numeric_grid_to_image( base_image_path='complex_ui.jpg', output_path='complex_ui_gridded.jpg', cell_size=75, label_background_color=(255, 255, 0, 180), # Yellow background for visibility font_size=16, text_opacity=200 # More visible text ) ``` ### Response #### Success Response (200) N/A (This is a Python function that modifies an image file) #### Response Example N/A ``` -------------------------------- ### Add Alphanumeric Grid to Image Source: https://context7.com/quinny1187/gridgpt/llms.txt Generates an image with a transparent alphanumeric grid overlay. Customize cell size, grid color, text opacity, and label background. Recommended cell sizes are 100 for 4K and 50 for 1080p images. ```python from PIL import Image, ImageDraw, ImageFont def add_alpha_numeric_grid_to_image( base_image_path, output_path, cell_size=100, grid_color='black', text_opacity=128, text_offset_x=5, label_background_color=(255, 255, 255, 128), font_size=20 ): """Adds an alphanumeric grid overlay to an image for ChatGPT Vision coordinate detection.""" pass # Example: Process a 4K screenshot for GUI automation add_alpha_numeric_grid_to_image( base_image_path='screenshot.jpg', output_path='screenshot_with_grid.jpg', cell_size=100 # Recommended for 4K images ) # Example: Process a 1080p image with smaller cells for precision add_alpha_numeric_grid_to_image( base_image_path='desktop_capture.png', output_path='desktop_with_grid.jpg', cell_size=50, # Recommended for 1080p images text_opacity=100, # Slightly more transparent labels grid_color='white' # White grid for dark backgrounds ) # Example: High-visibility configuration for complex images add_alpha_numeric_grid_to_image( base_image_path='complex_ui.jpg', output_path='complex_ui_gridded.jpg', cell_size=75, label_background_color=(255, 255, 0, 180), # Yellow background for visibility font_size=16, text_opacity=200 # More visible text ) ``` -------------------------------- ### Extract Grid Cell Response Source: https://context7.com/quinny1187/gridgpt/llms.txt Extracts the grid cell identifier string from the ChatGPT API response object. ```python grid_cells = response.choices[0].message.content print(f"Object located at grid cells: {grid_cells}") ``` === COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.