### Install Linux Dependencies Source: https://github.com/davormar/rustautogui/blob/main/README.md For Linux users, install the necessary development libraries for X11 and XTest. ```bash sudo apt-get update sudo apt-get install libx11-dev libxtst-dev ``` -------------------------------- ### Install Linux System Dependencies Source: https://context7.com/davormar/rustautogui/llms.txt On Linux, install system dependencies for X11 and input event tracking. For OpenCL support, also install the OpenCL ICD loader and utilities. ```bash sudo apt-get update sudo apt-get install libx11-dev libxtst-dev ``` ```bash sudo apt install ocl-icd-opencl-dev clinfo ``` -------------------------------- ### Install RustAutoGUI Crate Source: https://context7.com/davormar/rustautogui/llms.txt Add the RustAutoGUI crate to your Cargo.toml file. Choose the feature set that best suits your needs, with 'full' being the default. ```toml rustautogui = "2.5.0" ``` ```toml rustautogui = { version = "2.5.0", features = ["lite"] } ``` ```toml rustautogui = { version = "2.5.0", features = ["opencl"] } ``` -------------------------------- ### RustAutoGui::new Source: https://context7.com/davormar/rustautogui/llms.txt Initializes a new RustAutoGui instance, setting up screen, keyboard, and mouse subsystems. Debug mode can be enabled to log timing information and save debug images. ```APIDOC ## RustAutoGui::new ### Description Creates a new `RustAutoGui` instance, initializing the screen, keyboard, and mouse subsystems. Pass `true` to enable debug mode, which prints timing information and saves segmented template images to a `debug/` folder. ### Method `RustAutoGui::new(debug: bool) -> Result ` ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body None ### Request Example ```rust use rustautogui; fn main() { // debug = false (production use) let mut gui = rustautogui::RustAutoGui::new(false).unwrap(); // debug = true (prints algorithm timings and saves debug images) let mut gui_debug = rustautogui::RustAutoGui::new(true).unwrap(); // Query screen dimensions (useful on Linux multi-monitor setups) let (width, height) = gui.get_screen_size(); println!("Screen: {}x{}", width, height); // e.g. Screen: 1920x1080 // Toggle debug and warnings at runtime gui.change_debug_state(true); gui.set_suppress_warnings(true); // also: RUSTAUTOGUI_SUPPRESS_WARNINGS=1 env var } ``` ### Response #### Success Response (200) `RustAutoGui` instance #### Response Example None ``` -------------------------------- ### Get Mouse Position Source: https://github.com/davormar/rustautogui/blob/main/README.md Retrieves the current coordinates of the mouse cursor on the screen. This function returns a tuple of (x, y) coordinates. ```rust rustautogui.get_mouse_position().unwrap(); // returns (x,y) coordinate of mouse ``` -------------------------------- ### Initialize RustAutoGui Context Source: https://context7.com/davormar/rustautogui/llms.txt Create a new RustAutoGui instance, optionally enabling debug mode for detailed output and image saving. Screen dimensions can be queried, and debug/warning states can be changed at runtime. ```rust use rustautogui; fn main() { // debug = false (production use) let mut gui = rustautogui::RustAutoGui::new(false).unwrap(); // debug = true (prints algorithm timings and saves debug images) let mut gui_debug = rustautogui::RustAutoGui::new(true).unwrap(); // Query screen dimensions (useful on Linux multi-monitor setups) let (width, height) = gui.get_screen_size(); println!("Screen: {}x{}", width, height); // e.g. Screen: 1920x1080 // Toggle debug and warnings at runtime gui.change_debug_state(true); gui.set_suppress_warnings(true); // also: RUSTAUTOGUI_SUPPRESS_WARNINGS=1 env var } ``` -------------------------------- ### Initialize RustAutoGui Source: https://github.com/davormar/rustautogui/blob/main/README.md Import and initialize the RustAutoGui library. The optional boolean argument enables debug logging. ```rust use rustautogui; let mut rustautogui = rustautogui::RustAutoGui::new(false); // arg: debug ``` -------------------------------- ### OpenCL Device Management in Rust Source: https://github.com/davormar/rustautogui/blob/main/README.md Demonstrates how to list available OpenCL devices and change the active device within RustAutoGUI. Note that changing the device resets prepared data. ```rust gui.list_devices(); gui.change_ocl_device(1); // selects device on index = 1 ``` -------------------------------- ### prepare_template_from_file Source: https://context7.com/davormar/rustautogui/llms.txt Loads an image file from disk, converts it to grayscale, and precomputes data structures for template matching. This must be called before `find_image_on_screen`. The search can be restricted to a specific region. ```APIDOC ## prepare_template_from_file ### Description Loads an image file, converts it to grayscale, and precomputes the data structures needed for template matching. Must be called before `find_image_on_screen`. The `region` parameter restricts the search area to `(x, y, width, height)`; `None` searches the full screen. ### Method `prepare_template_from_file(&mut self, path: &str, region: Option<(i32, i32, u32, u32)>, match_mode: MatchMode) -> Result<(), Error>` ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body None ### Request Example ```rust use rustautogui::{RustAutoGui, MatchMode}; fn main() { let mut gui = RustAutoGui::new(false).unwrap(); // Load with Segmented NCC (best for small, simple templates) gui.prepare_template_from_file( "button.png", Some((0, 0, 960, 540)), // search top-left quadrant only MatchMode::Segmented, ).unwrap(); // Load with FFT NCC (best for large templates or templates close to region size) gui.prepare_template_from_file( "dialog.png", None, // search full screen MatchMode::FFT, ).unwrap(); // Custom threshold variant — manually controls segmentation fineness // threshold 0.0–0.85; higher = finer segments, potentially slower gui.prepare_template_from_file_custom( "icon.png", Some((100, 100, 800, 600)), MatchMode::Segmented, 0.5, // threshold ).unwrap(); } ``` ### Response #### Success Response (200) None #### Response Example None ``` -------------------------------- ### Rust Keyboard Input and Commands Source: https://github.com/davormar/rustautogui/blob/main/README.md Demonstrates various ways to input text and execute keyboard commands using RustAutoGUI. Supports direct string input, single key presses, multi-key combinations, and key down/up events. Note that only US keyboard layout is fully supported. ```rust rustautogui.keyboard_input("test!@#24").unwrap(); // input string, or better say, do the sequence of key presses ``` ```rust rustautogui.keyboard_command("backspace").unwrap(); // press a keyboard button ``` ```rust rustautogui.keyboard_multi_key("shift", "control", Some("t")).unwrap(); // Executed multiple key press at same time. third argument is optional ``` ```rust rustautogui.key_down("backspace").unwrap(); // press a keyboard button down only ``` ```rust rustautogui.key_up("backspace").unwrap(); // press a keyboard button down only ``` -------------------------------- ### Load Single Image Template from File Source: https://github.com/davormar/rustautogui/blob/main/README.md Prepares a template image from a file path for screen searching. Supports specifying a search region and match mode. The `prepare_template_from_file` function is an alias for `load_and_prepare_template` and will be deprecated. ```rust rustautogui.prepare_template_from_file( // returns Result<(), String> "template.png", // template_path: &str path to the image file on disk Some((0,0,1000,1000)), // region: Option<(u32, u32, u32, u32)> region of monitor to search (x, y, width, height) rustautogui::MatchMode::Segmented, // match_mode: rustautogui::MatchMode search mode (Segmented or FFT) ).unwrap(); ``` -------------------------------- ### Using OpenCL GPU Acceleration with RustAutoGUI Source: https://context7.com/davormar/rustautogui/llms.txt Enables GPU-accelerated template matching using `MatchMode::SegmentedOcl` and `MatchMode::SegmentedOclV2`. Requires the `opencl` feature. V1 is robust with automatic thresholding, while V2 is faster but needs manual threshold tuning. FFT falls back to CPU. ```rust #[cfg(feature = "opencl")] use rustautogui::{RustAutoGui, MatchMode}; #[cfg(feature = "opencl")] fn main() { let mut gui = RustAutoGui::new(false).unwrap(); // List available OpenCL devices gui.list_devices(); // Device 0: NVIDIA GeForce RTX 3080 | 10240 MB | 68 CUs | 1710 MHz // Switch to a specific device (resets all prepared data) // gui.change_ocl_device(1).unwrap(); // V1 — robust, automatic threshold gui.store_template_from_file( "target.png", None, MatchMode::SegmentedOcl, "target_v1", ).unwrap(); // V2 — faster with well-tuned threshold (avoid on integrated/low-end GPUs) gui.store_template_from_file_custom( "target2.png", Some((500, 100, 1000, 700)), MatchMode::SegmentedOclV2, "target_v2", 0.8, // threshold: tune for your specific image ).unwrap(); // CPU fallback template stored alongside GPU ones gui.store_template_from_file( "target3.png", None, MatchMode::Segmented, // runs on CPU "target_cpu", ).unwrap(); // Search API is identical regardless of GPU/CPU mode if let Some(locs) = gui.find_stored_image_on_screen(0.9, "target_v1").unwrap() { println!("V1 GPU found at ({}, {})", locs[0].0, locs[0].1); } if let Some(locs) = gui.find_stored_image_on_screen(0.9, "target_v2").unwrap() { println!("V2 GPU found at ({}, {})", locs[0].0, locs[0].1); } } ``` -------------------------------- ### find_image_on_screen_and_move_mouse Source: https://context7.com/davormar/rustautogui/llms.txt Finds a prepared template on screen and moves the mouse cursor to the center of the best match. It returns the same match information as `find_image_on_screen`. ```APIDOC ## `find_image_on_screen_and_move_mouse` — Find template and move mouse to it Combines `find_image_on_screen` with mouse movement: finds the template and moves the cursor to the center of the highest-confidence match. Returns the same `Vec<(x, y, correlation)>` as `find_image_on_screen`. ### Parameters - `precision`: f32 - The minimum correlation threshold for a match. - `duration_secs`: f32 - The time in seconds over which to move the mouse. ### Returns - Result>, Box> - A vector of matches (x, y, correlation) or None if no match is found. ``` -------------------------------- ### Add RustAutoGUI to Project Source: https://github.com/davormar/rustautogui/blob/main/README.md Add the RustAutoGUI crate to your project's Cargo.toml file. Specify features like 'opencl', 'lite', or 'dev' as needed. ```toml rustautogui = "2.5.0" ``` ```toml rustautogui = { version = "2.5.0", features = ["opencl"] } ``` ```toml rustautogui = { version = "2.5.0", features = ["lite"] } ``` ```toml rustautogui = { version = "2.5.0", features = ["dev"] } ``` -------------------------------- ### Load Single Image Template from ImageBuffer Source: https://github.com/davormar/rustautogui/blob/main/README.md Prepares a template image from an `ImageBuffer` for screen searching. Accepts RGB, RGBA, or Luma formats. Allows specifying a search region and match mode. The search covers the whole screen if no region is provided. ```rust rustautogui.prepare_template_from_imagebuffer( // returns Result<(), String> img_buffer, // image: ImageBuffer> -- Accepts RGB/RGBA/Luma(black and white) None, // region: Option<(u32, u32, u32, u32)> searches whole screen when None rustautogui::MatchMode::FFT, // match_mode: rustautogui::MatchMode search mode (Segmented or FFT) ).unwrap(); ``` -------------------------------- ### Choosing the Right MatchMode Algorithm in RustAutoGUI Source: https://context7.com/davormar/rustautogui/llms.txt Selects the template matching algorithm impacting speed and accuracy. `Segmented` is best for small templates in large regions, while `FFT` is consistent for large or complex templates. OpenCL variants offer GPU acceleration. ```rust use rustautogui::MatchMode; // MatchMode::Segmented // - Custom multithreaded Segmented NCC algorithm // - Best for: small templates, low visual complexity, tight search regions // - Speed can be up to 10x faster than FFT in favorable cases // - Can also be slower by orders of magnitude on complex images // MatchMode::FFT // - FFT-based Normalized Cross-Correlation // - Best for: large templates, high visual complexity, templates near region size // - Consistent, predictable performance with no large speed variance // MatchMode::SegmentedOcl (requires "opencl" feature) // - GPU-accelerated Segmented NCC, V1 algorithm // - Recommended for general GPU use; automatic thresholding works well // MatchMode::SegmentedOclV2 (requires "opencl" feature) // - GPU-accelerated Segmented NCC, V2 algorithm // - Use with _custom functions and manual threshold tuning // - Not recommended for integrated or low-end GPUs fn select_mode(template_size_px: u32, region_size_px: u32) -> MatchMode { if template_size_px < 10_000 && template_size_px < region_size_px / 4 { MatchMode::Segmented // small template in large region → Segmented wins } else { MatchMode::FFT // large template or region → FFT is more consistent } } ``` -------------------------------- ### Handle RustAutoGUI Automation Errors Source: https://context7.com/davormar/rustautogui/llms.txt Demonstrates how to use `match` to handle specific `AutoGuiError` variants like `OutOfBoundsError`, `AliasError`, and `UnSupportedKey`, as well as a general catch-all for other errors during automation. ```rust use rustautogui::{RustAutoGui, MatchMode}; use rustautogui::errors::AutoGuiError; fn run_automation() -> Result<(), AutoGuiError> { let mut gui = RustAutoGui::new(false)?; gui.prepare_template_from_file("btn.png", None, MatchMode::Segmented)?; match gui.find_image_on_screen(0.9)? { Some(locs) => { gui.move_mouse_to_pos(locs[0].0, locs[0].1, 0.5)?; gui.left_click()?; gui.keyboard_input("confirmed")?; gui.keyboard_command("return")?; } None => eprintln!("Template not found; skipping click") } Ok(()) } fn main() { match run_automation() { Ok(()) => println!("Automation complete"), Err(AutoGuiError::OutOfBoundsError(msg)) => eprintln!("Coords out of range: {}", msg), Err(AutoGuiError::AliasError(msg)) => eprintln!("Unknown alias: {}", msg), Err(AutoGuiError::UnSupportedKey(key)) => eprintln!("Key not supported: {}", key), Err(e) => eprintln!("Automation error: {}", e), } } ``` -------------------------------- ### loop_find_stored_image_on_screen_and_move_mouse Source: https://github.com/davormar/rustautogui/blob/main/README.md Continuously searches for a stored template, moves the mouse to the best match, and stops on find or timeout. Timeout of 0 is an infinite loop. ```APIDOC ## loop_find_stored_image_on_screen_and_move_mouse ### Description Continuously searches for a stored template image using its alias. If found, the mouse cursor is moved to the center of the best match. The search continues until the image is found or the specified timeout is reached. A timeout of 0 initiates an infinite loop. ### Method `loop_find_stored_image_on_screen_and_move_mouse(precision: f64, moving_time: f64, timeout: u64, alias: &str)` ### Parameters - `precision` (f64): The minimum correlation threshold for a match. - `moving_time` (f64): The duration in seconds for the mouse movement. - `timeout` (u64): The maximum time in seconds to search. A value of 0 means an infinite loop. - `alias` (&str): The alias of the stored template. ### Returns `Result>, String>`: Returns a Result containing an Option of found locations or an error string. ``` -------------------------------- ### Custom Image Preparation with Threshold Source: https://github.com/davormar/rustautogui/blob/main/README.md Uses custom image preparation functions (e.g., `prepare_template_from_file_custom`) which include an additional threshold parameter for manual control over segmentation granularity. This is only relevant for `Segmented` match modes. ```rust rustautogui.prepare_template_from_file_custom( "template.png", Some((0,0,1000,1000)), rustautogui::MatchMode::Segmented, 100 // custom threshold value ).unwrap(); ``` -------------------------------- ### Load Template from File Source: https://context7.com/davormar/rustautogui/llms.txt Precompute template matching data structures from an image file. Specify a search region and match mode (Segmented or FFT). A custom threshold can be used for finer control over segmentation. ```rust use rustautogui::{RustAutoGui, MatchMode}; fn main() { let mut gui = RustAutoGui::new(false).unwrap(); // Load with Segmented NCC (best for small, simple templates) gui.prepare_template_from_file( "button.png", Some((0, 0, 960, 540)), // search top-left quadrant only MatchMode::Segmented, ).unwrap(); // Load with FFT NCC (best for large templates or templates close to region size) gui.prepare_template_from_file( "dialog.png", None, // search full screen MatchMode::FFT, ).unwrap(); // Custom threshold variant — manually controls segmentation fineness // threshold 0.0–0.85; higher = finer segments, potentially slower gui.prepare_template_from_file_custom( "icon.png", Some((100, 100, 800, 600)), MatchMode::Segmented, 0.5, // threshold ).unwrap(); } ``` -------------------------------- ### Find Image on Screen and Move Mouse Source: https://github.com/davormar/rustautogui/blob/main/README.md Finds an image on the screen, returns its pixel coordinates, and moves the mouse to the highest correlation location. Requires a pre-prepared template. ```rust let found_locations: Option> = rustautogui.find_image_on_screen_and_move_mouse(0.9, 1.0).unwrap(); // args: precision , moving_time // executes find_image_on_screen() and moves mouse to the center of the highest correlation location ``` -------------------------------- ### find_stored_image_on_screen_and_move_mouse Source: https://github.com/davormar/rustautogui/blob/main/README.md Finds a stored template image, moves the mouse to the best match, and returns the found locations. ```APIDOC ## find_stored_image_on_screen_and_move_mouse ### Description Searches for a stored template image using its alias. If a match is found, the mouse cursor is moved to the center of the highest correlation location. Returns the pixel coordinates of all found templates. ### Method `find_stored_image_on_screen_and_move_mouse(precision: f64, moving_time: f64, alias: &str)` ### Parameters - `precision` (f64): The minimum correlation threshold for a match. - `moving_time` (f64): The duration in seconds for the mouse movement. - `alias` (&str): The alias of the stored template to search for. ### Returns `Option>`: An Option containing a vector of tuples representing found locations, or None if no match is found. ``` -------------------------------- ### Find Stored Image on Screen and Move Mouse Source: https://github.com/davormar/rustautogui/blob/main/README.md Searches for a pre-stored template image on the screen using its alias and moves the mouse to the found location. Requires the template to be stored beforehand. ```rust rustautogui .find_stored_image_on_screen_and_move_mouse(0.9, 1.0, "test2") // precision, moving_time, alias (&str) .unwrap(); ``` -------------------------------- ### Screenshot Function Source: https://context7.com/davormar/rustautogui/llms.txt Captures the full screen and saves it to a specified file path. ```APIDOC ## `save_screenshot` — Capture and save the screen Captures the full screen and saves it to the specified file path. Useful for debugging, testing, or building dynamic template images at runtime. ### `save_screenshot(file_path: &str)` Saves a screenshot of the entire screen to the given file path. ``` -------------------------------- ### Custom Image Preparation with Threshold Control Source: https://github.com/davormar/rustautogui/blob/main/README.md Use custom variants of image loading functions (suffixed with `_custom`) to manually control the threshold for finer segmentation, which can improve performance in specific scenarios. This is only relevant for `Segmented` match modes. ```APIDOC ## Custom Image Preparation Functions ### Description These functions provide manual control over the threshold parameter during image preparation, allowing for finer tuning of segmentation compared to automatic estimation. They are available for file, ImageBuffer, and raw encoded byte inputs, and are identified by the `_custom` suffix. ### Usage These functions mirror their standard counterparts but include an additional `threshold` parameter. The `threshold` value influences the detail of image segmentation, with higher values leading to finer segmentation and lower values to coarser segmentation. This parameter is only effective when using `rustautogui::MatchMode::Segmented`. ### Example (Conceptual - specific function signatures would follow the pattern of standard functions with an added threshold parameter) ```rust // Example for a hypothetical prepare_template_from_file_custom function: rustautogui.prepare_template_from_file_custom( "template.png", Some((0,0,1000,1000)), rustautogui::MatchMode::Segmented, 0.8 // Example threshold value ).unwrap(); // Example for a hypothetical store_template_from_imagebuffer_custom function: rustautogui.store_template_from_imagebuffer_custom( img_buffer, None, rustautogui::MatchMode::Segmented, "custom_button", 0.9 // Example threshold value ).unwrap(); ``` ### Threshold Explanation - **Higher Threshold**: Finer segmentation, more detailed image. - **Lower Threshold**: Coarser segmentation, faster processing. - **Internal Meaning**: Represents the correlation between the segmented image and the original template. ``` -------------------------------- ### Loop Find Stored Image on Screen and Move Mouse Source: https://github.com/davormar/rustautogui/blob/main/README.md Continuously searches for a pre-stored template image on the screen using its alias, moves the mouse to it, and stops when found or a timeout is reached. A timeout of 0 initiates an infinite loop. ```rust rustautogui .loop_find_stored_image_on_screen_and_move_mouse(0.95, 1.0, 15, "stars") // precision, moving_time, timeout, alias .unwrap(); ``` -------------------------------- ### Load Single Image Template from Raw Encoded Bytes Source: https://github.com/davormar/rustautogui/blob/main/README.md Prepares a template image from raw encoded bytes for screen searching. Supports specifying a search region and match mode. The search covers the whole screen if no region is provided. ```rust rustautogui.prepare_template_from_raw_encoded( // returns Result<(), String> img_raw // img_raw: &[u8] - encoded raw bytes None, // region: Option<(u32, u32, u32, u32)> rustautogui::MatchMode::FFT, // match_mode: rustautogui::MatchMode search mode (Segmented or FFT) ).unwrap(); ``` -------------------------------- ### loop_find_image_on_screen_and_move_mouse Source: https://github.com/davormar/rustautogui/blob/main/README.md Continuously searches for an image, moves the mouse to the best match, and stops when found or timeout is reached. Timeout of 0 is an infinite loop. ```APIDOC ## loop_find_image_on_screen_and_move_mouse ### Description Continuously searches for a template image on the screen. If found, it moves the mouse to the center of the best match and returns the found locations. The search continues until the image is found or the specified timeout is reached. A timeout of 0 initiates an infinite loop. ### Method `loop_find_image_on_screen_and_move_mouse(precision: f64, moving_time: f64, timeout: u64)` ### Parameters - `precision` (f64): The minimum correlation threshold for a match. - `moving_time` (f64): The time in seconds for the mouse movement. - `timeout` (u64): The maximum time in seconds to search. A value of 0 means an infinite loop. ### Returns `Result>, String>`: Returns a Result containing an Option of found locations or an error string. ``` -------------------------------- ### `loop_find_image_on_screen` / `loop_find_image_on_screen_and_move_mouse` Source: https://context7.com/davormar/rustautogui/llms.txt Repeatedly captures and searches the screen until the template is found or the timeout (in seconds) elapses. A timeout of `0` creates an infinite loop. Both variants emit a warning when timeout is 0 (suppressible). ```APIDOC ## `loop_find_image_on_screen` / `loop_find_image_on_screen_and_move_mouse` — Poll until template appears Repeatedly captures and searches the screen until the template is found or the timeout (in seconds) elapses. A timeout of `0` creates an infinite loop. Both variants emit a warning when timeout is 0 (suppressible). ```rust use rustautogui::{RustAutoGui, MatchMode}; fn main() { let mut gui = RustAutoGui::new(false).unwrap(); gui.prepare_template_from_file("loading_spinner.png", None, MatchMode::FFT).unwrap(); // Wait up to 30 seconds for the spinner to disappear... (find its presence) match gui.loop_find_image_on_screen(0.90, 30).unwrap() { Some(locs) => println!("Spinner appeared at ({}, {})", locs[0].0, locs[0].1), None => println!("Timed out"), } gui.prepare_template_from_file("ok_button.png", None, MatchMode::Segmented).unwrap(); // Wait up to 15 seconds, then move mouse to OK button and click if let Some(locs) = gui.loop_find_image_on_screen_and_move_mouse(0.93, 1.0, 15).unwrap() { println!("Found and moved to ({}, {})", locs[0].0, locs[0].1); gui.left_click().unwrap(); } } ``` ``` -------------------------------- ### Loop Find Image on Screen and Move Mouse Source: https://github.com/davormar/rustautogui/blob/main/README.md Continuously searches for an image on the screen, moves the mouse to it, and stops when found or a timeout is reached. A timeout of 0 initiates an infinite loop. Requires a pre-prepared template. ```rust rustautogui .loop_find_image_on_screen_and_move_mouse(0.95, 1.0, 15) // args: precision, moving_time and timeout .unwrap(); ``` -------------------------------- ### Find Template and Move Mouse Source: https://context7.com/davormar/rustautogui/llms.txt The `find_image_on_screen_and_move_mouse` function combines template searching with mouse cursor movement. It finds the best match and moves the cursor to its center, returning the same match data as `find_image_on_screen`. This is useful for automating interactions with UI elements. ```rust use rustautogui::{RustAutoGui, MatchMode}; fn main() { let mut gui = RustAutoGui::new(false).unwrap(); gui.prepare_template_from_file( "submit_btn.png", Some((0, 500, 1920, 580)), MatchMode::Segmented, ).unwrap(); // Find template and move mouse there over 0.8 seconds match gui.find_image_on_screen_and_move_mouse(0.92, 0.8).unwrap() { Some(locs) => { println!("Moved mouse to ({}, {})", locs[0].0, locs[0].1); gui.left_click().unwrap(); } None => eprintln!("Button not found"), } } ``` -------------------------------- ### prepare_template_from_imagebuffer Source: https://context7.com/davormar/rustautogui/llms.txt Loads an image template from an in-memory `ImageBuffer`. This is useful when templates are generated programmatically or loaded through a custom pipeline. Supports RGB, RGBA, and Luma formats. ```APIDOC ## prepare_template_from_imagebuffer ### Description Accepts an in-memory `ImageBuffer>` in RGB, RGBA, or Luma (grayscale) format. Useful when the template is generated programmatically or loaded via a custom pipeline. ### Method `prepare_template_from_imagebuffer(&mut self, img: ImageBuffer>, region: Option<(i32, i32, u32, u32)>, match_mode: MatchMode) -> Result<(), Error>` ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body None ### Request Example ```rust use rustautogui::{RustAutoGui, MatchMode, imgtools}; fn main() { let mut gui = RustAutoGui::new(false).unwrap(); // Load an RGBA image from disk for demonstration, then pass it as a buffer let img = imgtools::load_image_rgba("template.png").unwrap(); gui.prepare_template_from_imagebuffer( img, Some((0, 0, 1280, 720)), MatchMode::Segmented, ).unwrap(); // Custom threshold variant let img2 = imgtools::load_image_rgba("template2.png").unwrap(); gui.prepare_template_from_imagebuffer_custom( img2, None, MatchMode::Segmented, 0.3, ).unwrap(); } ``` ### Response #### Success Response (200) None #### Response Example None ``` -------------------------------- ### prepare_template_from_raw_encoded Source: https://context7.com/davormar/rustautogui/llms.txt Loads a single template from encoded image bytes. This is useful for embedding template images directly in binaries or receiving them over a network. It supports various image formats compatible with the `image` crate. ```APIDOC ## `prepare_template_from_raw_encoded` — Load a single template from encoded bytes Decodes an image from a raw byte slice (any format supported by the `image` crate: PNG, JPEG, BMP, etc.) using `image::load_from_memory`. Ideal for embedding template images directly in binary or receiving them over a network. ### Parameters - `template_bytes`: &[u8] - The raw encoded image bytes. - `region`: Option<(i32, i32, i32, i32)> - Optional region to search within (x, y, width, height). - `match_mode`: MatchMode - The matching algorithm to use. ### Returns - Result<(), Box> - Ok on success, Err on failure. ``` -------------------------------- ### Load Template from Encoded Bytes Source: https://context7.com/davormar/rustautogui/llms.txt Use `prepare_template_from_raw_encoded` to load an image template directly from a byte slice. This is useful for embedding images within your binary or receiving them over a network. `prepare_template_from_raw_encoded_custom` allows specifying a custom correlation threshold. ```rust use rustautogui::{RustAutoGui, MatchMode}; fn main() { let mut gui = RustAutoGui::new(false).unwrap(); // Embed template bytes at compile time let template_bytes: &[u8] = include_bytes!("../assets/submit_button.png"); gui.prepare_template_from_raw_encoded( template_bytes, Some((400, 300, 800, 600)), MatchMode::FFT, ).unwrap(); // With custom threshold gui.prepare_template_from_raw_encoded_custom( template_bytes, None, MatchMode::Segmented, 0.6, ).unwrap(); } ``` -------------------------------- ### Configure RustAutoGUI Warnings Source: https://github.com/davormar/rustautogui/blob/main/README.md Shows how to suppress or activate warnings from RustAutoGUI. This can be done either through environment variables in PowerShell, CMD, or by calling a method within your Rust code. ```powershell $env:RUSTAUTOGUI_SUPPRESS_WARNINGS="1" #to turn off warnings $env:RUSTAUTOGUI_SUPPRESS_WARNINGS="0" #to activate warnings ``` ```cmd set RUSTAUTOGUI_SUPPRESS_WARNINGS=1 #to turn off warnings set RUSTAUTOGUI_SUPPRESS_WARNINGS=0 #to activate warnings ``` ```bash export RUSTAUTOGUI_SUPPRESS_WARNINGS=1 #to turn off warnings export RUSTAUTOGUI_SUPPRESS_WARNINGS=0 #to activate warnings ``` ```rust let mut rustautogui = RustAutoGui::new(false).unwrap(); rustautogui.set_suppress_warnings(true); ``` -------------------------------- ### Store Multiple Named Templates Source: https://context7.com/davormar/rustautogui/llms.txt Store prepared template data in a `HashMap` using an alias. This allows for simultaneous storage and independent searching of multiple templates. Functions like `store_template_from_file`, `store_template_from_imagebuffer`, and `store_template_from_raw_encoded_custom` mirror their `prepare_*` counterparts but include an `alias` parameter. ```rust use rustautogui::{RustAutoGui, MatchMode, imgtools}; fn main() { let mut gui = RustAutoGui::new(false).unwrap(); // Store from file gui.store_template_from_file( "ok_button.png", Some((600, 400, 800, 400)), MatchMode::Segmented, "ok_btn", ).unwrap(); // Store from ImageBuffer let img = imgtools::load_image_rgba("cancel_button.png").unwrap(); gui.store_template_from_imagebuffer( img, None, MatchMode::FFT, "cancel_btn", ).unwrap(); // Store from raw bytes with custom threshold let raw = include_bytes!("../assets/close_icon.png"); gui.store_template_from_raw_encoded_custom( raw, Some((1800, 0, 120, 40)), MatchMode::Segmented, "close_icon", 0.4, ).unwrap(); // Search for each stored template independently if let Some(locs) = gui.find_stored_image_on_screen(0.9, "ok_btn").unwrap() { println!("OK button at ({}, {}), corr={:.3}", locs[0].0, locs[0].1, locs[0].2); } if let Some(locs) = gui.find_stored_image_on_screen(0.9, "cancel_btn").unwrap() { println!("Cancel button found: {} matches", locs.len()); } } ``` -------------------------------- ### Helper Function Source: https://github.com/davormar/rustautogui/blob/main/README.md A utility function to print the current mouse position. ```APIDOC ## Helper Function ### Print Mouse Position * **Description**: Prints the current coordinates of the mouse cursor to the console. Useful for debugging and determining target coordinates. * **Method**: `print_mouse_position().unwrap();` ``` -------------------------------- ### prepare_template_from_raw_encoded_custom Source: https://context7.com/davormar/rustautogui/llms.txt Loads a single template from encoded image bytes with a custom threshold. This function allows fine-tuning the matching sensitivity for template recognition. ```APIDOC ## `prepare_template_from_raw_encoded_custom` — Load a single template from encoded bytes with custom threshold Decodes an image from a raw byte slice and applies a custom correlation threshold for matching. This provides more control over the detection sensitivity. ### Parameters - `template_bytes`: &[u8] - The raw encoded image bytes. - `region`: Option<(i32, i32, i32, i32)> - Optional region to search within (x, y, width, height). - `match_mode`: MatchMode - The matching algorithm to use. - `threshold`: f32 - The custom correlation threshold (e.g., 0.6). ### Returns - Result<(), Box> - Ok on success, Err on failure. ``` -------------------------------- ### Press Special Keyboard Key Source: https://context7.com/davormar/rustautogui/llms.txt Simulates pressing a single special keyboard key. Refer to `Keyboard_commands.md` for a full list of cross-platform key names. Use `key_down` and `key_up` for keys that need to be held. ```rust use rustautogui::RustAutoGui; fn main() { let mut gui = RustAutoGui::new(false).unwrap(); gui.keyboard_command("return").unwrap(); // Enter/Return gui.keyboard_command("escape").unwrap(); // Escape gui.keyboard_command("tab").unwrap(); // Tab gui.keyboard_command("backspace").unwrap(); // Backspace gui.keyboard_command("up_arrow").unwrap(); // Arrow up gui.keyboard_command("f5").unwrap(); // F5 (refresh) gui.keyboard_command("delete").unwrap(); // Delete // key_down / key_up for held-key scenarios gui.key_down("shift").unwrap(); gui.keyboard_command("home").unwrap(); // Shift+Home = select to line start gui.key_up("shift").unwrap(); } ``` -------------------------------- ### find_image_on_screen_and_move_mouse Source: https://github.com/davormar/rustautogui/blob/main/README.md Finds an image on the screen, moves the mouse to the center of the highest correlation match, and returns the found locations. ```APIDOC ## find_image_on_screen_and_move_mouse ### Description Executes `find_image_on_screen()` and, if a match is found, moves the mouse cursor to the center of the location with the highest correlation. Returns the pixel coordinates of found templates. ### Method `find_image_on_screen_and_move_mouse(precision: f64, moving_time: f64)` ### Parameters - `precision` (f64): The minimum correlation threshold for a match. - `moving_time` (f64): The time in seconds it takes for the mouse to move to the target location. ### Returns `Option>`: An Option containing a vector of tuples representing found locations, or None if no match is found. ``` -------------------------------- ### `find_stored_image_on_screen` / `loop_find_stored_image_on_screen` Source: https://context7.com/davormar/rustautogui/llms.txt Searches for a template that was previously stored with `store_template_*` under the given alias. The active single-template state is temporarily swapped out and restored after the search, so both stored and single templates coexist safely. ```APIDOC ## `find_stored_image_on_screen` / `loop_find_stored_image_on_screen` — Search using a named stored template Searches for a template that was previously stored with `store_template_*` under the given alias. The active single-template state is temporarily swapped out and restored after the search, so both stored and single templates coexist safely. ```rust use rustautogui::{RustAutoGui, MatchMode}; fn main() { let mut gui = RustAutoGui::new(false).unwrap(); gui.store_template_from_file("yes.png", None, MatchMode::Segmented, "yes").unwrap(); gui.store_template_from_file("no.png", None, MatchMode::Segmented, "no").unwrap(); // One-shot search if let Some(locs) = gui.find_stored_image_on_screen(0.9, "yes").unwrap() { println!("Yes button at ({}, {})", locs[0].0, locs[0].1); } // Loop search with timeout match gui.loop_find_stored_image_on_screen(0.92, 20, "no") { Ok(Some(locs)) => println!("No button appeared at ({}, {})", locs[0].0, locs[0].1), Ok(None) => println!("Not found"), Err(e) => eprintln!("Error: {}", e), } // Find and move mouse directly if let Some(locs) = gui.find_stored_image_on_screen_and_move_mouse(0.9, 1.0, "yes").unwrap() { println!("Moved to ({}, {})", locs[0].0, locs[0].1); gui.left_click().unwrap(); } } ``` ``` -------------------------------- ### Move the mouse cursor with move_mouse_to_pos, move_mouse_to, and move_mouse Source: https://context7.com/davormar/rustautogui/llms.txt Three movement functions cover absolute positioning, absolute with optional axes, and relative movement. `moving_time` controls animation duration in seconds; `0.0` moves instantly. ```rust use rustautogui::RustAutoGui; fn main() { let mut gui = RustAutoGui::new(false).unwrap(); // Absolute move to (960, 540) over 1 second gui.move_mouse_to_pos(960, 540, 1.0).unwrap(); // Move only horizontally (keep current Y) over 0.5 seconds gui.move_mouse_to(Some(100), None, 0.5).unwrap(); // Relative move: 200px right, 100px down over 0.3 seconds // Negative X = left, negative Y = up gui.move_mouse(200, 100, 0.3).unwrap(); // Get current mouse position let (x, y) = gui.get_mouse_position().unwrap(); println!("Mouse is at ({}, {})", x, y); } ``` -------------------------------- ### Drag Mouse to Position Source: https://context7.com/davormar/rustautogui/llms.txt Use these functions to simulate dragging the mouse. Ensure `moving_time` is sufficiently high (e.g., > 0.2s) to avoid OS race conditions, especially on macOS. ```rust use rustautogui::RustAutoGui; fn main() { let mut gui = RustAutoGui::new(false).unwrap(); // Drag to absolute position (e.g., move a file icon) gui.drag_mouse_to_pos(800, 400, 1.5).unwrap(); // Drag to absolute position with optional axes (None keeps current axis) gui.drag_mouse_to(Some(300), None, 1.0).unwrap(); // Drag relative to current position: 150px right, 50px up gui.drag_mouse(150, -50, 1.2).unwrap(); } ``` -------------------------------- ### `move_mouse_to_pos` / `move_mouse_to` / `move_mouse` Source: https://context7.com/davormar/rustautogui/llms.txt Three movement functions cover absolute positioning, absolute with optional axes, and relative movement. `moving_time` controls animation duration in seconds; `0.0` moves instantly. ```APIDOC ## `move_mouse_to_pos` / `move_mouse_to` / `move_mouse` — Move the mouse cursor Three movement functions cover absolute positioning, absolute with optional axes, and relative movement. `moving_time` controls animation duration in seconds; `0.0` moves instantly. ```rust use rustautogui::RustAutoGui; fn main() { let mut gui = RustAutoGui::new(false).unwrap(); // Absolute move to (960, 540) over 1 second gui.move_mouse_to_pos(960, 540, 1.0).unwrap(); // Move only horizontally (keep current Y) over 0.5 seconds gui.move_mouse_to(Some(100), None, 0.5).unwrap(); // Relative move: 200px right, 100px down over 0.3 seconds // Negative X = left, negative Y = up gui.move_mouse(200, 100, 0.3).unwrap(); // Get current mouse position let (x, y) = gui.get_mouse_position().unwrap(); println!("Mouse is at ({}, {})", x, y); } ``` ``` -------------------------------- ### loop_find_image_on_screen Source: https://github.com/davormar/rustautogui/blob/main/README.md Continuously searches for an image on the screen until it's found or a timeout is reached. A timeout of 0 initiates an infinite loop. ```APIDOC ## loop_find_image_on_screen ### Description Repeatedly searches for a template image on the screen with a given precision until the image is found or the specified timeout in seconds is reached. A timeout value of 0 will result in an infinite loop. ### Method `loop_find_image_on_screen(precision: f64, timeout: u64)` ### Parameters - `precision` (f64): The minimum correlation threshold for a match. - `timeout` (u64): The maximum time in seconds to search for the image. A value of 0 means the search will continue indefinitely. ### Returns `Result>, String>`: Returns a Result containing an Option of found locations or an error string. ``` -------------------------------- ### General Functions Source: https://github.com/davormar/rustautogui/blob/main/README.md Provides functions for debugging and screen information. ```APIDOC ## General Functions ### Get Screen Size * **Description**: Retrieves the dimensions of the display. * **Method**: `rustautogui.get_screen_size()` * **Returns**: A tuple `(x, y)` representing the screen width and height. ### Change Debug State * **Description**: Enables or disables debug mode. * **Method**: `rustautogui.change_debug_state(state: bool)` * **Parameters**: * `state` (bool) - `true` to enable debug mode, `false` to disable. ### Set Suppress Warnings * **Description**: Turns warning messages on or off. * **Method**: `rustautogui.set_suppress_warning(state: bool)` * **Parameters**: * `state` (bool) - `true` to suppress warnings, `false` to show them. ### Save Screenshot * **Description**: Saves a screenshot of the current screen to a file. * **Method**: `rustautogui.save_screenshot(filename: &str)` * **Parameters**: * `filename` (&str) - The name of the file to save the screenshot to. * **Example**: `rustautogui.save_screenshot("test.png").unwrap();` ```