### Install Linux Dependencies Source: https://docs.rs/rustautogui/latest/rustautogui/index.html For Linux users, install the necessary development libraries for X11 and XTest before using the rustautogui crate. ```bash sudo apt-get update sudo apt-get install libx11-dev libxtst-dev ``` -------------------------------- ### Initialize and Use RustAutoGui Source: https://docs.rs/rustautogui/latest/rustautogui/struct.RustAutoGui.html Demonstrates initializing RustAutoGui, getting screen size, preparing image templates for search, finding images, and performing various mouse and keyboard actions. ```Rust fn main() { #[cfg(not(feature = "lite"))] { use rustautogui; use rustautogui::imgtools; // initialize autogui let mut gui = rustautogui::RustAutoGui::new(false).unwrap(); // displays (x, y) size of screen // especially useful on linux where screen from all monitors is grabbed gui.get_screen_size(); { // load the image searching for. Region is Option<(startx, starty, width, height)> of search. Matchmode FFT or Segmented (not implemented before 1.0 version), max segments, only important for Segmented match mode gui.prepare_template_from_file( "test.png", Some((0, 0, 500, 300)), rustautogui::MatchMode::FFT, ) .unwrap(); } // or another way to prepare template { let img = imgtools::load_image_rgba("test.png").unwrap(); // just loading this way for example gui.prepare_template_from_imagebuffer( img, Some((0, 0, 700, 500)), rustautogui::MatchMode::Segmented, ) .unwrap(); } // or segmented variant with no region gui.prepare_template_from_file("test.png", None, rustautogui::MatchMode::FFT) .unwrap(); // change prepare template settings, like region, matchmode or max segments // automatically move mouse to found template position, execute movement for 1 second gui.find_image_on_screen_and_move_mouse(0.9, 1.0).unwrap(); //move mouse to position (in this case to bottom left of stanard 1920x1080 monitor), move the mouse for 1 second gui.move_mouse_to_pos(1920, 1080, 1.0).unwrap(); // execute left click, move the mouse to x (500), y (500) position for 1 second, release mouse click // used for actions like moving icons or files // suggestion: dont use very small moving time values, especially on mac gui.drag_mouse(500, 500, 1.0).unwrap(); // execute mouse left click gui.left_click().unwrap(); // execute mouse right click gui.right_click().unwrap(); // execute mouse middle click gui.middle_click().unwrap(); // execute a double (left) mouse click gui.double_click().unwrap(); // mouse scrolls gui.scroll_down(1).unwrap(); gui.scroll_up(5).unwrap(); gui.scroll_right(8).unwrap(); gui.scroll_left(10).unwrap(); // input keyboard string gui.keyboard_input("test.com").unwrap(); // press a keyboard command, in this case enter(return) gui.keyboard_command("return").unwrap(); // two key press gui.keyboard_multi_key("alt", "tab", None).unwrap(); // three key press gui.keyboard_multi_key("shift", "control", Some("e")) .unwrap(); // maybe you would want to loop search until image is found and break the loop then loop { let pos = gui.find_image_on_screen_and_move_mouse(0.9, 1.0).unwrap(); match pos { Some(_) => break, None => (), } } } } ``` -------------------------------- ### Install OpenCL ICD and Utilities (Linux) Source: https://docs.rs/rustautogui/latest/rustautogui/index.html Install the OpenCL ICD and the 'clinfo' utility on Debian-based Linux systems to verify OpenCL device detection. ```bash sudo apt install ocl-icd-opencl-dev 1. Install OpenCL ICD: `sudo apt install ocl-icd-opencl-dev` 2. If clinfo is not installed: `sudo apt install clinfo` ``` -------------------------------- ### Automate Chrome Launch and GitHub Navigation Source: https://docs.rs/rustautogui/latest/rustautogui/struct.RustAutoGui.html This example demonstrates launching Chrome, typing a URL, and navigating to GitHub. It includes keyboard commands, input typing, and image searching. ```Rust // press windows/super key rustautogui.keyboard_command("win").unwrap(); thread::sleep(time::Duration::from_millis(500)); // write in chrome rustautogui.keyboard_input("chrome").unwrap(); thread::sleep(time::Duration::from_millis(500)); // run chrome with clicking return/enter rustautogui.keyboard_command("return").unwrap(); thread::sleep(time::Duration::from_millis(500)); // open new tab with ctrl+t rustautogui.keyboard_multi_key("ctrl", "t", None).unwrap(); thread::sleep(time::Duration::from_millis(500)); // input github in url (url input area selected automatically by new tab opening) rustautogui .keyboard_input("https://github.com/DavorMar/rustautogui") .unwrap(); thread::sleep(time::Duration::from_millis(500)); // return/enter to open github rustautogui.keyboard_command("return").unwrap(); thread::sleep(time::Duration::from_millis(500)); // loop till image of star is found or timeout of 15 seconds is hit rustautogui .loop_find_stored_image_on_screen_and_move_mouse(0.95, 1.0, 15, "stars") .unwrap(); thread::sleep(time::Duration::from_millis(500)); // rustautogui.left_click().unwrap(); thread::sleep(time::Duration::from_millis(500)); // scroll down 80 times to the bottom of the page where small terms button is rustautogui.scroll_down(80).unwrap(); // loop till image found with timeout of 15 seconds rustautogui .loop_find_image_on_screen_and_move_mouse(0.95, 1.0, 15) .unwrap(); thread::sleep(time::Duration::from_millis(500)); rustautogui.left_click().unwrap(); ``` -------------------------------- ### Basic GUI Automation with RustAutoGui Source: https://docs.rs/rustautogui/latest/rustautogui/struct.RustAutoGui.html This snippet demonstrates initializing RustAutoGui, getting screen size, preparing image templates for searching, and performing various mouse and keyboard actions. ```Rust 1fn main() { #[cfg(not(feature = "lite"))] { use rustautogui; use rustautogui::imgtools; // initialize autogui let mut gui = rustautogui::RustAutoGui::new(false).unwrap(); // displays (x, y) size of screen // especially useful on linux where screen from all monitors is grabbed gui.get_screen_size(); { // load the image searching for. Region is Option<(startx, starty, width, height)> of search. Matchmode FFT or Segmented (not implemented before 1.0 version), max segments, only important for Segmented match mode gui.prepare_template_from_file( "test.png", Some((0, 0, 500, 300)), rustautogui::MatchMode::FFT, ) .unwrap(); } // or another way to prepare template { let img = imgtools::load_image_rgba("test.png").unwrap(); // just loading this way for example gui.prepare_template_from_imagebuffer( img, Some((0, 0, 700, 500)), rustautogui::MatchMode::Segmented, ) .unwrap(); } // or segmented variant with no region gui.prepare_template_from_file("test.png", None, rustautogui::MatchMode::FFT) .unwrap(); // change prepare template settings, like region, matchmode or max segments // automatically move mouse to found template position, execute movement for 1 second gui.find_image_on_screen_and_move_mouse(0.9, 1.0).unwrap(); //move mouse to position (in this case to bottom left of stanard 1920x1080 monitor), move the mouse for 1 second gui.move_mouse_to_pos(1920, 1080, 1.0).unwrap(); // execute left click, move the mouse to x (500), y (500) position for 1 second, release mouse click // used for actions like moving icons or files // suggestion: dont use very small moving time values, especially on mac gui.drag_mouse(500, 500, 1.0).unwrap(); // execute mouse left click gui.left_click().unwrap(); // execute mouse right click gui.right_click().unwrap(); // execute mouse middle click gui.middle_click().unwrap(); // execute a double (left) mouse click gui.double_click().unwrap(); // mouse scrolls gui.scroll_down(1).unwrap(); gui.scroll_up(5).unwrap(); gui.scroll_right(8).unwrap(); gui.scroll_left(10).unwrap(); // input keyboard string gui.keyboard_input("test.com").unwrap(); // press a keyboard command, in this case enter(return) gui.keyboard_command("return").unwrap(); // two key press gui.keyboard_multi_key("alt", "tab", None).unwrap(); // three key press gui.keyboard_multi_key("shift", "control", Some("e")) .unwrap(); // maybe you would want to loop search until image is found and break the loop then loop { let pos = gui.find_image_on_screen_and_move_mouse(0.9, 1.0).unwrap(); match pos { Some(_) => break, None => (), } } } } ``` -------------------------------- ### Automate Opening GitHub Page with RustAutoGui Source: https://docs.rs/rustautogui/latest/rustautogui/struct.RustAutoGui.html This example demonstrates a complete workflow: initializing RustAutoGui, loading and preparing image templates, simulating keyboard inputs to open a browser and navigate to a specific URL, and then searching for an image on the page. It includes steps for opening the Windows search, typing 'chrome', launching it, opening a new tab, inputting a URL, and finally searching for an image on the loaded page. ```Rust fn main() { #[cfg(not(feature = "lite"))] { use rustautogui::RustAutoGui; use std::thread; use std::time; let mut rustautogui = RustAutoGui::new(false).unwrap(); // initialize let (screen_w, screen_h) = rustautogui.get_screen_size(); // load, process and store image of star(can be image of project name text) // regions are written in this way to cover any screen size, not hardcoding them rustautogui .store_template_from_file( "/home/davor/Pictures/stars.png", Some(( (0.2 * screen_w as f32) as u32, // start x 0, // start y (0.5 * screen_w as f32) as u32, // width (0.4 * screen_h as f32) as u32, // height )), rustautogui::MatchMode::Segmented, "stars", ) .unwrap(); // just for example doing single prepare, but you would want to store it also // load, process and store image of terms rustautogui .prepare_template_from_file( "/home/davor/Pictures/terms.png", Some(( (0.1 * screen_w as f32) as u32, // start x (0.7 * screen_h as f32) as u32, // start y (0.5 * screen_w as f32) as u32, // width (0.3 * screen_h as f32) as u32, // height )), rustautogui::MatchMode::Segmented, ) .unwrap(); // press windows/super key rustautogui.keyboard_command("win").unwrap(); thread::sleep(time::Duration::from_millis(500)); // write in chrome rustautogui.keyboard_input("chrome").unwrap(); thread::sleep(time::Duration::from_millis(500)); // run chrome with clicking return/enter rustautogui.keyboard_command("return").unwrap(); thread::sleep(time::Duration::from_millis(500)); // open new tab with ctrl+t rustautogui.keyboard_multi_key("ctrl", "t", None).unwrap(); thread::sleep(time::Duration::from_millis(500)); // input github in url (url input area selected automatically by new tab opening) rustautogui .keyboard_input("https://github.com/DavorMar/rustautogui") .unwrap(); thread::sleep(time::Duration::from_millis(500)); // return/enter to open github rustautogui.keyboard_command("return").unwrap(); thread::sleep(time::Duration::from_millis(500)); // loop till image of star is found or timeout of 15 seconds is hit rustautogui .loop_find_stored_image_on_screen_and_move_mouse(0.95, 1.0, 15, "stars") .unwrap(); thread::sleep(time::Duration::from_millis(500)); // rustautogui.left_click().unwrap(); thread::sleep(time::Duration::from_millis(500)); // scroll down 80 times to the bottom of the page where small terms button is rustautogui.scroll_down(80).unwrap(); // loop till image found with timeout of 15 seconds rustautogui .loop_find_image_on_screen_and_move_mouse(0.95, 1.0, 15) .unwrap(); thread::sleep(time::Duration::from_millis(500)); rustautogui.left_click().unwrap(); } } ``` -------------------------------- ### Storing and Preparing Image Templates Source: https://docs.rs/rustautogui/latest/rustautogui/struct.RustAutoGui.html This example shows how to store an image template from a file with a specified region and match mode, and how to prepare another template from a file. ```Rust 4fn main() { #[cfg(not(feature = "lite"))] { use rustautogui::RustAutoGui; use std::thread; use std::time; let mut rustautogui = RustAutoGui::new(false).unwrap(); // initialize let (screen_w, screen_h) = rustautogui.get_screen_size(); // load, process and store image of star(can be image of project name text) // regions are written in this way to cover any screen size, not hardcoding them rustautogui .store_template_from_file( "/home/davor/Pictures/stars.png", Some(( (0.2 * screen_w as f32) as u32, // start x 0, // start y (0.5 * screen_w as f32) as u32, // width (0.4 * screen_h as f32) as u32, // height )), rustautogui::MatchMode::Segmented, "stars", ) .unwrap(); // just for example doing single prepare, but you would want to store it also // load, process and store image of terms rustautogui .prepare_template_from_file( "/home/davor/Pictures/terms.png", Some(( (0.1 * screen_w as f32) as u32, // start x ``` -------------------------------- ### Automate Browser Interaction with RustAutoGui Source: https://docs.rs/rustautogui/latest/rustautogui/struct.RustAutoGui.html This example demonstrates a full workflow of opening a browser, navigating to a URL, and interacting with web elements using RustAutoGui. It includes opening the Windows search, typing a browser name, launching it, opening a new tab, entering a URL, and confirming the navigation. ```Rust fn main() { let rustautogui = rustautogui::RustAutoGUI::new(); // find image on screen and move mouse rustautogui .find_image_on_screen_and_move_mouse( "./images/star.png", 0.95, 1.0, rustautogui::MatchMode::FuzzyMoster, (0.7 * screen_h as f32) as u32, // start y (0.5 * screen_w as f32) as u32, // width (0.3 * screen_h as f32) as u32, // height ) .unwrap(); // press windows/super key rustautogui.keyboard_command("win").unwrap(); thread::sleep(time::Duration::from_millis(500)); // write in chrome rustautogui.keyboard_input("chrome").unwrap(); thread::sleep(time::Duration::from_millis(500)); // run chrome with clicking return/enter rustautogui.keyboard_command("return").unwrap(); thread::sleep(time::Duration::from_millis(500)); // open new tab with ctrl+t rustautogui.keyboard_multi_key("ctrl", "t", None).unwrap(); thread::sleep(time::Duration::from_millis(500)); // input github in url (url input area selected automatically by new tab opening) rustautogui .keyboard_input("https://github.com/DavorMar/rustautogui") .unwrap(); thread::sleep(time::Duration::from_millis(500)); // return/enter to open github rustautogui.keyboard_command("return").unwrap(); thread::sleep(time::Duration::from_millis(500)); // loop till image of star is found or timeout of 15 seconds is hit rustautogui .loop_find_stored_image_on_screen_and_move_mouse(0.95, 1.0, 15, "stars") .unwrap(); thread::sleep(time::Duration::from_millis(500)); // rustautogui.left_click().unwrap(); thread::sleep(time::Duration::from_millis(500)); // scroll down 80 times to the bottom of the page where small terms button is rustautogui.scroll_down(80).unwrap(); // loop till image found with timeout of 15 seconds rustautogui .loop_find_image_on_screen_and_move_mouse(0.95, 1.0, 15) .unwrap(); thread::sleep(time::Duration::from_millis(500)); rustautogui.left_click().unwrap(); } ``` -------------------------------- ### Automate Browser Interaction with RustAutoGui Source: https://docs.rs/rustautogui/latest/rustautogui/struct.RustAutoGui.html This example demonstrates a full workflow of opening a browser, navigating to a URL, and interacting with web elements using RustAutoGui. It includes opening the Windows search, typing a browser name, launching it, opening a new tab, entering a URL, and confirming the navigation. ```Rust // press windows/super key rustautogui.keyboard_command("win").unwrap(); thread::sleep(time::Duration::from_millis(500)); // write in chrome rustautogui.keyboard_input("chrome").unwrap(); thread::sleep(time::Duration::from_millis(500)); // run chrome with clicking return/enter rustautogui.keyboard_command("return").unwrap(); thread::sleep(time::Duration::from_millis(500)); // open new tab with ctrl+t rustautogui.keyboard_multi_key("ctrl", "t", None).unwrap(); thread::sleep(time::Duration::from_millis(500)); // input github in url (url input area selected automatically by new tab opening) rustautogui .keyboard_input("https://github.com/DavorMar/rustautogui") .unwrap(); thread::sleep(time::Duration::from_millis(500)); // return/enter to open github rustautogui.keyboard_command("return").unwrap(); thread::sleep(time::Duration::from_millis(500)); // loop till image of star is found or timeout of 15 seconds is hit rustautogui .loop_find_stored_image_on_screen_and_move_mouse(0.95, 1.0, 15, "stars") .unwrap(); thread::sleep(time::Duration::from_millis(500)); // rustautogui.left_click().unwrap(); thread::sleep(time::Duration::from_millis(500)); // scroll down 80 times to the bottom of the page where small terms button is rustautogui.scroll_down(80).unwrap(); // loop till image found with timeout of 15 seconds rustautogui .loop_find_image_on_screen_and_move_mouse(0.95, 1.0, 15) .unwrap(); thread::sleep(time::Duration::from_millis(500)); rustautogui.left_click().unwrap(); } } ``` -------------------------------- ### Initialize and Get Screen Size with RustAutoGui Source: https://docs.rs/rustautogui/latest/rustautogui/struct.RustAutoGui.html Initializes the RustAutoGui and retrieves the screen dimensions. This is particularly useful on Linux systems where the screen encompasses all monitors. ```Rust 1fn main() { #[cfg(not(feature = "lite"))] { use rustautogui; use rustautogui::imgtools; // initialize autogui let mut gui = rustautogui::RustAutoGui::new(false).unwrap(); // displays (x, y) size of screen // especially useful on linux where screen from all monitors is grabbed gui.get_screen_size(); ``` -------------------------------- ### Image Recognition and Mouse Movement with RustAutoGui Source: https://docs.rs/rustautogui/latest/rustautogui/struct.RustAutoGui.html This example shows how to find stored images on screen, move the mouse to them, and click. It includes looping until an image is found or a timeout occurs. ```Rust // loop till image of star is found or timeout of 15 seconds is hit rustautogui .loop_find_stored_image_on_screen_and_move_mouse(0.95, 1.0, 15, "stars") .unwrap(); thread::sleep(time::Duration::from_millis(500)); // rustautogui.left_click().unwrap(); thread::sleep(time::Duration::from_millis(500)); ``` ```Rust // loop till image found with timeout of 15 seconds rustautogui .loop_find_image_on_screen_and_move_mouse(0.95, 1.0, 15) .unwrap(); thread::sleep(time::Duration::from_millis(500)); rustautogui.left_click().unwrap(); ``` -------------------------------- ### Automate Opening GitHub Page with RustAutoGui Source: https://docs.rs/rustautogui/latest/rustautogui/struct.RustAutoGui.html This example demonstrates how to automate opening a specific GitHub page using RustAutoGui. It includes initializing the GUI, loading and preparing image templates, simulating keyboard inputs to launch a browser and navigate to a URL, and using image recognition to find elements on the page. ```Rust fn main() { #[cfg(not(feature = "lite"))] { use rustautogui::RustAutoGui; use std::thread; use std::time; let mut rustautogui = RustAutoGui::new(false).unwrap(); // initialize let (screen_w, screen_h) = rustautogui.get_screen_size(); // load, process and store image of star(can be image of project name text) // regions are written in this way to cover any screen size, not hardcoding them rustautogui .store_template_from_file( "/home/davor/Pictures/stars.png", Some(( (0.2 * screen_w as f32) as u32, // start x 0, // start y (0.5 * screen_w as f32) as u32, // width (0.4 * screen_h as f32) as u32, // height )), // end region rustautogui::MatchMode::Segmented, "stars", ) .unwrap(); // just for example doing single prepare, but you would want to store it also // load, process and store image of terms rustautogui .prepare_template_from_file( "/home/davor/Pictures/terms.png", Some(( (0.1 * screen_w as f32) as u32, // start x (0.7 * screen_h as f32) as u32, // start y (0.5 * screen_w as f32) as u32, // width (0.3 * screen_h as f32) as u32, // height )), // end region rustautogui::MatchMode::Segmented, ) .unwrap(); // press windows/super key rustautogui.keyboard_command("win").unwrap(); thread::sleep(time::Duration::from_millis(500)); // write in chrome rustautogui.keyboard_input("chrome").unwrap(); thread::sleep(time::Duration::from_millis(500)); // run chrome with clicking return/enter rustautogui.keyboard_command("return").unwrap(); thread::sleep(time::Duration::from_millis(500)); // open new tab with ctrl+t rustautogui.keyboard_multi_key("ctrl", "t", None).unwrap(); thread::sleep(time::Duration::from_millis(500)); // input github in url (url input area selected automatically by new tab opening) rustautogui .keyboard_input("https://github.com/DavorMar/rustautogui") .unwrap(); thread::sleep(time::Duration::from_millis(500)); // return/enter to open github rustautogui.keyboard_command("return").unwrap(); thread::sleep(time::Duration::from_millis(500)); // loop till image of star is found or timeout of 15 seconds is hit rustautogui .loop_find_stored_image_on_screen_and_move_mouse(0.95, 1.0, 15, "stars") .unwrap(); thread::sleep(time::Duration::from_millis(500)); // rustautogui.left_click().unwrap(); thread::sleep(time::Duration::from_millis(500)); // scroll down 80 times to the bottom of the page where small terms button is rustautogui.scroll_down(80).unwrap(); // loop till image found with timeout of 15 seconds rustautogui .loop_find_image_on_screen_and_move_mouse(0.95, 1.0, 15) .unwrap(); thread::sleep(time::Duration::from_millis(500)); rustautogui.left_click().unwrap(); } } ``` -------------------------------- ### Automate Browser Navigation with RustAutoGui Source: https://docs.rs/rustautogui/latest/rustautogui/struct.RustAutoGui.html This example demonstrates automating browser actions like opening Chrome, typing URLs, and navigating pages using keyboard commands and input. ```Rust // press windows/super key rustautogui.keyboard_command("win").unwrap(); thread::sleep(time::Duration::from_millis(500)); // write in chrome rustautogui.keyboard_input("chrome").unwrap(); thread::sleep(time::Duration::from_millis(500)); // run chrome with clicking return/enter rustautogui.keyboard_command("return").unwrap(); thread::sleep(time::Duration::from_millis(500)); // open new tab with ctrl+t rustautogui.keyboard_multi_key("ctrl", "t", None).unwrap(); thread::sleep(time::Duration::from_millis(500)); // input github in url (url input area selected automatically by new tab opening) rustautogui .keyboard_input("https://github.com/DavorMar/rustautogui") .unwrap(); thread::sleep(time::Duration::from_millis(500)); // return/enter to open github rustautogui.keyboard_command("return").unwrap(); ``` -------------------------------- ### General RustAutoGUI Functions Source: https://docs.rs/rustautogui/latest/rustautogui/index.html Provides examples for common utility functions. Debug mode can be enabled to log detailed information and save images. Warnings can be suppressed. ```rust rustautogui.get_screen_size(); // returns (x, y) size of display ``` ```rust rustautogui.change_debug_state(true); // change debugging ``` ```rust rustautogui.set_suppress_warning(true); // turn off warnings ``` ```rust rustautogui.save_screenshot("test.png").unwrap(); //saves screen screenshot ``` -------------------------------- ### Initialize and Get Screen Size Source: https://docs.rs/rustautogui/latest/rustautogui/struct.RustAutoGui.html Initializes the RustAutoGui and retrieves the screen dimensions. This is particularly useful on Linux systems where the screen might encompass multiple monitors. ```Rust let mut gui = rustautogui::RustAutoGui::new(false).unwrap(); gui.get_screen_size(); ``` -------------------------------- ### Mouse Clicks with RustAutoGui Source: https://docs.rs/rustautogui/latest/rustautogui/struct.RustAutoGui.html Provides examples for performing mouse clicks using the `RustAutoGui` library. You can execute left, right, or middle clicks. ```Rust gui.click(button: MouseClick).unwrap(); ``` ```Rust gui.left_click().unwrap(); ``` -------------------------------- ### Install OpenCL Drivers (Linux) Source: https://docs.rs/rustautogui/latest/rustautogui/index.html Install specific OpenCL drivers for Nvidia, AMD, or Intel GPUs on Linux systems. Follow official documentation for AMD ROCm. ```bash For Nvidia GPUs: `sudo apt install nvidia-opencl-icd` For AMD GPUs: Follow official ROCm docs for your distro and GPU For Intel GPUs: `sudo apt install intel-opencl-icd` ``` -------------------------------- ### new Source: https://docs.rs/rustautogui/latest/rustautogui/struct.RustAutoGui.html Initializes a new RustAutoGui instance, setting up screen, keyboard, and mouse functionalities. ```APIDOC ## new ### Description Initializes the screen, keyboard, and mouse functionalities, assigning them to a new `RustAutoGui` struct. All other struct fields are initialized to 0 or None. ### Method Signature `pub fn new(debug: bool) -> Result` ### Parameters - `debug`: `bool` - A flag to enable or disable debug mode. ### Returns - `Result` - A new `RustAutoGui` instance if initialization is successful, otherwise an `AutoGuiError`. ``` -------------------------------- ### Get Mouse Position Source: https://docs.rs/rustautogui/latest/rustautogui/index.html Retrieves the current coordinates of the mouse cursor. ```APIDOC ## Get Mouse Position ### Description Retrieves the current (x, y) coordinates of the mouse cursor on the screen. ### Method - `rustautogui.get_mouse_position()`: Returns a tuple `(x, y)` representing the current mouse coordinates. ### Example ```rust let (x, y) = rustautogui.get_mouse_position().unwrap(); println!("Current mouse position: ({}, {})", x, y); ``` ``` -------------------------------- ### Initialize and Find Image with RustAutoGui Source: https://docs.rs/rustautogui/latest/rustautogui/struct.RustAutoGui.html Demonstrates initializing RustAutoGui, preparing image templates for searching, and finding an image on the screen. It also shows how to move the mouse to the found image's location. ```Rust 1fn main() { #[cfg(not(feature = "lite"))] { use rustautogui; use rustautogui::imgtools; // initialize autogui let mut gui = rustautogui::RustAutoGui::new(false).unwrap(); // displays (x, y) size of screen // especially useful on linux where screen from all monitors is grabbed gui.get_screen_size(); { // load the image searching for. Region is Option<(startx, starty, width, height)> of search. Matchmode FFT or Segmented (not implemented before 1.0 version), max segments, only important for Segmented match mode gui.prepare_template_from_file( "test.png", Some((0, 0, 500, 300)), rustautogui::MatchMode::FFT, ) .unwrap(); } // or another way to prepare template { let img = imgtools::load_image_rgba("test.png").unwrap(); // just loading this way for example gui.prepare_template_from_imagebuffer( img, Some((0, 0, 700, 500)), rustautogui::MatchMode::Segmented, ) .unwrap(); } // or segmented variant with no region gui.prepare_template_from_file("test.png", None, rustautogui::MatchMode::FFT) .unwrap(); // change prepare template settings, like region, matchmode or max segments // automatically move mouse to found template position, execute movement for 1 second gui.find_image_on_screen_and_move_mouse(0.9, 1.0).unwrap(); //move mouse to position (in this case to bottom left of stanard 1920x1080 monitor), move the mouse for 1 second gui.move_mouse_to_pos(1920, 1080, 1.0).unwrap(); // execute left click, move the mouse to x (500), y (500) position for 1 second, release mouse click // used for actions like moving icons or files // suggestion: dont use very small moving time values, especially on mac gui.drag_mouse(500, 500, 1.0).unwrap(); // execute left click gui.left_click().unwrap(); // execute mouse right click gui.right_click().unwrap(); // execute mouse middle click gui.middle_click().unwrap(); // execute a double (left) mouse click gui.double_click().unwrap(); // mouse scrolls gui.scroll_down(1).unwrap(); gui.scroll_up(5).unwrap(); gui.scroll_right(8).unwrap(); gui.scroll_left(10).unwrap(); // input keyboard string gui.keyboard_input("test.com").unwrap(); // press a keyboard command, in this case enter(return) gui.keyboard_command("return").unwrap(); // two key press gui.keyboard_multi_key("alt", "tab", None).unwrap(); // three key press gui.keyboard_multi_key("shift", "control", Some("e")) .unwrap(); // maybe you would want to loop search until image is found and break the loop then loop { let pos = gui.find_image_on_screen_and_move_mouse(0.9, 1.0).unwrap(); match pos { Some(_) => break, None => (), } } } } ``` -------------------------------- ### Get Mouse Position Source: https://docs.rs/rustautogui/latest/rustautogui/index.html Retrieves the current coordinates of the mouse cursor on the screen. ```rust rustautogui.get_mouse_position().unwrap(); // returns (x,y) coordinate of mouse ``` -------------------------------- ### Error::cause Method (Deprecated) Source: https://docs.rs/rustautogui/latest/rustautogui/errors/enum.AutoGuiError.html Deprecated method for getting the error cause. Replaced by `Error::source` which supports downcasting. ```rust fn cause(&self) -> Option<&dyn Error> { self.source() } ``` -------------------------------- ### Error::description Method (Deprecated) Source: https://docs.rs/rustautogui/latest/rustautogui/errors/enum.AutoGuiError.html Deprecated method for getting an error description. Use the `Display` implementation or `to_string()` instead. ```rust fn description(&self) -> &str { "AutoGuiError" } ``` -------------------------------- ### Initialize and Use RustAutoGui Source: https://docs.rs/rustautogui/latest/rustautogui/struct.RustAutoGui.html Demonstrates the initialization of RustAutoGui and various common operations like screen size retrieval, image template preparation, mouse movement, and clicks. Ensure the 'lite' feature is not enabled for full functionality. ```Rust 1fn main() { #[cfg(not(feature = "lite"))] { use rustautogui; use rustautogui::imgtools; // initialize autogui let mut gui = rustautogui::RustAutoGui::new(false).unwrap(); // displays (x, y) size of screen // especially useful on linux where screen from all monitors is grabbed gui.get_screen_size(); { // load the image searching for. Region is Option<(startx, starty, width, height)> of search. Matchmode FFT or Segmented (not implemented before 1.0 version), max segments, only important for Segmented match mode gui.prepare_template_from_file( "test.png", Some((0, 0, 500, 300)), rustautogui::MatchMode::FFT, ) .unwrap(); } // or another way to prepare template { let img = imgtools::load_image_rgba("test.png").unwrap(); // just loading this way for example gui.prepare_template_from_imagebuffer( img, Some((0, 0, 700, 500)), rustautogui::MatchMode::Segmented, ) .unwrap(); } // or segmented variant with no region gui.prepare_template_from_file("test.png", None, rustautogui::MatchMode::FFT) .unwrap(); // change prepare template settings, like region, matchmode or max segments // automatically move mouse to found template position, execute movement for 1 second gui.find_image_on_screen_and_move_mouse(0.9, 1.0).unwrap(); //move mouse to position (in this case to bottom left of stanard 1920x1080 monitor), move the mouse for 1 second gui.move_mouse_to_pos(1920, 1080, 1.0).unwrap(); // execute left click, move the mouse to x (500), y (500) position for 1 second, release mouse click // used for actions like moving icons or files // suggestion: dont use very small moving time values, especially on mac gui.drag_mouse(500, 500, 1.0).unwrap(); // execute mouse left click gui.left_click().unwrap(); // execute mouse right click gui.right_click().unwrap(); // execute mouse middle click gui.middle_click().unwrap(); // execute a double (left) mouse click gui.double_click().unwrap(); // mouse scrolls gui.scroll_down(1).unwrap(); gui.scroll_up(5).unwrap(); gui.scroll_right(8).unwrap(); gui.scroll_left(10).unwrap(); // input keyboard string gui.keyboard_input("test.com").unwrap(); // press a keyboard command, in this case enter(return) gui.keyboard_command("return").unwrap(); // two key press gui.keyboard_multi_key("alt", "tab", None).unwrap(); // three key press gui.keyboard_multi_key("shift", "control", Some("e")) .unwrap(); // maybe you would want to loop search until image is found and break the loop then loop { let pos = gui.find_image_on_screen_and_move_mouse(0.9, 1.0).unwrap(); match pos { Some(_) => break, None => (), } } } } ``` -------------------------------- ### Automate Browser Interaction with RustAutoGui Source: https://docs.rs/rustautogui/latest/rustautogui/struct.RustAutoGui.html This snippet demonstrates a full workflow of opening a browser, navigating to a URL, and interacting with web elements using keyboard commands and image searching. It includes opening the Windows key, searching for Chrome, typing a URL, and scrolling. ```Rust // press windows/super key rustautogui.keyboard_command("win").unwrap(); thread::sleep(time::Duration::from_millis(500)); // write in chrome rustautogui.keyboard_input("chrome").unwrap(); thread::sleep(time::Duration::from_millis(500)); // run chrome with clicking return/enter rustautogui.keyboard_command("return").unwrap(); thread::sleep(time::Duration::from_millis(500)); // open new tab with ctrl+t rustautogui.keyboard_multi_key("ctrl", "t", None).unwrap(); thread::sleep(time::Duration::from_millis(500)); // input github in url (url input area selected automatically by new tab opening) rustautogui .keyboard_input("https://github.com/DavorMar/rustautogui") .unwrap(); thread::sleep(time::Duration::from_millis(500)); // return/enter to open github rustautogui.keyboard_command("return").unwrap(); thread::sleep(time::Duration::from_millis(500)); // loop till image of star is found or timeout of 15 seconds is hit rustautogui .loop_find_stored_image_on_screen_and_move_mouse(0.95, 1.0, 15, "stars") .unwrap(); thread::sleep(time::Duration::from_millis(500)); // rustautogui.left_click().unwrap(); thread::sleep(time::Duration::from_millis(500)); // scroll down 80 times to the bottom of the page where small terms button is rustautogui.scroll_down(80).unwrap(); // loop till image found with timeout of 15 seconds rustautogui .loop_find_image_on_screen_and_move_mouse(0.95, 1.0, 15) .unwrap(); thread::sleep(time::Duration::from_millis(500)); rustautogui.left_click().unwrap(); } } ``` -------------------------------- ### Import and Initialize RustAutoGui Source: https://docs.rs/rustautogui/latest/rustautogui/index.html Import the rustautogui crate and create a new instance. The constructor accepts a boolean argument for enabling debug mode. ```rust use rustautogui; let mut rustautogui = rustautogui::RustAutoGui::new(false); // arg: debug ``` -------------------------------- ### Prepare Template from File (FFT) Source: https://docs.rs/rustautogui/latest/rustautogui/struct.RustAutoGui.html Loads an image file to be used as a template for searching on the screen. Allows specifying a region of interest and the matching mode (FFT). ```Rust { // load the image searching for. Region is Option<(startx, starty, width, height)> of search. Matchmode FFT or Segmented (not implemented before 1.0 version), max segments, only important for Segmented match mode gui.prepare_template_from_file( "test.png", Some((0, 0, 500, 300)), rustautogui::MatchMode::FFT, ) .unwrap(); } ``` -------------------------------- ### Prepare Template from File Source: https://docs.rs/rustautogui/latest/rustautogui/struct.RustAutoGui.html Loads an image file to be used as a template for searching on screen. Supports specifying a region of interest and match mode. ```Rust gui.prepare_template_from_file( "test.png", Some((0, 0, 500, 300)), rustautogui::MatchMode::FFT, ) .unwrap(); ``` ```Rust gui.prepare_template_from_file("test.png", None, rustautogui::MatchMode::FFT) .unwrap(); ``` -------------------------------- ### Prepare Image Template from File (No Region) with RustAutoGui Source: https://docs.rs/rustautogui/latest/rustautogui/struct.RustAutoGui.html Prepares an image template from a file without specifying a region of interest, allowing the search to cover the entire screen. ```Rust // or segmented variant with no region gui.prepare_template_from_file("test.png", None, rustautogui::MatchMode::FFT) .unwrap(); ``` -------------------------------- ### Prepare Template from File Source: https://docs.rs/rustautogui/latest/rustautogui/struct.RustAutoGui.html Loads an image file to be used as a template for image searching. You can specify a region of the screen to search within and the matching mode (FFT or Segmented). ```Rust gui.prepare_template_from_file( "test.png", Some((0, 0, 500, 300)), rustautogui::MatchMode::FFT, ) .unwrap(); ``` ```Rust gui.prepare_template_from_file("test.png", None, rustautogui::MatchMode::FFT) .unwrap(); ``` -------------------------------- ### Prepare Template from File with Dynamic Region Source: https://docs.rs/rustautogui/latest/rustautogui/struct.RustAutoGui.html Prepares an image file as a template for searching, with a search region defined dynamically based on screen width. This ensures the search area scales with the screen resolution. ```Rust let (screen_w, screen_h) = rustautogui.get_screen_size(); rustautogui .prepare_template_from_file( "/home/davor/Pictures/terms.png", Some(( (0.1 * screen_w as f32) as u32, // start x ``` -------------------------------- ### RustAutoGui::new Source: https://docs.rs/rustautogui/latest/rustautogui/struct.RustAutoGui.html Initializes a new instance of RustAutoGui. The boolean parameter can be used to disable or enable features, potentially for a 'lite' version. ```APIDOC ## RustAutoGui::new ### Description Initializes a new instance of the RustAutoGui automation tool. The `lite` parameter can be used to control feature availability. ### Method `RustAutoGui::new(lite: bool) -> Result>` ### Parameters * `lite` (bool) - If true, may enable a 'lite' mode with reduced features. ``` -------------------------------- ### Prepare Template from File Source: https://docs.rs/rustautogui/latest/rustautogui/struct.RustAutoGui.html Loads an image from a file to be used as a template for screen searching. You can specify a region of the screen to search within and the matching mode. ```APIDOC ## prepare_template_from_file ### Description Loads an image from a file to be used as a template for screen searching. Allows specifying a search region and matching mode. ### Method RustAutoGui::prepare_template_from_file ### Parameters - **file_path** (string) - Required - The path to the image file. - **region** (Option<(u32, u32, u32, u32)>) - Optional - The region of the screen to search within (x, y, width, height). - **match_mode** (MatchMode) - Required - The mode to use for matching the template (e.g., FFT, Segmented). ### Response None (returns Result<(), Error>) ``` -------------------------------- ### Prepare Image Template from File Source: https://docs.rs/rustautogui/latest/rustautogui/struct.RustAutoGui.html Loads an image from a file to be used as a template for searching on the screen. You can specify a region to search within and the matching mode. ```Rust gui.prepare_template_from_file("test.png", Some((0, 0, 500, 300)), rustautogui::MatchMode::FFT).unwrap(); ``` ```Rust gui.prepare_template_from_file("test.png", None, rustautogui::MatchMode::FFT).unwrap(); ``` -------------------------------- ### Prepare Template from File (No Region) Source: https://docs.rs/rustautogui/latest/rustautogui/struct.RustAutoGui.html Prepares a template from a file without specifying a search region, using FFT matching mode. ```Rust gui.prepare_template_from_file("test.png", None, rustautogui::MatchMode::FFT) .unwrap(); ```