### RustAutoGui Library Setup and Configuration Source: https://docs.rs/rustautogui/latest/src/rustautogui/lib.rs.html This snippet shows the basic setup of the RustAutoGui library, including conditional compilation for features like 'lite' and 'opencl', and the declaration of public modules. It also defines the `MatchMode` enum for image matching strategies. ```rust #![allow(unused_doc_comments, unused_imports)] #![doc = include_str!("../README.md")] #[cfg(all(feature = "lite", feature = "opencl"))] compile_error!("Features `lite` and `opencl` cannot be enabled at the same time."); // Regular private modules #[cfg(not(any(test, feature = "dev")))] mod core; #[cfg(not(any(test, feature = "dev")))] mod data; // Public modules during testing #[cfg(any(test, feature = "dev"))] pub mod core; #[cfg(any(test, feature = "dev"))] pub mod data; pub mod errors; pub mod imgtools; mod rustautogui_impl; #[cfg(not(feature = "lite"))] use data::*; use crate::errors::*; use std::{collections::HashMap, env}; #[cfg(not(feature = "lite"))] use core::template_match; use core::{ keyboard::Keyboard, mouse::{mouse_position, Mouse, MouseScroll}, screen::Screen, }; // opencl stuff #[cfg(feature = "opencl")] use crate::data::{DevicesInfo, OpenClData}; #[cfg(feature = "opencl")] use ocl::{enums, Buffer, Context, Kernel, Program, Queue}; pub use core::mouse::mouse_position::print_mouse_position; pub use core::mouse::MouseClick; #[cfg(not(feature = "lite"))] const DEFAULT_ALIAS: &str = "default_rsgui_!#123#! D"; #[cfg(not(feature = "lite"))] const DEFAULT_BCKP_ALIAS: &str = "bckp_tmpl_.#!123!#."; /// Matchmode Segmented correlation and Fourier transform correlation #[derive(PartialEq, Debug)] #[cfg(not(feature = "lite"))] pub enum MatchMode { Segmented, FFT, #[cfg(feature = "opencl")] SegmentedOcl, #[cfg(feature = "opencl")] SegmentedOclV2, } #[cfg(not(feature = "lite"))] impl Clone for MatchMode { fn clone(&self) -> Self { match self { MatchMode::Segmented => MatchMode::Segmented, MatchMode::FFT => MatchMode::FFT, #[cfg(feature = "opencl")] MatchMode::SegmentedOcl => MatchMode::SegmentedOcl, #[cfg(feature = "opencl")] MatchMode::SegmentedOclV2 => MatchMode::SegmentedOclV2, } } } ``` -------------------------------- ### Install Linux Dependencies Source: https://docs.rs/rustautogui/latest/index.html For Linux users, install the necessary development libraries for X11 and XTest before using the rustautogui crate. ```bash sudo apt-get update sudo apt-get install libx11-dev libxtst-dev ``` -------------------------------- ### Launch Chrome and Navigate to GitHub Source: https://docs.rs/rustautogui/latest/rustautogui/struct.RustAutoGui.html This snippet demonstrates launching Chrome, typing a URL, and navigating to a GitHub repository. It includes examples of keyboard commands, inputting text, and multi-key combinations. ```rust // press windows/super key rustautogui.keyboard_command("win").unwrap(); thread::sleep(time::Duration::from_millis(500)); // write in chrome rustautogui.keyboard_input("chrome").unwrap(); thread::sleep(time::Duration::from_millis(500)); // run chrome with clicking return/enter rustautogui.keyboard_command("return").unwrap(); thread::sleep(time::Duration::from_millis(500)); // open new tab with ctrl+t rustautogui.keyboard_multi_key("ctrl", "t", None).unwrap(); thread::sleep(time::Duration::from_millis(500)); // input github in url (url input area selected automatically by new tab opening) rustautogui .keyboard_input("https://github.com/DavorMar/rustautogui") .unwrap(); thread::sleep(time::Duration::from_millis(500)); // return/enter to open github rustautogui.keyboard_command("return").unwrap(); ``` -------------------------------- ### Initialize and Get Screen Size with RustAutoGUI Source: https://docs.rs/rustautogui/latest/rustautogui/struct.RustAutoGui.html Initializes the RustAutoGUI instance and retrieves the screen dimensions. Useful for understanding the available screen real estate, especially on multi-monitor setups. ```rust 1fn main() { #[cfg(not(feature = "lite"))] { use rustautogui; use rustautogui::imgtools; // initialize autogui let mut gui = rustautogui::RustAutoGui::new(false).unwrap(); // displays (x, y) size of screen // especially useful on linux where screen from all monitors is grabbed gui.get_screen_size(); // ... rest of the code ... } } ``` -------------------------------- ### Initialize and Get Screen Size Source: https://docs.rs/rustautogui/latest/rustautogui/struct.RustAutoGui.html Initializes the RustAutoGui instance and retrieves the screen dimensions. This is useful for understanding the available screen real estate, especially on multi-monitor setups. ```rust 1fn main() { 2 #[cfg(not(feature = "lite"))] 3 { 4 use rustautogui; 5 use rustautogui::imgtools; 6 // initialize autogui 7 let mut gui = rustautogui::RustAutoGui::new(false).unwrap(); 8 9 // displays (x, y) size of screen 10 // especially useful on linux where screen from all monitors is grabbed 11 gui.get_screen_size(); 12 13 { 14 // load the image searching for. Region is Option<(startx, starty, width, height)> of search. Matchmode FFT or Segmented (not implemented before 1.0 version), max segments, only important for Segmented match mode 15 gui.prepare_template_from_file( 16 "test.png", 17 Some((0, 0, 500, 300)), 18 rustautogui::MatchMode::FFT, 19 ) 20 .unwrap(); 21 } 22 // or another way to prepare template 23 { 24 let img = imgtools::load_image_rgba("test.png").unwrap(); // just loading this way for example 25 gui.prepare_template_from_imagebuffer( 26 img, 27 Some((0, 0, 700, 500)), 28 rustautogui::MatchMode::Segmented, 29 ) 30 .unwrap(); 31 } 32 33 // or segmented variant with no region 34 gui.prepare_template_from_file("test.png", None, rustautogui::MatchMode::FFT) 35 .unwrap(); 36 37 // change prepare template settings, like region, matchmode or max segments 38 39 // automatically move mouse to found template position, execute movement for 1 second 40 gui.find_image_on_screen_and_move_mouse(0.9, 1.0).unwrap(); 41 42 //move mouse to position (in this case to bottom left of stanard 1920x1080 monitor), move the mouse for 1 second 43 gui.move_mouse_to_pos(1920, 1080, 1.0).unwrap(); 44 45 // execute left click, move the mouse to x (500), y (500) position for 1 second, release mouse click 46 // used for actions like moving icons or files 47 // suggestion: dont use very small moving time values, especially on mac 48 gui.drag_mouse(500, 500, 1.0).unwrap(); 49 50 // execute mouse left click 51 gui.left_click().unwrap(); 52 53 // execute mouse right click 54 gui.right_click().unwrap(); 55 56 // execute mouse middle click 57 gui.middle_click().unwrap(); 58 59 // execute a double (left) mouse click 60 gui.double_click().unwrap(); 61 62 // mouse scrolls 63 gui.scroll_down(1).unwrap(); 64 gui.scroll_up(5).unwrap(); 65 gui.scroll_right(8).unwrap(); 66 gui.scroll_left(10).unwrap(); 67 68 // input keyboard string 69 gui.keyboard_input("test.com").unwrap(); 70 71 // press a keyboard command, in this case enter(return) 72 gui.keyboard_command("return").unwrap(); 73 74 // two key press 75 gui.keyboard_multi_key("alt", "tab", None).unwrap(); 76 77 // three key press 78 gui.keyboard_multi_key("shift", "control", Some("e")) 79 .unwrap(); 80 81 // maybe you would want to loop search until image is found and break the loop then 82 loop { 83 let pos = gui.find_image_on_screen_and_move_mouse(0.9, 1.0).unwrap(); 84 match pos { 85 Some(_) => break, 86 None => (), 87 } 88 } 89 } 90} ``` -------------------------------- ### Automating Chrome with RustAutoGUI Source: https://docs.rs/rustautogui/latest/rustautogui/struct.RustAutoGui.html This example demonstrates a sequence of actions to automate opening Chrome, searching for a URL, and interacting with the page. It includes pressing the Windows key, typing a search query, opening a new tab, navigating to a URL, and scrolling. ```rust // press windows/super key rustautogui.keyboard_command("win").unwrap(); thread::sleep(time::Duration::from_millis(500)); // write in chrome rustautogui.keyboard_input("chrome").unwrap(); thread::sleep(time::Duration::from_millis(500)); // run chrome with clicking return/enter rustautogui.keyboard_command("return").unwrap(); thread::sleep(time::Duration::from_millis(500)); // open new tab with ctrl+t rustautogui.keyboard_multi_key("ctrl", "t", None).unwrap(); thread::sleep(time::Duration::from_millis(500)); // input github in url (url input area selected automatically by new tab opening) rustautogui .keyboard_input("https://github.com/DavorMar/rustautogui") .unwrap(); thread::sleep(time::Duration::from_millis(500)); // return/enter to open github rustautogui.keyboard_command("return").unwrap(); thread::sleep(time::Duration::from_millis(500)); // loop till image of star is found or timeout of 15 seconds is hit rustautogui .loop_find_stored_image_on_screen_and_move_mouse(0.95, 1.0, 15, "stars") .unwrap(); thread::sleep(time::Duration::from_millis(500)); // rustautogui.left_click().unwrap(); thread::sleep(time::Duration::from_millis(500)); // scroll down 80 times to the bottom of the page where small terms button is rustautogui.scroll_down(80).unwrap(); // loop till image found with timeout of 15 seconds rustautogui .loop_find_image_on_screen_and_move_mouse(0.95, 1.0, 15) .unwrap(); thread::sleep(time::Duration::from_millis(500)); rustautogui.left_click().unwrap(); } } ``` -------------------------------- ### Automate Opening GitHub in Chrome with RustAutoGui Source: https://docs.rs/rustautogui/latest/rustautogui/struct.RustAutoGui.html This example demonstrates how to automate opening a specific GitHub URL in Chrome using RustAutoGui. It includes steps for initializing the GUI, loading image templates, controlling the keyboard to launch applications and input URLs, and finding images on screen. ```rust 4fn main() { 5 #[cfg(not(feature = "lite"))] 6 { 7 use rustautogui::RustAutoGui; 8 use std::thread; 9 use std::time; 10 let mut rustautogui = RustAutoGui::new(false).unwrap(); // initialize 11 let (screen_w, screen_h) = rustautogui.get_screen_size(); 13 rustautogui 14 .store_template_from_file( 15 "/home/davor/Pictures/stars.png", 16 Some(( 17 (0.2 * screen_w as f32) as u32, // start x 18 0, // start y 19 (0.5 * screen_w as f32) as u32, // width 20 (0.4 * screen_h as f32) as u32, // height 21 )), 22 rustautogui::MatchMode::Segmented, 23 "stars", 24 ) 25 .unwrap(); 26 rustautogui 27 .prepare_template_from_file( 28 "/home/davor/Pictures/terms.png", 29 Some(( 30 (0.1 * screen_w as f32) as u32, // start x 31 (0.7 * screen_h as f32) as u32, // start y 32 (0.5 * screen_w as f32) as u32, // width 33 (0.3 * screen_h as f32) as u32, // height 34 )), 35 rustautogui::MatchMode::Segmented, 36 ) 37 .unwrap(); 38 rustautogui.keyboard_command("win").unwrap(); 39 thread::sleep(time::Duration::from_millis(500)); 40 rustautogui.keyboard_input("chrome").unwrap(); 41 thread::sleep(time::Duration::from_millis(500)); 42 rustautogui.keyboard_command("return").unwrap(); 43 thread::sleep(time::Duration::from_millis(500)); 44 rustautogui.keyboard_multi_key("ctrl", "t", None).unwrap(); 45 thread::sleep(time::Duration::from_millis(500)); 46 rustautogui 47 .keyboard_input("https://github.com/DavorMar/rustautogui") 48 .unwrap(); 49 thread::sleep(time::Duration::from_millis(500)); 50 rustautogui.keyboard_command("return").unwrap(); 51 thread::sleep(time::Duration::from_millis(500)); 52 rustautogui 53 .loop_find_stored_image_on_screen_and_move_mouse(0.95, 1.0, 15, "stars") 54 .unwrap(); 55 thread::sleep(time::Duration::from_millis(500)); 56 thread::sleep(time::Duration::from_millis(500)); 57 rustautogui.scroll_down(80).unwrap(); 58 rustautogui 59 .loop_find_image_on_screen_and_move_mouse(0.95, 1.0, 15) 60 .unwrap(); 61 thread::sleep(time::Duration::from_millis(500)); 62 rustautogui.left_click().unwrap(); 63 } 64} ``` -------------------------------- ### Automate Opening GitHub in Chrome Source: https://docs.rs/rustautogui/latest/rustautogui/struct.RustAutoGui.html This example demonstrates automating the process of opening a specific GitHub URL in Chrome. It includes steps for initializing RustAutoGui, preparing image templates, controlling the keyboard to launch applications and navigate web pages, and finding images on the screen. ```rust 4fn main() { 5 #[cfg(not(feature = "lite"))] 6 { 7 use rustautogui::RustAutoGui; 8 use std::thread; 9 use std::time; 10 let mut rustautogui = RustAutoGui::new(false).unwrap(); // initialize 11 let (screen_w, screen_h) = rustautogui.get_screen_size(); 12 // load, process and store image of star(can be image of project name text) 13 // regions are written in this way to cover any screen size, not hardcoding them 14 rustautogui 15 .store_template_from_file( 16 "/home/davor/Pictures/stars.png", 17 Some(( 18 (0.2 * screen_w as f32) as u32, // start x 19 0, // start y 20 (0.5 * screen_w as f32) as u32, // width 21 (0.4 * screen_h as f32) as u32, // height 22 )), 23 rustautogui::MatchMode::Segmented, 24 "stars", 25 ) 26 .unwrap(); 27 28 // just for example doing single prepare, but you would want to store it also 29 // load, process and store image of terms 30 rustautogui 31 .prepare_template_from_file( 32 "/home/davor/Pictures/terms.png", 33 Some(( 34 (0.1 * screen_w as f32) as u32, // start x 35 (0.7 * screen_h as f32) as u32, // start y 36 (0.5 * screen_w as f32) as u32, // width 37 (0.3 * screen_h as f32) as u32, // height 38 )), 39 rustautogui::MatchMode::Segmented, 40 ) 41 .unwrap(); 42 43 // press windows/super key 44 rustautogui.keyboard_command("win").unwrap(); 45 46 thread::sleep(time::Duration::from_millis(500)); 47 48 // write in chrome 49 rustautogui.keyboard_input("chrome").unwrap(); 50 51 thread::sleep(time::Duration::from_millis(500)); 52 53 // run chrome with clicking return/enter 54 rustautogui.keyboard_command("return").unwrap(); 55 56 thread::sleep(time::Duration::from_millis(500)); 57 58 // open new tab with ctrl+t 59 rustautogui.keyboard_multi_key("ctrl", "t", None).unwrap(); 60 61 thread::sleep(time::Duration::from_millis(500)); 62 63 // input github in url (url input area selected automatically by new tab opening) 64 rustautogui 65 .keyboard_input("https://github.com/DavorMar/rustautogui") 66 .unwrap(); 67 68 thread::sleep(time::Duration::from_millis(500)); 69 70 // return/enter to open github 71 rustautogui.keyboard_command("return").unwrap(); 72 73 thread::sleep(time::Duration::from_millis(500)); 74 75 // loop till image of star is found or timeout of 15 seconds is hit 76 rustautogui 77 .loop_find_stored_image_on_screen_and_move_mouse(0.95, 1.0, 15, "stars") 78 .unwrap(); 79 80 thread::sleep(time::Duration::from_millis(500)); 81 // rustautogui.left_click().unwrap(); 82 thread::sleep(time::Duration::from_millis(500)); 83 84 // scroll down 80 times to the bottom of the page where small terms button is 85 rustautogui.scroll_down(80).unwrap(); 86 87 // loop till image found with timeout of 15 seconds 88 rustautogui 89 .loop_find_image_on_screen_and_move_mouse(0.95, 1.0, 15) 90 .unwrap(); 91 92 thread::sleep(time::Duration::from_millis(500)); 93 rustautogui.left_click().unwrap(); 94 } 95} ``` -------------------------------- ### Find and Click Image on Screen Source: https://docs.rs/rustautogui/latest/rustautogui/struct.RustAutoGui.html This example shows how to find a stored image on the screen using a confidence threshold and timeout, then move the mouse to it. It also includes scrolling down the page and finding a generic image. ```rust rustautogui .loop_find_stored_image_on_screen_and_move_mouse(0.95, 1.0, 15, "stars") .unwrap(); thread::sleep(time::Duration::from_millis(500)); // rustautogui.left_click().unwrap(); thread::sleep(time::Duration::from_millis(500)) ``` ```rust rustautogui.scroll_down(80).unwrap(); // loop till image found with timeout of 15 seconds rustautogui .loop_find_image_on_screen_and_move_mouse(0.95, 1.0, 15) .unwrap(); thread::sleep(time::Duration::from_millis(500)); rustautogui.left_click().unwrap(); ``` -------------------------------- ### Initialize RustAutoGUI and Get Screen Size Source: https://docs.rs/rustautogui/latest/src/find_image_move_mouse_and_input/find_image_move_mouse_and_input.rs.html Initializes the RustAutoGUI instance and retrieves the screen dimensions. This is particularly useful on Linux systems where the screen might encompass multiple monitors. ```rust fn main() { #[cfg(not(feature = "lite"))] { use rustautogui; use rustautogui::imgtools; // initialize autogui let mut gui = rustautogui::RustAutoGui::new(false).unwrap(); // displays (x, y) size of screen // especially useful on linux where screen from all monitors is grabbed gui.get_screen_size(); ``` -------------------------------- ### Find Image on Screen and Move Mouse Source: https://docs.rs/rustautogui/latest/rustautogui/struct.RustAutoGui.html This example shows how to repeatedly search for a stored image on the screen within a given timeout and move the mouse to its location. It also includes scrolling functionality. ```rust rustautogui .loop_find_stored_image_on_screen_and_move_mouse(0.95, 1.0, 15, "stars") .unwrap(); thread::sleep(time::Duration::from_millis(500)); // rustautogui.left_click().unwrap(); thread::sleep(time::Duration::from_millis(500)); // scroll down 80 times to the bottom of the page where small terms button is rustautogui.scroll_down(80).unwrap(); // loop till image found with timeout of 15 seconds rustautogui .loop_find_image_on_screen_and_move_mouse(0.95, 1.0, 15) .unwrap(); thread::sleep(time::Duration::from_millis(500)); rustautogui.left_click().unwrap(); ``` -------------------------------- ### Setup OpenCL Device Source: https://docs.rs/rustautogui/latest/src/rustautogui/lib.rs.html Configures and selects an OpenCL device for accelerated operations. It scores available devices based on memory, compute units, and clock frequency, prioritizing GPUs. If a specific device ID is provided, it attempts to use that device; otherwise, it selects the highest-scoring GPU. ```rust #[cfg(feature = "opencl")] fn setup_opencl(device_id: Option) -> Result { let context = Context::builder().build()?; let available_devices = context.devices(); let device_count = available_devices.len(); let mut device_list: Vec = Vec::new(); let mut highest_score = 0; let mut best_device_index = 0; let mut max_workgroup_size = 0; for (i, device) in available_devices.into_iter().enumerate() { let device_type = device.info(enums::DeviceInfo::Type)?.to_string(); let workgroup_size: u32 = device .info(enums::DeviceInfo::MaxWorkGroupSize)? .to_string() .parse() .map_err(|_| AutoGuiError::OSFailure("Failed to read GPU data".to_string()))?; let global_mem: u64 = device .info(enums::DeviceInfo::GlobalMemSize)? .to_string() .parse() .map_err(|_| AutoGuiError::OSFailure("Failed to read GPU data".to_string()))?; let compute_units: u32 = device .info(enums::DeviceInfo::MaxComputeUnits)? .to_string() .parse() .map_err(|_| AutoGuiError::OSFailure("Failed to read GPU data".to_string()))?; let clock_frequency = device .info(enums::DeviceInfo::MaxClockFrequency)? .to_string() .parse() .map_err(|_| AutoGuiError::OSFailure("Failed to read GPU data".to_string()))?; let device_vendor = device.info(enums::DeviceInfo::Vendor)?.to_string(); let device_name = device.info(enums::DeviceInfo::Name)?.to_string(); let global_mem_gb = global_mem / 1_048_576; let score = global_mem_gb as u32 * 2 + compute_units * 10 + clock_frequency; let gui_device = DevicesInfo::new( device, i as u32, global_mem_gb as u32, clock_frequency, compute_units, device_vendor, device_name, score, ); device_list.push(gui_device); match device_id { Some(x) => { if x as usize > device_count { return Err(ocl::Error::from("No device found for the given index"))?; } if i == x as usize { highest_score = score; best_device_index = i; max_workgroup_size = workgroup_size; } } None => { if score >= highest_score && device_type.contains("GPU") { highest_score = score; best_device_index = i; max_workgroup_size = workgroup_size; } } } } let used_device = context.devices()[best_device_index as usize]; let queue = Queue::new(&context, used_device, None)?; let program_source = template_match::opencl_kernel::OCL_KERNEL; let program = Program::builder().src(program_source).build(&context)?; let opencl_data = OpenClData { device_list: device_list, ocl_program: program, ocl_context: context, ocl_queue: queue, ocl_buffer_storage: HashMap::new(), ocl_kernel_storage: HashMap::new(), ocl_workgroup_size: max_workgroup_size, }; Ok(opencl_data) } ``` -------------------------------- ### Initialize and Use RustAutoGui Source: https://docs.rs/rustautogui/latest/rustautogui/struct.RustAutoGui.html Demonstrates initializing RustAutoGui, preparing image templates, finding images on screen, and performing various mouse and keyboard actions. Includes screen size retrieval, template preparation with different modes and regions, mouse movement, clicking, dragging, scrolling, and keyboard input. ```rust fn main() { #[cfg(not(feature = "lite"))] { use rustautogui; use rustautogui::imgtools; // initialize autogui let mut gui = rustautogui::RustAutoGui::new(false).unwrap(); // displays (x, y) size of screen // especially useful on linux where screen from all monitors is grabbed gui.get_screen_size(); { // load the image searching for. Region is Option<(startx, starty, width, height)> of search. Matchmode FFT or Segmented (not implemented before 1.0 version), max segments, only important for Segmented match mode gui.prepare_template_from_file( "test.png", Some((0, 0, 500, 300)), rustautogui::MatchMode::FFT, ) .unwrap(); } // or another way to prepare template { let img = imgtools::load_image_rgba("test.png").unwrap(); // just loading this way for example gui.prepare_template_from_imagebuffer( img, Some((0, 0, 700, 500)), rustautogui::MatchMode::Segmented, ) .unwrap(); } // or segmented variant with no region gui.prepare_template_from_file("test.png", None, rustautogui::MatchMode::FFT) .unwrap(); // change prepare template settings, like region, matchmode or max segments // automatically move mouse to found template position, execute movement for 1 second gui.find_image_on_screen_and_move_mouse(0.9, 1.0).unwrap(); //move mouse to position (in this case to bottom left of stanard 1920x1080 monitor), move the mouse for 1 second gui.move_mouse_to_pos(1920, 1080, 1.0).unwrap(); // execute left click, move the mouse to x (500), y (500) position for 1 second, release mouse click // used for actions like moving icons or files // suggestion: dont use very small moving time values, especially on mac gui.drag_mouse(500, 500, 1.0).unwrap(); // execute mouse left click gui.left_click().unwrap(); // execute mouse right click gui.right_click().unwrap(); // execute mouse middle click gui.middle_click().unwrap(); // execute a double (left) mouse click gui.double_click().unwrap(); // mouse scrolls gui.scroll_down(1).unwrap(); gui.scroll_up(5).unwrap(); gui.scroll_right(8).unwrap(); gui.scroll_left(10).unwrap(); // input keyboard string gui.keyboard_input("test.com").unwrap(); // press a keyboard command, in this case enter(return) gui.keyboard_command("return").unwrap(); // two key press gui.keyboard_multi_key("alt", "tab", None).unwrap(); // three key press gui.keyboard_multi_key("shift", "control", Some("e")) .unwrap(); // maybe you would want to loop search until image is found and break the loop then loop { let pos = gui.find_image_on_screen_and_move_mouse(0.9, 1.0).unwrap(); match pos { Some(_) => break, None => (), } } } } ``` -------------------------------- ### Screen::new Source: https://docs.rs/rustautogui/latest/src/rustautogui/core/screen/linux/mod.rs.html Initializes a new Screen instance for Linux. It opens the X11 display, retrieves the root window, and captures screen dimensions. ```APIDOC ## Screen::new ### Description Initializes a new Screen instance for Linux. It opens the X11 display, retrieves the root window, and captures screen dimensions. ### Method `new()` ### Parameters None ### Returns `Screen` - A new instance of the Screen struct. ``` -------------------------------- ### Example Calling a Documented Function Source: https://docs.rs/rustautogui/latest/scrape-examples-help.html This example demonstrates how to call a documented function from another crate. Rustdoc scrapes such examples to include in the documentation of the called function. ```rust // examples/ex.rs fn main() { a_crate::a_func(); } ``` -------------------------------- ### RustAutoGui::new Source: https://docs.rs/rustautogui/latest/rustautogui/struct.RustAutoGui.html Initializes the RustAutoGui struct, setting up screen, keyboard, and mouse functionalities. All other struct fields are initialized to zero or None. ```APIDOC ## RustAutoGui::new ### Description Initializes the screen, keyboard, and mouse that are assigned to a new rustautogui struct. All the other struct fields are initiated as 0 or None. ### Signature ```rust pub fn new(debug: bool) -> Result ``` ### Parameters * `debug` (bool) - If true, enables debug mode for the GUI automation. ### Returns A `Result` containing the initialized `RustAutoGui` struct or an `AutoGuiError` if initialization fails. ``` -------------------------------- ### Get the deprecated cause for ImageProcessingError Source: https://docs.rs/rustautogui/latest/rustautogui/errors/enum.ImageProcessingError.html Provides a deprecated method to get the underlying cause of the error. It is recommended to use `Error::source` instead. ```rust fn cause(&self) -> Option<&dyn Error> ``` -------------------------------- ### Get the deprecated description for ImageProcessingError Source: https://docs.rs/rustautogui/latest/rustautogui/errors/enum.ImageProcessingError.html Provides a deprecated method to get a string description of the error. It is recommended to use the Display implementation or `to_string()` instead. ```rust fn description(&self) -> &str ``` -------------------------------- ### Launch Chrome and Navigate to GitHub Source: https://docs.rs/rustautogui/latest/rustautogui/struct.RustAutoGui.html This snippet demonstrates launching Chrome, typing a URL, and navigating to GitHub. It includes keyboard commands for opening the Windows key, typing search queries, and submitting them. ```rust rustautogui.keyboard_command("win").unwrap(); thread::sleep(time::Duration::from_millis(500)); // write in chrome rustautogui.keyboard_input("chrome").unwrap(); thread::sleep(time::Duration::from_millis(500)); // run chrome with clicking return/enter rustautogui.keyboard_command("return").unwrap(); thread::sleep(time::Duration::from_millis(500)); // open new tab with ctrl+t rustautogui.keyboard_multi_key("ctrl", "t", None).unwrap(); thread::sleep(time::Duration::from_millis(500)); // input github in url (url input area selected automatically by new tab opening) rustautogui .keyboard_input("https://github.com/DavorMar/rustautogui") .unwrap(); thread::sleep(time::Duration::from_millis(500)); // return/enter to open github rustautogui.keyboard_command("return").unwrap(); ``` -------------------------------- ### Initialize Linux Screen Capture Source: https://docs.rs/rustautogui/latest/src/rustautogui/core/screen/linux/mod.rs.html Opens the X11 display and retrieves screen dimensions and the root window. Panics if the display cannot be opened, often due to Wayland or missing X11. ```rust unsafe { // open the display (usually ":0"). This display pointer will be passed // to mouse and keyboard structs aswell let display: *mut _XDisplay = XOpenDisplay(ptr::null()); if display.is_null() { panic!("Error grabbing display. Unable to open X display. Possible x11 issue, check if it is activated and that you're not running wayland"); } // get root window let screen = XDefaultScreen(display); let root = XRootWindow(display, screen); let screen_width = XDisplayWidth(display, screen); let screen_height = XDisplayHeight(display, screen); #[cfg(not(feature = "lite"))] let img_data = ScreenImgData { pixel_data: vec![0u8; (screen_width * screen_height * 4) as usize], screen_region_width: 0, screen_region_height: 0, }; Screen { screen_width: screen_width, screen_height: screen_height, display: display, root_window: root, #[cfg(not(feature = "lite"))] screen_data: img_data, } } ``` -------------------------------- ### Get Mouse Position Source: https://docs.rs/rustautogui/latest/index.html Retrieves the current coordinates of the mouse cursor on the screen. ```rust rustautogui.get_mouse_position().unwrap(); // returns (x,y) coordinate of mouse ``` -------------------------------- ### Get Screen Dimensions Source: https://docs.rs/rustautogui/latest/src/rustautogui/core/screen/linux/mod.rs.html Returns the full screen dimensions (width, height) including all monitors. ```rust pub fn dimension(&self) -> (i32, i32) { let dimensions = (self.screen_width, self.screen_height); dimensions } ``` -------------------------------- ### dev_setup_opencl Source: https://docs.rs/rustautogui/latest/src/rustautogui/lib.rs.html Sets up OpenCL with optional device selection. This function is available when the 'dev' feature is enabled. ```APIDOC ## dev_setup_opencl ### Description Sets up OpenCL with optional device selection. This function is available when the 'dev' feature is enabled. ### Signature `pub fn dev_setup_opencl(device_id: Option) -> Result` ### Parameters - `device_id` (Option): An optional identifier for the OpenCL device to use. If `None`, the best available GPU device is selected. ``` -------------------------------- ### Get the alignment of a pointer Source: https://docs.rs/rustautogui/latest/rustautogui/errors/enum.ImageProcessingError.html Defines the ALIGN constant for pointer alignment. Part of the Pointable trait. ```rust const ALIGN: usize ``` -------------------------------- ### Basic Function Definition for Scraping Source: https://docs.rs/rustautogui/latest/scrape-examples-help.html This is a simple public function definition in Rust. Rustdoc can scrape examples that call this function. ```rust // src/lib.rs pub fn a_func() {} ``` -------------------------------- ### Get Screen Size Source: https://docs.rs/rustautogui/latest/src/rustautogui/lib.rs.html Retrieves the current screen dimensions. Returns a tuple containing the width and height of the screen in pixels. ```rust pub fn get_screen_size(&mut self) -> (i32, i32) { self.screen.dimension() } ``` -------------------------------- ### Get Region Dimensions Source: https://docs.rs/rustautogui/latest/src/rustautogui/core/screen/linux/mod.rs.html Returns the dimensions (width, height) of the currently configured screen region. This is typically set when a template is pre-calculated. ```rust pub fn region_dimension(&self) -> (u32, u32) { let dimensions = ( self.screen_data.screen_region_width, self.screen_data.screen_region_height, ); dimensions } ``` -------------------------------- ### Keyboard::new Source: https://docs.rs/rustautogui/latest/src/rustautogui/core/keyboard/linux/mod.rs.html Creates a new Keyboard instance. It initializes the keymap based on the detected keyboard layout. ```APIDOC ## Keyboard::new ### Description Creates a new keyboard instance. The display object is needed as an argument. The keymap is generated upon initialization. ### Signature ```rust pub fn new(screen: *mut _XDisplay) -> Self ``` ### Parameters * `screen` (*mut _XDisplay): A pointer to the X11 display object. ``` -------------------------------- ### Get Keycode from String Source: https://docs.rs/rustautogui/latest/src/rustautogui/core/keyboard/linux/mod.rs.html Converts a string representation of a key into its corresponding keycode and shifted state. This is a crucial step before sending keyboard input. ```rust pub unsafe fn get_keycode(&self, key: &str) -> Result<(u32, &bool), AutoGuiError> { let (value, shifted) = get_keymap_key(self, key)?; let mut keysym_to_keycode = HashMap::new(); let key_cstring = CString::new(value.clone())?; let key_cstring = key_cstring.as_ptr(); let keysym = XStringToKeysym(key_cstring); if keysym == 0 { return Err(AutoGuiError::OSFailure( "Failed to convert xstring to keysym. Keysym received is 0".to_string(), )); } if !keysym_to_keycode.contains_key(&keysym) { let keycode = XKeysymToKeycode(self.screen, keysym) as u32; keysym_to_keycode.insert(keysym, keycode); } let keycode = keysym_to_keycode[&keysym]; if keycode == 0 { return Err(AutoGuiError::OSFailure( "Failed to convert keysym to keycode. Keycode received is 0".to_string(), )); } Ok((keycode, shifted)) } ``` -------------------------------- ### Initialize and Prepare Template Source: https://docs.rs/rustautogui/latest/rustautogui/struct.RustAutoGui.html Initializes the RustAutoGui and prepares an image template for searching. Supports different preparation methods and match modes. ```rust 1fn main() { #[cfg(not(feature = "lite"))] { use rustautogui; use rustautogui::imgtools; // initialize autogui let mut gui = rustautogui::RustAutoGui::new(false).unwrap(); // displays (x, y) size of screen // especially useful on linux where screen from all monitors is grabbed gui.get_screen_size(); { // load the image searching for. Region is Option<(startx, starty, width, height)> of search. Matchmode FFT or Segmented (not implemented before 1.0 version), max segments, only important for Segmented match mode gui.prepare_template_from_file( "test.png", Some((0, 0, 500, 300)), rustautogui::MatchMode::FFT, ) .unwrap(); } // or another way to prepare template { let img = imgtools::load_image_rgba("test.png").unwrap(); // just loading this way for example gui.prepare_template_from_imagebuffer( img, Some((0, 0, 700, 500)), rustautogui::MatchMode::Segmented, ) .unwrap(); } // or segmented variant with no region gui.prepare_template_from_file("test.png", None, rustautogui::MatchMode::FFT) .unwrap(); // change prepare template settings, like region, matchmode or max segments // automatically move mouse to found template position, execute movement for 1 second gui.find_image_on_screen_and_move_mouse(0.9, 1.0).unwrap(); //move mouse to position (in this case to bottom left of stanard 1920x1080 monitor), move the mouse for 1 second gui.move_mouse_to_pos(1920, 1080, 1.0).unwrap(); // execute left click, move the mouse to x (500), y (500) position for 1 second, release mouse click // used for actions like moving icons or files // suggestion: dont use very small moving time values, especially on mac gui.drag_mouse(500, 500, 1.0).unwrap(); // execute mouse left click gui.left_click().unwrap(); // execute mouse right click gui.right_click().unwrap(); // execute mouse middle click gui.middle_click().unwrap(); // execute a double (left) mouse click gui.double_click().unwrap(); // mouse scrolls gui.scroll_down(1).unwrap(); gui.scroll_up(5).unwrap(); gui.scroll_right(8).unwrap(); gui.scroll_left(10).unwrap(); // input keyboard string gui.keyboard_input("test.com").unwrap(); // press a keyboard command, in this case enter(return) gui.keyboard_command("return").unwrap(); // two key press gui.keyboard_multi_key("alt", "tab", None).unwrap(); // three key press gui.keyboard_multi_key("shift", "control", Some("e")) .unwrap(); // maybe you would want to loop search until image is found and break the loop then loop { let pos = gui.find_image_on_screen_and_move_mouse(0.9, 1.0).unwrap(); match pos { Some(_) => break, None => (), } } } } ``` -------------------------------- ### Get Keymap Entry for Linux Source: https://docs.rs/rustautogui/latest/src/rustautogui/core/keyboard/mod.rs.html Retrieves a keymap entry for a given key string on Linux. Returns an error if the key is not found in the keymap. ```rust #[cfg(target_os = "linux")] fn get_keymap_key<'a>(target: &'a Keyboard, key: &str) -> Result<&'a (String, bool), AutoGuiError> { let values = target .keymap .get(key) .ok_or(AutoGuiError::UnSupportedKey(format!( "{} key/command is not supported", key )))?; Ok(values) } ``` -------------------------------- ### RustAutoGui Initialization Source: https://docs.rs/rustautogui/latest/src/rustautogui/lib.rs.html Initializes the RustAutoGui struct, setting up screen, keyboard, and mouse functionalities. It also handles OpenCL initialization if the feature is enabled and checks for environment variables to suppress warnings. ```APIDOC ## new ### Description Initiation of screen, keyboard and mouse that are assigned to new rustautogui struct. All the other struct fields are initiated as 0 or None. ### Signature ```rust pub fn new(debug: bool) -> Result ``` ### Parameters - `debug` (bool) - Flag to enable debug mode. ### Returns - `Result` - Returns a `RustAutoGui` instance on success or an `AutoGuiError` on failure. ``` -------------------------------- ### Get Mouse Position (Rust) Source: https://docs.rs/rustautogui/latest/src/rustautogui/rustautogui_impl/mouse_impl.rs.html Retrieves the current mouse cursor coordinates (x, y). This function uses platform-specific implementations for Linux, Windows, and macOS. ```rust pub fn get_mouse_position(&self) -> Result<(i32, i32), AutoGuiError> { #[cfg(target_os = "linux")] return self.mouse.get_mouse_position(); #[cfg(target_os = "windows")] return Ok(Mouse::get_mouse_position()); #[cfg(target_os = "macos")] return Mouse::get_mouse_position(); } ``` -------------------------------- ### Prepare Template from Image Buffer with RustAutoGui Source: https://docs.rs/rustautogui/latest/rustautogui/struct.RustAutoGui.html Prepares a template for screen searching using an existing image buffer. This method allows for more flexibility by loading the image using `imgtools::load_image_rgba` first. ```rust // or another way to prepare template { let img = imgtools::load_image_rgba("test.png").unwrap(); // just loading this way for example gui.prepare_template_from_imagebuffer( img, Some((0, 0, 700, 500)), rustautogui::MatchMode::Segmented, ) .unwrap(); } ``` -------------------------------- ### Get Keymap Entry for Windows/macOS Source: https://docs.rs/rustautogui/latest/src/rustautogui/core/keyboard/mod.rs.html Retrieves a keymap entry for a given key string on Windows or macOS. Returns an error if the key is not found in the keymap. ```rust #[cfg(any(target_os = "macos", target_os = "windows"))] fn get_keymap_key<'a>(target: &'a Keyboard, key: &str) -> Result<&'a (u16, bool), AutoGuiError> { let values = target .keymap .get(key) .ok_or(AutoGuiError::UnSupportedKey(format!( "{} key/command is not supported", key )))?; Ok(values) } ``` -------------------------------- ### Initialize Keyboard Instance Source: https://docs.rs/rustautogui/latest/src/rustautogui/core/keyboard/linux/mod.rs.html Creates a new `Keyboard` instance, initializing the keymap based on the detected keyboard layout. The display object is required for X11 interactions. ```rust pub fn new(screen: *mut _XDisplay) -> Self { // for future development let is_us_layout: bool = Self::is_us_layout(); let keymap = Keyboard::create_keymap(is_us_layout); Self { keymap: keymap, screen: screen, } } ``` -------------------------------- ### Get TypeId of a static type Source: https://docs.rs/rustautogui/latest/rustautogui/errors/enum.ImageProcessingError.html Implements the Any trait's type_id method, which retrieves the TypeId of a statically known type. This is a blanket implementation for any type T. ```rust fn type_id(&self) -> TypeId ```