### Install nut.js with npm Source: https://nutjs.dev/docs/installation Install the core nut.js package using npm. Ensure Node.js 22.x or later is installed. ```bash npm install @nut-tree/nut-js ``` -------------------------------- ### Install Azure OCR Plugin Source: https://nutjs.dev/docs/search/text Installs the `@nut-tree/plugin-azure` package for cloud-based OCR using Azure AI Vision. ```bash npm install @nut-tree/plugin-azure ``` -------------------------------- ### Start Screen Recording Source: https://nutjs.dev/docs/api/nut-js/v5.1.1/nut-tree-nut-js/screenclass Starts recording the system's main display. Optional configuration options can be provided. ```typescript async startRecording(options?: RECORDING_OPTIONS): Promise ``` -------------------------------- ### Verify nut.js Installation Source: https://nutjs.dev/docs/installation Create and run a test script to verify that nut.js is installed correctly and can access mouse position. This script requires Node.js and tsx to be installed. ```typescript import { mouse } from "@nut-tree/nut-js"; const position = await mouse.getPosition(); console.log(`Mouse position: ${position.x}, ${position.y}`); ``` -------------------------------- ### startRecording Source: https://nutjs.dev/docs/api/nut-js/v5.1.1/nut-tree-provider-interfaces/screenrecordingproviderinterface Starts screen recording. Optionally accepts recording options. This method returns a Promise that resolves when the recording has started. ```APIDOC ## startRecording ### Description Starts screen recording. ### Method `startRecording(options?: RECORDING_OPTIONS): Promise` ### Parameters #### Path Parameters - **options** (RECORDING_OPTIONS) - Optional - Recording options. ### Returns `Promise` ``` -------------------------------- ### Start Screen Streaming Source: https://nutjs.dev/docs/api/nut-js/v5.1.1/nut-tree-nut-js/screenclass Starts streaming the system's main display. Optional configuration options can be provided. ```typescript async startStreaming(options?: STREAMING_OPTIONS): Promise ``` -------------------------------- ### Verify npm Private Registry Setup Source: https://nutjs.dev/docs Run this command to verify your npm setup for the private registry. It should return the ID you created for your access token. ```shell npm whoami --registry=https://pkg.nutjs.dev ``` -------------------------------- ### Start Monitoring All Events Source: https://nutjs.dev/docs/api/nut-js/v5.1.1/nut-tree-nut-js/systemclass Starts monitoring all system events. ```typescript System.startMonitoringAllEvents() ``` -------------------------------- ### Start Monitoring Input Events Source: https://nutjs.dev/docs/api/nut-js/v5.1.1/nut-tree-nut-js/systemclass Starts monitoring input events. ```typescript System.startMonitoringInputEvents() ``` -------------------------------- ### Install Local OCR Plugin Source: https://nutjs.dev/docs/search/text Installs the `@nut-tree/plugin-ocr` package, which uses Tesseract for local OCR processing. ```bash npm install @nut-tree/plugin-ocr ``` -------------------------------- ### startStreaming Source: https://nutjs.dev/docs/api/nut-js/v5.1.1/nut-tree-provider-interfaces/screenrecordingproviderinterface Starts screen streaming. Optionally accepts streaming options. This method returns a Promise that resolves when the streaming has started. ```APIDOC ## startStreaming ### Description Starts screen streaming. ### Method `startStreaming(options?: STREAMING_OPTIONS): Promise` ### Parameters #### Path Parameters - **options** (STREAMING_OPTIONS) - Optional - Streaming options. ### Returns `Promise` ``` -------------------------------- ### Centralized Configuration Setup for nut.js Source: https://nutjs.dev/docs/configuration Configure all nut.js modules (keyboard, mouse, screen) once at the start of your script or test setup. This ensures consistent behavior for all subsequent automation operations. ```typescript import { keyboard, mouse, screen } from "@nut-tree/nut-js"; // -- Setup -- keyboard.config.autoDelayMs = 50; mouse.config.mouseSpeed = 1500; mouse.config.autoDelayMs = 50; screen.config.resourceDirectory = "./images"; screen.config.confidence = 0.9; screen.config.autoHighlight = true; // -- Automation -- // All operations below use the config above await screen.find(imageResource("target.png")); await keyboard.type("Hello!"); ``` -------------------------------- ### Start Monitoring Window Events Source: https://nutjs.dev/docs/api/nut-js/v5.1.1/nut-tree-nut-js/systemclass Starts monitoring window events. ```typescript System.startMonitoringWindowEvents() ``` -------------------------------- ### Install Element Inspector Plugin Source: https://nutjs.dev/docs/search/element Install the necessary plugin for element search functionality. This command is run in your terminal. ```bash npm install @nut-tree/element-inspector ``` -------------------------------- ### Setup Element Inspector and Import Platform Elements Source: https://nutjs.dev/docs/search/element Activate the element inspector plugin and import platform-specific element factories. This setup is required before performing element searches. ```typescript import { getActiveWindow } from "@nut-tree/nut-js"; import { useElementInspector } from "@nut-tree/element-inspector"; useElementInspector(); // For macOS: import { elements } from "@nut-tree/element-inspector/macos"; // For Windows: // import { elements } from "@nut-tree/element-inspector/win"; // For Linux: // import { elements } from "@nut-tree/element-inspector/linux"; const win = await getActiveWindow(); ``` -------------------------------- ### Import and Use NL Matcher Plugin Source: https://nutjs.dev/docs/search/image Before using image search functionalities, ensure the `@nut-tree/nl-matcher` plugin is installed and imported. This setup is required for all image-related operations. ```typescript import { screen, imageResource } from "@nut-tree/nut-js"; import { useNlMatcher } from "@nut-tree/nl-matcher"; useNlMatcher(); ``` -------------------------------- ### Configure Nut.js in Test Framework Setup Source: https://nutjs.dev/docs/configuration Set global Nut.js configurations in a test framework's setup phase, such as Playwright's `beforeAll`. This ensures consistent settings for all tests. ```typescript import { test } from "@playwright/test"; import { keyboard, mouse, screen } from "@nut-tree/nut-js"; test.beforeAll(() => { keyboard.config.autoDelayMs = 50; mouse.config.autoDelayMs = 50; mouse.config.mouseSpeed = 1500; screen.config.resourceDirectory = "./test-images"; screen.config.confidence = 0.9; }); ``` -------------------------------- ### Start Recording Method Source: https://nutjs.dev/docs/api/nut-js/v5.1.1/nut-tree-provider-interfaces/screenrecordingproviderinterface Implements the startRecording method to initiate screen recording. It accepts optional recording options and returns a Promise that resolves upon completion. ```typescript startRecording(options?: RECORDING_OPTIONS): Promise ``` -------------------------------- ### Start Monitoring Accessibility Events Source: https://nutjs.dev/docs/api/nut-js/v5.1.1/nut-tree-nut-js/systemclass Starts monitoring accessibility events. ```typescript System.startMonitoringAccessibilityEvents() ``` -------------------------------- ### Install Template Matcher for Image Matching Source: https://nutjs.dev/docs/migrations To enable image matching functionality in nut.js v2.0.0 and later, you must install a separate provider package like @nut-tree/template-matcher. ```bash npm install @nut-tree/template-matcher ``` -------------------------------- ### Automate Form Filling with Mouse and Keyboard Source: https://nutjs.dev/docs/mouse Demonstrates a practical example of automating form filling by moving the mouse, clicking, and typing using Nut.js. Requires importing mouse, keyboard, Button, Key, Point, and straightTo from '@nut-tree/nut-js'. ```typescript import { mouse, keyboard, Button, Key, Point, straightTo } from "@nut-tree/nut-js"; // Click the username field await mouse.move(straightTo(new Point(400, 200))); await mouse.click(Button.LEFT); await keyboard.type("john.doe@example.com"); // Tab to password field await keyboard.pressKey(Key.Tab); await keyboard.type("secretpassword"); // Click submit button await mouse.move(straightTo(new Point(400, 350))); await mouse.click(Button.LEFT); ``` -------------------------------- ### Execute Custom Mouse Movements with Sine Wave Source: https://nutjs.dev/docs/mouse Demonstrates how to use the `alongSineWaveTo` function to move the mouse. Shows examples with default and customized wave parameters, and integration with image search for dynamic targets. Requires importing mouse and Point from '@nut-tree/nut-js'. ```typescript // Move to a specific point along a sine wave await mouse.move(alongSineWaveTo(new Point(500, 300))); // Customize the wave parameters await mouse.move(alongSineWaveTo( new Point(800, 400), 30, // amplitude - height of the wave 6, // frequency - number of wave cycles 500 // numPoints - smoothness of the path )); // Combine with image search for dynamic targets await mouse.move( alongSineWaveTo( centerOf(screen.find(imageResource("button.png"))) ) ); ``` -------------------------------- ### Find Menu Item by Title and Click Source: https://nutjs.dev/docs/search/element Locate a menu item by its accessible title and then interact with it. This example finds the 'File' menu item and clicks it. ```typescript import { getActiveWindow, mouse, centerOf, straightTo, Button as MouseButton } from "@nut-tree/nut-js"; import { useElementInspector } from "@nut-tree/element-inspector"; useElementInspector(); import { elements } from "@nut-tree/element-inspector/macos"; const win = await getActiveWindow(); // Find menu item by title const fileMenu = await win.find(elements.menuItem({ title: "File" })); // Click on it using the element's region await mouse.move(straightTo(centerOf(fileMenu.region))); await mouse.click(MouseButton.LEFT); ``` -------------------------------- ### startMonitoringAccessibilityEvents Source: https://nutjs.dev/docs/api/element-inspector/v1.1.1/lib/accessibilitymonitor Initiates the process of monitoring accessibility events. Ensure an emitter is connected before starting. ```APIDOC ## startMonitoringAccessibilityEvents ### Description Starts monitoring accessibility events. ### Method `startMonitoringAccessibilityEvents(): void` ``` -------------------------------- ### Start Streaming Method Source: https://nutjs.dev/docs/api/nut-js/v5.1.1/nut-tree-provider-interfaces/screenrecordingproviderinterface Implements the startStreaming method to begin screen streaming. It accepts optional streaming options and returns a Promise that resolves upon completion. ```typescript startStreaming(options?: STREAMING_OPTIONS): Promise ``` -------------------------------- ### Start Accessibility Event Monitoring Source: https://nutjs.dev/docs/api/element-inspector/v1.1.1/lib/accessibilitymonitor Initiates the process of monitoring accessibility events for all connected and monitored windows. This should be called after setting up the emitter and any windows to monitor. ```typescript startMonitoringAccessibilityEvents(): void ``` -------------------------------- ### Get All Windows Source: https://nutjs.dev/docs/api/nut-js/v5.1.1/nut-tree-nut-js/getwindows Retrieves a promise that resolves to an array of all open windows. This function requires no arguments. ```typescript const getWindows: () => Promise ``` -------------------------------- ### ScreenClass Methods Source: https://nutjs.dev/docs/api/nut-js/v5.1.1/nut-tree-nut-js/screenclass Provides access to methods for interacting with the system's main display, including capturing screenshots, getting pixel colors, managing recordings, finding elements, and retrieving screen dimensions. ```APIDOC ## ScreenClass `ScreenClass` class provides methods to access screen content of a systems main display. ### capture `capture(fileName: string, fileFormat?: FileType, filePath?: string, fileNamePrefix?: string, fileNamePostfix?: string): Promise` `capture` captures a screenshot of a systems main display. #### Parameters - `fileName` (string) - Basename for the generated screenshot - `fileFormat` (FileType) - Optional - The `FileType` for the generated screenshot - `filePath` (string) - Optional - The output path for the generated screenshot (Default: `cwd`) - `fileNamePrefix` (string) - Optional - Filename prefix for the generated screenshot (Default: empty) - `fileNamePostfix` (string) - Optional - Filename postfix for the generated screenshot (Default: empty) #### Returns `Promise` ### colorAt `colorAt(point: Point | Promise): Promise` `colorAt` returns RGBA color values for a certain pixel at `Point` p. #### Parameters - `point` (Point | Promise) - Location to query color information from #### Returns `Promise` ### discardRecording `discardRecording(): Promise` `discardRecording` discards the current screen recording without saving. #### Returns `Promise` ### find `find(searchInput: RegionResultFindInput | Promise, params?: OptionalSearchParameters): Promise` `find` will search for a single occurrence of a given search input on a systems main screen. #### Parameters - `searchInput` (RegionResultFindInput | Promise) - A `FindInput` instance - `params` (OptionalSearchParameters) - Optional - `OptionalSearchParameters` which are used to fine tune search region and / or match confidence #### Returns `Promise` ### findAll `findAll(searchInput: RegionResultFindInput | Promise, params?: OptionalSearchParameters): Promise` `findAll` will search for every occurrence of a given search input on a systems main screen. #### Parameters - `searchInput` (RegionResultFindInput | Promise) - A `FindInput` instance to search for - `params` (OptionalSearchParameters) - Optional - `OptionalSearchParameters` which are used to fine tune search region and / or match confidence #### Returns `Promise` ### grab `grab(): Promise` `grab` grabs screen content of a systems main display. #### Returns `Promise` ### height `height(): Promise` `height` returns the main screen height. This refers to the hardware resolution. Screens with higher pixel density (e.g. retina displays in MacBooks) might have a higher height in in actual pixels. #### Returns `Promise` ### highlight `highlight(regionToHighlight: Region | Promise): Promise` `highlight` highlights a screen `Region` for a certain duration by overlaying it with an opaque highlight window. #### Parameters - `regionToHighlight` (Region | Promise) - The `Region` to highlight #### Returns `Promise` ### on `on(event: E, listener: unknown[unknown]): this` `on` registers a callback which is triggered once a certain searchInput image is found. #### Parameters - `event` (E) - The event to listen for. Currently supported events are `WindowEvents` - `listener` (unknown[unknown]) - #### Returns `this` ### startRecording `startRecording(options?: RECORDING_OPTIONS): Promise` `startRecording` starts screen recording. #### Parameters - `options` (RECORDING_OPTIONS) - Optional #### Returns `Promise` ### startStreaming `startStreaming(options?: STREAMING_OPTIONS): Promise` `startStreaming` starts screen streaming. #### Parameters - `options` (STREAMING_OPTIONS) - Optional #### Returns `Promise` ### stopRecording `stopRecording(options?: STOP_RECORDING_OPTIONS): Promise` `stopRecording` stops screen recording. #### Parameters - `options` (STOP_RECORDING_OPTIONS) - Optional #### Returns `Promise` ### stopStreaming `stopStreaming(options?: STOP_STREAMING_OPTIONS): Promise` `stopStreaming` stops screen streaming. #### Parameters - `options` (STOP_STREAMING_OPTIONS) - Optional #### Returns `Promise` ### waitFor `waitFor(descriptor: SettledDescriptor): Promise` `waitFor` waits until window events settle globally (no events fire for the configured quiet period). Attach the listener before the triggering action: `const p = screen.waitFor(settled()); await launchApp(); await p;`. #### Parameters - `descriptor` (SettledDescriptor) - A `SettledDescriptor` created via `settled` #### Returns `Promise` ### width `width(): Promise` `width` returns the main screen width. This refers to the hardware resolution. Screens with higher pixel density (e.g. retina displays in MacBooks) might have a higher width in in actual pixels. #### Returns `Promise` ``` -------------------------------- ### Click on a Found Image Source: https://nutjs.dev/docs/search/image After finding an image using `screen.find()`, you can interact with it, such as clicking. This example demonstrates moving the mouse to the center of the found image and performing a left-click. ```typescript import { screen, mouse, imageResource, centerOf, straightTo, Button } from "@nut-tree/nut-js"; import { useNlMatcher } from "@nut-tree/nl-matcher"; useNlMatcher(); // Find and click a button const button = await screen.find(imageResource("submit-button.png")); await mouse.move(straightTo(centerOf(button))); await mouse.click(Button.LEFT); // Or as a one-liner await mouse.move(straightTo(centerOf(await screen.find(imageResource("submit-button.png"))))); await mouse.click(Button.LEFT); ``` -------------------------------- ### Import Image Matching Provider Source: https://nutjs.dev/docs/migrations After installing an image matching provider package, import it into your TypeScript code to register it with nut.js. ```typescript import { screen } from "@nut-tree/nut-js"; import "@nut-tree/template-matcher"; ``` -------------------------------- ### Open Application via Spotlight (macOS) Source: https://nutjs.dev/docs/keyboard Automate opening applications on macOS by triggering Spotlight, typing the application name, and pressing Enter. Includes a short delay to allow Spotlight to appear. ```typescript import { keyboard, Key } from "@nut-tree/nut-js"; // Open Spotlight await keyboard.type(Key.LeftSuper, Key.Space); // Wait for Spotlight to appear await new Promise(r => setTimeout(r, 500)); // Type application name await keyboard.type("calculator"); // Press Enter to open await keyboard.type(Key.Enter); ``` -------------------------------- ### Get Windows Source: https://nutjs.dev/docs/window Retrieve the currently focused window or a list of all open windows. Useful for identifying and interacting with specific application windows. ```APIDOC ## getActiveWindow ### Description Get the currently focused window. ### Method `getActiveWindow()` ### Returns Promise ## getWindows ### Description Get all open windows. ### Method `getWindows()` ### Returns Promise ``` -------------------------------- ### configure Source: https://nutjs.dev/docs/api/plugin-azure/v2.1.1/api Configures the Azure plugin with the provided settings. This is the primary function for initializing the plugin. ```APIDOC ## configure ### Description Configures the Azure plugin with the provided settings. This is the primary function for initializing the plugin. ### Function Signature function configure(configuration: AzurePluginConfiguration) ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body * **configuration** (AzurePluginConfiguration) - Required - The configuration object for the Azure plugin. ``` -------------------------------- ### Get Window Elements Source: https://nutjs.dev/docs/api/nut-js/v5.1.1/nut-tree-shared/windowinterface Retrieves window elements, optionally limiting the maximum number of elements returned. Use this to get a list of all or a subset of elements in the window. ```typescript getElements(maxElements?: number): Promise ``` -------------------------------- ### Basic Automation Script with nut.js Source: https://nutjs.dev/docs/getting-started This script demonstrates moving the mouse to the center of the screen and typing text using nut.js. Ensure necessary imports are included. ```typescript import { mouse, keyboard, screen, straightTo, Point } from "@nut-tree/nut-js"; // Move mouse to center of screen const screenWidth = await screen.width(); const screenHeight = await screen.height(); await mouse.move(straightTo(new Point(screenWidth / 2, screenHeight / 2))); // Type some text await keyboard.type("Hello from nut.js!"); ``` -------------------------------- ### pixelWithColor Source: https://nutjs.dev/docs/api/nut-js/v5.1.1/nut-tree-nut-js/pixelwithcolor Gets information about a pixel with a specific RGBA color. ```APIDOC ## pixelWithColor ### Description Gets information about a pixel with a specific RGBA color. ### Method `pixelWithColor` ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body None ### Parameters - **color** (RGBA) - Required - The RGBA color to search for. ### Returns `object` - Information about the pixel. ### Request Example ```typescript const pixelInfo = await pixelWithColor([255, 0, 0, 255]); ``` ### Response #### Success Response (200) - **object** - Information about the pixel. #### Response Example ```json { "x": 100, "y": 200, "color": [255, 0, 0, 255] } ``` ``` -------------------------------- ### Configure Keyboard, Mouse, and Screen Modules Source: https://nutjs.dev/docs/configuration Set configuration properties directly on the respective module's config object. Changes take effect immediately. ```typescript import { keyboard, mouse, screen } from "@nut-tree/nut-js"; // Configure each module directly keyboard.config.autoDelayMs = 50; mouse.config.mouseSpeed = 1500; screen.config.confidence = 0.9; ``` -------------------------------- ### Get Log Provider Source: https://nutjs.dev/docs/api/nut-js/v5.1.1/nut-tree-provider-interfaces/providerregistry Retrieves the registered LogProviderInterface. Use this for logging. ```typescript getLogProvider(): LogProviderInterface ``` -------------------------------- ### Get Window Finder Source: https://nutjs.dev/docs/api/nut-js/v5.1.1/nut-tree-provider-interfaces/providerregistry Retrieves the registered WindowFinderInterface. Use this to find windows. ```typescript getWindowFinder(): WindowFinderInterface ``` -------------------------------- ### useDefaultWindowProvider Source: https://nutjs.dev/docs/api/nut-js/v5.1.1/nut-tree-nut-js/usedefaultwindowprovider Sets up the default window provider for Nut.js. This function should be called once at the beginning of your script to enable window management functionalities. ```APIDOC ## useDefaultWindowProvider ### Description Initializes the default window provider for the Nut.js library. This is essential for enabling features that interact with application windows, such as finding, manipulating, or capturing them. ### Method ```typescript useDefaultWindowProvider(): void ``` ### Parameters This function does not accept any parameters. ### Returns This function does not return any value (`void`). ### Usage Example ```typescript import { useDefaultWindowProvider } from "nut-js"; // Call this function once at the start of your script useDefaultWindowProvider(); // Now you can use other Nut.js window-related functions // For example: // const windows = await Window.all(); // console.log(windows); ``` ``` -------------------------------- ### click Source: https://nutjs.dev/docs/api/nut-js/v5.1.1/nut-tree-provider-interfaces/keyboardproviderinterface Simulates a single key press and release action at the OS level. ```APIDOC ## click ### Description Allows to press a single key via OS level keyboard event. ### Method `click(keys: Key[]): Promise` ### Parameters #### Path Parameters - **keys** (Key[]) - Required - Array of `Key`s to click ### Returns `Promise` ``` -------------------------------- ### Get Screen Provider Source: https://nutjs.dev/docs/api/nut-js/v5.1.1/nut-tree-provider-interfaces/providerregistry Retrieves the registered ScreenProviderInterface. Use this to interact with the screen. ```typescript getScreen(): ScreenProviderInterface ``` -------------------------------- ### getWindows Function Source: https://nutjs.dev/docs/api/nut-js/v5.1.1/nut-tree-nut-js Retrieves a list of all open OS windows. ```APIDOC ## Function: getWindows ### Description Retrieves a list of all open OS windows. ### Parameters (No parameters documented in the source text) ### Returns (No return type documented in the source text) ### Example (No example provided in the source text) ``` -------------------------------- ### Get Image Processor Source: https://nutjs.dev/docs/api/nut-js/v5.1.1/nut-tree-provider-interfaces/providerregistry Retrieves the registered ImageProcessor. Use this to process images. ```typescript getImageProcessor(): ImageProcessor ``` -------------------------------- ### application Source: https://nutjs.dev/docs/api/element-inspector/v1.1.1/macos/elements Creates a WindowElementQuery for finding Application elements. Accepts a partial description to match elements like title or value. ```APIDOC ## application ### Description Creates a WindowElementQuery for finding Application elements. ### Method N/A (SDK Method) ### Endpoint N/A (SDK Method) ### Parameters #### Path Parameters N/A #### Query Parameters N/A #### Request Body - **description** (Omit) - Required - Partial description to match elements (title, value, etc.) ### Request Example N/A ### Response #### Success Response (200) - **WindowElementQuery** - An object for further element matching and interaction. #### Response Example N/A ``` -------------------------------- ### Get Window Provider Source: https://nutjs.dev/docs/api/nut-js/v5.1.1/nut-tree-provider-interfaces/providerregistry Retrieves the registered WindowProviderInterface. Use this to interact with application windows. ```typescript getWindow(): WindowProviderInterface ``` -------------------------------- ### Get Vision Finder Source: https://nutjs.dev/docs/api/nut-js/v5.1.1/nut-tree-provider-interfaces/providerregistry Retrieves the registered VisionFinderInterface. Use this for advanced vision-based finding. ```typescript getVisionFinder(): VisionFinderInterface ``` -------------------------------- ### useBoltWindows() Source: https://nutjs.dev/docs/api/bolt/v3.0.0/nut-tree-bolt/useboltwindows Initializes or enables window functionalities within the Bolt framework. This is a void function and does not return any value. ```APIDOC ## useBoltWindows() ### Description Initializes or enables window functionalities within the Bolt framework. This is a void function and does not return any value. ### Signature useBoltWindows(): void ``` -------------------------------- ### Get Text Finder Source: https://nutjs.dev/docs/api/nut-js/v5.1.1/nut-tree-provider-interfaces/providerregistry Retrieves the registered TextFinderInterface. Use this to find text on the screen. ```typescript getTextFinder(): TextFinderInterface ``` -------------------------------- ### Get Mouse Provider Source: https://nutjs.dev/docs/api/nut-js/v5.1.1/nut-tree-provider-interfaces/providerregistry Retrieves the registered MouseProviderInterface. Use this to simulate mouse input. ```typescript getMouse(): MouseProviderInterface ``` -------------------------------- ### matte Source: https://nutjs.dev/docs/api/element-inspector/v1.1.1/macos/elements Creates a WindowElementQuery for finding Matte elements. Use this to find 'Matte' UI components within the application window. ```APIDOC ## matte ### Description Creates a WindowElementQuery for finding Matte elements. ### Parameters #### Path Parameters * **description** (Omit) - Required - Partial description to match elements (title, value, etc.) ### Returns * **WindowElementQuery** ``` -------------------------------- ### Get Keyboard Provider Source: https://nutjs.dev/docs/api/nut-js/v5.1.1/nut-tree-provider-interfaces/providerregistry Retrieves the registered KeyboardProviderInterface. Use this to simulate keyboard input. ```typescript getKeyboard(): KeyboardProviderInterface ``` -------------------------------- ### Get Input Monitor Source: https://nutjs.dev/docs/api/nut-js/v5.1.1/nut-tree-provider-interfaces/providerregistry Retrieves the registered InputMonitor. Use this to monitor input actions. ```typescript getInputMonitor(): InputMonitor ``` -------------------------------- ### useDefaultClipboardProvider Source: https://nutjs.dev/docs/api/nut-js/v5.1.1/nut-tree-nut-js/usedefaultclipboardprovider Sets up the default clipboard provider for Nut.js. This function is typically called once during the initialization of your Nut.js environment. ```APIDOC ## useDefaultClipboardProvider ### Description Sets up the default clipboard provider for Nut.js. This function is typically called once during the initialization of your Nut.js environment. ### Signature useDefaultClipboardProvider(): void ### Parameters This function does not accept any parameters. ### Returns void ``` -------------------------------- ### Get Image Writer Source: https://nutjs.dev/docs/api/nut-js/v5.1.1/nut-tree-provider-interfaces/providerregistry Retrieves the registered ImageWriter. Use this to write image data. ```typescript getImageWriter(): ImageWriter ``` -------------------------------- ### useOcrPlugin Source: https://nutjs.dev/docs/api/plugin-ocr/v4.0.2/nut-tree-plugin-ocr Initializes and returns the OCR plugin instance, making its functionalities available for use. ```APIDOC ## useOcrPlugin ### Description Initializes the OCR plugin and returns an instance that can be used to access OCR functionalities. ### Function Signature `useOcrPlugin(): OcrPlugin` ### Returns - **OcrPlugin** - An instance of the OCR plugin. ``` -------------------------------- ### Get Image Reader Source: https://nutjs.dev/docs/api/nut-js/v5.1.1/nut-tree-provider-interfaces/providerregistry Retrieves the registered ImageReader. Use this to read image data. ```typescript getImageReader(): ImageReader ``` -------------------------------- ### Get Image Finder Source: https://nutjs.dev/docs/api/nut-js/v5.1.1/nut-tree-provider-interfaces/providerregistry Retrieves the registered ImageFinderInterface. Use this to find images on the screen. ```typescript getImageFinder(): ImageFinderInterface ``` -------------------------------- ### useSelenium Source: https://nutjs.dev/docs/api/selenium-bridge/v2.0.0/nut-tree-selenium-bridge/useselenium Initializes the Selenium Bridge with configuration options, including a Selenium WebDriver instance. ```APIDOC ## useSelenium ### Description Initializes the Selenium Bridge with configuration options, including a Selenium WebDriver instance. ### Signature useSelenium(options: SeleniumBridgeConfigOptions): void ### Parameters #### `options` (`SeleniumBridgeConfigOptions`) - **Description**: Configuration containing the Selenium WebDriver instance. - **Required**: true ``` -------------------------------- ### appBar Source: https://nutjs.dev/docs/api/element-inspector/v1.1.1/win/elements Creates a WindowElementQuery for finding AppBar elements. Allows partial description matching. ```APIDOC ## appBar ### Description Creates a WindowElementQuery for finding AppBar elements. Allows partial description matching. ### Method appBar ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body * **description** (Omit) - Partial description to match elements (title, value, etc.). ### Request Example ```json { "description": { "title": "My App Bar" } } ``` ### Response #### Success Response (200) * **WindowElementQuery** - An object representing the query for AppBar elements. #### Response Example ```json { "query": "appBar(description: { title: \"My App Bar\" })" } ``` ``` -------------------------------- ### Get Color Finder Source: https://nutjs.dev/docs/api/nut-js/v5.1.1/nut-tree-provider-interfaces/providerregistry Retrieves the registered ColorFinderInterface. Use this to find colors on the screen. ```typescript getColorFinder(): ColorFinderInterface ``` -------------------------------- ### Get Clipboard Provider Source: https://nutjs.dev/docs/api/nut-js/v5.1.1/nut-tree-provider-interfaces/providerregistry Retrieves the registered ClipboardProviderInterface. Use this to interact with the system clipboard. ```typescript getClipboard(): ClipboardProviderInterface ``` -------------------------------- ### type Source: https://nutjs.dev/docs/api/nut-js/v5.1.1/nut-tree-provider-interfaces/keyboardproviderinterface Simulates typing a string of text using OS level keyboard events. ```APIDOC ## type ### Description Allows to type a given text via OS level keyboard events. ### Method `type(input: string): Promise` ### Parameters #### Path Parameters - **input** (string) - Required - The text to type ### Returns `Promise` ``` -------------------------------- ### Get Screen Size Source: https://nutjs.dev/docs/api/plugin-rdp/v1.0.1/nut-tree-plugin-rdp/rdpclient Retrieves the dimensions (width and height) of the RDP session's screen. ```typescript getScreenSize(): ScreenSize ``` -------------------------------- ### connect Source: https://nutjs.dev/docs/api/plugin-rdp/v1.0.1/nut-tree-plugin-rdp Establishes an RDP connection using the provided connection parameters. ```APIDOC ## connect ### Description Establishes an RDP connection. ### Method connect ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body - **ConnectionParameters** (object) - Required - Parameters required to establish an RDP connection. ### Request Example ```json { "ConnectionParameters": { "host": "your_rdp_host", "port": 3389, "username": "your_username", "password": "your_password" } } ``` ### Response #### Success Response (200) - **RdpClient** (object) - An object representing the established RDP client connection. #### Response Example ```json { "rdpClient": "{ ... RDP Client Object ... }" } ``` ``` -------------------------------- ### Get Mouse Position Source: https://nutjs.dev/docs/api/plugin-rdp/v1.0.1/nut-tree-plugin-rdp/rdpclient Retrieves the current position of the mouse cursor within the RDP session. ```typescript getMousePosition(): Point ``` -------------------------------- ### inputMethodWindow Source: https://nutjs.dev/docs/api/element-inspector/v1.1.1/linux/elements Creates a WindowElementQuery for finding InputMethodWindow elements. Allows partial description matching. ```APIDOC ## inputMethodWindow ### Description Creates a WindowElementQuery for finding InputMethodWindow elements. ### Parameters #### Query Parameters - **description** (Omit) - Required - Partial description to match elements (title, value, etc.) ### Returns `WindowElementQuery` ``` -------------------------------- ### systemWide Source: https://nutjs.dev/docs/api/element-inspector/v1.1.1/macos/elements Creates a WindowElementQuery for finding SystemWide elements. This function allows you to specify partial descriptions to match elements based on their properties. ```APIDOC ## systemWide ### Description Creates a WindowElementQuery for finding SystemWide elements. ### Method Not applicable (SDK function) ### Endpoint Not applicable (SDK function) ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body None ### Parameters #### description `Omit` - Partial description to match elements (title, value, etc.). This parameter is optional. ### Returns `WindowElementQuery` - An object that can be used to further query and interact with the found SystemWide elements. ``` -------------------------------- ### Get Window Event Monitor Source: https://nutjs.dev/docs/api/nut-js/v5.1.1/nut-tree-provider-interfaces/providerregistry Retrieves the registered WindowEventMonitor. Use this to monitor window events. ```typescript getWindowEventMonitor(): WindowEventMonitor ``` -------------------------------- ### Get Window Element Inspector Source: https://nutjs.dev/docs/api/nut-js/v5.1.1/nut-tree-provider-interfaces/providerregistry Retrieves the registered ElementInspectionProviderInterface. Use this to inspect window elements. ```typescript getWindowElementInspector(): ElementInspectionProviderInterface ``` -------------------------------- ### connect Source: https://nutjs.dev/docs/api/plugin-rdp/v1.0.1/nut-tree-plugin-rdp/connect Establishes an RDP connection using provided parameters and returns an object with methods to manage the connection and remote control. ```APIDOC ## connect ### Description Establishes an RDP connection using provided parameters and returns an object with methods to manage the connection and remote control. ### Method ```typescript connect(connectionParams: ConnectionParameters): Promise<{ disconnect: () => void; isConnected: () => boolean; useRemote: () => void; useRemoteClipboard: () => void; useRemoteKeyboard: () => void; useRemoteMouse: () => void; useRemoteScreen: () => void }> ``` ### Parameters #### Path Parameters - None #### Query Parameters - None #### Request Body - **connectionParams** (ConnectionParameters) - Required - The connection parameters (host, port, credentials, etc.) ### Response #### Success Response - **disconnect**: () => void - A function to disconnect the RDP session. - **isConnected**: () => boolean - A function that returns true if the RDP session is connected, false otherwise. - **useRemote**: () => void - A function to enable general remote control. - **useRemoteClipboard**: () => void - A function to enable remote clipboard synchronization. - **useRemoteKeyboard**: () => void - A function to enable remote keyboard input. - **useRemoteMouse**: () => void - A function to enable remote mouse input. - **useRemoteScreen**: () => void - A function to enable remote screen sharing. ### Request Example ```typescript // Example usage (assuming ConnectionParameters is defined elsewhere) const connection = await connect({ host: 'your_rdp_host', port: 3389, username: 'your_username', password: 'your_password' }); if (connection.isConnected()) { console.log('Connected to RDP server'); connection.useRemoteKeyboard(); } // To disconnect later: // connection.disconnect(); ``` ### Response Example ```json { "disconnect": "[function]", "isConnected": "[function]", "useRemote": "[function]", "useRemoteClipboard": "[function]", "useRemoteKeyboard": "[function]", "useRemoteMouse": "[function]", "useRemoteScreen": "[function]" } ``` ``` -------------------------------- ### getWindows Source: https://nutjs.dev/docs/api/nut-js/v5.1.1/nut-tree-provider-interfaces/windowproviderinterface Returns a list of window handles for further processing. ```APIDOC ## getWindows ### Description Returns a list of window handles for further processing. These window handles may serve as input to e.g. `getWindowTitle`. ### Method `getWindows(): Promise` ### Returns - **Promise** - A promise that resolves to an array of window handles. ``` -------------------------------- ### Get Screen Recording Provider Source: https://nutjs.dev/docs/api/nut-js/v5.1.1/nut-tree-provider-interfaces/providerregistry Retrieves the registered ScreenRecordingProviderInterface. Use this for screen recording functionality. ```typescript getScreenRecordingProvider(): ScreenRecordingProviderInterface ``` -------------------------------- ### Window Class Methods Source: https://nutjs.dev/docs/api/nut-js/v5.1.1/nut-tree-nut-js/window This section details the methods available on the Window class for interacting with OS windows. ```APIDOC ## find ### Description Searches for a single occurrence of a given search input in the current window. ### Method `find(searchInput: object | Promise): Promise` ### Parameters #### Parameters - **searchInput** (object | Promise) - Required - A `WindowedFindInput` instance ### Returns `Promise` ``` ```APIDOC ## findAll ### Description Searches for multiple occurrences of a given search input in the current window. ### Method `findAll(searchInput: object | Promise): Promise` ### Parameters #### Parameters - **searchInput** (object | Promise) - Required - A `WindowedFindInput` instance ### Returns `Promise` ``` ```APIDOC ## focus ### Description Brings the window to the foreground and gives it focus. ### Method `focus(): Promise` ### Returns `Promise` ``` ```APIDOC ## getElements ### Description Retrieves the UI element tree of the window. ### Method `getElements(maxElements?: number): Promise` ### Parameters #### Parameters - **maxElements** (number) - Optional - Maximum number of elements to retrieve ### Returns `Promise` ``` ```APIDOC ## getRegion ### Description Returns the `Region` occupied by the window, clamped to screen boundaries. ### Method `getRegion(): Promise` ### Returns `Promise` ``` ```APIDOC ## getTitle ### Description Returns the title of the window. ### Method `getTitle(): Promise` ### Returns `Promise` ``` ```APIDOC ## minimize ### Description Minimizes the window. ### Method `minimize(): Promise` ### Returns `Promise` ``` ```APIDOC ## move ### Description Moves the window to a new origin `Point`. ### Method `move(newOrigin: Point): Promise` ### Parameters #### Parameters - **newOrigin** (Point) - Required - The target `Point` for the window's new position ### Returns `Promise` ``` ```APIDOC ## off ### Method `off(event: E, listener: unknown[unknown]): this` ### Parameters #### Parameters - **event** (E) - Required - **listener** (unknown[unknown]) - Required ### Returns `this` ``` ```APIDOC ## on ### Description Registers either: An event listener for accessibility events (from `TypedEmitter`) or a find-hook callback triggered when a `WindowElementQuery` is matched. ### Method `on(event: E, listener: unknown[unknown]): this` ### Parameters #### Parameters - **event** (E) - Required - **listener** (unknown[unknown]) - Required ### Returns `this` ``` ```APIDOC ## once ### Method `once(event: E, listener: unknown[unknown]): this` ### Parameters #### Parameters - **event** (E) - Required - **listener** (unknown[unknown]) - Required ### Returns `this` ``` ```APIDOC ## removeAllListeners ### Method `removeAllListeners(event?: E): this` ### Parameters #### Parameters - **event** (E) - Optional ### Returns `this` ``` ```APIDOC ## removeListener ### Method `removeListener(event: E, listener: unknown[unknown]): this` ### Parameters #### Parameters - **event** (E) - Required - **listener** (unknown[unknown]) - Required ### Returns `this` ``` ```APIDOC ## resize ### Description Resizes the window to a new `Size`. ### Method `resize(newSize: Size): Promise` ### Parameters #### Parameters - **newSize** (Size) - Required - The target `Size` for the window ### Returns `Promise` ``` ```APIDOC ## restore ### Description Restores a minimized window. ### Method `restore(): Promise` ### Returns `Promise` ``` ```APIDOC ## waitFor ### Description Waits until accessibility events for this window settle (no events fire for the configured quiet period). Attach the listener before the triggering action: ```ts const p = window.waitFor(settled()); await button.click(); await p; ``` ### Method `waitFor(descriptor: SettledDescriptor): Promise` ### Parameters #### Parameters - **descriptor** (SettledDescriptor) - Required - A `SettledDescriptor` created via `settled` ### Returns `Promise` ``` -------------------------------- ### Get Accessibility Event Monitor Source: https://nutjs.dev/docs/api/nut-js/v5.1.1/nut-tree-provider-interfaces/providerregistry Retrieves the registered AccessibilityEventMonitor. Use this to monitor accessibility events. ```typescript getAccessibilityEventMonitor(): AccessibilityEventMonitor ``` -------------------------------- ### Get Window Title Source: https://nutjs.dev/docs/api/nut-js/v5.1.1/nut-tree-nut-js/window Returns the title of the window. This is useful for identifying or verifying specific windows. ```typescript await window.getTitle(); ``` -------------------------------- ### grabScreen Source: https://nutjs.dev/docs/api/nut-js/v5.1.1/nut-tree-provider-interfaces/screenproviderinterface Captures a screenshot of the system's main screen and returns it as an Image object, including its dimensions. ```APIDOC ## grabScreen ### Description Captures a screenshot of the system's main screen and returns it as an Image object, including its dimensions. ### Method `grabScreen(): Promise` ### Returns `Promise` - An Image object containing the screenshot data and dimensions. ``` -------------------------------- ### handle Source: https://nutjs.dev/docs/api/element-inspector/v1.1.1/macos/elements Creates a WindowElementQuery for finding Handle elements. Accepts a partial description to match elements like title or value. ```APIDOC ## handle ### Description Creates a WindowElementQuery for finding Handle elements. ### Method N/A (SDK Method) ### Endpoint N/A (SDK Method) ### Parameters #### Path Parameters N/A #### Query Parameters N/A #### Request Body - **description** (Omit) - Required - Partial description to match elements (title, value, etc.) ### Request Example N/A ### Response #### Success Response (200) - **WindowElementQuery** - An object for further element matching and interaction. #### Response Example N/A ``` -------------------------------- ### elementRegion Source: https://nutjs.dev/docs/api/selenium-bridge/v2.0.0/nut-tree-selenium-bridge/elementregion Gets the bounding region of a WebElement. This is useful for interacting with elements based on their position and size on the page. ```APIDOC ## elementRegion ### Description Gets the bounding region of a WebElement. This is useful for interacting with elements based on their position and size on the page. ### Signature elementRegion(element: WebElement): Promise ### Parameters #### Path Parameters - **element** (WebElement) - Required - The WebElement whose bounds to determine. ### Returns `Promise` - An object representing the region of the element, typically containing properties like x, y, width, and height. ``` -------------------------------- ### Get Clipboard Content Source: https://nutjs.dev/docs/api/plugin-rdp/v1.0.1/nut-tree-plugin-rdp/rdpclient Retrieves the current text content from the RDP session's clipboard. This is an asynchronous operation. ```typescript getClipboard(): Promise ``` -------------------------------- ### useDefaultWindowProvider Function Source: https://nutjs.dev/docs/api/nut-js/v5.1.1/nut-tree-nut-js Sets the default window provider. ```APIDOC ## Function: useDefaultWindowProvider ### Description Sets the default window provider. ### Parameters (No parameters documented in the source text) ### Returns (No return type documented in the source text) ### Example (No example provided in the source text) ``` -------------------------------- ### useContext Function Source: https://nutjs.dev/docs/api/playwright-bridge/v2.0.0/nut-tree-playwright-bridge A utility function to get the Playwright BrowserContext. This is useful for advanced configurations and interactions with the browser context. ```APIDOC ## useContext ### Description Provides access to the Playwright BrowserContext. ### Usage ```javascript const context = await useContext(); ``` ``` -------------------------------- ### Point Class Source: https://nutjs.dev/docs/api/nut-js/v5.1.1/nut-tree-shared/point Represents a point in 2D space with x and y coordinates. Includes a method to get a string representation. ```APIDOC ## Class: Point ### Description Represents a point in 2D space with `x` and `y` coordinates. ### Properties - **x** (`number`) - The x-coordinate of the point. - **y** (`number`) - The y-coordinate of the point. ### Methods #### toString `toString(): string` ##### Description Converts the point object to its string representation. ##### Returns - `string` - The string representation of the point. ``` -------------------------------- ### menu Source: https://nutjs.dev/docs/api/element-inspector/v1.1.1/win/elements Creates a WindowElementQuery for finding Menu elements. You can pass a partial description to match elements by properties like title or value. ```APIDOC ## menu ### Description Creates a WindowElementQuery for finding Menu elements. ### Method `menu(description: Omit): WindowElementQuery` ### Parameters #### description `Omit` - Partial description to match elements (title, value, etc.) ### Returns `WindowElementQuery` ``` -------------------------------- ### useDefaultKeyboardProvider Source: https://nutjs.dev/docs/api/nut-js/v5.1.1/nut-tree-nut-js/usedefaultkeyboardprovider Initializes the default keyboard provider. This function should be called to enable keyboard interactions within your Nut.js script. ```APIDOC ## useDefaultKeyboardProvider ### Description Initializes the default keyboard provider. This function should be called to enable keyboard interactions within your Nut.js script. ### Method `useDefaultKeyboardProvider()` ### Parameters This function does not take any parameters. ### Returns `void` ### Example ```typescript import { useDefaultKeyboardProvider } from "nut-js"; // Call this function to set up the default keyboard provider useDefaultKeyboardProvider(); // Now you can use keyboard functions like type, pressKey, etc. // await type(Key.A); ``` ``` -------------------------------- ### Get Window Event Emitter Source: https://nutjs.dev/docs/api/nut-js/v5.1.1/nut-tree-provider-interfaces/providerregistry Retrieves the registered window event emitter. Use this to listen for window-related events. ```typescript getWindowEventEmitter(): TypedEmitter ``` -------------------------------- ### Get Input Event Emitter Source: https://nutjs.dev/docs/api/nut-js/v5.1.1/nut-tree-provider-interfaces/providerregistry Retrieves the registered input event emitter. Use this to listen for input events. ```typescript getInputEventEmitter(): TypedEmitter ``` -------------------------------- ### window Source: https://nutjs.dev/docs/api/element-inspector/v1.1.1/win/elements Creates a WindowElementQuery for finding Window elements. You can pass a partial description to match elements by properties like title or value. ```APIDOC ## window ### Description Creates a WindowElementQuery for finding Window elements. ### Method `window(description: Omit): WindowElementQuery` ### Parameters #### description `Omit` - Partial description to match elements (title, value, etc.) ### Returns `WindowElementQuery` ``` -------------------------------- ### Get Accessibility Event Emitter Source: https://nutjs.dev/docs/api/nut-js/v5.1.1/nut-tree-provider-interfaces/providerregistry Retrieves the registered accessibility event bus. Use this to access accessibility-related events. ```typescript getAccessibilityEventEmitter(): AccessibilityEventBus ``` -------------------------------- ### Get Clipboard Content Source: https://nutjs.dev/docs/api/nut-js/v5.1.1/nut-tree-nut-js/clipboardclass Retrieves the current text content from the system clipboard. This method is asynchronous and returns a Promise. ```typescript getContent(): Promise ``` -------------------------------- ### useBoltWindows Source: https://nutjs.dev/docs/api/bolt/v3.0.0/nut-tree-bolt Manages and interacts with multiple windows in the Bolt system. ```APIDOC ## useBoltWindows ### Description Manages and interacts with multiple windows in the Bolt system. ### Method function ### Endpoint N/A ### Parameters N/A ### Request Example N/A ### Response N/A ``` -------------------------------- ### descriptionList Source: https://nutjs.dev/docs/api/element-inspector/v1.1.1/linux/elements Creates a WindowElementQuery for finding DescriptionList elements. Use this to query UI elements structured as description lists. ```APIDOC ## descriptionList ### Description Creates a `WindowElementQuery` for finding `DescriptionList` elements. Use this to query UI elements structured as description lists. ### Method `descriptionList(description: Omit): WindowElementQuery` ### Parameters #### Parameters - **description** (`Omit`) - Required - Partial description to match elements (title, value, etc.) ### Returns - `WindowElementQuery` ``` -------------------------------- ### useBoltKeyboard Function Source: https://nutjs.dev/docs/api/bolt/v3.0.0/nut-tree-bolt/useboltkeyboard Initializes and manages keyboard event listeners for Bolt applications. This function is typically called once during application setup. ```APIDOC ## useBoltKeyboard ### Description Initializes and manages keyboard event listeners for Bolt applications. This function is typically called once during application setup. ### Method function ### Signature useBoltKeyboard(): void ### Parameters This function does not accept any parameters. ### Returns This function does not return any value. ```