### HSTB.kluster.fqpr_project Module Source: https://github.com/noaa-ocs-hydrography/kluster/blob/master/HSTB/kluster/docbuild/index.html Details the functions and classes within the HSTB.kluster.fqpr_project module, which manages FQPR project settings. ```APIDOC ## HSTB.kluster.fqpr_project Module ### Description This module handles the management of FQPR projects, including settings and related operations. ### Functions * [Functions]: Lists the available functions for project management. ### Classes * [Classes]: Describes the classes used for FQPR project management. ``` -------------------------------- ### Run Kluster GUI Source: https://github.com/noaa-ocs-hydrography/kluster/blob/master/HSTB/kluster/docbuild/_sources/index.rst.txt Starts the Kluster graphical user interface (GUI) application. This command should be executed after activating the 'kluster_test' environment and installing the Kluster package. ```bash python -m HSTB.kluster ``` -------------------------------- ### HSTB.kluster.fqpr_generation Module Source: https://github.com/noaa-ocs-hydrography/kluster/blob/master/HSTB/kluster/docbuild/index.html Details the functions and classes within the HSTB.kluster.fqpr_generation module, responsible for creating FQPR datasets. ```APIDOC ## HSTB.kluster.fqpr_generation Module ### Description This module is responsible for the generation and creation of Fully Qualified Ping Records (FQPR) datasets. ### Functions * [Functions]: Details various functions available for generating FQPR data. ### Classes * [Classes]: Describes the classes used in the FQPR generation process. ``` -------------------------------- ### Install Kluster using Pip Source: https://github.com/noaa-ocs-hydrography/kluster/blob/master/HSTB/kluster/docbuild/_sources/index.rst.txt Installs the Kluster package from its GitHub repository using pip. This command fetches the latest version and installs it into the active Python environment. The '#egg=hstb.kluster' part specifies the package name for installation. ```bash pip install git+https://github.com/noaa-ocs-hydrography/kluster.git#egg=hstb.kluster ``` -------------------------------- ### HSTB.kluster.fqpr_convenience Module Source: https://github.com/noaa-ocs-hydrography/kluster/blob/master/HSTB/kluster/docbuild/index.html Details the functions available in the HSTB.kluster.fqpr_convenience module, which offers simplified ways to interact with FQPR data. ```APIDOC ## HSTB.kluster.fqpr_convenience Module ### Description This module provides a set of convenient functions for common operations on Fully Qualified Ping Records (FQPR). ### Functions This module primarily exposes functions for simplifying FQPR data manipulation. Refer to the module documentation for specific function signatures and usage. ``` -------------------------------- ### Visualize Entwine Source: https://github.com/noaa-ocs-hydrography/kluster/blob/master/HSTB/kluster/docbuild/api/HSTB.kluster.pdal_entwine.visualize_entwine.html Starts an HTTP server to visualize entwine point tiles in the specified directory using potree. Requires NodeJS and http-server to be installed. ```APIDOC ## POST /visualize_entwine ### Description Starts an HTTP server and uses potree to visualize the entwine point tiles in the given entwine_dir. Requires nodejs/http-server to be installed. ### Method POST ### Endpoint /visualize_entwine ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body - **entwine_dir** (str) - Required - Folder path to the entwine point tile directory ### Request Example ```json { "entwine_dir": "/path/to/your/entwine/tiles" } ``` ### Response #### Success Response (200) - **message** (str) - Indicates that the server has started and provides the URL to access the visualization. #### Response Example ```json { "message": "HTTP server started. Access visualization at http://localhost:8080/" } ``` ``` -------------------------------- ### HSTB.kluster.monitor Module Source: https://github.com/noaa-ocs-hydrography/kluster/blob/master/HSTB/kluster/docbuild/index.html Details the functions and classes within the HSTB.kluster.monitor module, providing monitoring capabilities for Kluster processes. ```APIDOC ## HSTB.kluster.monitor Module ### Description This module offers functionalities for monitoring the status and progress of Kluster processes, aiding in performance tracking and diagnostics. ### Functions * [Functions]: Details functions for monitoring Kluster operations. ### Classes * [Classes]: Describes classes related to monitoring interfaces or data structures. ``` -------------------------------- ### Install Kluster using Pip Source: https://github.com/noaa-ocs-hydrography/kluster/blob/master/README.md This pip command installs the Kluster package directly from its GitHub repository. It installs the 'hstb.kluster' module and expects it to be prefixed with 'hstb.kluster' in the installation string. ```bash pip install git+https://github.com/noaa-ocs-hydrography/kluster.git#egg=hstb.kluster ``` -------------------------------- ### HSTB.kluster.modules.orientation Module Source: https://github.com/noaa-ocs-hydrography/kluster/blob/master/HSTB/kluster/docbuild/index.html Details the functions within the HSTB.kluster.modules.orientation module, which handles orientation-related calculations. ```APIDOC ## HSTB.kluster.modules.orientation Module ### Description This module contains functions specifically designed for orientation calculations, likely related to sensor or vessel orientation in hydrographic surveys. ### Functions * [Functions]: Details the available orientation calculation functions. ``` -------------------------------- ### Instantiate and Start DirectoryMonitor in Python Source: https://github.com/noaa-ocs-hydrography/kluster/blob/master/HSTB/kluster/docbuild/api/HSTB.kluster.monitor.DirectoryMonitor.html This snippet demonstrates how to create an instance of the DirectoryMonitor class, specifying the directory path and whether to monitor recursively. It then starts the monitoring process. ```python from HSTB.kluster.monitor import DirectoryMonitor dm = DirectoryMonitor(r'C:\data_dirtj_patch_test', is_recursive=True) dm.start() ``` -------------------------------- ### HSTB.kluster.fqpr_actions Module Source: https://github.com/noaa-ocs-hydrography/kluster/blob/master/HSTB/kluster/docbuild/index.html Details the functions and classes within the HSTB.kluster.fqpr_actions module, defining operations that can be performed on FQPR data. ```APIDOC ## HSTB.kluster.fqpr_actions Module ### Description This module defines various actions that can be applied to FQPR datasets, facilitating data manipulation and processing workflows. ### Functions * [Functions]: Outlines the available actions as functions. ### Classes * [Classes]: Describes classes that might represent or encapsulate these actions. ``` -------------------------------- ### HSTB.kluster.fqpr_vessel Module Source: https://github.com/noaa-ocs-hydrography/kluster/blob/master/HSTB/kluster/docbuild/index.html Details the functions and classes within the HSTB.kluster.fqpr_vessel module, managing vessel-related information for FQPR data. ```APIDOC ## HSTB.kluster.fqpr_vessel Module ### Description This module is concerned with vessel information and its integration with FQPR data, crucial for accurate hydrographic surveys. ### Functions * [Functions]: Lists functions related to handling vessel data. ### Classes * [Classes]: Describes classes that represent vessel attributes or configurations. ``` -------------------------------- ### Kluster API Modules Source: https://github.com/noaa-ocs-hydrography/kluster/blob/master/HSTB/kluster/docbuild/index.html This section details the various modules available within the Kluster API. Each module contains specific functions and classes for hydrographic data processing. ```APIDOC ## Kluster API Modules Overview ### Description Provides access to various functionalities within the Kluster hydrographic data processing software. ### Modules * **fqpr_convenience**: Contains convenience functions for working with FQPR data. * **fqpr_generation**: Handles the generation of FQPR datasets. * **fqpr_project**: Manages FQPR project settings and operations. * **fqpr_intelligence**: Integrates intelligent processing capabilities for FQPR data. * **fqpr_actions**: Defines actions that can be performed on FQPR data. * **fqpr_vessel**: Deals with vessel information related to FQPR data. * **monitor**: Provides monitoring functionalities for Kluster processes. * **xarray_conversion**: Handles conversion of data to and from xarray format. * **orientation**: Module for orientation-related calculations. * **beampointingvector**: Module for handling beam pointing vectors. ``` -------------------------------- ### HSTB.kluster.fqpr_intelligence Module Source: https://github.com/noaa-ocs-hydrography/kluster/blob/master/HSTB/kluster/docbuild/index.html Details the functions and classes within the HSTB.kluster.fqpr_intelligence module, focusing on intelligent processing of FQPR data. ```APIDOC ## HSTB.kluster.fqpr_intelligence Module ### Description This module integrates intelligent processing capabilities for FQPR data, potentially including analysis and advanced features. ### Functions * [Functions]: Lists the functions related to intelligent processing. ### Classes * [Classes]: Describes the classes involved in intelligent data analysis. ``` -------------------------------- ### HSTB.kluster.xarray_conversion Module Source: https://github.com/noaa-ocs-hydrography/kluster/blob/master/HSTB/kluster/docbuild/index.html Details the functions and classes within the HSTB.kluster.xarray_conversion module, focused on converting data to and from the xarray format. ```APIDOC ## HSTB.kluster.xarray_conversion Module ### Description This module provides utilities for converting Kluster data structures to and from the popular xarray data format, enabling seamless integration with other scientific Python libraries. ### Functions * [Functions]: Lists functions for performing data conversions. ### Classes * [Classes]: Describes classes involved in the conversion process. ### Class Inheritance Diagram * [Class Inheritance Diagram]: Visual representation of the class hierarchy within this module. ``` -------------------------------- ### Process and Load Multibeam Data with Kluster Source: https://github.com/noaa-ocs-hydrography/kluster/blob/master/notebooks/example_export.ipynb Processes multibeam data from a specified file using `intel_process` and then selects the first dataset if multiple are returned. Alternatively, it reloads existing data using `reload_data` from a directory. ```python # we start with one of the preferred processing steps from the data_processing example _, fq = intel_process(r"C:\data_dir\0009_20170523_181119_FA2806.all") fq = fq[0] # this should just be a list of one element if you have just one sonar/day of data, take the first one to get the data ``` ```python # or we can just reload if you have data from before fq = reload_data(r"C:\data_dir\em2040_40111_05_23_2017") ``` -------------------------------- ### Start Python Interpreter Source: https://github.com/noaa-ocs-hydrography/kluster/blob/master/HSTB/kluster/docbuild/_sources/index.rst.txt Starts the Python interpreter within the active environment (likely inside a Docker container). This is usually done to interactively run Python commands or scripts. ```python python ``` -------------------------------- ### dask_find_or_start_client Source: https://github.com/noaa-ocs-hydrography/kluster/blob/master/HSTB/kluster/docbuild/api/HSTB.kluster.dask_helpers.dask_find_or_start_client.html This function either starts a new Dask client or returns an existing one, facilitating distributed processing. It supports both local and networked cluster configurations. ```APIDOC ## dask_find_or_start_client ### Description Either start or return a Dask client in local/networked cluster mode. Use this function whenever you need access to a new or existing cluster. ### Method PYTHON FUNCTION (Conceptual) ### Endpoint N/A (Python function) ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body None ### Request Example ```python from HSTB.kluster.dask_helpers import dask_find_or_start_client # Example: Start a local cluster with 4 workers and 8 threads per worker client = dask_find_or_start_client(number_of_workers=4, threads_per_worker=8) # Example: Connect to an existing Dask scheduler at a specific address # client = dask_find_or_start_client(address='tcp://127.0.0.1:8786') ``` ### Response #### Success Response - **Client instance** (dask.distributed.client.Client) - An instance of a Dask client representing either a local or networked cluster. #### Response Example ```json { "client_instance": "" } ``` ### Parameters * **address** (`Optional[str]`) – IP address:port for an existing or desired new Dask server instance. Will accept anything that `dask.distributed.get_client` accepts. * **number_of_workers** (`Optional[int]`) – Integer number of workers in the LocalCluster, only used when `address` is None (local cluster). * **threads_per_worker** (`Optional[int]`) – Integer number of threads per worker in the LocalCluster, only used when `address` is None (local cluster). * **memory_per_worker** (`Optional[str]`) – String representation of the memory allowed per worker, e.g., ‘10GB’. * **multiprocessing** (`bool`) – If True, will allow multiple workers; if False, will only use threads. * **silent** (`bool`) – Whether or not to print messages. * **logger** (`Optional[Logger]`) – If included, will print output messages to the provided logger. ``` -------------------------------- ### Understanding Kluster Grid Tiles and Data Source: https://github.com/noaa-ocs-hydrography/kluster/blob/master/notebooks/example_gridding.ipynb This snippet delves into the tile-based structure of Kluster grids. It demonstrates how to access grid tile information, including the tile size and the collection of tiles. For a specific tile, it shows how to access the raw data points and the gridded cell data. The example also illustrates computing the gridded depth data for a specific resolution. ```Python # Inspecting tile information print(bs.tile_size) print(bs.tiles) # Accessing data within a tile tile = bs.tiles[0][0] # Assuming first tile in the first row print(tile.data) # Raw points print(tile.cells) # Gridded cells # Computing gridded depth data for a specific resolution print(tile.cells[8.0]['depth'].compute()) ``` -------------------------------- ### Import Kluster Data Processing Modules Source: https://github.com/noaa-ocs-hydrography/kluster/blob/master/notebooks/example_export.ipynb Imports necessary modules for data processing and reloading within the Kluster library. These are typically used at the beginning of a script that interacts with Kluster data. ```python from HSTB.kluster.fqpr_convenience import reload_data from HSTB.kluster.fqpr_intelligence import intel_process ``` -------------------------------- ### HSTB.kluster.modules.beampointingvector Module Source: https://github.com/noaa-ocs-hydrography/kluster/blob/master/HSTB/kluster/docbuild/index.html Details the functions within the HSTB.kluster.modules.beampointingvector module, used for handling beam pointing vectors. ```APIDOC ## HSTB.kluster.modules.beampointingvector Module ### Description This module provides functionalities for managing and processing beam pointing vectors, which are essential for determining the spatial coverage of sonar data. ### Functions * [Functions]: Lists the functions related to beam pointing vectors. ``` -------------------------------- ### Open Project Source: https://github.com/noaa-ocs-hydrography/kluster/blob/master/HSTB/kluster/docbuild/api/HSTB.kluster.fqpr_project.FqprProject.html Opens an existing Kluster project from a project file. Optionally skips Dask client autostart. ```APIDOC ## POST /open_project ### Description Opens a Kluster project from a specified project file. A Dask client is typically required for conversion and processing tasks. ### Method POST ### Endpoint /open_project ### Parameters #### Request Body - **projfile** (str) - Required - Path to the project file (.proj). - **skip_dask** (bool) - Optional - If True, does not automatically start a Dask client. Defaults to False. ### Request Example ```json { "projfile": "/path/to/my_project.proj", "skip_dask": false } ``` ### Response #### Success Response (200) - **message** (str) - Confirmation message indicating the project has been opened. #### Response Example ```json { "message": "Project opened successfully." } ``` ``` -------------------------------- ### HSTB.kluster.fqpr_project.open_project Source: https://github.com/noaa-ocs-hydrography/kluster/blob/master/HSTB/kluster/docbuild/api/HSTB.kluster.fqpr_project.open_project.html Loads an existing fqpr_project file from a specified path. ```APIDOC ## open_project ### Description Loads from a saved fqpr_project file. ### Method POST (Assumed, as it modifies or retrieves project state) ### Endpoint /api/kluster/fqpr_project/open ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body - **project_path** (str) - Required - The file path to the saved FqprProject json file. ### Request Example ```json { "project_path": "/path/to/your/project.json" } ``` ### Response #### Success Response (200) - **FqprProject** (object) - An instance of the FqprProject class initialized from the loaded json file. #### Response Example ```json { "project_data": { ... } // Represents the loaded project data } ``` ``` -------------------------------- ### Create New Project Source: https://github.com/noaa-ocs-hydrography/kluster/blob/master/HSTB/kluster/docbuild/api/HSTB.kluster.fqpr_project.create_new_project.html Creates a new FqprProject by converting multibeam files, instantiating a new Fqpr, and loading it into the project. ```APIDOC ## create_new_project ### Description Create a new FqprProject by taking in multibeam files, converting them, making a new Fqpr instance and loading that Fqpr into a new FqprProject. ### Method POST (Assumed, as it creates a new resource) ### Endpoint /api/projects ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body - **output_folder** (Optional[str]) - An optional path to an output folder. If not provided, conversion will occur next to the mbes_files. ### Request Example ```json { "output_folder": "/path/to/output" } ``` ### Response #### Success Response (200) - **project_instance** (FqprProject) - The newly created project instance with one new Fqpr instance loaded. #### Response Example ```json { "project_instance": { "project_name": "new_project", "fqpr_instances": [ { "fqpr_id": "fqpr_1" } ] } } ``` ``` -------------------------------- ### Kluster API - orientation Module Source: https://github.com/noaa-ocs-hydrography/kluster/blob/master/HSTB/kluster/docbuild/api/HSTB.kluster.fqpr_actions.FqprAction.html API documentation for the orientation module. ```APIDOC ## HSTB.kluster.orientation Module ### Description Provides functionalities related to data orientation and transformations. ``` -------------------------------- ### New Project Initialization Source: https://github.com/noaa-ocs-hydrography/kluster/blob/master/HSTB/kluster/docbuild/api/HSTB.kluster.fqpr_project.FqprProject.html Initializes a new hydrographic project from a specified directory. The directory can be empty for a new project or contain existing converted data. ```APIDOC ## POST /new_project_from_directory ### Description Creates a new Kluster project in the specified directory. If the directory is empty, an empty project is created. If it contains data, it initializes a project with that data. ### Method POST ### Endpoint /new_project_from_directory ### Parameters #### Request Body - **directory_path** (str) - Required - Path to the directory for the new project. ### Request Example ```json { "directory_path": "/path/to/new_project_directory" } ``` ### Response #### Success Response (200) - **message** (str) - Confirmation message that the project was created. #### Response Example ```json { "message": "Project created successfully at /path/to/new_project_directory" } ``` ``` -------------------------------- ### Create and Manage Kluster Project Source: https://context7.com/noaa-ocs-hydrography/kluster/llms.txt Organizes multiple Fqpr instances and surface grids within a structured project directory. Allows adding and accessing different data objects. Saves project configuration to a JSON file. Dependencies include the FqprProject class. Inputs are the project path and paths to Fqpr data or surface grids. Automatically loads existing projects. ```python from HSTB.kluster.fqpr_project import FqprProject # Create new project in directory proj = FqprProject(project_path=r"C:\survey_project") # Add Fqpr instances to project for converted_path in [r"C:\data\em2040_40111_05_23_2017", r"C:\data\em2040_40111_05_24_2017"]: proj.add_fqpr(converted_path) # Access Fqpr instances print(proj.fqpr_instances) # Output: {'em2040_40111_05_23_2017': , 'em2040_40111_05_24_2017': } # Add surface grids to project proj.add_surface(r"C:\data\mygridpath") # Access surfaces print(proj.surface_instances) # Output: {'mygridpath': } # Save project (creates kluster_project.json in project directory) proj.save_project() # Open existing project proj = FqprProject(project_path=r"C:\survey_project") # Automatically loads kluster_project.json and references to Fqpr/surfaces ``` -------------------------------- ### Kluster API - fqpr_convenience Module Source: https://github.com/noaa-ocs-hydrography/kluster/blob/master/HSTB/kluster/docbuild/api/HSTB.kluster.fqpr_actions.FqprAction.html API documentation for the fqpr_convenience module. ```APIDOC ## HSTB.kluster.fqpr_convenience Module ### Description This module offers convenience functions for working with fQPR data, simplifying common operations. ``` -------------------------------- ### get_overlapping_timestamps Source: https://github.com/noaa-ocs-hydrography/kluster/blob/master/HSTB/kluster/docbuild/api/HSTB.kluster.fqpr_vessel.get_overlapping_timestamps.html Finds timestamps within a given start and end time from a list of timestamps. The first timestamp returned is the closest to the start time without exceeding it. ```APIDOC ## GET /kluster/fqpr_vessel/get_overlapping_timestamps ### Description Finds the timestamps in the provided list of timestamps that fall between starttime/endtime. Returns these timestamps as a list. The first timestamp in the list is always the nearest to the starttime without going over. ### Method GET ### Endpoint /kluster/fqpr_vessel/get_overlapping_timestamps ### Parameters #### Query Parameters - **timestamps** (list) - Required - A list of timestamps to filter. - **starttime** (int) - Required - An integer UTC timestamp in seconds representing the start of the range. - **endtime** (int) - Required - An integer UTC timestamp in seconds representing the end of the range. ### Request Example (No explicit request body for GET requests, parameters are typically passed as query parameters) ### Response #### Success Response (200) - **timestamps** (list) - A list of timestamps (integers in seconds) that fall within the specified starttime and endtime range. The first element is the nearest timestamp to starttime without going over. #### Response Example ```json { "timestamps": [ 1678886400, 1678886460, 1678886520 ] } ``` ``` -------------------------------- ### Kluster API - fqpr_convenience Module Source: https://github.com/noaa-ocs-hydrography/kluster/blob/master/HSTB/kluster/docbuild/api/HSTB.kluster.modules.visualizations.FqprVisualizations.html Documentation for the fqpr_convenience module within the Kluster API. ```APIDOC ## fqpr_convenience Module ### Description This module provides convenience functions for working with FPQR objects. ### Endpoint N/A (This is a module, not a REST endpoint) ### Parameters N/A ### Request Example N/A ### Response N/A ``` -------------------------------- ### return_xyzrph_from_mbes Function Source: https://github.com/noaa-ocs-hydrography/kluster/blob/master/HSTB/kluster/docbuild/api/HSTB.kluster.xarray_conversion.return_xyzrph_from_mbes.html Converts the first installation record from a multibeam file into the xyzrph format used by Kluster. This is particularly useful for loading Vessel View with initial installation parameters. ```APIDOC ## HSTB.kluster.xarray_conversion.return_xyzrph_from_mbes ### Description Converts the first installation record from a multibeam file into the xyzrph format used by Kluster. This function is currently used to load Vessel View with the first installation parameters. xyzrph is a dict of offsets/angles (e.g., ‘tx_x’) and values organized by the time of the installation record (e.g., SortedDict({‘1583306608’: ‘0.000’})). ### Method `return_xyzrph_from_mbes` ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body None ### Parameters * **mbesfil** (`str`) - Required - Path to a multibeam file. * **logger** (`Optional[Logger]`) - Optional - A logger object for logging text output to a specific file or handler. ### Request Example ```python # No direct request body for this function, parameters are passed as arguments # Example usage: # from HSTB.kluster.xarray_conversion import return_xyzrph_from_mbes # result = return_xyzrph_from_mbes('/path/to/your/multibeam.mb58') ``` ### Response #### Success Response (200) * _dict_ - Translated installation parameter record in the format used by Kluster. * _str_ - Sonar model number. * _int_ - Primary system serial number. #### Response Example ```json { "installation_parameters": { "1583306608": { "tx_x": "0.000", "tx_y": "0.000", "tx_z": "0.000", "tx_roll": "0.000", "tx_pitch": "0.000", "tx_heading": "0.000" } }, "sonar_model": "EM710", "serial_number": 12345 } ``` ``` -------------------------------- ### Kluster API - monitor Module Source: https://github.com/noaa-ocs-hydrography/kluster/blob/master/HSTB/kluster/docbuild/api/HSTB.kluster.fqpr_actions.FqprAction.html API documentation for the monitor module. ```APIDOC ## HSTB.kluster.monitor Module ### Description This module provides tools for monitoring Kluster processing jobs and data. ``` -------------------------------- ### Kluster API - orientation Module Source: https://github.com/noaa-ocs-hydrography/kluster/blob/master/HSTB/kluster/docbuild/api/HSTB.kluster.modules.visualizations.FqprVisualizations.html Documentation for the orientation module within the Kluster API. ```APIDOC ## orientation Module ### Description This module deals with orientation data and calculations. ### Endpoint N/A (This is a module, not a REST endpoint) ### Parameters N/A ### Request Example N/A ### Response N/A ``` -------------------------------- ### FQPR Project Module Functions and Classes Source: https://github.com/noaa-ocs-hydrography/kluster/blob/master/HSTB/kluster/docbuild/kluster_api/index.html Functions and classes for managing Kluster projects, including creation, opening, and reprocessing. ```APIDOC ## Functions and Classes in HSTB.kluster.fqpr_project ### Description This section details functionalities for creating, opening, and managing hydrographic projects within Kluster, including reprocessing existing FPQR data. ### Functions - **create_new_project**: Creates a new Kluster project. - **open_project**: Opens an existing Kluster project. - **reprocess_fqprs**: Reprocesses FPQR datasets within a project. - **return_project_data**: Retrieves data associated with a project. ### Classes - **FqprProject**: Represents and manages a Kluster project. ``` -------------------------------- ### Get Attributes from FQPR Source: https://github.com/noaa-ocs-hydrography/kluster/blob/master/HSTB/kluster/docbuild/kluster_api/fqpr_convenience.html Retrieves attributes from an FQPR instance. ```APIDOC ## GET /kluster/get_attributes_from_fqpr ### Description Takes in an FQPR instance and returns its attributes. ### Method GET ### Endpoint /kluster/get_attributes_from_fqpr ### Parameters #### Query Parameters - **fqpr_instance** (object) - Required - The FQPR instance to retrieve attributes from. ### Request Example ```json { "fqpr_instance": { "data_path": "path/to/your/fqpr_data" } } ``` ### Response #### Success Response (200) - **attributes** (object) - An object containing the attributes of the FQPR instance. #### Response Example ```json { "attributes": { "survey_date": "2023-10-27", "vessel_name": "R/V Explorer" } } ``` ``` -------------------------------- ### Get Vector Layers Source: https://github.com/noaa-ocs-hydrography/kluster/blob/master/HSTB/kluster/docbuild/api/HSTB.kluster.gdal_helpers.get_vector_layers.html Retrieves a list of all layer names from a GDAL-supported vector file. ```APIDOC ## GET /kluster/gdal_helpers/vector_layers ### Description Returns a list of all layer names in the given GDAL supported vector file. ### Method GET ### Endpoint /kluster/gdal_helpers/vector_layers ### Parameters #### Query Parameters - **vector_source** (string) - Required - Absolute path to the vector file ### Request Example ```json { "vector_source": "/path/to/your/vector/file.shp" } ``` ### Response #### Success Response (200) - **layers** (list) - A list of strings, where each string is a layer name. #### Response Example ```json { "layers": [ "layer1", "layer2" ] } ``` ``` -------------------------------- ### Kluster API - fqpr_convenience Module Source: https://github.com/noaa-ocs-hydrography/kluster/blob/master/HSTB/kluster/docbuild/api/HSTB.kluster.fqpr_actions.FqprActionContainer.html Documentation for the fqpr_convenience module, offering simplified access to common Kluster operations. ```APIDOC ## fqpr_convenience module ### Description Provides a set of convenient functions for common operations within the Kluster hydrographic processing workflow. This module aims to simplify user interactions with the Kluster API. ``` -------------------------------- ### GET /kluster/dask_helpers/get_number_of_workers Source: https://github.com/noaa-ocs-hydrography/kluster/blob/master/HSTB/kluster/docbuild/api/HSTB.kluster.dask_helpers.get_number_of_workers.html Retrieves the total number of workers from the Dask cluster using a provided client. ```APIDOC ## GET /kluster/dask_helpers/get_number_of_workers ### Description Retrieves the total number of workers from the Dask cluster. ### Method GET ### Endpoint /kluster/dask_helpers/get_number_of_workers ### Parameters #### Query Parameters - **client** (Client) - Required - The Dask client used to determine the number of workers. ### Response #### Success Response (200) - **total_workers** (int) - The total number of workers available in the Dask cluster. ``` -------------------------------- ### Run Kluster GUI Source: https://github.com/noaa-ocs-hydrography/kluster/blob/master/README.md This command launches the Kluster graphical user interface (GUI) by running Kluster as a Python module. It requires that the 'kluster_test' conda environment is activated. ```bash (kluster_test) C:>python -m HSTB.kluster ``` -------------------------------- ### HSTB.kluster.fqpr_vessel.carry_over_optional Source: https://github.com/noaa-ocs-hydrography/kluster/blob/master/HSTB/kluster/docbuild/api/HSTB.kluster.fqpr_vessel.carry_over_optional.html Populates optional and TPU parameters in new data with the latest existing entry from starting data. ```APIDOC ## HSTB.kluster.fqpr_vessel.carry_over_optional ### Description Populate the optional and tpu parameters in the new data with the latest existing entry. ### Method Python Function ### Endpoint N/A (Python Function) ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body - **starting_data** (dict) - Required - base dict to compare against - **new_data** (dict) - Required - new dict to compare with ### Request Example ```json { "starting_data": {}, "new_data": {} } ``` ### Response #### Success Response (dict) - **new_data** (dict) - The input new_data dictionary with tpu parameters potentially overridden. #### Response Example ```json { "example": "response body" } ``` ``` -------------------------------- ### Build Kluster Docker Image Source: https://github.com/noaa-ocs-hydrography/kluster/blob/master/HSTB/kluster/docbuild/_sources/index.rst.txt Builds a Docker image tagged as 'kluster/ubuntu' using the Dockerfile present in the current directory. This is used to containerize the Kluster application and its dependencies. ```bash docker build -t kluster/ubuntu . ``` -------------------------------- ### Get Attributes from Zarr Stores Source: https://github.com/noaa-ocs-hydrography/kluster/blob/master/HSTB/kluster/docbuild/kluster_api/fqpr_convenience.html Retrieves attributes from a list of directories containing FQPR generated Zarr stores. ```APIDOC ## GET /kluster/get_attributes_from_zarr_stores ### Description Takes in a list of paths to directories containing FQPR generated Zarr stores and returns their attributes. ### Method GET ### Endpoint /kluster/get_attributes_from_zarr_stores ### Parameters #### Query Parameters - **list_dir_paths** (array) - Required - A list of paths to directories containing Zarr stores. ### Request Example ```json { "list_dir_paths": [ "path/to/zarr_store_1", "path/to/zarr_store_2" ] } ``` ### Response #### Success Response (200) - **attributes** (object) - An object containing the combined attributes from the specified Zarr stores. #### Response Example ```json { "attributes": { "all_survey_dates": ["2023-10-27", "2023-10-28"], "common_vessel_name": "R/V Explorer" } } ``` ``` -------------------------------- ### GET /kluster/monitor/get_file_size Source: https://github.com/noaa-ocs-hydrography/kluster/blob/master/HSTB/kluster/docbuild/api/HSTB.kluster.monitor.get_file_size.html Retrieves the actual size of a file being written to disk, useful for monitoring file copying progress. ```APIDOC ## GET /kluster/monitor/get_file_size ### Description This endpoint returns the actual number of bytes written to a file that is currently being copied. Unlike standard file size checks which may report allocated size, this function is designed to accurately reflect the data transferred to disk. ### Method GET ### Endpoint /kluster/monitor/get_file_size ### Parameters #### Query Parameters - **filepath** (str) - Required - The full path to the file being monitored. ### Request Example ```json { "filepath": "/path/to/your/file.ext" } ``` ### Response #### Success Response (200) - **file_size** (int) - The size of the file in bytes as written to disk. #### Response Example ```json { "file_size": 1048576 } ``` ``` -------------------------------- ### Kluster API - visualizations Module Source: https://github.com/noaa-ocs-hydrography/kluster/blob/master/HSTB/kluster/docbuild/api/HSTB.kluster.fqpr_actions.FqprAction.html API documentation for the visualizations module. ```APIDOC ## HSTB.kluster.visualizations Module ### Description Provides functionalities for creating visualizations of hydrographic data. ``` -------------------------------- ### Get SV Files from Directory Source: https://github.com/noaa-ocs-hydrography/kluster/blob/master/HSTB/kluster/docbuild/api/HSTB.kluster.modules.svcorrect.get_sv_files_from_directory.html Retrieves a list of all files with supported extensions from a specified directory, with an option to search subdirectories. ```APIDOC ## GET /api/kluster/svcorrect/files ### Description Returns a list of all files that have an extension in the global variable supported_file_formats. Disable search_subdirs if you want to only search the given folder, not subfolders. ### Method GET ### Endpoint /api/kluster/svcorrect/files ### Parameters #### Query Parameters - **dir_path** (string) - Required - The path to the parent directory containing sv files. - **search_subdirs** (boolean) - Optional - If True searches all subfolders as well. Defaults to True. ### Response #### Success Response (200) - **files** (list) - A list of full file paths to all sv files with approved file extension. #### Response Example { "files": [ "/path/to/file1.sv", "/path/to/subdir/file2.sv" ] } ``` -------------------------------- ### Kluster API - fqpr_project Module Source: https://github.com/noaa-ocs-hydrography/kluster/blob/master/HSTB/kluster/docbuild/api/HSTB.kluster.fqpr_actions.FqprAction.html API documentation for the fqpr_project module. ```APIDOC ## HSTB.kluster.fqpr_project Module ### Description This module contains utilities for managing and interacting with Kluster projects. ``` -------------------------------- ### Player Class Source: https://github.com/noaa-ocs-hydrography/kluster/blob/master/HSTB/kluster/docbuild/kluster_api/visualizations.html A Matplotlib FuncAnimation player that allows control over animation playback, including start, stop, speed adjustment, and frame skipping. ```APIDOC ## Player ### Description Matplotlib FuncAnimation player that includes the ability to start/stop/speed up/slow down/skip frames. ### Method (Not specified, this is a class) ### Endpoint (Not applicable, this is a Python class) ### Parameters #### Path Parameters (None) #### Query Parameters (None) #### Request Body - **fig** (object) - Required - The Matplotlib figure to use for the player. - **func** (callable) - Required - The animation function. - **frames** (iterable, optional) - Frames to iterate over. - **init_func** (callable, optional) - Initialization function. ``` -------------------------------- ### HSTB.kluster.rotations.build_mounting_angle_mat Source: https://github.com/noaa-ocs-hydrography/kluster/blob/master/HSTB/kluster/docbuild/kluster_api/rotations.html Feeds build_rot_mat, difference being this takes in single floating point numbers for rpy as you get from a surveyed mount angle data point. ```APIDOC ## POST /kluster/rotations/build_mounting_angle_mat ### Description Builds a mounting angle rotation matrix from roll, pitch, and yaw values. ### Method POST ### Endpoint /kluster/rotations/build_mounting_angle_mat ### Parameters #### Request Body - **roll** (float) - Required - The roll angle. - **pitch** (float) - Required - The pitch angle. - **yaw** (float) - Required - The yaw angle. - **tstmp** (float) - Required - The timestamp. ### Request Example ```json { "roll": 0.5, "pitch": 1.2, "yaw": -0.3, "tstmp": 1678886400.0 } ``` ### Response #### Success Response (200) - **rotation_matrix** (list[list[float]]) - The generated rotation matrix. #### Response Example ```json { "rotation_matrix": [[1.0, 0.0, 0.0], [0.0, 0.98, -0.19], [0.0, 0.19, 0.98]] } ``` ``` -------------------------------- ### Kluster API Overview Source: https://github.com/noaa-ocs-hydrography/kluster/blob/master/HSTB/kluster/docbuild/api/HSTB.kluster.modules.sat.ExtinctionTest.html This section provides an overview of the Kluster API and lists the available modules for developers. ```APIDOC ## Kluster API This API provides functionalities for hydrographic data processing within the Kluster project. The following modules are available: * `fqpr_convenience` * `fqpr_generation` * `fqpr_project` * `fqpr_intelligence` * `fqpr_actions` * `fqpr_vessel` * `monitor` * `xarray_conversion` * `orientation` * `beampointingvector` * `svcorrect` * `georeference` * `tpu` * `backscatter` * `visualizations` * `sat` * `wobble` * `export` * `subset` * `rotations` * `pdal_entwine` * `dask_helpers` * `fqpr_helpers` * `gdal_helpers` * `numba_helpers` * `pydro_helpers` * `xarray_helpers` * `utc_helpers` * `logging_conf` Refer to the specific module documentation for detailed information on functions and classes. ``` -------------------------------- ### HSTB.kluster.modules.wobble.return_high_pass_filtered_slope Source: https://github.com/noaa-ocs-hydrography/kluster/blob/master/HSTB/kluster/docbuild/api/HSTB.kluster.modules.wobble.return_high_pass_filtered_slope.html Performs linear regression on acrosstrack/depth offsets to get ping slope. High pass filters the slope to retain only the signal related to attitude. ```APIDOC ## HSTB.kluster.modules.wobble.return_high_pass_filtered_slope ### Description Performs linear regression on acrosstrack/depth offsets to get ping slope. High pass filters the slope to retain only the signal related to attitude. ### Method *Not applicable - this is a function call* ### Endpoint *Not applicable - this is a function call* ### Parameters #### Path Parameters *None* #### Query Parameters *None* #### Request Body *None* ### Parameters - **y** (array) - Required - numpy array (ping, beam) for alongtrack offset rel RP - **z** (array) - Required - numpy array (ping, beam) for depth rel wline - **max_period** (float) - Required - float, max period of the attitude arrays (roll, pitch, heave) - **numtaps** (int) - Optional (default: 101) - filter length, must be odd ### Returns - **np.array** - numpy array (ping) high pass filtered slope for each ping - **np.array** - numpy array (ping) percent deviation of the model ### Example ```python import numpy as np from HSTB.kluster.modules.wobble import return_high_pass_filtered_slope # Assuming y, z, and max_period are defined numpy arrays and float respectively # y = np.random.rand(100, 2) # z = np.random.rand(100, 2) # max_period = 50.0 slope, deviation = return_high_pass_filtered_slope(y, z, max_period) print(f"Slope shape: {slope.shape}") print(f"Deviation shape: {deviation.shape}") ``` ``` -------------------------------- ### Kluster API - fqpr_helpers Module Source: https://github.com/noaa-ocs-hydrography/kluster/blob/master/HSTB/kluster/docbuild/api/HSTB.kluster.fqpr_actions.FqprAction.html API documentation for the fqpr_helpers module. ```APIDOC ## HSTB.kluster.fqpr_helpers Module ### Description General helper functions for working with fQPR data. ``` -------------------------------- ### build_entwine_points Source: https://github.com/noaa-ocs-hydrography/kluster/blob/master/HSTB/kluster/docbuild/api/HSTB.kluster.pdal_entwine.build_entwine_points.html This function takes exported LAS files and builds Entwine point tiles in a specified output folder. It requires the Pydro38_test environment and Entwine to be installed. ```APIDOC ## POST /kluster/build_entwine_points ### Description Builds Entwine point tiles from exported LAS files in a given output folder. ### Method POST ### Endpoint /kluster/build_entwine_points ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body - **input_las_directory** (str) - Required - Folder path to the directory of LAS files. - **output_folder** (str) - Required - Folder path to where you want to build the Entwine point tiles. ### Request Example ```json { "input_las_directory": "/path/to/las/files", "output_folder": "/path/to/output/tiles" } ``` ### Response #### Success Response (200) - **message** (str) - A confirmation message indicating the process has started or completed. #### Response Example ```json { "message": "Entwine point tiles are being built." } ``` ``` -------------------------------- ### Kluster API - pydro_helpers Module Source: https://github.com/noaa-ocs-hydrography/kluster/blob/master/HSTB/kluster/docbuild/api/HSTB.kluster.fqpr_actions.FqprAction.html API documentation for the pydro_helpers module. ```APIDOC ## HSTB.kluster.pydro_helpers Module ### Description Helper functions related to the Pydro library, likely for specific hydrographic tasks. ``` -------------------------------- ### Kluster API - fqpr_helpers Module Source: https://github.com/noaa-ocs-hydrography/kluster/blob/master/HSTB/kluster/docbuild/api/HSTB.kluster.modules.visualizations.FqprVisualizations.html Documentation for the fqpr_helpers module within the Kluster API. ```APIDOC ## fqpr_helpers Module ### Description This module contains helper functions specifically for FPQR operations. ### Endpoint N/A (This is a module, not a REST endpoint) ### Parameters N/A ### Request Example N/A ### Response N/A ``` -------------------------------- ### HSTB.kluster.xarray_conversion.BatchRead.return_xyz_prefixes_for_systems Source: https://github.com/noaa-ocs-hydrography/kluster/blob/master/HSTB/kluster/docbuild/api/HSTB.kluster.xarray_conversion.BatchRead.html Returns prefixes needed to get offsets/angles from self.xyzrph depending on dual-head configuration. Assumes self.raw_ping contains Datasets broken up by system. ```APIDOC ## GET /api/kluster/xarray_conversion/BatchRead/return_xyz_prefixes_for_systems ### Description Returns prefixes needed to get offsets/angles from self.xyzrph depending on dual-head configuration. Assumes self.raw_ping contains Datasets broken up by system. ### Method GET ### Endpoint /api/kluster/xarray_conversion/BatchRead/return_xyz_prefixes_for_systems ### Parameters ### Request Example ```json { "example": "request body" } ``` ### Response #### Success Response (200) - **prefixes** (List[List[str]]) - List of two element lists containing the prefixes needed for tx/rx offsets and angles. #### Response Example ```json { "prefixes": [ ["prefix1_tx", "prefix1_rx"], ["prefix2_tx", "prefix2_rx"] ] } ``` ``` -------------------------------- ### HSTB.kluster.fqpr_actions.build_nav_action Source: https://github.com/noaa-ocs-hydrography/kluster/blob/master/HSTB/kluster/docbuild/api/HSTB.kluster.fqpr_actions.build_nav_action.html Generates a new import navigation action from POSPac SBET files. ```APIDOC ## POST /kluster/fqpr_actions/build_nav_action ### Description Generates a new import navigation action from POSPac SBET files. This function is part of the `HSTB.kluster.fqpr_actions` module. ### Method POST ### Endpoint /kluster/fqpr_actions/build_nav_action ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body - **destination** (str) - Required - File path to the converted Fqpr instance. - **fqpr_instance** (Fqpr) - Required - Fqpr object that we are importing the nav into. - **navfiles** (list) - Required - List of post processed navigation files (.sbet). - **error_files** (list) - Required - List of post processed navigation error files (.smrmsg). - **log_files** (list) - Required - List of post processed navigation export files (.log). ### Request Example ```json { "destination": "/path/to/converted/fqpr", "fqpr_instance": "", "navfiles": [ "/path/to/navfile1.sbet", "/path/to/navfile2.sbet" ], "error_files": [ "/path/to/errorfile1.smrmsg" ], "log_files": [ "/path/to/logfile1.log" ] } ``` ### Response #### Success Response (200) - **FqprAction** (FqprAction) - The newly generated import processed navigation action. #### Response Example ```json { "action_type": "import_nav", "details": { "destination": "/path/to/converted/fqpr", "navfiles_processed": 2, "errorfiles_processed": 1, "logfiles_processed": 1 } } ``` ``` -------------------------------- ### Activate Conda Environment Source: https://github.com/noaa-ocs-hydrography/kluster/blob/master/README.md This command activates the previously created conda environment named 'kluster_test'. Once activated, any subsequent commands will utilize the packages installed within this environment. ```bash conda activate kluster_test ``` -------------------------------- ### Get Attributes from FQPR Instance Source: https://github.com/noaa-ocs-hydrography/kluster/blob/master/HSTB/kluster/docbuild/api/HSTB.kluster.fqpr_convenience.get_attributes_from_fqpr.html Retrieves attributes from an FQPR instance. It prioritizes attributes from the xyz_dat store but can fall back to reading from converted zarr stores. Optionally includes unique modes from each raw_ping. ```APIDOC ## GET /api/kluster/attributes ### Description Retrieves attributes from an FQPR (Fully Qualified PING Record) instance. This function attempts to get attributes primarily from the `xyz_dat` store. If not found, it reads from converted zarr stores, as attributes across these raw ping data stores are identical. An option `include_mode` allows for the inclusion of unique modes from each raw ping in the returned attributes. ### Method GET ### Endpoint `/api/kluster/attributes` ### Parameters #### Query Parameters - **fqpr_instance** (object) - Required - The FQPR instance from which to load attributes. - **include_mode** (boolean) - Optional - Defaults to `True`. If `True`, includes mode information in the returned attributes. ### Request Example (No request body for GET requests) ### Response #### Success Response (200) - **attributes** (dict) - A dictionary containing the attributes from the FQPR instance. Keys will include attribute names, and values will be the attribute data. If `include_mode` is True, a 'mode' key will also be present. #### Response Example ```json { "attributes": { "depth": [ 10.5, 11.2, 10.9 ], "quality": [ 1, 1, 0 ], "mode": [ "auto", "auto", "manual" ] } } ``` ```