### Install and Use an MSNoise Plugin Source: https://github.com/robelgium/msnoise/blob/master/doc/releasenotes/msnoise-1.4.md To use a plugin, install it as a Python package and declare its name in the project's configuration. This example shows how to invoke a command from an installed plugin. ```sh $ msnoise plugin amazing Usage: msnoise-script.py plugin amazing [OPTIONS] COMMAND [ARGS]... Example Amazing Plugin for MSNoise Options: --help Show this message and exit. Commands: sayhi A Very Polite Command ``` ```sh $ msnoise plugin amazing sayhi Hi ``` -------------------------------- ### Implement Plugin Installation Logic Source: https://github.com/robelgium/msnoise/blob/master/doc/plugins.md Creates the database table and populates it with default values. ```python from msnoise.api import * from .amazing_table_def import AmazingConfig from .default import default def main(): engine = get_engine() Session = sessionmaker(bind=engine) session = Session() AmazingConfig.__table__.create(bind=engine, checkfirst=True) for name in default.keys(): session.add(AmazingConfig(name=name,value=default[name][-1])) session.commit() ``` -------------------------------- ### Define Setup Entry Points Source: https://github.com/robelgium/msnoise/blob/master/doc/plugins.md Registers the plugin table definition in setup.py. ```python 'msnoise.plugins.table_def': [ 'AmazingConfig = msnoise_amazing.amazing_table_def:AmazingConfig', ], ``` -------------------------------- ### List MSNoise plugins via CLI Source: https://github.com/robelgium/msnoise/blob/master/doc/plugins.md Verify installed and configured plugins using the command line interface. ```sh $ msnoise plugin Usage: msnoise-script.py plugin [OPTIONS] COMMAND [ARGS]... Runs a command in a named plugin Options: --help Show this message and exit. Commands: amazing Example Amazing Plugin for MSNoise ``` -------------------------------- ### MSNoise Reset Command Example Source: https://github.com/robelgium/msnoise/blob/master/doc/releasenotes/msnoise-1.3.md Example of how to use the `msnoise reset` command to change job status. ```APIDOC ## `msnoise reset` Command The `msnoise reset` command is used to reset the status of jobs. ### Usage: `msnoise reset [OPTIONS] JOBTYPE` ### Description: Resets the job to "T"odo status. The `JOBTYPE` argument can be `CC` or `DTT`. By default, it only resets jobs that are currently "I"n progress. ### Options: - **-a, --all**: Resets all jobs, regardless of their current status. - **--help**: Shows this message and exit. ``` -------------------------------- ### msnoise admin Source: https://github.com/robelgium/msnoise/blob/master/doc/clickhelp/msnoise.md Starts the Web Admin interface. ```APIDOC ## msnoise admin ### Description Starts the Web Admin on http://localhost:5000 by default. ### Parameters #### Options - **-p, --port** (INTEGER) - Optional - Port to open ``` -------------------------------- ### Test MariaDB Server Source: https://github.com/robelgium/msnoise/blob/master/doc/installation.md Verify the MariaDB server installation by running the executable in the console. ```sh bin/mysqld --console ``` -------------------------------- ### Verify MSNoise Installation Source: https://github.com/robelgium/msnoise/blob/master/doc/installation.md Use these commands to check for missing dependencies and run the built-in test suite to ensure the environment is configured correctly. ```sh msnoise utils bugreport msnoise utils test ``` -------------------------------- ### Install Documentation Dependencies Source: https://github.com/robelgium/msnoise/blob/master/doc/installation.md Required packages for building the documentation using Conda and pip. ```sh conda install -c conda-forge "sphinx<6" sphinx_bootstrap_theme numpydoc sphinx-gallery pip install "sphinx_rtd_theme>1" pip install pillow==9.0.0 ``` -------------------------------- ### Register CLI Command for Plugin Source: https://github.com/robelgium/msnoise/blob/master/doc/plugins.md Adds the install command to the plugin definition. ```python @click.command() def install(): """ Create the Config table""" from .install import main main() amazing.add_command(install) ``` -------------------------------- ### Output File Path Structure Source: https://github.com/robelgium/msnoise/blob/master/ARCHITECTURE.md This example shows the standard output file path structure used in the msnoise project. It highlights the components like lineage names, step names, and output directories. Pay attention to the `lineage_names_upstream` and `step_name` for correct path construction. ```shell OUTPUT/preprocess_1/cc_1/filter_1/_output/daily/ZZ/YA.UV05.00_YA.UV06.00/2010-09-01.nc ─────────────────────────── ──────────── lineage_names_upstream step_name (= filter_1 for cc) ``` ```shell OUTPUT/...stack_1/refstack_1/_output/REF/ZZ/YA.UV05.00_YA.UV06.00.nc ``` ```shell OUTPUT/...refstack_1/mwcs_1/_output/1D_1D/ZZ/YA.UV05.00_YA.UV06.00.nc ``` ```shell # DVV aggregates: OUTPUT / upstream_lineage / dvv_step_name / _output / 1D_1D / dvv_CC_ZZ.nc ``` -------------------------------- ### Install MSNoise and Dependencies via Conda Source: https://github.com/robelgium/msnoise/blob/master/doc/installation.md Use these commands to install the necessary Python packages and the MSNoise framework from the conda-forge channel. ```sh conda install -c conda-forge flask-admin flask-wtf markdown folium pymysql logbook pandas pytables pip xarray conda install -c conda-forge obspy conda install -c conda-forge msnoise ``` -------------------------------- ### Example msnoise bugreport output Source: https://github.com/robelgium/msnoise/blob/master/doc/how_tos.md This is an example output from the msnoise bugreport script, showing system details and the status of required Python modules. An '[X]' indicates presence, while '[ ]' indicates absence. ```sh ************* Computer Report ************* ----------------+SYSTEM+------------------- Windows PC1577-as 10 10.0.17134 AMD64 Intel64 Family 6 Model 158 Stepping 9, GenuineIntel ----------------+PYTHON+------------------- Python:3.7.3 | packaged by conda-forge | (default, Jul 1 2019, 22:01:29) [MSC v.1900 64 bit (AMD64)] This script is at d:\pythonforsource\msnoise_stack\msnoise\msnoise\bugreport.py ---------------+MODULES+------------------- Required: [X] setuptools: 41.2.0 [X] numpy: 1.15.4 [X] scipy: 1.3.0 [X] pandas: 0.25.0 [X] matplotlib: 3.1.1 [X] sqlalchemy: 1.3.8 [X] obspy: 1.1.0 [X] click: 7.0 [X] pymysql: 0.9.3 [X] flask: 1.1.1 [X] flask_admin: 1.5.3 [X] markdown: 3.1.1 [X] wtforms: 2.2.1 [X] folium: 0.10.0 [X] jinja2: 2.10.1 Only necessary if you plan to build the doc locally: [X] sphinx: 2.2.0 [X] sphinx_bootstrap_theme: 0.7.1 Graphical Backends: (at least one is required) [ ] wx: not found [ ] pyqt: not found [ ] PyQt4: not found [X] PyQt5: present (no version) [ ] PySide: not found Not required, just checking: [X] json: 2.0.9 [X] psutil: 5.6.3 [ ] reportlab: not found [ ] configobj: not found [X] pkg_resources: present (no version) [ ] paramiko: not found [X] ctypes: 1.1.0 [X] pyparsing: 2.4.2 [X] distutils: 3.7.3 [X] IPython: 7.7.0 [ ] vtk: not found [ ] enable: not found [ ] traitsui: not found [ ] traits: not found [ ] scikits.samplerate: not found ``` -------------------------------- ### Install Development Version of MSNoise Source: https://github.com/robelgium/msnoise/blob/master/doc/installation.md Commands to remove the current version and install the latest master branch of MSNoise. ```sh pip uninstall msnoise pip install http://msnoise.org/master.zip ``` -------------------------------- ### Connect to MSNoise API and Get Configuration Source: https://github.com/robelgium/msnoise/blob/master/doc/releasenotes/msnoise-1.3.md Demonstrates how to connect to the MSNoise API and retrieve the configuration for a specific parameter like 'mov_stack'. This is useful for custom scripts or console interactions. ```python from msnoise.api import connect, get_config session = connect() print get_config(session, "mov_stack") ``` -------------------------------- ### Install Development Version of ObsPy Source: https://github.com/robelgium/msnoise/blob/master/doc/installation.md Commands to remove the current version and install the latest master branch of ObsPy. ```sh pip uninstall obspy pip install https://github.com/obspy/obspy/archive/master.zip ``` -------------------------------- ### Run MSNoise Compute CC with Threading Source: https://github.com/robelgium/msnoise/blob/master/doc/releasenotes/msnoise-1.4.md Example of using the -t flag with msnoise compute_cc to enable multi-threading. Adjust thread count based on available cores and system RAM. ```bash msnoise -t 4 compute_cc ``` -------------------------------- ### Run MSNoise Compute MWCS with Threading Source: https://github.com/robelgium/msnoise/blob/master/doc/releasenotes/msnoise-1.4.md Example of using the -t flag with msnoise compute_mwcs to enable multi-threading. Ensure sufficient RAM is available if loading large datasets. ```bash msnoise -t 16 compute_mwcs ``` -------------------------------- ### MSNoise Compute Command Execution Output Source: https://github.com/robelgium/msnoise/blob/master/doc/plugins.md Example output from running the MSNoise plugin compute command. It shows the processing of specific station pairs for a given day and the details of the retrieved seismic data streams. ```shell $ msnoise plugin amazing compute Processing YA.UV05 for day 2010-09-01 1 Trace(s) in Stream: YA.UV05.00.HHZ | 2010-09-01T00:00:00.000000Z - 2010-09-01T23:59:59.990000Z | 100.0 Hz, 8640000 samples Processing YA.UV06 for day 2010-09-01 1 Trace(s) in Stream: YA.UV06.00.HHZ | 2010-09-01T00:00:00.000000Z - 2010-09-01T23:59:59.990000Z | 100.0 Hz, 8640000 samples Processing YA.UV10 for day 2010-09-01 1 Trace(s) in Stream: YA.UV10.00.HHZ | 2010-09-01T00:00:00.000000Z - 2010-09-01T23:59:59.990000Z | 100.0 Hz, 8640000 samples ``` -------------------------------- ### Configure plugin entry points in setup.py Source: https://github.com/robelgium/msnoise/blob/master/doc/plugins.md Use setuptools entry points to register plugin commands with the MSNoise CLI. ```python from setuptools import setup, find_packages setup( name='msnoise_amazing', version='0.1a', packages=find_packages(), include_package_data=True, install_requires=['msnoise', 'obspy'], entry_points = { 'msnoise.plugins.commands': [ 'amazing = msnoise_amazing.plugin_definition:amazing', ], }, author = "Thomas Lecocq & MSNoise dev team", author_email = "Thomas.Lecocq@seismology.be", description = "An example plugin", license = "EUPL-1.1", url = "http://www.msnoise.org", keywords="amazing seismology" ) ``` -------------------------------- ### Display MSNoise help Source: https://github.com/robelgium/msnoise/blob/master/doc/releasenotes/msnoise-1.3.md Lists all available top-level commands and global options. ```sh msnoise --help Usage: msnoise-script.py [OPTIONS] COMMAND [ARGS]... Options: -t, --threads INTEGER Number of threads to use (only affects modules that are designed to do parallel processing) --help Show this message and exit. Commands: bugreport This command launches the Bug Report script. compute_cc Computes the CC jobs (based on the "New Jobs"... compute_dtt Computes the dt/t jobs based on the new MWCS... compute_mwcs Computes the MWCS based on the new stacked... compute_stretching [experimental] Computes the stretching based... config This command launches the Configurator. info Outputs general information about the current... install This command launches the installer. ipython Launches an ipython notebook in the current... new_jobs Determines if new CC jobs are to be defined plot Top level command to trigger different plots populate Rapidly scan the archive filenames and find... reset Resets the job to "T"odo. scan_archive Scan the archive and insert into the Data... stack Stacks the [REF] and/or [MOV] windows test Runs the test suite, should be executed in an... upgrade_db Upgrade the database from pre-1.3 to MSNoise... ``` -------------------------------- ### Build Documentation Source: https://github.com/robelgium/msnoise/blob/master/doc/installation.md Command to generate the HTML documentation. ```sh make html ``` -------------------------------- ### Register Plugin Job Types in setup.py Source: https://github.com/robelgium/msnoise/blob/master/doc/plugins.md This entry point in `setup.py` registers custom job types from a plugin. Ensure the path `msnoise_amazing.plugin_definition:register_job_types` correctly points to your registration function. ```python 'msnoise.plugins.jobtypes': [ 'register = msnoise_amazing.plugin_definition:register_job_types', ], ``` -------------------------------- ### Initialize MSNoise Project Source: https://context7.com/robelgium/msnoise/llms.txt Initializes a new MSNoise project, creating a database and configuration file. Use --auto-workflow to create default configurations and workflow steps. Requires a running server for MySQL/MariaDB and PostgreSQL. ```bash # Create a new project directory and initialize mkdir my_project && cd my_project # Initialize with SQLite (simplest, no server needed) msnoise db init --tech 1 --auto-workflow # Initialize with MySQL/MariaDB (requires running server) msnoise db init --tech 2 --auto-workflow # Follow prompts for: hostname, database name, username, password # Initialize with PostgreSQL msnoise db init --tech 3 --auto-workflow # After initialization, configure the project msnoise config set data_folder=/path/to/seismic/archive msnoise config set output_folder=./OUTPUT msnoise config set startdate=2010-01-01 msnoise config set enddate=2010-12-31 ``` -------------------------------- ### msnoise db init Source: https://github.com/robelgium/msnoise/blob/master/doc/clickhelp/msnoise.md Initializes the current folder as an MSNoise project. ```APIDOC ## msnoise db init ### Description Initializes the current folder to be a MSNoise Project by creating a database and a db.ini file. ### Parameters #### Options - **--tech** (TEXT) - Optional - Database technology: 1=SQLite 2=MySQL/MariaDB 3=PostgreSQL - **--auto-workflow** (BOOLEAN) - Optional - Automatically create all default config sets, workflow steps and links without prompting. ``` -------------------------------- ### Run the Simplest MSNoise Configuration Source: https://github.com/robelgium/msnoise/blob/master/doc/how_tos.md This recipe provides the most basic MSNoise run for rapidly checking data. It includes essential configuration steps and commands for cross-correlation and related analyses. ```sh msnoise db init --tech 1 msnoise config set startdate=2019-01-01 msnoise config set enddate=2019-02-01 msnoise config set overlap=0.5 msnoise config set mov_stack=1,5,10 msnoise scan_archive --path /path/to/archive --recursively msnoise populate --fromDA msnoise db update_loc_chan msnoise new_jobs --init msnoise admin # add 1 filter in the Filter table # or msnoise db execute "insert into filters (ref, low, mwcs_low, high, mwcs_high, mwcs_wlen, mwcs_step, used) values (1, 0.1, 0.1, 1.0, 1.0, 12.0, 4.0, 1)" msnoise cc compute_cc msnoise cc stack -r msnoise reset STACK msnoise cc stack -m msnoise cc dtt compute_mwcs msnoise cc dtt compute_dtt msnoise cc dtt compute_dvv msnoise cc dtt plot dvv ``` -------------------------------- ### Register Web Admin View Source: https://github.com/robelgium/msnoise/blob/master/doc/plugins.md Registers a new admin page via entry points in setup.py. ```python 'msnoise.plugins.admin_view': [ 'AmazingConfigView = msnoise_amazing.plugin_definition:AmazingConfigView', ], ``` -------------------------------- ### Node Drag Start Function Source: https://github.com/robelgium/msnoise/blob/master/msnoise/templates/admin/workflow_builder.html Initiates the drag behavior for a node by raising it to the top of the SVG paint order. ```javascript function dragstarted(event, d) { d3.select(this).raise(); // bring to front while dragging } ``` -------------------------------- ### Initialize MSNoise Database Session Source: https://github.com/robelgium/msnoise/blob/master/examples/README.rst Establishes a connection to the MSNoise database. This must be executed within a directory initialized with 'msnoise db init'. ```python from msnoise.api import db db = connect() ``` -------------------------------- ### Dump and Import MSNoise Configuration Source: https://github.com/robelgium/msnoise/blob/master/doc/how_tos.md Export database tables to CSV and import them into a new project location. ```sh msnoise db dump ``` ```sh msnoise db init msnoise db import config --force msnoise db import stations --force msnoise db import filters --force msnoise db import data_availability --force msnoise db import jobs --force ``` -------------------------------- ### Declare SQLAlchemy Tables Source: https://github.com/robelgium/msnoise/blob/master/ARCHITECTURE.md Always call declare_tables() to get a fresh Job class for DB queries. Do not use module-level imports. ```python schema = declare_tables() Job = schema.Job ``` -------------------------------- ### PBS Job Script for Parallel CC Source: https://github.com/robelgium/msnoise/blob/master/doc/how_tos.md Example job script for running MSNoise cross-correlation tasks in parallel across an array of nodes. ```bash #!/bin/bash #PBS -N MSNoise_PDF_CC #PBS -l walltime=01:00:00 #PBS -l select=1:ncpus=1:mem=1g #PBS -l place=shared #PBS -J 1-400 cd /scratch-a/thomas/2019_PDF source /space/hpc-home/thomas/.profile msnoise cc compute_cc ``` -------------------------------- ### Implement plugin logic in plugin_definition.py Source: https://github.com/robelgium/msnoise/blob/master/doc/plugins.md Define CLI commands using the click library to extend MSNoise functionality. ```python import click @click.group() def amazing(): """Example Amazing Plugin for MSNoise""" pass @click.command() def sayhi(): """A Very Polite Command""" print("Hi") amazing.add_command(sayhi) ``` -------------------------------- ### Handling No Configuration Sets Found Source: https://github.com/robelgium/msnoise/blob/master/msnoise/templates/admin/config_sets_index.html Displays a message and a link to create the first configuration set when no sets are found for a given category. This is a fallback UI element. ```html {{ category_info.display_name }} _No configuration sets found_ [Create first set]({{ url_for('.create_set', category=category_info.category) }}) ``` -------------------------------- ### Custom Plotting with MSNoise Source: https://github.com/robelgium/msnoise/blob/master/doc/releasenotes/msnoise-1.4.md Override default plot commands by using the -c argument before the plot command. Copy and edit plot scripts from the install directory to your project directory. ```bash msnoise -c plot distance ``` -------------------------------- ### Compute Wavelet Coherence Transform (WCT) in Python Source: https://context7.com/robelgium/msnoise/llms.txt Example of computing the Wavelet Coherence Transform (WCT) using Python, including parameters for smoothing, frequency, and wavelet type. ```python from msnoise.core.signal import xwt, compute_wct_dtt import numpy as np # Example wavelet coherence computation fs = 20.0 # Sampling rate t = np.arange(0, 100, 1/fs) ref_trace = np.sin(2 * np.pi * 0.5 * t) # Reference cur_trace = np.sin(2 * np.pi * 0.5 * (t * 1.001)) # Slightly stretched # Compute wavelet coherence WXamp, WXspec, WXangle, Wcoh, WXdt, freqs, coi = xwt( ref_trace, cur_trace, fs, ns=3, # Smoothing parameter (scale) nt=0.25, # Smoothing parameter (time) vpo=12, # Voices per octave freqmin=0.1, # Minimum frequency freqmax=8.0, # Maximum frequency nptsfreq=100, # Number of frequency points wavelet_type=("Morlet", 6.0) ) print(f"Frequency range: {freqs.min():.2f} - {freqs.max():.2f} Hz") print(f"WCT shape: {Wcoh.shape}") ``` -------------------------------- ### Manage Workflow and Configuration Source: https://context7.com/robelgium/msnoise/llms.txt CLI commands for managing configuration sets, workflow steps, and job statuses. ```bash # List all configuration sets msnoise config list-sets # Show details of a specific config set msnoise config show-set mwcs 1 # Create a new configuration set msnoise config create-set mwcs # Copy an existing config set msnoise config copy-set mwcs 1 mwcs 2 # List workflow steps msnoise config list-workflow-steps # Show workflow graph msnoise config show-workflow-graph # Create workflow links between steps msnoise config create-workflow-links --verbose # Reset jobs to Todo status msnoise reset cc --all # Get project info and job status msnoise info msnoise info --jobs ``` -------------------------------- ### MSNoise Configuration Management Source: https://context7.com/robelgium/msnoise/llms.txt Retrieves and updates configuration values programmatically. Supports getting single values, all global configurations, and merged parameters for the workflow. Configuration is stored in the database. ```python from msnoise.api import connect from msnoise.core.config import get_config, update_config, get_params db = connect() # Get a single configuration value maxlag = float(get_config(db, "maxlag")) print(f"Maximum lag time: {maxlag} seconds") # Get a boolean value autocorr = get_config(db, "autocorr", isbool=True) print(f"Autocorrelation enabled: {autocorr}") # Get all global configuration as dictionary all_config = get_config(db) print(f"Data folder: {all_config['data_folder']}") # Update a configuration value update_config(db, "maxlag", "120") # Get merged parameters for the full workflow (LayeredParams object) params = get_params(db) print(f"HPC mode: {params.global_.hpc}") print(f"Sampling rate: {params.global_.cc_sampling_rate}") db.close() ``` -------------------------------- ### Custom Time Series Plotting with MSNoise Source: https://github.com/robelgium/msnoise/blob/master/doc/releasenotes/msnoise-1.4.md Example of customizing a specific plot (ccftime) with additional arguments using the -c flag. Modify the plot script to change visualization behavior. ```bash msnoise -c plot ccftime YA.UV02 YA.UV06 -m 5 ``` -------------------------------- ### Compute MWCS (Moving Window Cross-Spectrum) Source: https://context7.com/robelgium/msnoise/llms.txt Commands for computing Moving Window Cross-Spectrum (MWCS) measurements, calculating dt/t, aggregating dv/v, and plotting results. Use 'compute_mwcs' to start. ```bash # Compute MWCS measurements msnoise cc dtt compute_mwcs ``` ```bash # Compute dt/t from MWCS measurements msnoise cc dtt compute_mwcs_dtt ``` ```bash # Aggregate dv/v across all station pairs msnoise cc dtt dvv compute_mwcs_dtt_dvv ``` ```bash # Plot MWCS results for a station pair msnoise cc dtt plot mwcs YA.UV05.00 YA.UV06.00 --show True ``` ```bash # Plot dt/t timeseries msnoise cc dtt plot mwcs_dtt_timing --filterid 1 --show True ``` -------------------------------- ### Configure MariaDB Database Source: https://github.com/robelgium/msnoise/blob/master/doc/installation.md Commands for setting up the database password, testing the connection, and creating the MSNoise database schema. ```sh mysqladmin -u root flush-privileges password "SECRET" mysql -u root -p CREATE DATABASE msnoise; SHOW DATABASES; quit; ``` -------------------------------- ### msnoise utils jupyter Source: https://github.com/robelgium/msnoise/blob/master/doc/clickhelp/msnoise.md Launches a Jupyter notebook environment in the current directory. ```APIDOC ## msnoise utils jupyter ### Description Launches an jupyter notebook in the current folder. ### Method CLI Command ### Endpoint N/A (CLI command) ### Parameters No specific parameters other than the command itself. ### Request Example ```sh msnoise utils jupyter ``` ### Response N/A (CLI command launches Jupyter Notebook server) ### Error Handling N/A (CLI command specific errors not detailed) ``` -------------------------------- ### Customize MSNoise Plotting Scripts Source: https://github.com/robelgium/msnoise/blob/master/doc/plotting.md Override default plot commands using the `-c` argument before the plot command. To modify scripts, copy them from the install directory to your project and adjust the MSNoise API import. ```python from ..api import * to from msnoise.api import * ``` -------------------------------- ### msnoise config list-workflow-steps Source: https://github.com/robelgium/msnoise/blob/master/doc/clickhelp/msnoise.md Lists all workflow steps. ```APIDOC ## msnoise config list-workflow-steps ### Description List all workflow steps. ### Method Not applicable (CLI command) ### Endpoint Not applicable (CLI command) ### Parameters #### Options - **--help** (boolean) - Optional - Show this message and exit. ``` -------------------------------- ### Manage MSNoise Web Admin Source: https://context7.com/robelgium/msnoise/llms.txt Commands to launch the MSNoise web-based administration interface. ```bash # Start the web admin on default port 5000 msnoise admin # Use a custom port msnoise admin --port 8080 # Then open http://localhost:5000 in your browser ``` -------------------------------- ### Display command-specific help Source: https://github.com/robelgium/msnoise/blob/master/doc/releasenotes/msnoise-1.3.md Provides detailed usage information and options for a specific command. ```sh msnoise reset --help Usage: msnoise-script.py reset [OPTIONS] JOBTYPE Resets the job to "T"odo. ARG is [CC] or [DTT]. By default only resets jobs "I"n progress. --all resets all jobs, whatever the flag value Options: -a, --all Reset all jobs --help Show this message and exit. ``` -------------------------------- ### Load and Analyze MSNoise Results Source: https://context7.com/robelgium/msnoise/llms.txt Demonstrates loading results using MSNoiseResult from database IDs, retrieving CCF interferograms and reference stacks, and discovering downstream results. ```python from msnoise.api import connect from msnoise.results import MSNoiseResult db = connect() # Build result object from step IDs result = MSNoiseResult.from_ids( db, preprocess=1, cc=1, filter=1, stack=1, refstack=1 ) print(result) # Shows available methods # Get CCF interferogram as xarray DataArray ccf = result.get_ccf( pair="YA.UV05.00:YA.UV06.00", components="ZZ", mov_stack=("1D", "1D"), # (window, step) format="xarray" ) # Plot interferogram import matplotlib.pyplot as plt ccf.plot(robust=True, figsize=(10, 8)) plt.title("Cross-Correlation Function vs Time") plt.savefig("interferogram.png") # Get reference stack ref = result.get_ref(pair="YA.UV05.00:YA.UV06.00", components="ZZ") ref.REF.plot() plt.savefig("reference.png") # Discover downstream results (e.g., MWCS, stretching branches) for branch in result.branches(): print(f"Available: {branch.category} - {branch.lineage_names[-1]}") db.close() ``` -------------------------------- ### Create Processing Jobs Source: https://context7.com/robelgium/msnoise/llms.txt Analyzes data availability to create jobs for the workflow. Use --init for initial preprocessing and CC jobs, or --after to create jobs for subsequent steps. ```bash # Create initial preprocessing and CC jobs msnoise new_jobs --init ``` ```bash # Create jobs for subsequent workflow steps (after CC is done) msnoise new_jobs --after cc ``` ```bash # Create stack jobs from completed CC jobs msnoise new_jobs --after stack ``` ```bash # Check job status msnoise info --jobs ``` -------------------------------- ### Run MSNoise Test Suite Source: https://github.com/robelgium/msnoise/blob/master/doc/clickhelp/msnoise.md Executes the test suite in a temporary folder. Use the --fast option for a quick smoke test that uses stub compute functions and completes in under 30 seconds without requiring seismic data. ```sh msnoise utils test --help ``` ```sh msnoise utils test --fast ``` ```sh msnoise utils test -p myprefix --tech 1 ``` ```sh msnoise utils test -c ``` -------------------------------- ### msnoise config create-all-sets Source: https://github.com/robelgium/msnoise/blob/master/doc/clickhelp/msnoise.md Creates a complete set of workflow configurations for each category. ```APIDOC ## msnoise config create-all-sets ### Description Create one configuration set for each workflow category. ### Method Not applicable (CLI command) ### Endpoint Not applicable (CLI command) ### Parameters #### Options - **--force** (boolean) - Optional - Force creation even if config sets already exist - **--dry-run** (boolean) - Optional - Show what would be created without actually creating it ``` -------------------------------- ### Develop MSNoise Plugins Source: https://context7.com/robelgium/msnoise/llms.txt Structure for creating custom MSNoise plugins and discovering them via CLI. ```python # Plugin structure example (in my_plugin/__init__.py) """ my_plugin ========= Custom MSNoise plugin for specialized analysis. """ from flask_admin.contrib.sqla import ModelView # Define plugin info name = "My Plugin" short_name = "myplugin" # Define custom configuration parameters def get_config(): return { "my_param": { "default": "10", "definition": "My custom parameter", } } # Define custom job types def get_jobs(): return { "MYPROCESS": { "description": "My custom processing step", "after": "CC", # Run after CC step } } # Register with MSNoise def register(app): """Called when MSNoise loads the plugin.""" pass ``` ```bash # Plugins are automatically discovered when installed via pip # List available plugins msnoise info ``` -------------------------------- ### Document Ready Event Handlers Source: https://github.com/robelgium/msnoise/blob/master/msnoise/templates/admin/psd-spectrogram.html Initializes the web interface by fetching network data and setting up event listeners for dropdown changes and refresh button clicks. These handlers trigger data population and image updates. ```javascript $(document).ready(function() { get_network_station(populate_network); $('#network').change(function() { $('#station') .find('option') .remove() .end(); get_network_station(populate_stations); }); $('#station').change(function() { $('#location') .find('option') .remove() .end(); $('#channel') .find('option') .remove() .end(); $('#file') .find('option') .remove() .end(); $('#date') .find('option') .remove() .end(); get_network_station(populate_locations); }); $('#location').change(function() { update_images(); }); $('#channel').change(function() { update_images(); }); $('#refresh').click(function() { update_images(); }); }); ``` -------------------------------- ### msnoise config create-workflow-steps-from-configs Source: https://github.com/robelgium/msnoise/blob/master/doc/clickhelp/msnoise.md Creates workflow steps automatically from all existing config sets. ```APIDOC ## msnoise config create-workflow-steps-from-configs ### Description Create workflow steps automatically from all existing config sets. This command scans all configuration sets in the database and creates corresponding workflow steps, sorted by natural workflow order. ### Method Not applicable (CLI command) ### Endpoint Not applicable (CLI command) ### Parameters #### Options - **-v, --verbose** (boolean) - Optional - Show detailed output - **--help** (boolean) - Optional - Show this message and exit. ``` -------------------------------- ### Initialize Graph and Update Stats on DOM Load Source: https://github.com/robelgium/msnoise/blob/master/msnoise/templates/admin/workflow_builder.html Sets up the workflow graph and initial statistics when the DOM is ready. Also configures event listeners for dynamic form element updates. ```javascript document.addEventListener('DOMContentLoaded', function() { console.log('DOM loaded, initializing graph...'); initializeGraph(); updateStats(); // Update form dropdowns based on workflow logic const fromStepSelect = document.getElementById('from_step_select'); const toStepSelect = document.getElementById('to_step_select'); if (fromStepSelect && toStepSelect && workflowData.nodes) { fromStepSelect.addEventListener('change', function() { const fromStepId = parseInt(this.value); const fromNode = workflowData.nodes.find(n => n.id === fromStepId); // Clear options toStepSelect.innerHTML = ''; if (fromNode) { const WORKFLOW_CHAINS = { 'global': ['preprocess', 'psd'], 'preprocess': ['cc'], 'cc': ['filter'], 'psd': ['psd_rms'], 'psd_rms': [], 'filter': ['stack'], 'stack': ['refstack'], 'refstack': ['mwcs', 'stretching', 'wavelet'], 'mwcs': ['mwcs_dtt'], 'mwcs_dtt': ['mwcs_dtt_dvv'], 'mwcs_dtt_dvv': [], 'stretching': ['stretching_dvv'], 'stretching_dvv': [], 'wavelet': ['wavelet_dtt'], 'wavelet_dtt': ['wavelet_dtt_dvv'], 'wavelet_dtt_dvv': [] }; const validNextCategories = WORKFLOW_CHAINS[fromNode.category] || []; workflowData.nodes.forEach(node => { if (node.id !== fromStepId && (validNextCategories.includes(node.category) || validNextCategories.length === 0)) { const existingLink = workflowData.edges.find(e => e.from === fromStepId && e.to === node.id ); const linkStatus = existingLink ? ' (already linked)' : ''; const option = document.createElement('option'); option.value = node.id; option.textContent = `${node.name} (${node.category}:${node.set_number})${linkStatus}`; if (existingLink) { option.style.color = '#6c757d'; option.style.fontStyle = 'italic'; } toStepSelect.appendChild(option); } }); } }); } }); } ``` -------------------------------- ### Generate Power Spectral Densities and Spectrograms with MSNoise Source: https://github.com/robelgium/msnoise/blob/master/doc/how_tos.md Use this recipe to rapidly check noise content and generate Power Spectral Densities (PSDs) and spectrograms. Ensure response files are correctly configured. ```sh msnoise db init --tech 1 msnoise config set startdate=2019-01-01 msnoise config set enddate=2019-02-01 msnoise config set response_path=/path/to/response_files msnoise scan_archive --path /path/to/archive --recursively msnoise populate --fromDA msnoise db update_loc_chan msnoise new_jobs --init --nocc msnoise admin # check the qc_* parameters msnoise qc compute_psd msnoise qc plot_psd YA.UV05.00.HHZ ``` -------------------------------- ### Define Default Plugin Parameters Source: https://github.com/robelgium/msnoise/blob/master/doc/plugins.md Stores default configuration values in an OrderedDict. ```python from collections import OrderedDict default = OrderedDict() default['parameter1'] = ["Some really useful text",'1'] default['parameter2'] = ["Some really useful text",'1'] default['parameter3'] = ["Some really useful text",'1'] default['parameter4'] = ["Some really useful text",'1'] default['question1'] = ["Is this a useful text [Y]/N",'Y'] ``` -------------------------------- ### msnoise populate Source: https://github.com/robelgium/msnoise/blob/master/doc/clickhelp/msnoise.md Scans archive filenames to find Network/Stations. ```APIDOC ## msnoise populate ### Description Rapidly scan the archive filenames and find Network/Stations, only works with known archive structures, or with a custom code provided by the user. ### Method Not applicable (CLI command) ### Endpoint Not applicable (CLI command) ### Parameters #### Options - **--fromDA** (boolean) - Optional - Populates the station table using network and station codes found in the data_availability table, overrides the default workflow step. - **--help** (boolean) - Optional - Show this message and exit. ``` -------------------------------- ### Connect to MSNoise Database Source: https://context7.com/robelgium/msnoise/llms.txt Establishes a database connection using SQLAlchemy. Reads connection parameters from 'db.ini'. Supports SQLite, MySQL/MariaDB, and PostgreSQL. Always close the session when done. ```python from msnoise.api import connect # Connect to database (reads db.ini from current directory) db = connect() # Alternatively, specify a custom db.ini path db = connect(inifile="/path/to/project/db.ini") # Use the session for queries from msnoise.msnoise_table_def import Station stations = db.query(Station).filter(Station.used == True).all() for sta in stations: print(f"{sta.net}.{sta.sta}: {sta.X}, {sta.Y}") # Always close when done db.close() ``` -------------------------------- ### Run MSNoise Plugin Command Source: https://context7.com/robelgium/msnoise/llms.txt This command is used to execute a plugin within the MSNoise framework. Ensure the plugin namespace is correctly specified. ```bash msnoise myplugin process ``` -------------------------------- ### msnoise config create-workflow-step Source: https://github.com/robelgium/msnoise/blob/master/doc/clickhelp/msnoise.md Creates a new workflow step interactively. ```APIDOC ## msnoise config create-workflow-step ### Description Create a new workflow step interactively. ### Method Not applicable (CLI command) ### Endpoint Not applicable (CLI command) ### Parameters #### Options - **--help** (boolean) - Optional - Show this message and exit. ``` -------------------------------- ### Display plot command help Source: https://github.com/robelgium/msnoise/blob/master/doc/releasenotes/msnoise-1.3.md Lists all available plotting sub-commands. ```sh msnoise plot --help Usage: msnoise-script.py plot [OPTIONS] COMMAND [ARGS]... Top level command to trigger different plots Options: --help Show this message and exit. Commands: ccftime Plots the dv/v (parses the dt/t results) data_availability Plots the Data Availability vs time distance Plots the REFs of all pairs vs distance dvv Plots the dv/v (parses the dt/t results) interferogram Plots the interferogram between sta1 and sta2... mwcs Plots the mwcs results between sta1 and sta2... station_map Plots the station map (very basic) ``` -------------------------------- ### MSNoise Main Commands Source: https://github.com/robelgium/msnoise/blob/master/doc/releasenotes/msnoise-1.3.md Overview of the primary commands available in the MSNoise CLI. ```APIDOC ## MSNoise CLI Commands MSNoise provides a top-level command-line interface for various operations. The general usage is `msnoise [COMMAND] [OPTIONS] [ARGS]`. ### Available Commands: - **install**: Installs MSNoise. - **config**: Configures MSNoise settings. - **populate**: Scans the archive and populates the station table. - **scan_archive**: Scans the archive for data. - **new_jobs**: Defines new jobs for processing. - **compute_cc**: Computes cross-correlations. - **stack**: Stacks seismic data windows. - **compute_mwcs**: Computes Moving Window Cross-Spectral analysis. - **compute_dtt**: Computes time-domain difference analysis. - **bugreport**: Launches the bug report script. - **ipython**: Launches an IPython notebook. - **reset**: Resets job status. - **info**: Outputs general information. - **test**: Runs the test suite. - **upgrade_db**: Upgrades the database. - **plot**: Triggers various plotting functions. ### Global Options: - **-t, --threads INTEGER**: Number of threads to use for parallel processing. - **--help**: Shows this message and exits. ```