### Example Usage of shmem_barrier_all Source: https://github.com/open-mpi/ompi/blob/main/docs/man-openshmem/man3/shmem_barrier_all.3.rst This example demonstrates how shmem_barrier_all can be used to ensure that setup operations performed by PE 0 are completed before other PEs proceed. All PEs wait for PE 0 to finish its setup. ```c++ setup_data() { if (shmem_my_pe() == 0) { setup(); } /* All PEs wait for PE 0 to complete setup(). */ shmem_barrier_all(); } ``` -------------------------------- ### Copy Example Component Directory Source: https://github.com/open-mpi/ompi/blob/main/ompi/mca/op/example/README.md To use this example as a template for your component, copy the 'example' directory to your new component's name (e.g., 'foo') and navigate into it. ```shell shell$ cd (top_ompi_dir)/ompi/mca/op shell$ cp -r example foo shell$ cd foo ``` -------------------------------- ### Install Open MPI Source: https://github.com/open-mpi/ompi/blob/main/contrib/build-mca-comps-outside-of-tree/README.md Standard procedure to download, configure, build, and install Open MPI. Ensure to use `--prefix` to specify the installation directory. ```bash $ cd $HOME $ wget https://www.open-mpi.org/software/ompi/vX.Y/downloads/openmpi-X.Y.Z.tar.bz2 [...lots of output...] $ tar jxf openmpi-X.Y.Z.tar.bz2 $ cd openmpi-X.Y.Z $ ./configure --prefix=/opt/openmpi ... [...lots of output...] $ make -j 4 install [...lots of output...] $ /opt/openmpi/bin/ompi_info | grep btl MCA btl: self (MCA vA.B, API vM.N, Component vX.Y.Z) MCA btl: sm (MCA vA.B, API vM.N, Component vX.Y.Z) MCA btl: tcp (MCA vA.B, API vM.N, Component vX.Y.Z) [where X.Y.Z, A.B, and M.N are appropriate for your version of Open MPI] $ ``` -------------------------------- ### Configure Open MPI with PMIx and PRRTE Installation Paths Source: https://github.com/open-mpi/ompi/blob/main/docs/installing-open-mpi/required-support-libraries.rst Specifies custom installation paths for PMIx and PRRTE when they are not found in the default search locations. This example assumes both are installed in '/opt/open-mpi-stuff'. ```sh ./configure --prefix=$HOME/openmpi-install \ --with-pmix=/opt/open-mpi-stuff \ --with-prrte=/opt/open-mpi-stuff ... ``` -------------------------------- ### Build and Install Automake Source: https://github.com/open-mpi/ompi/blob/main/docs/developers/gnu-autotools.rst Compile and install Automake from source. Ensure the PATH is re-indexed if necessary before installation. ```sh # Make $PATH be re-indexed if necessary, e.g., via "rehash" shell$ cd AUTOMAKE_DIRECTORY shell$ ./configure --prefix=PREFIX shell$ make all install ``` -------------------------------- ### Build and Install Libtool Source: https://github.com/open-mpi/ompi/blob/main/docs/developers/gnu-autotools.rst Compile and install Libtool from source. Ensure the PATH is re-indexed if necessary before installation. ```sh # Make $PATH be re-indexed if necessary, e.g., via "rehash" shell$ cd LIBTOOL_DIRECTORY shell$ ./configure --prefix=PREFIX shell$ make all install ``` -------------------------------- ### Build and Install Autoconf Source: https://github.com/open-mpi/ompi/blob/main/docs/developers/gnu-autotools.rst Compile and install Autoconf from source. Ensure the PATH is re-indexed if necessary before installation. ```sh # Make $PATH be re-indexed if necessary, e.g., via "rehash" shell$ cd AUTOCONF_DIRECTORY shell$ ./configure --prefix=PREFIX shell$ make all install ``` -------------------------------- ### View Installed HTML Docs Source: https://github.com/open-mpi/ompi/blob/main/docs/developers/sphinx.rst Open the installed HTML documentation in your default web browser. This command assumes Open MPI has been installed and the documentation is in the default location. ```sh shell$ open $prefix/share/doc/openmpi/html/index.html ``` -------------------------------- ### Fortran Example for Nonblocking Get and Summation Source: https://github.com/open-mpi/ompi/blob/main/docs/man-openshmem/man3/shmem_getmem_nbi.3.rst This Fortran example demonstrates using SHMEM_REAL_GET_NBI to fetch data from remote PEs and accumulate a sum locally after ensuring completion with SHMEM_QUIET. ```fortran PROGRAM REDUCTION REAL VALUES, SUM COMMON /C/ VALUES REAL WORK CALL START_PES(0) ! ALLOW ANY NUMBER OF PES VALUES = MY_PE() ! INITIALIZE IT TO SOMETHING CALL SHMEM_BARRIER_ALL SUM = 0.0 DO I = 0,NUM_PES()-1 CALL SHMEM_REAL_GET_NBI(WORK, VALUES, 1, I) CALL SHMEM_QUIET ! wait for delivery SUM = SUM + WORK ENDDO PRINT *, 'PE ', MY_PE(), ' COMPUTED SUM=', SUM CALL SHMEM_BARRIER_ALL END ``` -------------------------------- ### BuildNode Setup and Gist Creation Source: https://github.com/open-mpi/ompi/wiki/PRJenkinsSetupFirewall Commands to set up a log file, push it to a Gist for public access, and configure the relay system on the BuildNode. ```shell # Setup a log file touch $PATH_TO_OMPI_TESTS/log/your-log.md # Push this file to the Gist site # Make sure to note the URL returned, as you will need the ID for later # --public makes it publicly available, remove that option if you want it private $PATH_TO_OMPI_TESTS/jenkins/bin/gist.pl --tokenfile $TOKENFILE -cmd create --public $PATH_TO_OMPI_TESTS/log/your-log.md # Setup the configuration for the relay system. Start with the template. cp $PATH_TO_OMPI_TESTS/jenkins/bin/config.inc.sample $PATH_TO_OMPI_TESTS/jenkins/bin/config.inc $EDITOR $PATH_TO_OMPI_TESTS/jenkins/bin/config.inc # Setup a crontab entry for the following script crontab -e # This script removes gists older than N days (Defined in config.inc) 0 8 * * * $PATH_TO_OMPI_TESTS/jenkins/bin/clean-history.sh $PATH_TO_OMPI_TESTS/jenkins/bin/config.inc ``` -------------------------------- ### Example Usage of MPIX_Query_cuda_support in C Source: https://github.com/open-mpi/ompi/blob/main/docs/man-openmpi/man3/MPIX_Query_cuda_support.3.rst This example demonstrates how to use MPIX_Query_cuda_support to check for CUDA-aware support in an Open MPI installation. It includes necessary headers and uses a preprocessor macro to ensure compatibility. ```c #include #include #include /* Needed for CUDA-aware check */ int main(int argc, char *argv[]) { MPI_Init(&argc, &argv); bool happy = false; #if defined(OMPI_HAVE_MPI_EXT_CUDA) && OMPI_HAVE_MPI_EXT_CUDA happy = (bool) MPIX_Query_cuda_support(); #endif if (happy) { printf("This Open MPI installation has CUDA-aware support.\n"); } else { printf("This Open MPI installation does not have CUDA-aware support.\n"); } MPI_Finalize(); return 0; } ``` -------------------------------- ### Select Network Transport (BTL) Components Source: https://context7.com/open-mpi/ompi/llms.txt This example shows how to explicitly select which network transport (BTL) components to use. This is crucial for optimizing communication on different network fabrics or for specific deployment scenarios. ```bash # Use only shared memory and self (single node, no network) mpirun --mca btl self,sm -n 4 ./app # Exclude openib; use all others mpirun --mca btl ^openib -n 4 ./app # Use UCX as point-to-point transport mpirun --mca pml ucx -n 4 ./app ``` -------------------------------- ### MPI_Type_indexed Example Source: https://github.com/open-mpi/ompi/blob/main/docs/man-openmpi/man3/MPI_Type_indexed.3.rst Illustrates the creation of an indexed datatype with three blocks. The first block has 3 copies of oldtype starting at displacement 4*extent, and the second block has 1 copy starting at displacement 0. ```c int B[2] = {3, 1}; int D[2] = {4, 0}; MPI_Type_indexed(2, B, D, oldtype, &newtype); ``` -------------------------------- ### Example Hostfile Configuration Source: https://github.com/open-mpi/ompi/blob/main/docs/launching-apps/scheduling.rst Define hosts and optionally specify the number of slots (processes) per host. Comments and blank lines are ignored. ```shell # This is an example hostfile. Comments begin with #. # # Since no slots are specified, the number of slots defaults to the # number of processor cores available on the machine. foo.example.com # We want to allow launching a maximum of 2 processes on this host # (e.g., potentially because it has two processor cores): bar.example.com slots=2 ``` -------------------------------- ### RST Numbered List Example Source: https://github.com/open-mpi/ompi/blob/main/docs/developers/rst-for-markdown-expats.rst Demonstrates how to create numbered lists in RST using '#.' at the start of each line. The trailing period is crucial. ```rst #. Item number 1 #. The second item #. A third item ``` -------------------------------- ### Example: Print process binding using layout string format Source: https://github.com/open-mpi/ompi/blob/main/docs/man-openmpi/man3/OMPI_Affinity_str.3.rst This example shows how to use OMPI_Affinity_str with OMPI_AFFINITY_LAYOUT_FMT to display binding information as ASCII art representing the machine resource layout. It initializes MPI, gets the rank, and then calls OMPI_Affinity_str. ```c int rank; char ompi_bound[OMPI_AFFINITY_STRING_MAX]; char current_binding[OMPI_AFFINITY_STRING_MAX]; char exists[OMPI_AFFINITY_STRING_MAX]; MPI_Init(&argc, &argv); MPI_Comm_rank(MPI_COMM_WORLD, &rank); OMPI_Affinity_str(OMPI_AFFINITY_LAYOUT_FMT, ompi_bound, current_binding, exists); ``` -------------------------------- ### Basic configure.m4 Example for OMPI_CHECK_PACKAGE Source: https://github.com/open-mpi/ompi/wiki/devel-CreateComponent-before-sep-2010 A simple configure.m4 example demonstrating the usage of OMPI_CHECK_PACKAGE to check for necessary header and library files for a component. ```m4 AC_INIT([ompi], m4_PACKAGE_VERSION, [www.open-mpi.org/community/help/]) AC_CONFIG_SRCDIR([Makefile.am]) AC_CONFIG_MACRO_DIRS([m4]) AC_PROG_CXX AC_PROG_CC # Include Open MPI's common macros OMPI_WANT_PACKAGE_LIBS AS NEEDED # Check for the OpenFabrics header and library files OMPI_CHECK_PACKAGE([OPENFABRICS], [openfabrics], [yes]) # If the component needs to invoke AM_CONDITIONAL, define MCA___POST_CONFIG # MCA_ompi_foo_POST_CONFIG AC_CONFIG_FILES([Makefile]) AC_OUTPUT ``` -------------------------------- ### Example: Using MPI_Get_address for an array (Fortran) Source: https://github.com/open-mpi/ompi/blob/main/docs/man-openmpi/man3/MPI_Get_address.3.rst This example demonstrates how to use MPI_Get_address to get the memory addresses of elements within a Fortran array and calculate the difference between them. The difference in addresses, when divided by the size of the data type, indicates the number of elements between the two accessed locations. ```fortran REAL :: A(100,100) INTEGER(MPI_ADDRESS_KIND) :: I1, I2, DIFF CALL MPI_GET_ADDRESS(A(1,1), I1, IERROR) CALL MPI_GET_ADDRESS(A(10,10), I2, IERROR) DIFF = I2 - I1 ! The value of DIFF is 909*sizeofreal; the values of I1 and I2 are ! implementation dependent. ``` -------------------------------- ### Configure Open MPI Installation Path Source: https://github.com/open-mpi/ompi/blob/main/docs/launching-apps/ssh.rst Example of configuring Open MPI with a custom prefix path. This path needs to be added to PATH and LD_LIBRARY_PATH. ```sh shell$ ./configure --prefix=$HOME/my-openmpi ... ``` -------------------------------- ### Fortran Example Initialization Source: https://github.com/open-mpi/ompi/blob/main/docs/man-openshmem/man3/shmem_char_get.3.rst This Fortran example demonstrates initializing a program with a specified number of PEs and setting a variable based on the current PE's ID. It includes a barrier synchronization. ```fortran PROGRAM REDUCTION REAL VALUES, SUM COMMON /C/ VALUES REAL WORK CALL START_PES(0) ! ALLOW ANY NUMBER OF PES VALUES = MY_PE() ! INITIALIZE IT TO SOMETHING CALL SHMEM_BARRIER_ALL ``` -------------------------------- ### RST Anchor Definition Example Source: https://github.com/open-mpi/ompi/blob/main/docs/developers/rst-for-markdown-expats.rst Defines an anchor point in RST documentation. Anchor names typically start with 'label-' or end with '-label' for clarity. ```rst .. _ANCHOR_NAME: ``` -------------------------------- ### Build and Install Open MPI Source: https://github.com/open-mpi/ompi/blob/main/docs/installing-open-mpi/configure-cli-options/rpath-and-runpath.rst Standard commands to configure, build, and install Open MPI. This sets the stage for checking linker options. ```shell shell$ ./configure --prefix=/opt/openmpi/ ... ... shell$ make -j 32 all && make install ... ``` -------------------------------- ### RST Comment Block Example Source: https://github.com/open-mpi/ompi/blob/main/docs/developers/rst-for-markdown-expats.rst Shows how to create comments in RST that will not be included in the rendered output. Comments start with '.. ' and are terminated by a blank line and outdenting. ```rst .. Hello world. This is a comment. This whole block is a comment. You can leave it here in the final document, and it will not be included in the rendered output. Your comment can even include blank lines. You terminate a comment |mdash| just like most other things in RST |mdash| by a blank line and then outdenting back out to the same column as the first "." This line is no longer part of the comment. ``` -------------------------------- ### Fortran shmem_ptr Example Source: https://github.com/open-mpi/ompi/blob/main/docs/man-openshmem/man3/shmem_ptr.3.rst This Fortran program demonstrates calling shmem_ptr to get the address of a remote array and then writing to it from PE 0. Ensure SHMEM_BARRIER_ALL is used for synchronization. ```fortran PROGRAM REMOTEWRITE INCLUDE 'mpp/shmem.fh' INTEGER BIGD(100) SAVE BIGD INTEGER POINTEE(*) POINTER (PTR,POINTEE) CALL START_PES(0) IF (MY_PE() .EQ. 0) THEN ! initialize PE 1's BIGD array PTR = SHMEM_PTR(BIGD, 1) ! get address of PE 1's BIGD ! array DO I=1,100 POINTEE(I) = I ENDDO ENDIF CALL SHMEM_BARRIER_ALL IF (MY_PE() .EQ. 1) THEN PRINT *, 'BIGD on PE 1 is: ' PRINT *, BIGD ENDIF END ``` -------------------------------- ### MPI_Init C Example Source: https://github.com/open-mpi/ompi/blob/main/docs/man-openmpi/man3/MPI_Init.3.rst Example of how to use MPI_Init in a C program. ```c #include int main(int argc, char *argv[]) { MPI_Init(&argc, &argv); /* ...body of main MPI pogram... */ MPI_Finalize(); return 0; } ``` -------------------------------- ### Cron Job with Environment Initialization Source: https://github.com/open-mpi/ompi/wiki/Jenkins-Build-Agent Example of a cron entry that sources the user's profile before executing the agent keep-alive script, useful for non-transparent proxy setups. ```bash */5 * * * * . $HOME/.profile /home/someuser/jenkins/agent-keepalive.sh ``` -------------------------------- ### Prebind subset with strided cores and multiple CPUs per process Source: https://github.com/open-mpi/ompi/wiki/ProcessAffinity This example shows prebinding to a subset of physical CPUs (2,3,4,5,12,13) and then running mpirun with strided core assignment and multiple CPUs per process. It uses 3 processes. ```bash numactl --physcpubind=2,3,4,5,12,13 mpirun -np 3 -stride 2 -cpus-per-proc 2 -bysocket -bind-to-core hostname ``` -------------------------------- ### mpirun --prefix Example Source: https://github.com/open-mpi/ompi/blob/main/docs/launching-apps/prerequisites.rst Use the --prefix option with mpirun to specify the Open MPI installation directory. This is often useful in ssh-based environments where PATH and LD_LIBRARY_PATH may not be automatically propagated. ```shell shell$ mpirun --prefix /opt/openmpi-VERSION -n 4 a.out ``` -------------------------------- ### Configure for 64-bit build on Linux Source: https://github.com/open-mpi/ompi/wiki/MultiLib Specifies installation directories for a 64-bit build on Linux to support multilib. Use `--includedir` for 64-bit include files and `--libdir` for 64-bit libraries, relative to prefix and exec_prefix. ```bash % ./configure --prefix=/opt/openmpi --includedir='${prefix}/include/64' --libdir='${exec_prefix}/lib64' ``` -------------------------------- ### Build Error Example: Library not found Source: https://github.com/open-mpi/ompi/blob/main/docs/installing-open-mpi/required-support-libraries.rst This error can occur when Open MPI's configure script uses external installations of hwloc or Libevent, and the C and Fortran compilers have different library search paths. ```shell ... PPFC profile/pwin_unlock_f08.lo PPFC profile/pwin_unlock_all_f08.lo PPFC profile/pwin_unlock_wait_f08.lo FCLD libmpi_usempif08.la ld: library not found for -lhwloc collect2: error: ld returned 1 exit status make``2``: *** ``libmpi_usempif08.la`` Error 1 ``` -------------------------------- ### Install and Build Open MPI from Source Source: https://context7.com/open-mpi/ompi/llms.txt Standard autotools build process for Open MPI. Use --prefix to set the installation directory. VPATH builds are supported for out-of-tree compilation. ```sh # Binary install (macOS) brew install openmpi # or MacPorts port install openmpi # From source tarball tar xf openmpi-.tar.bz2 cd openmpi- # Configure with optional features; --prefix sets install location ./configure --prefix=/usr/local/openmpi \ --with-ucx=/path/to/ucx \ --with-pmix=/path/to/pmix \ --with-ft=ulfm \ 2>&1 | tee config.out make -j 8 all 2>&1 | tee make.out make install 2>&1 | tee install.out # Verify installation ompi_info | grep "Open MPI" # Open MPI: 5.0.x # VPATH (out-of-tree) build mkdir build && cd build ../configure --prefix=/usr/local/openmpi make -j 8 all install ``` -------------------------------- ### C/C++ Product Reduction Equivalent Calls Source: https://github.com/open-mpi/ompi/blob/main/docs/man-openshmem/man3/shmem_short_prod_to_all.3.rst This C/C++ example shows a more efficient call to `shmem_short_prod_to_all` that is semantically equivalent to multiple calls with individual elements. Ensure that pWrk and pSync arrays are managed correctly for repeated calls. ```c++ shmem_short_prod_to_all(target, source, 3, 0, 0, 8, pwrk, psync); shmem_short_prod_to_all(&(target[0]), &(source[0]), 1, 0, 0, 8, pwrk1, psync1); shmem_short_prod_to_all(&(target[1]), &(source[1]), 1, 0, 0, 8, pwrk2, psync2); shmem_short_prod_to_all(&(target[2]), &(source[2]), 1, 0, 0, 8, pwrk1, psync1); ``` -------------------------------- ### MPI_T Initialization and MCA Parameter Setting Example Source: https://github.com/open-mpi/ompi/blob/main/docs/man-openmpi/man3/MPI_T.3.rst This example demonstrates how to use the MPI_T interface to set MCA parameters (like 'pml' and 'btl') before initializing MPI. This allows for influencing how MPI components are selected during initialization. It shows initializing MPI_T, getting indices for cvars, allocating handles, writing values, reading them back, and finally freeing the handles before calling MPI_Init. ```APIDOC ## MPI_T Initialization and MCA Parameter Setting Example ### Description This example demonstrates how to use the MPI_T interface to set MCA parameters (like 'pml' and 'btl') before initializing MPI. This allows for influencing how MPI components are selected during initialization. It shows initializing MPI_T, getting indices for cvars, allocating handles, writing values, reading them back, and finally freeing the handles before calling MPI_Init. ### Code ```c int provided, index, count; MPI_T_cvar_handle pml_handle, btl_handle; char pml_value[64], btl_value[64]; MPI_T_init_thread(MPI_THREAD_SINGLE, &provided); MPI_T_cvar_get_index("pml", &index); MPI_T_cvar_handle_alloc(index, NULL, &pml_handle, &count); MPI_T_cvar_write(pml_handle, "ob1"); MPI_T_cvar_get_index("btl", &index); MPI_T_cvar_handle_alloc(index, NULL, &btl_handle, &count); MPI_T_cvar_write(btl_handle, "tcp,vader,self"); MPI_T_cvar_read(pml_handle, pml_value); MPI_T_cvar_read(btl_handle, btl_value); printf("Set value of cvars: PML: %s, BTL: %s\n", pml_value, btl_value); MPI_T_cvar_handle_free(&pml_handle); MPI_T_cvar_handle_free(&btl_handle); MPI_Init(NULL, NULL); // ... MPI_Finalize(); MPI_T_finalize(); ``` ```