### Running CCTBX Regression Tests on Unix

Source: https://github.com/cctbx/cctbx_project/blob/master/dox/rst/installation.txt

This sequence of commands first sources a comprehensive environment setup script and then executes specific regression test suites for CCTBX and SCITBX. The `--Quick` flag can be used to run a faster subset of tests.

```Csh
source setpaths_all.csh
libtbx.python $SCITBX_DIST/run_tests.py
libtbx.python $CCTBX_DIST/run_tests.py --Quick
```

--------------------------------

### Initializing CCTBX Build Directory on Unix

Source: https://github.com/cctbx/cctbx_project/blob/master/dox/rst/installation.txt

This sequence of commands creates a new directory for the CCTBX build, navigates into it, and then runs the `configure.py` script using the previously installed Python. This initializes the build environment and generates necessary configuration files like `setpaths.csh`.

```Shell
mkdir cctbx_build
cd cctbx_build
/your/choice/bin/python ../cctbx_sources/libtbx/configure.py mmtbx
```

--------------------------------

### Installing Python from Source on Unix

Source: https://github.com/cctbx/cctbx_project/blob/master/dox/rst/installation.txt

This snippet demonstrates how to manually install Python from a gzipped tarball on a Unix-like system. It involves decompressing the archive, navigating into the extracted directory, configuring the build with a custom installation prefix, and then compiling and installing Python.

```Shell
gunzip -c Python-2.7.2.tar.gz | tar xf -
cd Python-2.7.2
./configure --prefix=/your/choice
make
make install
```

--------------------------------

### Cloning CCTBX Project from Git Repository

Source: https://github.com/cctbx/cctbx_project/blob/master/dox/rst/installation.txt

This sequence of commands fetches and executes a C-shell script that sets up a CCTBX development environment. It clones the `cctbx_project` from GitHub and obtains third-party dependencies from nightly builds.

```Shell
wget https://raw.githubusercontent.com/cctbx/cctbx_project/master/libtbx/development/cctbx_svn_getting_started.csh
./cctbx_svn_getting_started.csh
```

--------------------------------

### Installing and Running Fable on Windows Systems

Source: https://github.com/cctbx/cctbx_project/blob/master/fable/doc/index.txt

This snippet outlines the steps for installing and running Fable on Windows systems. It involves executing the downloaded installer, running a batch file to set up environment paths, and finally running an example conversion using the `fable.cout` command.

```Batch
fable_win_xp.exe
fable_build\setpaths.bat
fable.cout --example
```

--------------------------------

### Running GLTBX wxPython Example

Source: https://github.com/cctbx/cctbx_project/blob/master/gltbx/README.txt

After successful compilation and installation, this command executes a wxPython example script located within the `gltbx` module. It demonstrates the functionality of the `gltbx` installation, particularly its integration with wxPython for graphical scene rendering.

```Shell
python ../gltbx/wx_scene.py
```

--------------------------------

### Initializing CCTBX Build Directory on Windows

Source: https://github.com/cctbx/cctbx_project/blob/master/dox/rst/installation.txt

This sequence of commands creates a build directory, navigates into it, and then runs the CCTBX configuration script using a specific Python executable path. This sets up the build environment and generates `setpaths.bat`.

```Batch
mkdir cctbx_build
cd cctbx_build
C:\python27\python.exe ..\cctbx_sources\libtbx\configure.py mmtbx
```

--------------------------------

### Running CCTBX Bootstrap Script

Source: https://github.com/cctbx/cctbx_project/blob/master/README.md

Executes the `bootstrap.py` script to initiate the CCTBX project installation and dependency setup. This command starts the build process for a development version of CCTBX.

```Python
python bootstrap.py
```

--------------------------------

### Compiling CCTBX Modules on Unix

Source: https://github.com/cctbx/cctbx_project/blob/master/dox/rst/installation.txt

This command initiates the compilation of CCTBX modules using the `make` utility. It internally invokes `libtbx.scons` and utilizes all available CPU cores by default to speed up the build process.

```Shell
make
```

--------------------------------

### Configuring and Building CCTBX from Sources

Source: https://github.com/cctbx/cctbx_project/blob/master/dox/rst/installation.txt

These commands outline the process for configuring and building the CCTBX project from a local source directory. It involves creating a build directory, running the configuration script with a specific Python interpreter, sourcing environment paths, and compiling the project.

```Shell
mkdir build
cd build
/your/choice/bin/python ../sources/cctbx_project/libtbx/configure.py mmtbx
source setpaths.csh
make
```

--------------------------------

### Loading Example PDB and MTZ Files (Python)

Source: https://github.com/cctbx/cctbx_project/blob/master/cctbx_website/html_files/script_rfactors.html

This code demonstrates how to locate and load example PDB and MTZ files that are distributed with the Phenix installation. It uses libtbx.load_env to find files within the Phenix regression test suite, providing a convenient way to access sample data for testing or demonstration purposes.

```Python
import os
import libtbx.load_env
pdb_file = libtbx.env.find_in_repositories(
  relative_path="phenix_regression/pdb/1yjp_h.pdb",
  test=os.path.isfile)
mtz_file = libtbx.env.find_in_repositories(
  relative_path="phenix_regression/reflection_files/1yjp.mtz",
  test=os.path.isfile)
assert (not None in [pdb_file, mtz_file])
```

--------------------------------

### Unpacking CCTBX Source Bundle on Unix

Source: https://github.com/cctbx/cctbx_project/blob/master/dox/rst/installation.txt

This command extracts the gzipped CCTBX source tarball on a Unix-like system. It creates a subdirectory named `cctbx_sources` containing all the necessary source files for manual compilation.

```Shell
tar zxf cctbx_bundle.tar.gz
```

--------------------------------

### Compiling CCTBX Modules on Windows

Source: https://github.com/cctbx/cctbx_project/blob/master/dox/rst/installation.txt

This command initiates the compilation of CCTBX modules on Windows using the `libtbx.scons` build system. By default, it will attempt to use all available CPU cores for the build process.

```Batch
libtbx.scons
```

--------------------------------

### Running CCTBX Regression Tests (Windows)

Source: https://github.com/cctbx/cctbx_project/blob/master/dox/rst/installation.txt

These commands run regression tests for SCITBX and CCTBX components after compilation. `setpaths_all.bat` sets up the environment, and subsequent commands execute specific test suites. The `--Quick` flag runs a subset of tests for faster execution.

```Batch
setpaths_all.bat
libtbx.python %SCITBX_DIST%\run_tests.py
libtbx.python %CCTBX_DIST%\run_tests.py --Quick
```

--------------------------------

### Installing and Running Fable on Unix Systems

Source: https://github.com/cctbx/cctbx_project/blob/master/fable/doc/index.txt

This snippet provides the command-line steps to download, extract, configure environment paths, and execute an example conversion using Fable on Unix-like operating systems. It utilizes `wget` for download, `perl` for extraction, and `source` to set up the necessary environment variables for `fable.cout`.

```Shell
wget http://cci.lbl.gov/fable_bundles/current/fable_bundle.selfx
perl fable_bundle.selfx
source fable_build/setpaths.sh
fable.cout --example
```

--------------------------------

### Installing LatticeRepLib

Source: https://github.com/cctbx/cctbx_project/blob/master/cctbx/uctbx/lrl/README.md

This sequence of shell commands outlines the steps to clone the LatticeRepLib repository from GitHub, switch to the cctbx_build branch, set up the conda environment paths, configure the project using libtbx, and finally build it within a cctbx installation.

```Shell
$ cd $modules
$ git clone git@github.com:dwpaley/LatticeRepLib
$ pushd LatticeRepLib; git checkout cctbx_build; popd
$ cd ../build
$ source conda_setpaths.sh
$ libtbx.configure LatticeRepLib
$ make
```

--------------------------------

### Example `process.phil` Configuration for `diffBragg.stills_process` (PHIL)

Source: https://github.com/cctbx/cctbx_project/blob/master/simtbx/diffBragg/README.md

This `process.phil` file provides a comprehensive configuration for `diffBragg.stills_process`. It defines parameters for spot finding (e.g., `threshold.algorithm`, `gain`, `global_threshold`), indexing (e.g., `method`, `known_symmetry`, `refinement_protocol`), and integration. It also includes settings for input, output, and dispatch, enabling fine-grained control over the data processing pipeline.

```PHIL
spotfinder {
  threshold.algorithm=dispersion
  threshold.dispersion.gain=28
  threshold.dispersion.global_threshold=40
  threshold.dispersion.kernel_size=[2,2]
  threshold.dispersion.sigma_strong=1
  threshold.dispersion.sigma_background=6
  filter.min_spot_size=2
}

indexing {
  method=fft1d
  known_symmetry.unit_cell=79.1,79.1,38.4,90,90,90
  known_symmetry.space_group=P43212
  refinement_protocol.mode=None
  stills {
    rmsd_min_px=4 
    refine_all_candidates=False 
  }
}

integration.summation.detector_gain=28
input.sync_reference_geom = False
output.composite_output = False
dispatch.hit_finder.enable=False
```

--------------------------------

### Starting Interactive CCTBX Docker Container (Bash)

Source: https://github.com/cctbx/cctbx_project/blob/master/xfel/small_cell/README.md

This command starts an interactive Docker container for CCTBX, configuring X11 display forwarding to the host IP and mounting the current working directory. This setup allows users to run graphical applications and access local files from within the container.

```Bash
docker run -it -e DISPLAY=192.168.1.225:0 -v $PWD:/cwd dwpaley/cctbx:20211012
```

--------------------------------

### Example SHELXL Instruction File for Structure Solution (SHELXL)

Source: https://github.com/cctbx/cctbx_project/blob/master/xfel/small_cell/README.md

This SHELXL .ins file provides an example setup for structure solution, specifically for 'mithrene' in space group C2/c. It defines the unit cell parameters, Z value, lattice type, symmetry operations, scattering factors, and displacement parameters for various atoms. It also specifies the reflection file format.

```SHELXL
TITL mithrene
CELL 1.032066 5.93762 7.32461 29.202 90 95.4404 90
ZERR 8 0 0 0 0 0 0
LATT 7
SYMM -X,+Y,0.5-Z
SFAC C H Se Ag
DISP Ag -0.2289 2.1223 13899.9697
DISP C 0.0087 0.0039 30.8059
DISP H -0 0 0.6385
DISP Se -2.5865 0.5775 2790.4387
UNIT 80 72 8 8
HKLF 4
```

--------------------------------

### Compiling and Running the ucif Example Parser

Source: https://github.com/cctbx/cctbx_project/blob/master/ucif/examples/ucif_example.txt

These shell commands provide instructions for cloning the `cctbx_project` repository, navigating to the `ucif` directory, building the example CIF parser using a shell script, and then executing the compiled parser with a sample CIF file. This demonstrates the end-to-end workflow for using the `ucif` library.

```Shell
git clone https://github.com/cctbx/cctbx_project.git
cd cctbx_project/ucif
sh build_cif_parser.sh
./cif_parser myfile.cif
```

--------------------------------

### Running CCTBX Scripts with libtbx.python

Source: https://github.com/cctbx/cctbx_project/blob/master/dox/rst/installation.txt

This command is used to execute Python scripts that import CCTBX modules. It ensures that the necessary CCTBX environment is correctly set up for the script to run.

```Shell
libtbx.python your_script.py
```

--------------------------------

### Unpacking CCTBX Source Bundle on Windows

Source: https://github.com/cctbx/cctbx_project/blob/master/dox/rst/installation.txt

This command executes the self-extracting CCTBX source bundle on Windows. It unpacks the source files into a subdirectory named `cctbx_sources` in the current directory.

```Batch
cctbx_bundle.exe
```

--------------------------------

### Cloning cxid9114 Repository and Fetching LFS Data

Source: https://github.com/cctbx/cctbx_project/blob/master/simtbx/diffBragg/README.md

This command sequence clones the `cxid9114` GitHub repository, which contains test data and scripts for diffBragg examples. It then navigates into the cloned directory and uses `git-lfs` to fetch large data files, which are essential for running the simulations and examples. Users on NERSC are advised to load the `git-lfs` module.

```Shell
cd $MOD
git clone https://github.com/dermen/cxid9114.git
# install git-lfs (if on nersc, just load the module
cd ~/Crystal/modules/cxid9114
module load git-lfs
cd ~/Crystal/modules/cxid9114
git lfs install
git lfs fetch
git lfs pull
```

--------------------------------

### Setting Up and Running cctbx.xfel Regression Tests

Source: https://github.com/cctbx/cctbx_project/blob/master/xfel/conda_envs/README.md

This snippet provides a comprehensive guide to set up the cctbx.xfel regression test environment. It involves cloning multiple Git repositories, installing Git LFS for large file handling, configuring the necessary modules with libtbx, and executing parallel tests with specified thread and process counts.

```Shell
$ cd modules
$ conda install -c conda-forge git-lfs
$ git clone https://gitlab.com/cctbx/xfel_regression.git
$ git clone https://github.com/nksauter/LS49.git
$ git clone https://gitlab.com/cctbx/ls49_big_data.git
$ cd xfel_regression
$ git lfs install --local
$ git lfs pull
$ cd ../uc_metrics
$ git lfs install --local
$ git lfs pull
$ cd ../ls49_big_data
$ git lfs install --local
$ git lfs pull
$ cd ../../
$ mkdir test; cd test
$ libtbx.configure xfel_regression LS49 ls49_big_data
$ export OMP_NUM_THREADS=4
$ libtbx.run_tests_parallel module=uc_metrics module=simtbx module=xfel_regression module=LS49 nproc=64
```

--------------------------------

### Configuring cctbx Project on Windows

Source: https://github.com/cctbx/cctbx_project/blob/master/sphinx/installation.rst

This snippet outlines the initial steps to configure the cctbx project. It creates a build directory, navigates into it, and then runs the configure.py script using a specific Python executable to set up the project for mmtbx.

```Batch
mkdir cctbx_build
cd cctbx_build
C:\python27\python.exe ..\cctbx_sources\libtbx\configure.py mmtbx
```

--------------------------------

### Complete Example: Scaling Parameters for cctbx.xfel

Source: https://github.com/cctbx/cctbx_project/blob/master/xfel/merging/README.md

This comprehensive example demonstrates a full set of parameters for the scaling and post-refinement process in cctbx.xfel. It includes input file specifications, detailed filtering and selection criteria, scaling model parameters, merging settings (relevant for the output format), post-refinement enablement, and output configuration, ensuring data is prepared for subsequent merging.

```cctbx parameters
input.path=<path to data subset>
input.experiments_suffix=.expt
input.reflections_suffix=.refl
dispatch.step_list=input balance model_scaling modify filter scale postrefine statistics_unitcell statistics_beam model_statistics statistics_resolution
input.parallel_file_load.method=uniform
filter.outlier.min_corr=-1
filter.algorithm=unit_cell
filter.unit_cell.value.relative_length_tolerance=0.03
select.algorithm=significance_filter
select.significance_filter.sigma=0.1
select.significance_filter.min_ct=200
select.significance_filter.max_ct=300
scaling.model=<model or mtz file>
scaling.resolution_scalar=0.96
merging.d_min=1.8
merging.merge_anomalous=True
postrefinement.enable=True
statistics.n_bins=20
output.do_timing=True
output.save_experiments_and_reflections=True
output.do_timing=True
output.output_dir=<path to folder for output data>
output.prefix<text to prepend to output files>
```

--------------------------------

### Sourcing CCTBX Environment in Windows

Source: https://github.com/cctbx/cctbx_project/blob/master/dox/rst/installation.txt

This command executes the `setpaths.bat` batch file, which configures the necessary environment variables and paths for CCTBX within the current Windows command prompt session. This is essential for subsequent build and execution steps.

```Batch
setpaths.bat
```

--------------------------------

### Installing IOTA Module

Source: https://github.com/cctbx/cctbx_project/blob/master/iota/README.md

This command installs the IOTA module from its new repository location. It places the module within the `modules/` directory of the current development environment, ensuring the latest version is available.

```bash
libtbx.install iota
```

--------------------------------

### Starting cctbx-compatible Python Environment

Source: https://github.com/cctbx/cctbx_project/blob/master/cctbx_website/html_files/doc_hlo_intro.html

This command initiates a Python interpreter specifically configured for cctbx, ensuring all necessary cctbx modules are accessible. It's crucial for running cctbx-related scripts and commands.

```Shell
libtbx.python
```

--------------------------------

### Installing Python on Linux/macOS from Source

Source: https://github.com/cctbx/cctbx_project/blob/master/sphinx/installation.rst

This snippet demonstrates how to manually install a specific Python version (e.g., 2.7.13) from its source tarball on Linux or macOS. It involves extracting the archive, navigating into the directory, configuring the build with a custom installation prefix, and then compiling and installing Python.

```Shell
tar -xf Python-2.7.13.tar.xz
cd Python-2.7.13
./configure --prefix=/your/choice
make
make install
```

--------------------------------

### Creating Symbolic Link for CCTBX Bootstrap Script

Source: https://github.com/cctbx/cctbx_project/blob/master/sphinx/installation.rst

This snippet shows how to replace the initially downloaded `bootstrap.py` with a symbolic link to the version located within the `modules` directory of the CCTBX installation. This ensures that the `bootstrap.py` in your installation directory stays synchronized with updates from the CCTBX source.

```Shell
cd <installation directory>
rm bootstrap.py
ln -s ./modules/cctbx_project/libtbx/auto_build/bootstrap.py
```

--------------------------------

### Compiling CCTBX with Specific CPU Cores on Windows

Source: https://github.com/cctbx/cctbx_project/blob/master/dox/rst/installation.txt

This command explicitly runs the `libtbx.scons` build system to compile CCTBX on Windows, allowing the user to specify the number of CPU cores to utilize for the compilation process using the `-j` flag.

```Batch
libtbx.scons -j N
```

--------------------------------

### Compiling CCTBX with Specific CPU Cores on Unix

Source: https://github.com/cctbx/cctbx_project/blob/master/dox/rst/installation.txt

This command explicitly runs the `libtbx.scons` build system to compile CCTBX, allowing the user to specify the number of CPU cores to utilize for the compilation process. The `-j` flag controls the parallelism.

```Shell
libtbx.scons -j 4
```

--------------------------------

### Installing websocket_server for cctbx.python

Source: https://github.com/cctbx/cctbx_project/blob/master/crys3d/hklviewer/ReadMe.txt

This command installs the `websocket_server` package, which is a prerequisite for `crys3d.hklview` to function correctly, enabling communication for the web-based visualization.

```Shell
cctbx.python -m pip install websocket_server
```

--------------------------------

### Importing Core Dependencies for Multiprocessing Setup - Python

Source: https://github.com/cctbx/cctbx_project/blob/master/cctbx_website/html_files/doc_programming_tips_3.html

This snippet imports essential modules for the multiprocessing example: `print_function` for Python 2/3 compatibility, `sys` for system-specific parameters, and `group_args` from `libtbx` for returning structured data.

```Python
from __future__ import print_function
import sys
from libtbx import group_args
```

--------------------------------

### Initializing cctbx Environment Paths

Source: https://github.com/cctbx/cctbx_project/blob/master/sphinx/installation.rst

This command executes the setpaths.bat file, which is generated during the configuration step. It initializes the current shell or process with the necessary environment variables and paths required for cctbx.

```Batch
setpaths.bat
```

--------------------------------

### Creating Panel Group Files (Python)

Source: https://github.com/cctbx/cctbx_project/blob/master/simtbx/diffBragg/README.md

This Python script demonstrates how to programmatically create different panel group mapping files (`single_panel.txt`, `cspad_32panel.txt`, `cspad_quads.txt`). These files define how detector panels are grouped for geometry refinement in `diffBragg`, with examples for treating all panels as one group, 32 groups, or 4 quadrants.

```Python
with open("single_panel.txt", "w") as o:
    for pid in range(64):
        o.write("%d %d\n" % (pid, 0))

with open("cspad_32panel.txt", "w") as o:
    for pid in range(64):
        groupid = int(pid/2)
        o.write("%d %d\n" % (pid, groupid))

with open("cspad_quads.txt", "w") as o:
    for pid in range(64):
        groupid = int(pid/16)
        o.write("%d %d\n" % (pid, groupid))
```

--------------------------------

### Example Output of Sites Coordinates in Python

Source: https://github.com/cctbx/cctbx_project/blob/master/scitbx/examples/rigid_body_refinement_core.txt

This snippet shows an example of the output when printing the `sites` list. It illustrates the typical format of the coordinates as a list of Python tuples, each representing a 3D point with floating-point values.

```Python
[(10.949, 12.815, 15.189),
   (10.404999999999999, 13.954000000000001, 15.917),
   (10.779, 15.262, 15.227)]
```

--------------------------------

### Viewing Simulated Background Image with DIALS

Source: https://github.com/cctbx/cctbx_project/blob/master/simtbx/diffBragg/README.md

This snippet first installs a custom image format reader (`FormatD9114.py`) required by `dxtbx` to properly interpret the simulated data. Subsequently, it uses `dials.image_viewer` to display the generated `mybackground.h5` image, setting the brightness for optimal viewing.

```Shell
dxtbx.install_format  -u $MOD/cxid9114/format/FormatD9114.py
dials.image_viewer mybackground.h5 brightness=75
```

--------------------------------

### Running Fable Example in Docker (Bash)

Source: https://github.com/cctbx/cctbx_project/blob/master/fable/docker/README.md

Runs Fable inside the Docker container to generate an example output. It mounts the current working directory (`$(pwd)`) into the container to allow Fable to write `fable.cout` and `fable_cout.cpp` directly to the host filesystem.

```bash
docker run -i -w $(pwd) --volume=$(pwd):$(pwd):rw -t ${DOCK_NAME} --example
```

--------------------------------

### Complete Example: Merging Parameters for cctbx.xfel

Source: https://github.com/cctbx/cctbx_project/blob/master/xfel/merging/README.md

This example provides a complete set of parameters for merging scaled data using cctbx.xfel. It specifies multiple input paths for scaled subsets, defines the dispatch list for merging operations, includes scaling model and resolution parameters, and configures output settings. It also highlights the use of the ev11 error model for merging.

```cctbx parameters
input.path=<path to scaling output for subset 1>
input.path=<path to scaling output for subset 2, etc.>
input.experiments_suffix=.expt
input.reflections_suffix=.refl
dispatch.step_list=input model_scaling statistics_unitcell statistics_beam model_statistics statistics_resolution group errors_merge statistics_intensity merge statistics_intensity_cxi
input.parallel_file_load.method=uniform
scaling.model=<model or mtz file>
scaling.resolution_scalar=0.96
statistics.n_bins=20
merging.d_min=1.8
merging.merge_anomalous=True
merging.error.model=ev11
output.do_timing=True
output.output_dir=<path to folder for output data>
output.prefix<text to prepend to output files>
```

--------------------------------

### Example Configuration for cctbx.xfel.merge (PHIL)

Source: https://github.com/cctbx/cctbx_project/blob/master/xfel/merging/README.md

This PHIL file provides a comprehensive example of configuring `cctbx.xfel.merge` for data processing. It sets parameters for input file paths, filtering criteria (e.g., outlier correlation, unit cell tolerance), selection algorithms, scaling options (e.g., reference model, resolution scalar), and merging parameters (e.g., d_min, error model, anomalous merging).

```PHIL
input.path=<path to data>
input.experiments_suffix=.expt
input.reflections_suffix=.refl
filter.outlier.min_corr=0.1
filter.algorithm=unit_cell
filter.unit_cell.value.relative_length_tolerance=0.03
select.algorithm=significance_filter
select.significance_filter.sigma=0.1
select.significance_filter.min_ct=200
select.significance_filter.max_ct=300
scaling.model=<model.pdb or model.mtz>
scaling.resolution_scalar=0.96
merging.d_min=1.8
merging.error.model=ev11
merging.merge_anomalous=True
postrefinement.enable=True
statistics.n_bins=20
output.do_timing=True
output.output_dir=<path to folder for output data>
output.prefix<text to prepend to output files>
```

--------------------------------

### Using Absolute Python Imports (Good Example)

Source: https://github.com/cctbx/cctbx_project/blob/master/libtbx/development/dev_guidelines.txt

This example demonstrates the recommended practice of using absolute imports in Python. Absolute imports prevent confusion, make code more portable, and clearly indicate the origin of imported modules, improving readability and maintainability.

```Python
from iotbx import reflection_file_reader
```

--------------------------------

### Sourcing CCTBX Paths and Starting Python in Docker

Source: https://github.com/cctbx/cctbx_project/blob/master/dox/rst/debug.md

These commands are executed inside the Docker container. The first sources the CCTBX environment paths, ensuring that `libtbx.python` and other CCTBX tools are available. The second command then starts the `libtbx.python` interpreter, which is the CCTBX-specific Python executable.

```Bash
source /img/build_debug/conda_setpaths.sh
libtbx.python
```

--------------------------------

### Compiling cctbx Project

Source: https://github.com/cctbx/cctbx_project/blob/master/sphinx/installation.rst

This command initiates the compilation of the cctbx project using libtbx.scons, which is a wrapper around SCons. It compiles the entire project without specifying the number of parallel jobs.

```Shell
libtbx.scons
```

--------------------------------

### Creating a New cctbx-base Conda Environment

Source: https://github.com/cctbx/cctbx_project/blob/master/README.md

This command creates a new Conda environment named 'my_env' and installs the 'cctbx-base' package from the 'conda-forge' channel. This provides a minimal cctbx installation without additional GUI dependencies.

```Shell
conda create -n my_env -c conda-forge cctbx-base
```

--------------------------------

### Slicing Python List to Get Elements After N

Source: https://github.com/cctbx/cctbx_project/blob/master/cctbx_website/html_files/doc_programming_tips_2.html

Illustrates array slicing `[N:]` in Python to obtain all elements from a list starting from a specific index `N` to the end. This example gets elements from index 2 onwards.

```Python
print(a[2:])
```

--------------------------------

### Downloading Faulty Geometry File (wget)

Source: https://github.com/cctbx/cctbx_project/blob/master/simtbx/diffBragg/README.md

This command downloads a pre-prepared faulty experimental geometry file (`badGeo.expt`) from a specified URL. This file is used to simulate real experimental errors associated with CSPAD geometry for subsequent processing and refinement with diffBragg.

```Bash
wget https://smb.slac.stanford.edu/~dermen/diffBragg/badGeo.expt
```

--------------------------------

### Downloading quick_detresid.py Script (Shell)

Source: https://github.com/cctbx/cctbx_project/blob/master/simtbx/diffBragg/README.md

This shell command uses `wget` to download the `quick_detresid.py` Python script from a specified URL. This script is essential for analyzing the results of the `diffBragg` processing by calculating and visualizing detector residuals.

```bash
wget https://smb.slac.stanford.edu/~dermen/diffBragg/quick_detresid.py
```

--------------------------------

### Installing DIALS/CCTBX and GSASII with Conda/Mamba

Source: https://github.com/cctbx/cctbx_project/blob/master/xfel/small_cell/README.md

This snippet outlines the steps for a developer installation of DIALS/CCTBX and GSASII within a Conda environment. It addresses a specific issue with the `svn` conda-forge distribution by aliasing it to the system's default version, ensuring GSASII can be properly installed and its path added to Python's site-packages.

```Bash
svn --version # make sure there is a working system default version
old_svn=$(which svn)
conda activate $PWD/conda_base
mamba install svn -c conda-forge
ln -sf $old_svn $(which svn)
mamba install gsas2pkg -c briantoby -c conda-forge
echo $PWD/conda_base/GSASII > conda_base/lib/python3.*/site-packages/GSASII.pth
```

--------------------------------

### Adding Python Binaries to PATH (csh)

Source: https://github.com/cctbx/cctbx_project/blob/master/sphinx/installation.rst

This command adds the specified Python installation's binary directory to the system's PATH environment variable using the csh shell. This allows executables within `/your/choice/bin` to be run directly from the command line without specifying their full path.

```csh
set path=(/your/choice/bin $path)
```

--------------------------------

### Downloading and Running CCTBX Bootstrap Script (Linux/macOS)

Source: https://github.com/cctbx/cctbx_project/blob/master/sphinx/installation.rst

This sequence of commands creates a new directory, navigates into it, downloads the `bootstrap.py` script using `wget` from the CCTBX GitHub repository, and then executes it with Python. This script is used to download CCTBX sources and manage the build process.

```Shell
mkdir <installation directory>
cd <installation directory>
wget https://raw.githubusercontent.com/cctbx/cctbx_project/master/libtbx/auto_build/bootstrap.py
python ./bootstrap.py
```

--------------------------------

### Using Specific Python Imports (Good Example)

Source: https://github.com/cctbx/cctbx_project/blob/master/libtbx/development/dev_guidelines.txt

This example illustrates the preferred method of importing specific modules and then referencing their attributes. This approach avoids name clashes and makes it clear where symbols originate, enhancing code clarity and preventing subtle bugs.

```Python
from iotbx import mtz
mtz_object = mtz.object(file_name="any.mtz")
```

--------------------------------

### Downloading `process.phil` Configuration File (Shell)

Source: https://github.com/cctbx/cctbx_project/blob/master/simtbx/diffBragg/README.md

This command uses `wget` to download the `process.phil` configuration file from a specified URL. This file contains essential parameters for configuring `diffBragg.stills_process`, including spot finding, indexing, and integration settings, which are critical for subsequent data processing.

```Shell
wget https://smb.slac.stanford.edu/~dermen/diffBragg/process.phil
```

--------------------------------

### Slicing Python List with Negative Start Index

Source: https://github.com/cctbx/cctbx_project/blob/master/cctbx_website/html_files/doc_programming_tips_2.html

Explains slicing a Python list using a negative start index `[-1:]`, which retrieves elements from the end of the list. In this case, it gets the last element.

```Python
print( a[-1:])
```

--------------------------------

### Initializing a Python List

Source: https://github.com/cctbx/cctbx_project/blob/master/cctbx_website/html_files/doc_programming_tips_2.html

Demonstrates the initialization of a simple Python list with three integer values, which serves as the base array for subsequent slicing examples.

```Python
a = [5,6,7]
```

--------------------------------

### Initializing CCTBX Environment (csh)

Source: https://github.com/cctbx/cctbx_project/blob/master/sphinx/installation.rst

This command sources the `setpaths.csh` file, typically found in the `build` directory of a CCTBX installation. Sourcing this file initializes the current shell's environment variables, such as PATH and PYTHONPATH, to correctly point to the CCTBX binaries and libraries, making CCTBX commands available.

```csh
source setpaths.csh
```

--------------------------------

### Phil User File Default Output Example

Source: https://github.com/cctbx/cctbx_project/blob/master/libtbx/phil/doc.txt

This snippet shows an example of a Phil user file output, typically generated by an application (e.g., `minimization.quick --show_defaults`). It displays only the most relevant parameters, classified by the developer as expert_level = 0, without their associated attributes, resulting in a more concise view for end-users.

```Phil
minimization.parameters {
  method = *bfgs conjugate_gradient
  max_iterations = 10
}
```

--------------------------------

### Compiling cctbx Project with Multiple CPUs

Source: https://github.com/cctbx/cctbx_project/blob/master/sphinx/installation.rst

This command compiles the cctbx project using libtbx.scons, leveraging multiple CPU cores. 'N' should be replaced with the number of available CPUs to speed up the compilation process.

```Shell
libtbx.scons -j N
```

--------------------------------

### Unpacking CCTBX Source Bundle (Windows)

Source: https://github.com/cctbx/cctbx_project/blob/master/sphinx/installation.rst

On Windows, executing `cctbx_bundle.exe` in a new, empty directory unpacks the self-extracting archive containing the CCTBX source code. This creates a `cctbx_sources` subdirectory, providing the necessary files for building CCTBX on Windows.

```Shell
cctbx_bundle.exe
```

--------------------------------

### Creating Faulty Geometry File (Python)

Source: https://github.com/cctbx/cctbx_project/blob/master/simtbx/diffBragg/README.md

This Python script programmatically creates a `badGeo.expt` file using the `cxid9114` and `dxtbx` libraries. It initializes an `ExperimentList` with a `CSPAD2` detector model and saves it as a JSON file, providing an alternative to downloading the geometry file.

```Python
from cxid9114.geom.multi_panel import CSPAD2
from dxtbx.model import Experiment, ExperimentList
El = ExperimentList()
E = Experiment()
E.detector = CSPAD2
El.append(E)
El.as_json("badGeo.expt")
```

--------------------------------

### Generating a Background Image for Simulation

Source: https://github.com/cctbx/cctbx_project/blob/master/simtbx/diffBragg/README.md

This command uses the `d9114_mpi_sims.py` script to create a background image, which is a prerequisite for simulating diffraction patterns. It specifies the output directory as the current directory and names the background file `mybackground.h5`, enabling the background creation mode.

```Shell
libtbx.python $MOD/cxid9114/sim/d9114_mpi_sims.py -odir . --bg-name mybackground.h5 --make-background   --sad
```

--------------------------------

### LBFGS Target Evaluator Class Example in scitbx (Python)

Source: https://github.com/cctbx/cctbx_project/blob/master/cctbx/examples/unit_cell_refinement.txt

This Python class serves as a `target_evaluator` for the `scitbx.lbfgs.run` minimizer. It demonstrates the required structure for an LBFGS target evaluator, including an `__init__` method to initialize parameters and start the minimization, a `compute_functional_and_gradients` method to return the current functional value and its gradients, and an optional `callback_after_step` method for monitoring minimization progress. The `x` attribute holds the parameters to be refined.

```Python
class refinery:

    def __init__(self):
      self.x = flex.double([0])
      scitbx.lbfgs.run(target_evaluator=self)

    def compute_functional_and_gradients(self):
      f = 0
      g = flex.double([0])
      return f, g

    def callback_after_step(self, minimizer):
      pass
```

--------------------------------

### Downloading Panel Group File (wget)

Source: https://github.com/cctbx/cctbx_project/blob/master/simtbx/diffBragg/README.md

This command downloads a pre-defined panel group mapping file (`cspad_32panel.txt`) for the CSPAD detector. This file is crucial for `diffBragg.geometry_refiner` as it specifies how individual panels are grouped for refinement, which is necessary because `diffBragg` does not directly interpret `dxtbx` detector hierarchies.

```Bash
wget https://smb.slac.stanford.edu/~dermen/diffBragg/cspad_32panel.txt
```

--------------------------------

### Slicing Python List to Get First N Elements

Source: https://github.com/cctbx/cctbx_project/blob/master/cctbx_website/html_files/doc_programming_tips_2.html

Shows how to use array slicing `[:N]` in Python to extract the first `N` elements from a list. This example retrieves the first two elements of the list `a`.

```Python
print(a[:2])
```

--------------------------------

### Running DiffBragg Stills Process (Shell)

Source: https://github.com/cctbx/cctbx_project/blob/master/simtbx/diffBragg/README.md

This shell command executes the `diffBragg.stills_process` program. It enables CUDA for GPU acceleration (`DIFFBRAGG_USE_CUDA=1`), uses `srun` for job submission with 2 CPU cores, specifies the PHIL configuration file (`process.phil`), provides input image paths (`"poly_images/job*/*.h5"`), sets the multiprocessing method to MPI with 4 devices, enables integration, and defines the output directory (`poly_images/procPoly`). Command line parameters override PHIL file settings.

```bash
DIFFBRAGG_USE_CUDA=1 srun -c2  diffBragg.stills_process process.phil "poly_images/job*/*.h5" mp.method=mpi num_devices=4  dispatch.integrate=True output.output_dir=poly_images/procPoly
```

--------------------------------

### Configuring diffBragg Logging to Stderr (Python)

Source: https://github.com/cctbx/cctbx_project/blob/master/simtbx/diffBragg/README.md

This Python snippet configures the `diffBragg.main` logger to output debug-level messages to standard error. It's a basic setup to enable verbose logging for debugging `diffBragg` operations within a script.

```Python
import logging
dblogger = logging.getLogger("diffBragg.main")
dblogger.setLevel(logging.DEBUG)
```

--------------------------------

### Sourcing CCTBX Environment in Csh

Source: https://github.com/cctbx/cctbx_project/blob/master/dox/rst/installation.txt

This command sources the `setpaths.csh` script, which sets up the necessary environment variables and paths for CCTBX in the current csh shell session. This is crucial for subsequent compilation and execution of CCTBX tools.

```Csh
source setpaths.csh
```

--------------------------------

### Setting up and Testing simtbx with LS49 (Shell)

Source: https://github.com/cctbx/cctbx_project/blob/master/simtbx/README.md

This snippet provides a sequence of shell commands to set up the development environment for cctbx + dials, clone the LS49 repository, configure and build LS49, download necessary big data, and finally run parallel tests for both LS49 and simtbx. It requires a Linux system with at least 16 cores and specific environment variables like DEVELOPER_DIR, BUILD_LS49, DATA_DIR, and TEST to be set.

```Shell
cd ${DEVELOPER_DIR} # the normal location for bootstrap.py and the modules directory for source code
python bootstrap.py hot update --builder=dials
cd modules
git clone git@github.com:nksauter/LS49.git

cd ${BUILD_LS49} # create and change to an empty build directory; use conda if appropriate
python ../modules/cctbx_project/libtbx/configure.py LS49 prime iota --enable_openmp_if_possible=True --use_conda
source setpaths.sh
make
export OMP_NUM_THREADS=16 # or higher

cd ${DATA_DIR} # create an empty directory and download big data file from Nick Sauter
tar xzf ls49.tgz
export LS49_BIG_DATA=${DATA_DIR}/ls49_big_data

cd ${TEST} # create an empty directory for test execution
rm *
libtbx.run_tests_parallel module=LS49 nproc=15
rm *
libtbx.run_tests_parallel module=simtbx nproc=5
```

--------------------------------

### Initializing iotbx.map_model_manager in Python

Source: https://github.com/cctbx/cctbx_project/blob/master/cctbx_website/html_files/doc_hlo_intro.html

This snippet imports the `map_model_manager` class from `iotbx.map_model_manager` and then creates an initialized instance of it, assigned to the variable `mmm`. This object serves as an empty container for managing map and model data within cctbx.

```Python
from iotbx.map_model_manager import map_model_manager
mmm=map_model_manager()
```

--------------------------------

### Running CCTBX Python Scripts on Unix

Source: https://github.com/cctbx/cctbx_project/blob/master/dox/rst/installation.txt

This command executes a Python script within the CCTBX environment, ensuring that all necessary CCTBX modules and dependencies are correctly loaded. It acts as a wrapper around the standard Python interpreter.

```Shell
libtbx.python your_script.py
```

--------------------------------

### Downloading diffBragg Geometry Configuration File (PHIL) - Bash

Source: https://github.com/cctbx/cctbx_project/blob/master/simtbx/diffBragg/README.md

This command downloads the `geom.phil` configuration file from a specified URL. This file contains parameters necessary for configuring the geometry refinement process in diffBragg, such as optimization methods, restraint settings, and simulation parameters.

```Bash
wget https://smb.slac.stanford.edu/~dermen/diffBragg/geom.phil
```

--------------------------------

### Preparing Model for Explicit Origin Shift in cctbx Python

Source: https://github.com/cctbx/cctbx_project/blob/master/cctbx_website/html_files/doc_maps_boxing.html

This snippet re-applies the shift to set the model's origin back to zero, serving as a setup step to clearly demonstrate the effect of `shift_any_model_to_match` from a known starting point.

```Python
m.shift_model_and_set_crystal_symmetry(shift_to_apply)
print(m)
```

--------------------------------

### Example cctbx.xfel Data Locator File

Source: https://github.com/cctbx/cctbx_project/blob/master/xfel/conda_envs/README.md

This snippet shows the content of a simple locator file (`example.loc`) used by cctbx.xfel and dials commands. It specifies the experiment name, run number, and detector address, enabling the software to find and process specific crystallographic data. More complex detectors may require additional options.

```Text
experiment=cxi78513
run=22
detector_address=CxiDs1.0:Cspad.0
```

--------------------------------

### Setting Initial Minimizer Value - Python

Source: https://github.com/cctbx/cctbx_project/blob/master/cctbx_website/html_files/script_lbfgs_no_curvature.html

This line of code initializes the internal 'x' attribute of the minimizer object with the starting value provided by the 'calculator'. This ensures that the minimization process begins from the specified initial point, which in this example is (0,0).

```Python
self.x = self.calculator.x
```

--------------------------------

### Running cctbx Regression Tests

Source: https://github.com/cctbx/cctbx_project/blob/master/sphinx/installation.rst

This snippet demonstrates how to run regression tests for cctbx and scitbx. It first initializes all necessary paths using setpaths_all.bat and then executes specific test scripts for scitbx and cctbx (with a quick option) using libtbx.python.

```Batch
setpaths_all.bat
libtbx.python %SCITBX_DIST%\run_tests.py
libtbx.python %CCTBX_DIST%\run_tests.py --Quick
```

--------------------------------

### Downloading CCTBX Bootstrap Script with curl (macOS)

Source: https://github.com/cctbx/cctbx_project/blob/master/sphinx/installation.rst

On macOS, where `wget` might not be available by default, this command uses `curl` to download the `bootstrap.py` script from the CCTBX GitHub repository and saves it as `bootstrap.py` in the current directory. This script is essential for initiating the CCTBX build process.

```Shell
curl https://raw.githubusercontent.com/cctbx/cctbx_project/master/libtbx/auto_build/bootstrap.py > bootstrap.py
```

--------------------------------

### Identifying and Exporting CCTBX Modules Path

Source: https://github.com/cctbx/cctbx_project/blob/master/simtbx/diffBragg/README.md

This snippet demonstrates how to programmatically find the root directory of your CCTBX installation's modules and then export it as an environment variable named `MOD`. This variable is crucial for subsequent commands to locate necessary scripts and data within the CCTBX environment.

```Shell
libtbx.python -c 'import cctbx;print(cctbx.__file__.split("/cctbx_project")[0])'
/Users/dermen/CrystalNew/modules
export MOD=/Users/dermen/CrystalNew/modules
```

--------------------------------

### Running CCTBX Regression Tests (csh)

Source: https://github.com/cctbx/cctbx_project/blob/master/sphinx/installation.rst

This snippet demonstrates how to run regression tests for CCTBX. It first sources `setpaths_all.csh` to set up the complete CCTBX environment, then executes test scripts for `SCITBX` and `CCTBX` using `libtbx.python`, with the `--Quick` option for faster execution of CCTBX tests.

```csh
source build/setpaths_all.csh
libtbx.python $SCITBX_DIST/run_tests.py
libtbx.python $CCTBX_DIST/run_tests.py --Quick
```

--------------------------------

### Generating Example Map and Model with cctbx Python

Source: https://github.com/cctbx/cctbx_project/blob/master/cctbx_website/html_files/doc_hlo_model_map_manager.html

This snippet demonstrates how to generate a sample crystallographic map and model using the iotbx.map_model_manager for tutorial purposes. It initializes the manager, generates a default map and model, and then writes them to 'map.mrc' and 'model.pdb' files respectively. This provides the necessary input files for subsequent manager operations.

```Python
from iotbx.map_model_manager import map_model_manager      #   load in the map_model_manager
mmm=map_model_manager()         #   get an initialized instance of the map_model_manager
mmm.generate_map()              #   get a model from a small library model and calculate a map for it
mmm.write_map("map.mrc")        #   write out a map in ccp4/mrc format
mmm.write_model("model.pdb")    #   write out a model in PDB format
```

--------------------------------

### Retrieving `flex.double` Array 'Last' Corner (Actual End) in Python

Source: https://github.com/cctbx/cctbx_project/blob/master/cctbx_website/html_files/doc_low_flex_advanced.html

This example shows how to get the actual end coordinates of a `flex.double()` array by passing `False` to the `last()` method. This returns the coordinates of the last available element in each dimension, representing the true corner of the map data.

```Python
print(map_data.last(False)) # prints (29, 39, 31) corner of available map
```

--------------------------------

### Adding Python Executable to PATH in Csh

Source: https://github.com/cctbx/cctbx_project/blob/master/dox/rst/installation.txt

This command adds the specified Python binary directory to the shell's PATH environment variable in a csh shell. This allows Python executables to be run without specifying their full path, making it convenient for command-line usage.

```Csh
set path=(/your/choice/bin $path)
```

--------------------------------

### Executing direct_methods_light.py Script via Command Line

Source: https://github.com/cctbx/cctbx_project/blob/master/iotbx/examples/direct_methods_light.txt

This command demonstrates how to run the `direct_methods_light.py` script using `iotbx.python`, passing `vj1132Isup2.hkl` as the required first command-line argument for diffraction data processing. This is the typical way to invoke the script from a shell.

```Shell
iotbx.python direct_methods_light.py vj1132Isup2.hkl
```

--------------------------------

### Retrieving CCTBX Map Last Grid Point (Python)

Source: https://github.com/cctbx/cctbx_project/blob/master/cctbx_website/html_files/doc_maps_intro.html

This snippet shows how to get the last available grid point in the map using `last(False)` and the start of the next unit cell using `last()` (or `last(True)`). Understanding these points is crucial for iterating over or defining map boundaries.

```Python
print(map_data.last(False))     # prints (29, 39, 31) last available point
print(map_data.last())          # prints (30, 40, 32) start of next unit cell
```

--------------------------------

### Executing diffBragg Geometry Refinement with CUDA - Bash

Source: https://github.com/cctbx/cctbx_project/blob/master/simtbx/diffBragg/README.md

This command initiates the diffBragg geometry refinement process, leveraging CUDA for accelerated computation. It uses the `geom.phil` configuration file and specifies command-line overrides for the output detector name (`optGeo.expt`), input data pickle (`hopper_process_summary.pkl`), maximum L-BFGS iterations (`2000`), number of devices (`4`), maximum processes (`20`), and output directory (`geom`).

```Bash
DIFFBRAGG_USE_CUDA=1 srun -c2 diffBragg.geometry_refiner --phil geom.phil --cmdlinePhil  optimized_detector_name=optGeo.expt input_pkl=poly_images/procBad/hopper_process_summary.pkl lbfgs_maxiter=2000 num_devices=4 max_process=20 outdir=geom
```

--------------------------------

### Slicing Python List with Negative Start and Positive End Index

Source: https://github.com/cctbx/cctbx_project/blob/master/cctbx_website/html_files/doc_programming_tips_2.html

Highlights a non-obvious behavior when slicing with a negative start index and a positive end index. This combination often results in an empty list if the effective start index is greater than or equal to the end index.

```Python
print( a[-1:1])
```

--------------------------------

### Chaining X-ray Structure Display Methods in CCTBX (Python)

Source: https://github.com/cctbx/cctbx_project/blob/master/iotbx/examples/direct_methods_light.txt

This example demonstrates method chaining, allowing multiple display methods to be called sequentially on the same 'xray_structure' object in a single line. It first shows the structure summary and then immediately lists all scatterers.

```Python
xray_structure.show_summary().show_scatterers()
```