### Install EXtra Toolkit Source: https://context7.com/european-xfel/extra/llms.txt Install the EXtra toolkit on the Maxwell cluster using modules or via pip. Development installation includes tests and documentation. ```bash module load exfel exfel-python ``` ```bash pip install euxfel-extra ``` ```bash pip install -e '.[tests,docs]' ``` -------------------------------- ### Install EXtra with Pip Source: https://github.com/european-xfel/extra/blob/master/docs/index.md Install the euxfel-extra package using pip. This allows for self-installation of the library. ```bash pip install euxfel-extra ``` -------------------------------- ### Quick Example: Using the Proposal API Source: https://github.com/european-xfel/extra/blob/master/docs/proposal.md Demonstrates how to open a proposal, retrieve general information, sample tables, DAQ data, and DAMNIT data. ```python from extra.proposal import Proposal # Open a proposal proposal = Proposal(1234) # Print general information about the proposal proposal.info() # Get a table of samples and the runs they have proposal.samples_table() # Get DAQ data from extra-data proposal.data() # Get DAMNIT data through the DAMNIT API proposal.damnit() ``` -------------------------------- ### Install EXtra in Editable Mode Source: https://github.com/european-xfel/extra/blob/master/README.md Use this command to install the package in editable mode during development, including test and documentation dependencies. ```bash pip install -e '.[tests,docs]' ``` -------------------------------- ### Synchronize Installation Script Source: https://github.com/european-xfel/extra/blob/master/README.md Use rsync to update the install-extra.sh script on the Maxwell server. This ensures the latest version of the script is used for deployment. ```bash rsync -a --progress docs/install-extra.sh xsoft@max-exfl-display.desy.de:/home/xsoft ``` -------------------------------- ### Run Tests with Pytest Source: https://github.com/european-xfel/extra/blob/master/README.md Execute all tests in the project using pytest. Ensure pytest is installed. ```bash python -m pytest . ``` -------------------------------- ### Serve Documentation Locally Source: https://github.com/european-xfel/extra/blob/master/README.md Build and serve the project documentation locally using mkdocs. This command automatically rebuilds the docs as you edit. ```bash mkdocs serve ``` -------------------------------- ### Import Karabo Bridge Client Source: https://github.com/european-xfel/extra/blob/master/docs/karabo-bridge.md Import the Client class from the extra.karabo_bridge module to begin using the Karabo bridge. ```python from extra.karabo_bridge import Client ``` -------------------------------- ### Proposal Object Usage Source: https://github.com/european-xfel/extra/blob/master/docs/proposal.md Demonstrates how to instantiate and use the Proposal object to access various proposal-related data and functionalities. ```APIDOC ## Proposal API Usage Example ### Description This section provides a Python code example demonstrating the basic usage of the `Proposal` class from the `extra.proposal` module. ### Method N/A (Illustrative Python code) ### Endpoint N/A (Illustrative Python code) ### Parameters N/A (Illustrative Python code) ### Request Example ```python from extra.proposal import Proposal # Open a proposal by its ID proposal = Proposal(1234) # Print general information about the proposal proposal.info() # Get a table of samples and their associated runs proposal.samples_table() # Retrieve DAQ data using extra-data proposal.data() # Access DAMNIT data via the DAMNIT API proposal.damnit() ``` ### Response N/A (Illustrative Python code execution results in output to console or data structures) #### Success Response (N/A) N/A #### Response Example N/A ``` -------------------------------- ### Access Proposal and Run Metadata Source: https://context7.com/european-xfel/extra/llms.txt Retrieve proposal information and specific run references using the EXtra proposal module. ```python from extra.proposal import Proposal, RunReference # Access proposal information proposal = Proposal(2222) # List all runs for run_ref in proposal.runs: print(f"Run {run_ref.run_number}: {run_ref.start_time}") # Get specific run reference run_ref = RunReference(proposal=2222, run=42) print(f"Run path: {run_ref.path}") ``` -------------------------------- ### Access Spectrometer Calibration Applications Source: https://context7.com/european-xfel/extra/llms.txt Import specialized calibration applications for spectrometers. ```python from extra.applications import CookieboxCalibration, Grating2DCalibration ``` -------------------------------- ### Importing the Damnit class Source: https://github.com/european-xfel/extra/blob/master/docs/damnit.md Initializes access to the DAMNIT API by importing the Damnit class from the extra.damnit module. ```python from extra.damnit import Damnit ``` -------------------------------- ### Calibrate eTOF and Grating Spectrometer Source: https://context7.com/european-xfel/extra/llms.txt Initialize and calibrate Cookiebox eTOF and 2D grating spectrometer devices using calibration runs. ```python cookiebox = CookieboxCalibration( run=calibration_run, tof_source="SQS_AQS_DIGITIZER/ADC/1", mono_source="SQS_MDL_MONO/MDL/PHOTON_ENERGY" ) # Calibrate using known photon energies cookiebox.calibrate(energies=[280, 300, 320, 340]) # Apply calibration to data calibrated = cookiebox.apply(data_run) # 2D grating spectrometer calibration grating = Grating2DCalibration( calibration_run=cal_run, camera_source="SQS_CAM_DEVICE/CAM/IMAGE" ) # Fit calibration parameters grating.calibrate() # Plot calibration results grating.plot_calibration_data() ``` -------------------------------- ### Initialize and Use XGM Component Source: https://context7.com/european-xfel/extra/llms.txt Initialize the XGM component to access X-ray Gas Monitor data. Auto-detects the device or allows explicit specification. Retrieve photon energy, wavelength, and pulse-resolved data. Supports multi-SASE configurations and provides visualization and summary statistics. ```python from extra.data import open_run from extra.components import XGM run = open_run(proposal=2222, run=42) # Initialize XGM - auto-detects if only one XGM in the run xgm = XGM(run) # Or specify explicitly with device name, alias, or unique substring xgm = XGM(run, device="SPB_XTD9_XGM/XGM/DOOCS") xgm = XGM(run, "spb") # unique substring match # Get photon energy and wavelength photon_energy = xgm.photon_energy(with_units=True) # Returns pint.Quantity in keV wavelength = xgm.wavelength() # In nanometers # Get pulse-resolved energy data pulse_energy = xgm.pulse_energy() # 2D DataArray (trainId, pulseIndex) in µJ pulse_energy_series = xgm.pulse_energy(series=True) # 1D Series, NaN entries dropped # Get pulse counts per train pulse_counts = xgm.pulse_counts() max_pulses = xgm.max_npulses() # For XGMs with multi-SASE data (SASE 1 and SASE 3) xgm_sa1 = XGM(run, default_sase=1) sa1_energy = xgm_sa1.pulse_energy(sase=1) sa3_energy = xgm_sa1.pulse_energy(sase=3) # Visualization xgm.plot() # Overview with heatmap and energy plots xgm.plot_pulse_energy() # Heatmap of pulse energies xgm.plot_energy_per_train() # Mean train energy with rolling average xgm.info() # Print summary statistics ``` -------------------------------- ### Loading Calibration Data by Conditions Source: https://github.com/european-xfel/extra/blob/master/docs/calibration.md Demonstrates how to find and load calibration constants for a specific detector type based on given conditions and an event timestamp. ```APIDOC ## Load Calibration Data by Conditions ### Description This section shows how to retrieve calibration constants by specifying detector conditions and an event timestamp. It uses the `CalibrationData.from_condition` method. ### Method N/A (This describes a programmatic API usage, not a direct HTTP method) ### Endpoint N/A ### Parameters #### Path Parameters N/A #### Query Parameters N/A #### Request Body N/A ### Request Example ```python from extra.calibration import CalibrationData, LPDConditions lpd_cd = CalibrationData.from_condition( LPDConditions(memory_cells=200, sensor_bias_voltage=250), "FXE_DET_LPD1M-1", event_at="2022-05-22T02:00:00", ) # Load one constant for all found modules offset = lpd_cd["Offset"].ndarray() ``` ### Response #### Success Response (200) N/A (This describes programmatic data loading, not an HTTP response) #### Response Example ```python # Example of accessing loaded data offset = lpd_cd["Offset"].ndarray() ``` ``` -------------------------------- ### Perform Analysis Utility Tasks Source: https://context7.com/european-xfel/extra/llms.txt Use helper functions for Gaussian fitting, enhanced image plotting, and array manipulation. ```python from extra.utils import ( gaussian, fit_gaussian, gaussian2d, lorentzian, imshow2, hyperslicer2, ridgeplot, find_nearest_index, find_nearest_value, reorder_axes_to_shape ) import numpy as np import matplotlib.pyplot as plt # Gaussian fitting xdata = np.linspace(-5, 5, 100) ydata = gaussian(xdata, y0=0, A=10, mu=0, sigma=1) + np.random.normal(0, 0.5, 100) # Fit with automatic initial guess popt = fit_gaussian(ydata, xdata) # Returns [y0, A, mu, sigma] y0, A, mu, sigma = popt # Fit with constraints popt = fit_gaussian(ydata, xdata, A_sign=1) # Force upward peak popt = fit_gaussian(ydata, xdata, nans_on_failure=True) # Return NaNs on failure # Plot fit plt.plot(xdata, ydata, 'o', label='Data') plt.plot(xdata, gaussian(xdata, *popt, norm=False), label='Fit') # Enhanced image display image = np.random.rand(100, 200) imshow2(image) # Auto vmin/vmax, aspect ratio, colorbar imshow2(image, lognorm=True, colorbar=False) # Log scale, no colorbar # Works with xarray DataArrays import xarray as xr da = xr.DataArray(image, dims=['y', 'x']) imshow2(da) # Uses xarray plotting with labels # Interactive image stack viewer (requires mpl_interactions) images = np.random.rand(50, 100, 200) # (frames, y, x) hyperslicer2(images) # Interactive slider and play buttons # Ridgeline plot for sequences spectra = np.random.rand(10, 500) # 10 spectra fig = ridgeplot(spectra, overlap=0.5, xlabel="Energy", stack_label="Scan step") # Array utilities arr = np.array([1.1, 2.3, 3.7, 4.2]) idx = find_nearest_index(arr, 3.5) # Returns 2 val = find_nearest_value(arr, 3.5) # Returns 3.7 # Reorder array axes to match target shape data = np.random.rand(128, 512, 64) # (pixels_y, pixels_x, cells) reordered = reorder_axes_to_shape(data, (64, 128, 512)) # (cells, pixels_y, pixels_x) ``` -------------------------------- ### Loading Calibration Data by Report Source: https://github.com/european-xfel/extra/blob/master/docs/calibration.md Illustrates how to load a group of calibration constants associated with a specific CalCat report number. ```APIDOC ## Load Calibration Data by Report ### Description This section demonstrates how to load a set of calibration constants identified by a specific CalCat report number using the `CalibrationData.from_report` method. ### Method N/A (This describes a programmatic API usage, not a direct HTTP method) ### Endpoint N/A ### Parameters #### Path Parameters N/A #### Query Parameters N/A #### Request Body N/A ### Request Example ```python from extra.calibration import CalibrationData agipd_cd = CalibrationData.from_report(3757) ``` ### Response #### Success Response (200) N/A (This describes programmatic data loading, not an HTTP response) #### Response Example ```python # Example of using the loaded calibration data agipd_cd.display_markdown_table() ``` ``` -------------------------------- ### Access Detector Properties Source: https://context7.com/european-xfel/extra/llms.txt Retrieve basic metadata and configuration settings from an acquisition object. ```python print(f"Sampling rate: {adq.sampling_rate / 1e9:.2f} GHz") print(f"Trace duration: {adq.trace_duration * 1e6:.1f} µs") print(f"Interleaved: {adq.interleaved}") ``` -------------------------------- ### Perform Unit Conversions Source: https://context7.com/european-xfel/extra/llms.txt Utilize the configured pint unit registry for XFEL-specific physical quantity conversions. ```python from extra import ureg # Wavelength to energy conversion wavelength = 0.1 * ureg.nm energy = wavelength.to("keV", "xfel") print(f"{wavelength} = {energy:.2f}") # Create quantities pulse_energy = 100 * ureg.uJ photon_energy = 9.3 * ureg.keV ``` -------------------------------- ### Configure Detector Geometry Source: https://context7.com/european-xfel/extra/llms.txt Load, assemble, and export detector geometry configurations. ```python from extra.geom import AGIPD_1MGeometry, LPD_1MGeometry, DSSC_1MGeometry # Load geometry from file geom = AGIPD_1MGeometry.from_crystfel_geom("/path/to/agipd.geom") # Or from quad positions geom = AGIPD_1MGeometry.from_quad_positions( quad_pos=[(-525, 625), (-550, -10), (520, -160), (542.5, 475)] ) # Assemble detector image assembled = geom.position_modules(module_data) # (modules, ss, fs) -> (y, x) # Get pixel positions x, y, z = geom.get_pixel_positions() # Write geometry files geom.write_crystfel_geom("output.geom") ``` -------------------------------- ### Displaying Calibration Data Table Source: https://github.com/european-xfel/extra/blob/master/docs/calibration.md Shows how to display the selected calibration constants in a Markdown table format, useful for Jupyter notebooks. ```APIDOC ## Displaying Calibration Data Table ### Description This section explains how to render the loaded calibration constants as a Markdown table, which is particularly useful for visualization within Jupyter notebooks. ### Method N/A (This describes a programmatic API usage, not a direct HTTP method) ### Endpoint N/A ### Parameters #### Path Parameters N/A #### Query Parameters N/A #### Request Body N/A ### Request Example ```python # Assuming 'agipd_cd' is a CalibrationData object loaded previously agipd_cd.display_markdown_table() ``` ### Response #### Success Response (200) N/A (This describes programmatic output, not an HTTP response) #### Response Example ```markdown | Modules | BadPixelsDark | Noise | Offset | ThresholdsDark | |--------:|:----------------------------------------------------------------------------------------|:----------------------------------------------------------------------------------------|:----------------------------------------------------------------------------------------|:----------------------------------------------------------------------------------------| | 0 | [2023-07-29 19:34](https://in.xfel.eu/calibration/calibration_constant_versions/172433) | [2023-07-29 19:34](https://in.xfel.eu/calibration/calibration_constant_versions/172424) | [2023-07-29 19:34](https://in.xfel.eu/calibration/calibration_constant_versions/172409) | [2023-07-29 19:34](https://in.xfel.eu/calibration/calibration_constant_versions/172442) | | 1 | [2023-07-29 19:34](https://in.xfel.eu/calibration/calibration_constant_versions/172402) | [2023-07-29 19:34](https://in.xfel.eu/calibration/calibration_constant_versions/172395) | [2023-07-29 19:34](https://in.xfel.eu/calibration/calibration_constant_versions/172390) | [2023-07-29 19:34](https://in.xfel.eu/calibration/calibration_constant_versions/172416) | | 2 | [2023-07-29 19:34](https://in.xfel.eu/calibration/calibration_constant_versions/172425) | [2023-07-29 19:34](https://in.xfel.eu/calibration/calibration_constant_versions/172410) | [2023-07-29 19:34](https://in.xfel.eu/calibration/calibration_constant_versions/172388) | [2023-07-29 19:34](https://in.xfel.eu/calibration/calibration_constant_versions/172435) | | ... | | | | | | 14 | [2023-07-29 19:34](https://in.xfel.eu/calibration/calibration_constant_versions/172407) | [2023-07-29 19:34](https://in.xfel.eu/calibration/calibration_constant_versions/172399) | [2023-07-29 19:34](https://in.xfel.eu/calibration/calibration_constant_versions/172392) | [2023-07-29 19:34](https://in.xfel.eu/calibration/calibration_constant_versions/172418) | | 15 | [2023-07-29 19:34](https://in.xfel.eu/calibration/calibration_constant_versions/172421) | [2023-07-29 19:34](https://in.xfel.eu/calibration/calibration_constant_versions/172415) | [2023-07-29 19:34](https://in.xfel.eu/calibration/calibration_constant_versions/172406) | [2023-07-29 19:34](https://in.xfel.eu/calibration/calibration_constant_versions/172429) | ``` ``` -------------------------------- ### Grating2DCalibration Source: https://github.com/european-xfel/extra/blob/master/docs/applications/grating.md Documentation for the 2D Grating calibration application. ```APIDOC ## Grating2DCalibration ### Description Calibration application for 2D grating data. ### Reference extra.applications.Grating2DCalibration ``` -------------------------------- ### Grating1DCalibration Source: https://github.com/european-xfel/extra/blob/master/docs/applications/grating.md Documentation for the 1D Grating calibration application. ```APIDOC ## Grating1DCalibration ### Description Calibration application for 1D grating data. ### Reference extra.applications.Grating1DCalibration ``` -------------------------------- ### Import AGIPD 1M Geometry Source: https://github.com/european-xfel/extra/blob/master/docs/detector-geometry.md Import the AGIPD_1MGeometry class from the extra.geom module to handle the geometry of AGIPD 1M detectors. Refer to the EXtra-geom documentation for detailed usage. ```python from extra.geom import AGIPD_1MGeometry ``` -------------------------------- ### Load calibration constants by report Source: https://github.com/european-xfel/extra/blob/master/docs/calibration.md Retrieves a group of constants associated with a specific characterization process report. ```python from extra.calibration import CalibrationData agipd_cd = CalibrationData.from_report(3757) ``` -------------------------------- ### Load calibration constants by condition Source: https://github.com/european-xfel/extra/blob/master/docs/calibration.md Retrieves calibration data for a specific detector type and time using a condition object. ```python from extra.calibration import CalibrationData, LPDConditions lpd_cd = CalibrationData.from_condition( LPDConditions(memory_cells=200, sensor_bias_voltage=250), "FXE_DET_LPD1M-1", event_at="2022-05-22T02:00:00", ) # Load one constant for all found modules offset = lpd_cd["Offset"].ndarray() ``` -------------------------------- ### Importing open_run from EXtra-data Source: https://github.com/european-xfel/extra/blob/master/docs/reading-data.md Use this import to access the primary function for opening data runs. ```python from extra.data import open_run ``` -------------------------------- ### Manage Calibration Data Source: https://context7.com/european-xfel/extra/llms.txt Load, filter, and manipulate detector calibration constants from CalCat or metadata files. ```python from extra.calibration import ( CalibrationData, AGIPDConditions, JUNGFRAUConditions, LPDConditions, DSSCConditions, BadPixels ) # Define detector operating conditions conditions = AGIPDConditions( sensor_bias_voltage=300, memory_cells=352, acquisition_rate=1.1, gain_setting=0, gain_mode=0, source_energy=9.3 ) # Look up calibration constants cal_data = CalibrationData.from_condition( conditions, detector_name="SPB_DET_AGIPD1M-1", calibrations=["Offset", "Noise", "BadPixelsDark"], event_at="2024-01-15" # Point in time for constants ) # Access constants by type offset = cal_data["Offset"] # MultiModuleConstant offset_mod0 = cal_data["Offset", 0] # SingleConstant for module 0 # Load constant data as numpy array offset_array = offset.ndarray() # Shape: (n_modules, cells, gain, pixels_y, pixels_x) offset_xr = offset.xarray() # With dimension labels # Load constants used for an already-corrected run cal_data = CalibrationData.from_correction( proposal=2222, run=42, detector_name="SPB_DET_AGIPD1M-1" ) # Or from YAML metadata file cal_data = CalibrationData.from_correction( "/path/to/proc/r0042/calibration_metadata_SPB_DET_AGIPD1M-1.yml" ) # Display summary table in Jupyter cal_data.display_markdown_table() # Select specific modules cal_subset = cal_data.select_modules([0, 1, 2, 3]) cal_subset = cal_data.select_modules(qm_names=["Q1M1", "Q1M2"]) # Require specific calibration types (drops modules missing them) cal_filtered = cal_data.require_calibrations(["Offset", "Noise"]) # Merge multiple CalibrationData objects merged = cal_data1.merge(cal_data2) # Access module information print(cal_data.module_nums) print(cal_data.aggregator_names) # e.g., ['AGIPD00', 'AGIPD01', ...] print(cal_data.pdu_names) # Physical detector unit names # Interpret bad pixel masks mask = cal_data["BadPixelsDark"].ndarray() is_noisy = (mask & BadPixels.NOISE_OUT_OF_THRESHOLD) > 0 is_dead = (mask & BadPixels.NO_DARK_DATA) > 0 ``` -------------------------------- ### Display calibration constants table Source: https://github.com/european-xfel/extra/blob/master/docs/calibration.md Renders a markdown table of selected constants within a Jupyter notebook environment. ```python agipd_cd.display_markdown_table() ``` -------------------------------- ### Work with Pulse Pattern Components Source: https://context7.com/european-xfel/extra/llms.txt Utilize components for X-ray, optical laser, and combined pump-probe pulse patterns. Access pulse IDs, masks, and statistics. Supports specifying SASE beamlines or PPL seeds and checking for interleaved patterns. ```python from extra.data import open_run from extra.components import XrayPulses, OpticalLaserPulses, PumpProbePulses, MachinePulses run = open_run(proposal=2222, run=42) # X-ray pulses from SASE beamlines xray_pulses = XrayPulses(run) # Or specify SASE beamline explicitly xray_pulses = XrayPulses(run, sase=1) # Get pulse IDs as labeled pandas Series pulse_ids = xray_pulses.pulse_ids() # MultiIndex: (trainId, pulseIndex) pulse_ids_array = xray_pulses.pulse_ids(labelled=False) # numpy array # Check if pulse pattern is constant if xray_pulses.is_constant_pattern(): npulses = xray_pulses.pulse_counts().iloc[0] print(f"Constant pattern with {npulses} pulses per train") # Get pulse mask (boolean array for bunch pattern table) mask = xray_pulses.pulse_mask() # xarray DataArray (trainId, pulseId) # Get pulse statistics pulse_counts = xray_pulses.pulse_counts() rep_rates = xray_pulses.pulse_repetition_rates() # In Hz train_durations = xray_pulses.train_durations() # In seconds # Optical laser pulses (PPL) ppl = OpticalLaserPulses(run) # Or specify PPL seed by instrument name ppl = OpticalLaserPulses(run, ppl_seed="SQS") # Pump-probe combined patterns ppp = PumpProbePulses(run, pulse_offset=0) # PPL aligned with first FEL pulse ppp = PumpProbePulses(run, bunch_table_offset=10) # PPL offset by 10 bunch slots # Access FEL and PPL masks separately fel_mask = ppp.pulse_mask(field='fel') ppl_mask = ppp.pulse_mask(field='ppl') # Check if SA1 and SA3 are interleaved is_interleaved = xray_pulses.is_sa1_interleaved_with_sa3() # Print pattern summary xray_pulses.info() ``` -------------------------------- ### Process ADQ Digitizer Data Source: https://context7.com/european-xfel/extra/llms.txt Handle raw traces, pulse-separated data, and signal edge detection for fast ADQ digitizers. ```python from extra.data import open_run from extra.components import AdqRawChannel run = open_run(proposal=2222, run=42) # Initialize channel (auto-detects digitizer) adq = AdqRawChannel(run, channel="1A") # or "1_A" # With explicit digitizer and configuration adq = AdqRawChannel(run, channel="2B", digitizer="SQS_DIGITIZER_UTC1/ADC/1:network", cm_period=8, # Common mode correction period baselevel=0, # Pull baseline to this value first_pulse_offset=10000, # Sample where first pulse starts sample_dim='time') # Use time coordinates instead of samples # Get raw traces by train train_data = adq.train_data() # xarray DataArray (trainId, sample/time) train_data_np = adq.train_data(labelled=False) # numpy array # Get pulse-separated data pulse_data = adq.pulse_data() # (pulse MultiIndex, sample) pulse_data = adq.pulse_data(pulse_dim='pulseTime') # Time-based pulse coordinates # Unstack pulses back into (train, pulse, sample) unstacked = adq.unstack_pulses(pulse_data) # Find signal edges (for ToF spectroscopy) from extra.signal import dled edges_df = adq.train_edges(threshold=0.5) # DataFrame with edge positions edges_df = adq.pulse_edges(threshold=0.5, max_edges=20) # Get edges as ragged arrays result = adq.train_edge_array(threshold=0.5) # result is xarray.Dataset with 'edges' and 'amplitudes' # Process existing data with edge finding data = adq.train_data(labelled=False) edges, amplitudes = adq.find_edge_array(data, labelled=False, threshold=0.5) # Manual data processing raw = adq.raw_samples_key.ndarray() corrected = adq.correct_common_mode(raw, cm_period=8, baseline=slice(0, 1000)) reshaped = adq.reshape_to_pulses(corrected) ``` -------------------------------- ### Read XFEL Data with EXtra-data Source: https://context7.com/european-xfel/extra/llms.txt Open and select data runs using the `extra.data` library. Access specific sources, filter trains by ID, and retrieve data as xarray.DataArray. Iterate over trains to process data. ```python from extra.data import open_run, DataCollection, by_id # Open a run by proposal and run number run = open_run(proposal=2222, run=42) # Access sources in the run print(run.all_sources) # Select specific trains subset = run.select_trains(by_id[1000:1100]) # Get data from a specific source and key xgm_data = run["SA2_XTD1_XGM/XGM/DOOCS", "data.intensityTD"] data_array = xgm_data.xarray() # Returns xarray.DataArray with train IDs # Iterate over trains for train_id, data in run.trains(): print(f"Train {train_id}: {len(data)} sources") ``` -------------------------------- ### Access Delay Line Detector Data Source: https://context7.com/european-xfel/extra/llms.txt Initialize DLD components to retrieve reconstructed hits, raw edge data, and pulse information. ```python from extra.data import open_run from extra.components import DelayLineDetector, DldPulses run = open_run(proposal=2222, run=42) # Initialize DLD component dld = DelayLineDetector(run) # Or specify detector source explicitly dld = DelayLineDetector(run, detector="SQS_AQS_DLD1/DET/TOP") # Get reconstructed hits as DataFrame hits = dld.hits() # hits DataFrame has columns: x, y, t, m (method) with MultiIndex (trainId, pulseId, hitIndex) # Filter by reconstruction quality (lower method = safer) safe_hits = dld.hits(max_method=10) # Use different pulse dimension hits_by_time = dld.hits(pulse_dim='pulseTime') # Get raw edge data edges = dld.edges() # Series indexed by (trainId, pulseId, edgeIndex, channel) # Add aligned pulse-resolved data to hits xgm = XGM(run) pulse_energy = xgm.pulse_energy(series=True) hits_with_energy = dld.hits(extra_columns={'pulse_energy': pulse_energy}) # Or insert columns after the fact dld.insert_aligned_columns(hits, {'pulse_energy': pulse_energy}) # Get reconstruction signals for diagnostics signals = dld.signals() # Select subset of trains dld_subset = dld.select_trains(by_id[1000:1100]) # Access trigger information triggers = dld.triggers() # DLD-specific pulse information dld_pulses = DldPulses(run[f"{dld.detector_name}:output"]) pulse_info = dld_pulses.pulse_ids() ``` -------------------------------- ### Fitting Functions Source: https://github.com/european-xfel/extra/blob/master/docs/utilities.md Provides functions for fitting data, specifically Gaussian distributions. ```APIDOC ## Fitting Functions This section covers utility functions for data fitting. ### `extra.utils.fit_gaussian` **Description**: Fits a Gaussian function to the provided data. ``` -------------------------------- ### Detector and Module Metadata Source: https://github.com/european-xfel/extra/blob/master/docs/calibration.md Endpoints for retrieving physical detector unit (PDU) mapping and module metadata. ```APIDOC ## DetectorData ### Description Provides access to detector metadata and mapping information for physical detector units (PDUs). ## DetectorModule ### Description Provides access to specific module-level metadata within the calibration framework. ``` -------------------------------- ### Mathematical Functions Source: https://github.com/european-xfel/extra/blob/master/docs/utilities.md Provides mathematical functions including Gaussian and Lorentzian distributions. ```APIDOC ## Mathematical Functions This section covers mathematical utility functions. ### `extra.utils.gaussian` **Description**: Computes a Gaussian function. ### `extra.utils.gaussian2d` **Description**: Computes a 2D Gaussian function. ### `extra.utils.lorentzian` **Description**: Computes a Lorentzian function. ``` -------------------------------- ### Analyze Motor Scans Source: https://context7.com/european-xfel/extra/llms.txt Detect and analyze steps in motor scans, bin data by steps, and visualize results. ```python from extra.data import open_run from extra.components import Scan run = open_run(proposal=2222, run=42) # Create scan from motor source scan = Scan(run["MOTOR/MCMOTORYFACE"]) # Or from a specific key scan = Scan(run["MOTOR/DEVICE"]["actualPosition"]) # Custom detection parameters for difficult scans scan = Scan(run["MOTOR/DEVICE"], resolution=0.001, # Detection resolution min_trains=100, # Minimum trains per step intra_step_filtering=1.5) # Filter noisy trains # Access scan properties print(f"Steps: {len(scan.steps)}") print(f"Positions: {scan.positions}") print(f"Train IDs per step: {scan.positions_train_ids}") # Visualize scan detection scan.plot() # Shows motor positions with detected steps highlighted scan.info() # Print detailed scan information # Bin data by scan steps roi_intensity = run["DETECTOR/DATA"]["intensity"].xarray() binned = scan.bin_by_steps(roi_intensity, uncertainty_method="std") # binned is a DataArray with coordinates: # - position: motor positions for each step # - counts: number of trains in each step # - uncertainty: standard deviation or standard error # Plot binned data directly scan.plot_bin_by_steps(roi_intensity, title="ROI vs Motor Position") # Split any data object by scan steps for step_data in scan.split_by_steps(run): print(f"Step with {len(step_data.train_ids)} trains") # Group data for flexible aggregation grouped = scan.group_data(roi_intensity) # Returns xarray GroupBy mean_per_step = grouped.mean() std_per_step = grouped.std() ``` -------------------------------- ### Load EXtra Python Module Source: https://github.com/european-xfel/extra/blob/master/docs/index.md Load the EXtra Python module on the Maxwell cluster. This is the default environment for Jupyterhub. ```bash module load exfel exfel-python ``` -------------------------------- ### Plotting Functions Source: https://github.com/european-xfel/extra/blob/master/docs/utilities.md Provides functions for data visualization and plotting. ```APIDOC ## Plotting Functions This section covers utility functions for plotting. ### `extra.utils.imshow2` **Description**: Displays an image with enhanced features. ### `extra.utils.hyperslicer2` **Description**: Provides advanced slicing capabilities for hyperspectral data. ### `extra.utils.ridgeplot` **Description**: Generates a ridge plot for visualizing distributions. ``` -------------------------------- ### Bad Pixel Interpretation Source: https://github.com/european-xfel/extra/blob/master/docs/calibration.md Documentation for interpreting values found in calibration masks. ```APIDOC ## BadPixels ### Description Defines the values used in calibration masks (e.g., image.mask or data.mask). Zeros represent good pixels, while non-zero values indicate specific issues or characteristics such as NON_STANDARD_SIZE. ``` -------------------------------- ### Array Functions Source: https://github.com/european-xfel/extra/blob/master/docs/utilities.md Provides functions for manipulating and searching within arrays. ```APIDOC ## Array Functions This section covers utility functions for array manipulation. ### `extra.utils.find_nearest_index` **Description**: Finds the index of the element nearest to a given value in an array. ### `extra.utils.find_nearest_value` **Description**: Finds the value of the element nearest to a given value in an array. ### `extra.utils.reorder_axes_to_shape` **Description**: Reorders the axes of an array to match a specified shape. ``` === COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.