### Install Dependencies

Source: https://github.com/caltech-ipac/irsa-tutorials/blob/main/tutorials/cloud_access/euclid-cloud-access.md

Installs necessary Python libraries for accessing Euclid Q1 data, including s3fs, astropy, and astroquery.

```python
# Uncomment the next line to install dependencies if needed.
# !pip install s3fs astropy 'astroquery>=0.4.10' matplotlib
```

--------------------------------

### Install Required Packages

Source: https://github.com/caltech-ipac/irsa-tutorials/blob/main/tutorials/spherex/spherex_intro.md

Installs necessary Python packages for the tutorial, including numpy, matplotlib, astropy, pyvo, and firefly-client. Uncomment the line if packages are not already installed.

```ipython3
# !pip install numpy matplotlib astropy pyvo firefly-client
```

--------------------------------

### Install Dependencies

Source: https://github.com/caltech-ipac/irsa-tutorials/blob/main/tutorials/cloud_access/cloud-access-intro.md

Installs necessary Python libraries for cloud access and data manipulation, including astropy, pandas, pyvo, and s3fs.

```ipython3
# Uncomment the next line to install dependencies if needed.
# !pip install astropy hpgeom pandas pyarrow 'pyvo>=1.5' s3fs matplotlib astroquery
```

--------------------------------

### Install Dependencies

Source: https://github.com/caltech-ipac/irsa-tutorials/blob/main/tutorials/parquet-catalog-demos/wise-allwise-catalog-demo.md

Installs necessary Python libraries for data analysis, including pandas, pyarrow, matplotlib, hpgeom, and astropy. This is a prerequisite for running the subsequent code examples.

```ipython3
# Uncomment the next line to install dependencies if needed.
# !pip install 'pandas>=1.5.2' 'pyarrow>=10.0.1' matplotlib hpgeom astropy
```

--------------------------------

### Install Dependencies

Source: https://github.com/caltech-ipac/irsa-tutorials/blob/main/tutorials/euclid_access/4_Euclid_intro_PHZ_catalog.md

Installs necessary Python libraries including matplotlib, astropy, and astroquery for astronomical data analysis.

```ipython3
# Uncomment the next line to install dependencies if needed.
# !pip install matplotlib 'astropy>=5.3' 'astroquery>=0.4.10' fsspec firefly_client
```

--------------------------------

### Install Dependencies

Source: https://github.com/caltech-ipac/irsa-tutorials/blob/main/tutorials/euclid_access/2_Euclid_intro_MER_catalog.md

Installs necessary Python libraries including numpy, matplotlib, and astroquery. Ensure astroquery is version 0.4.10 or newer.

```python
# Uncomment the next line to install dependencies if needed
# !pip install numpy matplotlib 'astroquery>=0.4.10'
```

--------------------------------

### Install Dependencies

Source: https://github.com/caltech-ipac/irsa-tutorials/blob/main/tutorials/euclid_access/1_Euclid_intro_MER_images.md

Installs necessary Python libraries including numpy, astropy, matplotlib, astroquery, and sep for data processing and visualization.

```python
# Uncomment the next line to install dependencies if needed.
# !pip install numpy 'astropy>=5.3' matplotlib 'astroquery>=0.4.10' 'sep>=1.4' fsspec
```

--------------------------------

### Install Dependencies

Source: https://github.com/caltech-ipac/irsa-tutorials/blob/main/tutorials/parallelize/Parallelize_Convolution.md

Installs necessary libraries for the tutorial, including dask, numpy, ray, and scipy. This is a prerequisite for running the parallelization examples.

```bash
#!/bin/bash
# Uncomment the next line to install dependencies if needed.
# !pip install dask[distributed] numpy ray scipy
```

--------------------------------

### Install Dependencies

Source: https://github.com/caltech-ipac/irsa-tutorials/blob/main/tutorials/euclid_access/3_Euclid_intro_1D_spectra.md

Installs necessary Python libraries including matplotlib, astropy, and astroquery. Ensure astroquery is version 0.4.10 or newer.

```python
# Uncomment the next line to install dependencies if needed
# !pip install matplotlib astropy 'astroquery>=0.4.10'
```

--------------------------------

### Install Dependencies

Source: https://github.com/caltech-ipac/irsa-tutorials/blob/main/tutorials/euclid_access/5_Euclid_intro_SPE_catalog.md

Installs necessary Python libraries including matplotlib, astropy, and astroquery. Ensure astroquery is version 0.4.10 or newer.

```ipython3
# Uncomment the next line to install dependencies if needed
# !pip install matplotlib astropy 'astroquery>=0.4.10'
```

--------------------------------

### Install Dependencies

Source: https://github.com/caltech-ipac/irsa-tutorials/blob/main/tutorials/cosmodc2/cosmoDC2_TAP_access.md

Installs necessary Python libraries (numpy, matplotlib, pyvo) for accessing and visualizing astronomical data. This is a prerequisite for running the tutorial.

```ipython3
# Uncomment the next line to install dependencies if needed.
# !pip install numpy matplotlib pyvo
```

--------------------------------

### Install Dependencies

Source: https://github.com/caltech-ipac/irsa-tutorials/blob/main/tutorials/irsa-sia-examples/sia_2mass_allsky.md

Installs necessary Python packages including matplotlib, astropy, and pyvo if they are not already present.

```bash
#!/bin/bash
# Uncomment the next line to install dependencies if needed.
# !pip install matplotlib astropy pyvo
```

--------------------------------

### Install Dependencies

Source: https://github.com/caltech-ipac/irsa-tutorials/blob/main/tutorials/parquet-catalog-demos/neowise-source-table-strategies.md

Installs necessary Python libraries for data processing, including hpgeom, pandas, and pyarrow. This is a prerequisite for running the subsequent code examples.

```ipython3
# Uncomment the next line to install dependencies if needed.
# !pip install hpgeom pandas pyarrow
```

--------------------------------

### Setup Catalog Paths and Query Filters

Source: https://github.com/caltech-ipac/irsa-tutorials/blob/main/tutorials/parquet-catalog-demos/wise-allwise-catalog-demo.md

Defines the S3 bucket and folder for the AllWISE catalog, sets up a pyarrow S3FileSystem for accessing the data, and establishes query parameters such as magnitude limits, sky coordinates (RA/Dec), and polygon definitions.

```ipython3
bucket = "nasa-irsa-wise"
folder = "wise/allwise/catalogs/p3as_psd/healpix_k5"
parquet_root = f"{bucket}/{folder}/wise-allwise.parquet"

fs = S3FileSystem(region="us-west-2", anonymous=True)  # the bucket is in region us-west-2

w1mpro_min = 10.0
ra_min, ra_max = 15, 25  # deg
dec_min, dec_max = 62, 72  # deg
polygon_corners = [(ra_min, dec_min), (ra_min, dec_max), (ra_max, dec_max), (ra_max, dec_min)]
radius = 5 * u.arcmin.to(u.deg)

order = 5  # the catalog is partitioned by HEALPix pixel index at order 5
nside = hp.order_to_nside(order)  # number of order 5 pixels along one side of a base pixel
```

--------------------------------

### Install Dependencies

Source: https://github.com/caltech-ipac/irsa-tutorials/blob/main/tutorials/irsa-sia-examples/sia_cosmos.md

Installs necessary Python libraries including matplotlib, astropy, and pyvo if they are not already present.

```bash
# Uncomment the next line to install dependencies if needed.
# !pip install matplotlib astropy pyvo
```

--------------------------------

### Install Dependencies

Source: https://github.com/caltech-ipac/irsa-tutorials/blob/main/tutorials/irsa-sia-examples/siav2_seip.md

Installs necessary Python libraries including matplotlib, astropy, and pyvo for astronomical data processing and visualization.

```bash
# Uncomment the next line to install dependencies if needed.
# !pip install matplotlib astropy pyvo
```

--------------------------------

### Install and Import Modules

Source: https://github.com/caltech-ipac/irsa-tutorials/blob/main/tutorials/openuniversesims/openuniverse2024_roman_simulated_wideareasurvey.md

Installs necessary libraries like astropy, numpy, and s3fs for data analysis and S3 bucket interaction. Imports these libraries for use in the script.

```ipython3
# Uncomment the next line to install dependencies if needed.
# !pip install astropy numpy s3fs
```

```ipython3
#Import modules
from astropy.io import fits
import numpy as np
import s3fs  # browse buckets
```

--------------------------------

### Example Usage for Downloading Simulations

Source: https://github.com/caltech-ipac/irsa-tutorials/blob/main/tutorials/roman_simulations/roman_hlss_number_density.md

An example of how to call the `download_simulations` function. It sets the download directory and specifies whether to download all available simulation files or just the first one.

```ipython3
# Set download_all to True to download all files, or False to download only the first file
# only a single downloaded file is required to run the notebook in its entirety, 
# however section 4 will be more accuate with all 10 files

# Downloading 10 files, each 3GB in size, may take between 1 and 30 minutes,
# or even longer, depending on your proximity to the data.

download_dir = 'downloaded_hdf5_files'
download_simulations(download_dir, download_all=False)
```

--------------------------------

### Install Firefly Client and Dependencies

Source: https://github.com/caltech-ipac/irsa-tutorials/blob/main/tutorials/firefly/NEOWISE_light_curve_demo.md

Installs the necessary firefly_client and astropy libraries for the tutorial. This is a prerequisite for running the subsequent code cells.

```ipython3
# Uncomment the next line to install dependencies if needed.
# !pip install firefly_client astropy
```

--------------------------------

### Install Dependencies

Source: https://github.com/caltech-ipac/irsa-tutorials/blob/main/tutorials/euclid_access/Euclid_ERO.md

Installs necessary Python packages including astropy, astroquery, photutils, and sep for astronomical data analysis.

```bash
#!/bin/bash
pip install tqdm numpy matplotlib astropy 'photutils>=2.0' 'astroquery>=0.4.10' 'sep>=1.4' 'firefly_client>=3.2'
```

--------------------------------

### Install Dependencies

Source: https://github.com/caltech-ipac/irsa-tutorials/blob/main/tutorials/firefly/OpenUniverse2024Preview_Firefly.md

Installs necessary Python libraries for the tutorial, including astropy, numpy, s3fs, firefly_client, and reproject. This is a prerequisite for running the subsequent code cells.

```ipython3
# Uncomment the next line to install dependencies if needed.
# !pip install numpy astropy s3fs firefly_client reproject
```

--------------------------------

### Install Dependencies

Source: https://github.com/caltech-ipac/irsa-tutorials/blob/main/tutorials/parquet-catalog-demos/neowise-source-table-lightcurves.md

Installs necessary Python libraries for data analysis, including astropy, astroquery, hpgeom, matplotlib, pandas, pyarrow, and pyvo. This is a prerequisite for running the subsequent code.

```ipython3
# Uncomment the next line to install dependencies if needed.
# !pip install astropy astroquery hpgeom matplotlib pandas pyarrow pyvo
```

--------------------------------

### Environment Setup and Data Directory Creation

Source: https://github.com/caltech-ipac/irsa-tutorials/blob/main/tutorials/euclid_access/Euclid_ERO.md

Sets up the environment by creating an output data directory if it doesn't exist. It also defines the search radius and cutout size for fetching astronomical data, and specifies the target of interest using SkyCoord.

```Python
# create output directory
if os.path.exists("./data/"):
    print("Output directory already created.")
else:
    print("Creating data directory.")
    os.mkdir("./data/")

search_radius = 1.5 * u.arcmin # search radius
cutout_size = 1.5 * u.arcmin # cutout size

coord = SkyCoord.from_name('NGC 6397')
```

--------------------------------

### Install and Import Modules

Source: https://github.com/caltech-ipac/irsa-tutorials/blob/main/tutorials/openuniversesims/openuniverse2024_roman_simulated_timedomainsurvey.md

Installs and imports necessary Python libraries for data analysis, including astropy, matplotlib, numpy, pandas, pyarrow, s3fs, and scipy. It also sets up an S3 file system client and filters specific warnings.

```ipython3
# Uncomment the next line to install dependencies if needed.
# !pip install astropy matplotlib numpy pandas pyarrow s3fs scipy
```

```python
# Import modules
import warnings

import astropy.units as u
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
from astropy.coordinates import SkyCoord
from astropy.io import fits
from astropy.nddata import Cutout2D
from astropy.nddata.utils import NoOverlapError
from astropy.table import Table
from astropy.wcs import WCS, FITSFixedWarning
from matplotlib import animation
from scipy.ndimage import rotate

# Needed to access data in the cloud
import s3fs
s3 = s3fs.S3FileSystem(anon=True)  # create an S3 client

# Filter out the FITSFixedWarning, which is consequenceless and gets thrown every time you deal with a WCS
# in a Roman openuniverse simulated image using astropy.
warnings.simplefilter('ignore', category=FITSFixedWarning)
```

--------------------------------

### Install Dependencies

Source: https://github.com/caltech-ipac/irsa-tutorials/blob/main/tutorials/roman_simulations/roman_hlss_number_density.md

Installs necessary Python libraries for the tutorial, including h5py, pandas, matplotlib, numpy, and requests. This is typically run once before executing the main code.

```python
# Uncomment the next line to install dependencies if needed.
# !pip install h5py pandas matplotlib numpy requests
```

--------------------------------

### General Data Analysis Techniques

Source: https://github.com/caltech-ipac/irsa-tutorials/blob/main/index.md

Covers miscellaneous but useful techniques for users analyzing IRSA-curated data, including parallel processing examples.

```Python
tutorials/parallelize/Parallelize_Convolution
```

--------------------------------

### Access IRSA Data via VO Protocols

Source: https://github.com/caltech-ipac/irsa-tutorials/blob/main/index.md

Demonstrates accessing IRSA's on-premises data holdings using Virtual Observatory (VO) protocols. This includes examples for querying different sky surveys and datasets.

```Python
tutorials/irsa-sia-examples/sia_2mass_allsky
tutorials/irsa-sia-examples/sia_allwise_atlas
tutorials/irsa-sia-examples/sia_cosmos
tutorials/irsa-sia-examples/siav2_seip
tutorials/cosmodc2/cosmoDC2_TAP_access.md
```

--------------------------------

### Initialize S3 File System

Source: https://github.com/caltech-ipac/irsa-tutorials/blob/main/tutorials/cloud_access/euclid-cloud-access.md

Creates an S3 file system client using s3fs for browsing AWS S3 buckets anonymously.

```python
s3 = s3fs.S3FileSystem(anon=True)
```

--------------------------------

### Install Dependencies

Source: https://github.com/caltech-ipac/irsa-tutorials/blob/main/tutorials/firefly/SEDs_in_Firefly.md

Installs necessary Python packages including numpy, astropy, pyvo, and firefly_client. This command should be run before starting the Jupyterlab session.

```python
# Uncomment the next line to install dependencies if needed.
# !pip install numpy astropy pyvo firefly_client
```

--------------------------------

### Initialize Firefly Client and Show Image

Source: https://github.com/caltech-ipac/irsa-tutorials/blob/main/tutorials/euclid_access/4_Euclid_intro_PHZ_catalog.md

Initializes a Firefly client to connect to the IRSA viewer and displays the MER FITS image, aligning it with WCS.

```ipython3
fc = FireflyClient.make_client('https://irsa.ipac.caltech.edu/irsaviewer')

fc.show_fits(url=filename)

fc.align_images(lock_match=True)
```

--------------------------------

### Import Libraries

Source: https://github.com/caltech-ipac/irsa-tutorials/blob/main/tutorials/cloud_access/euclid-cloud-access.md

Imports essential Python libraries for cloud data access, astronomical data handling, querying, and visualization.

```python
import s3fs
from astropy.coordinates import SkyCoord
import astropy.units as u
from astropy.visualization import ImageNormalize, PercentileInterval, AsinhStretch
from astropy.io import fits
from astropy.nddata import Cutout2D
from astropy.wcs import WCS
from astropy.table import Table
from astroquery.ipac.irsa import Irsa
from matplotlib import pyplot as plt
import json
```

--------------------------------

### List Top-Level Directories

Source: https://github.com/caltech-ipac/irsa-tutorials/blob/main/tutorials/cloud_access/euclid-cloud-access.md

Lists the top-level directories within the specified S3 bucket, showing the 'q1' directory containing Euclid Q1 data products.

```python
s3.ls(f'{BUCKET_NAME}/q1')
```

--------------------------------

### List Available Euclid Catalogs

Source: https://github.com/caltech-ipac/irsa-tutorials/blob/main/tutorials/euclid_access/4_Euclid_intro_PHZ_catalog.md

Queries IRSA to list all available catalogs related to the 'euclid' project, providing an overview of the data products.

```ipython3
Irsa.list_catalogs(filter='euclid')
```

--------------------------------

### Install Dependencies

Source: https://github.com/caltech-ipac/irsa-tutorials/blob/main/tutorials/irsa-sia-examples/sia_allwise_atlas.md

Installs necessary Python libraries including matplotlib, astropy, and pyvo for astronomical data processing and visualization.

```python
# Uncomment the next line to install dependencies if needed.
# !pip install matplotlib astropy pyvo
```

--------------------------------

### Initializing Firefly Client

Source: https://github.com/caltech-ipac/irsa-tutorials/blob/main/tutorials/firefly/OpenUniverse2024Preview_Firefly.md

Initializes a Firefly client for interactive visualization. It provides options for JupyterLab environments using `make_lab_client()` and other environments (Jupyter Notebook, Python shell) using `make_client()`. Requires the `jupyter-firefly-extensions` package and a running Firefly server, accessible via the FIREFLY_URL environment variable or passed directly.

```ipython3
# Uncomment when using within Jupyter Lab with jupyter_firefly_extensions installed
# fc = FireflyClient.make_lab_client()

# Uncomment for contexts other than above 
fc = FireflyClient.make_client(url="https://irsa.ipac.caltech.edu/irsaviewer")

fc.reinit_viewer() # to clean the state, if this cell ran earlier
```

--------------------------------

### List MER Images

Source: https://github.com/caltech-ipac/irsa-tutorials/blob/main/tutorials/cloud_access/euclid-cloud-access.md

Lists the first 10 MER (Multi-Epoch) image files available in the S3 bucket, which are stored as FITS files.

```python
s3.ls(f'{BUCKET_NAME}/q1/MER')[:10]
```

--------------------------------

### List Files for a Specific Instrument

Source: https://github.com/caltech-ipac/irsa-tutorials/blob/main/tutorials/cloud_access/euclid-cloud-access.md

Lists files for a specific instrument (e.g., VIS for visible imaging) within a given MER tile ID.

```python
s3.ls(f'{BUCKET_NAME}/q1/MER/102018211/VIS')
```

--------------------------------

### Download and Inspect Euclid MER Mosaic

Source: https://github.com/caltech-ipac/irsa-tutorials/blob/main/tutorials/euclid_access/1_Euclid_intro_MER_images.md

Retrieves the URL and estimated size for a VIS bandpass Euclid MER mosaic image. It then downloads the file, opens it using `astropy.io.fits`, and prints the file information and header.

```ipython3
filename = science_images[science_images['energy_bandpassname'] == 'VIS']['access_url'][0]
filesize = science_images[science_images['energy_bandpassname'] == 'VIS']['access_estsize'][0] / 1000000
print(filename)

print(f'Please note this image is {filesize} GB. With 230 Mbps internet download speed, it takes about 1 minute to download.')
```

```ipython3
fname = download_file(filename, cache=True)
hdu_mer_irsa = fits.open(fname)
print(hdu_mer_irsa.info())

header_mer_irsa = hdu_mer_irsa[0].header
```

```ipython3
# download_path = 'data'
# hdu_mer_irsa.writeto(os.path.join(download_path, 'MER_image_VIS.fits'), overwrite=True)
```

```ipython3
header_mer_irsa
```

```ipython3
im_mer_irsa = hdu_mer_irsa[0].data

print(im_mer_irsa.shape)
```

```ipython3
plt.imshow(im_mer_irsa[0:1200,0:1200], cmap='gray', origin='lower',
           norm=ImageNormalize(im_mer_irsa[0:1200,0:1200], interval=PercentileInterval(99.9), stretch=AsinhStretch()))
colorbar = plt.colorbar()
```

```ipython3
# # Full MER mosaic, may take a minute for python to create this image
# plt.imshow(im_mer_irsa, cmap='gray', origin='lower', norm=ImageNormalize(im_mer_irsa, interval=PercentileInterval(99.9), stretch=AsinhStretch()))
# colorbar = plt.colorbar()
```

--------------------------------

### List Euclid Catalogs

Source: https://github.com/caltech-ipac/irsa-tutorials/blob/main/tutorials/cloud_access/euclid-cloud-access.md

Lists all Euclid catalogs provided by IRSA, with an option to retrieve full details. This helps in identifying relevant catalogs for data access.

```ipython3
catalogs = Irsa.list_catalogs(full=True, filter='euclid')
catalogs
```

--------------------------------

### Import Libraries for Catalog Querying

Source: https://github.com/caltech-ipac/irsa-tutorials/blob/main/tutorials/cloud_access/cloud-access-intro.md

Imports necessary Python libraries for interacting with astronomical catalogs, including HPGeom for spatial indexing, Pandas for data manipulation, and PyArrow for reading Parquet files from S3.

```ipython3
import hpgeom
import pandas as pd
import pyarrow.dataset
import pyarrow.fs
from matplotlib import colors
```

--------------------------------

### Locate and Get Image URL

Source: https://github.com/caltech-ipac/irsa-tutorials/blob/main/tutorials/irsa-sia-examples/sia_2mass_allsky.md

Iterates through the search results to find the first H-band image and retrieves its data URL.

```python
for i in range(len(im_table)):
    if im_table[i]['band'] == 'H':
        break
print(im_table[i].getdataurl())
```

--------------------------------

### Initialize S3 FileSystem

Source: https://github.com/caltech-ipac/irsa-tutorials/blob/main/tutorials/cloud_access/cloud-access-intro.md

Creates an anonymous S3 file system client using the s3fs library for accessing public S3 buckets.

```ipython3
import s3fs

# create an S3 client
s3 = s3fs.S3FileSystem(anon=True)
```

--------------------------------

### List Table Columns

Source: https://github.com/caltech-ipac/irsa-tutorials/blob/main/tutorials/euclid_access/5_Euclid_intro_SPE_catalog.md

Retrieves and prints the number of columns in a specified table. It also shows how to get a full list of columns with descriptions.

```python
columns_info = Irsa.list_columns(catalog=table_mer)
print(len(columns_info))
```

```python
Irsa.list_columns(catalog=table_1dspectra, full=True)
```

```python
# Full list of columns and their description
columns_info
```

--------------------------------

### Locate and Get Image URL

Source: https://github.com/caltech-ipac/irsa-tutorials/blob/main/tutorials/irsa-sia-examples/sia_cosmos.md

Iterates through the image table to find the first image associated with the 'i+' band and retrieves its direct download URL.

```python
for i in range(len(im_table)):
    if im_table[i]['band_name'] == 'i+':
        break
print(im_table[i].getdataurl())
```

--------------------------------

### Get Firefly Display URL

Source: https://github.com/caltech-ipac/irsa-tutorials/blob/main/tutorials/firefly/SEDs_in_Firefly.md

Provides a web link to open the Firefly viewer directly in a browser, useful if the automatically opened tab has issues.

```ipython3
fc.display_url()
```

--------------------------------

### Initialize and Connect to Firefly Client

Source: https://github.com/caltech-ipac/irsa-tutorials/blob/main/tutorials/spherex/spherex_intro.md

Initializes a FireflyClient instance to connect to a Firefly viewer. This allows programmatic control and visualization of astronomical data.

```ipython3
from IPython.display import display
from herschelhelp_dl.firefly import FireflyClient

fc = FireflyClient.make_client(url="https://irsa.ipac.caltech.edu/irsaviewer")

fc.reinit_viewer()
```

--------------------------------

### Construct File URI for Spectrum

Source: https://github.com/caltech-ipac/irsa-tutorials/blob/main/tutorials/euclid_access/4_Euclid_intro_PHZ_catalog.md

Constructs the full URI for the spectral data file by joining the IRSA TAP URL with the 'uri' from the `result_spectra` table.

```ipython3
file_uri = urllib.parse.urljoin(Irsa.tap_url, result_spectra['uri'][0])
file_uri
```

--------------------------------

### Access WCS Information

Source: https://github.com/caltech-ipac/irsa-tutorials/blob/main/tutorials/spherex/spherex_intro.md

Retrieves and displays all keywords starting with 'WCSNAME' from the FITS header. This is used to identify different World Coordinate System (WCS) definitions present in the file.

```ipython3
# Assuming spectral_image_header is a FITS header object

spectral_image_header['WCSNAME*']
```

--------------------------------

### Get CPU Count

Source: https://github.com/caltech-ipac/irsa-tutorials/blob/main/tutorials/parallelize/Parallelize_Convolution.md

Retrieves and prints the number of logical CPUs available on the system using the psutil library. This information is used to configure parallel processing tasks.

```python
num_cpus = psutil.cpu_count(logical=True)
print(num_cpus)
```

--------------------------------

### Query Spectra using TAP and ASQL

Source: https://github.com/caltech-ipac/irsa-tutorials/blob/main/tutorials/euclid_access/4_Euclid_intro_PHZ_catalog.md

Constructs an ASQL query to retrieve spectral data for a given object ID from the 1D spectra table using IRSA's TAP service.

```ipython3
adql_object = f"SELECT * FROM {table_1dspectra} WHERE objectid = {obj_id}"

## Pull the data on this particular galaxy
result_spectra = Irsa.query_tap(adql_object).to_table()
result_spectra
```

--------------------------------

### Example Usage: Load and Process Astronomical Data

Source: https://github.com/caltech-ipac/irsa-tutorials/blob/main/tutorials/roman_simulations/roman_hlss_number_density.md

Demonstrates the practical application of the `read_hdf5_to_pandas` function. It specifies the file path, columns to retain, and columns to convert to a more memory-efficient format before loading.

```ipython3
#don't change this, the above code downloads the files to this directory
file_path = 'downloaded_hdf5_files/Roman_small_V2_0.hdf5'

#out of consideration for the size of these files and the amount of memory required to work with them
# we only keep the columns that we are going to use and convert some to 32bit instead of 64 where 
# the higher precision is not necessary to the science
columns_to_keep = ['RA', 'DEC', 'redshift_observed', 'flux_Halpha6563']
columns_to_convert = ['redshift_observed', 'flux_Halpha6563']

df = read_hdf5_to_pandas(file_path, columns_to_keep, columns_to_convert)

```

--------------------------------

### Define Target Coordinates

Source: https://github.com/caltech-ipac/irsa-tutorials/blob/main/tutorials/firefly/SEDs_in_Firefly.md

Defines the celestial coordinates for a target source using Astropy's SkyCoord object. This specific example uses coordinates from a research paper (J052736.37+344940.6).

```python
target = SkyCoord(ra="05h27m36.37s", dec="+34d49m40.6s")
```

--------------------------------

### Get Rubin Coadd HTTPS URL

Source: https://github.com/caltech-ipac/irsa-tutorials/blob/main/tutorials/firefly/OpenUniverse2024Preview_Firefly.md

Generates the HTTPS URL for the previously selected Rubin coadd image, enabling direct access to the file.

```ipython3
coadd_s3_fpath_rubin = get_rubin_coadd_fpath(filter_rubin)
https_url(coadd_s3_fpath_rubin)
```

--------------------------------

### Inspect Rubin Coadd

Source: https://github.com/caltech-ipac/irsa-tutorials/blob/main/tutorials/firefly/OpenUniverse2024Preview_Firefly.md

Loads and inspects a simulated Rubin coadd for a chosen filter ('r' in this example), making the data and WCS accessible for further analysis and visualization.

```ipython3
filter_rubin = 'r'
coadd_rubin = get_rubin_coadd(filter_rubin)
```

--------------------------------

### Query IRSA for Spitzer SEIP Mosaics and Cloud Access

Source: https://github.com/caltech-ipac/irsa-tutorials/blob/main/tutorials/cloud_access/cloud-access-intro.md

Uses PyVO's SIA2Service to search the IRSA SIA service for Spitzer SEIP Super Mosaics within a specified coordinate region. It then retrieves and displays the cloud access information for the first five overlapping files.

```ipython3
irsa_SIA = pyvo.dal.SIA2Service("https://irsa.ipac.caltech.edu/SIA")
seip_results = irsa_SIA.search((coords, size), collection="spitzer_seip")

# view cloud access info for the first few files overlapping the search area
seip_results["cloud_access"][:5]
```

--------------------------------

### List Available IRSA Collections (Optional)

Source: https://github.com/caltech-ipac/irsa-tutorials/blob/main/tutorials/cloud_access/cloud-access-intro.md

This commented-out cell shows how to list all available datasets (collections) from IRSA that can be used with the SIA search. Note that not all listed datasets are guaranteed to be in cloud storage.

```ipython3
# from astroquery.ipac.irsa import Irsa

# Irsa.list_collections()
```

--------------------------------

### Get Maximum Records and Hard Limit

Source: https://github.com/caltech-ipac/irsa-tutorials/blob/main/tutorials/cosmodc2/cosmoDC2_TAP_access.md

Retrieves and prints the default maximum number of records returned by the service ('maxrec') and the hard limit ('hardlimit'). These settings control the size of query results.

```ipython3
service.maxrec

print(service.hardlimit)
```

--------------------------------

### Initialize IRSA TAP Service

Source: https://github.com/caltech-ipac/irsa-tutorials/blob/main/tutorials/firefly/SEDs_in_Firefly.md

Establishes a connection to the IRSA Table Access Protocol (TAP) service, which is used for querying astronomical catalogs. This setup is typically done once at the beginning of a session.

```ipython3
irsa_tap = pyvo.dal.TAPService("https://irsa.ipac.caltech.edu/TAP")
```

--------------------------------

### Inspect Catalog Schema Fields

Source: https://github.com/caltech-ipac/irsa-tutorials/blob/main/tutorials/cloud_access/cloud-access-intro.md

Demonstrates how to find specific columns within the loaded schema, such as HEALPix partitioning columns, and how to access detailed information about a particular field, including its metadata.

```ipython3
[name for name in schema.names if "healpix" in name]
# the result shows that both orders k=0 and k=5 are available, but typical use cases only need k=5
```

```ipython3
# this will display basic information like name and type
schema.field("ext_flg")
```

```ipython3
# units and descriptions are in the field's metadata attribute
schema.field("ext_flg").metadata
```

--------------------------------

### Load and Read Spectrum Data

Source: https://github.com/caltech-ipac/irsa-tutorials/blob/main/tutorials/cloud_access/euclid-cloud-access.md

Opens a FITS file from an S3 bucket using Astropy's `fits.open` and retrieves a specific spectrum extension. It then reads the spectrum data into an Astropy Table.

```ipython3
with fits.open(f's3://{BUCKET_NAME}/{spec_fpath_key}', fsspec_kwargs={'anon': True}) as hdul:
    spec_hdu = hdul[object_hdu_idx]
    spec_tbl = Table.read(spec_hdu)
```

--------------------------------

### Access IRSA Data from Cloud Storage

Source: https://github.com/caltech-ipac/irsa-tutorials/blob/main/index.md

Guides users on accessing IRSA-curated datasets available in Amazon Web Services (AWS) S3 cloud storage buckets. Covers introductory concepts and specific dataset demonstrations.

```Python
tutorials/cloud_access/cloud-access-intro
tutorials/parquet-catalog-demos/wise-allwise-catalog-demo
tutorials/parquet-catalog-demos/neowise-source-table-strategies
tutorials/parquet-catalog-demos/neowise-source-table-lightcurves
tutorials/openuniversesims/openuniverse2024_roman_simulated_timedomainsurvey
tutorials/openuniversesims/openuniverse2024_roman_simulated_wideareasurvey
tutorials/cloud_access/euclid-cloud-access
```

--------------------------------

### Get Rubin Coadd File Path

Source: https://github.com/caltech-ipac/irsa-tutorials/blob/main/tutorials/firefly/OpenUniverse2024Preview_Firefly.md

Constructs the S3 file path for a simulated Rubin Observatory coadded image given a specific filter. This function is a helper for retrieving the data.

```ipython3
def get_rubin_coadd_fpath(filter):
    coadd_fname_root = f"deepCoadd_calexp_2877_19_{filter}_DC2_u_descdm_preview_data_step3_2877_19_w_2024_12_20240403T150003Z.fits"
    coadd_fpath = f"{BUCKET_NAME}/{RUBIN_COADD_PATH}/{filter}/{coadd_fname_root}"
    return coadd_fpath
```

--------------------------------

### Example Usage: Inspect Roman Coadd

Source: https://github.com/caltech-ipac/irsa-tutorials/blob/main/tutorials/firefly/OpenUniverse2024Preview_Firefly.md

Demonstrates how to use the defined functions to retrieve and inspect a simulated Roman coadd. It sets a sky coordinate and filter, then calls `get_roman_coadd` to fetch the data.

```ipython3
coord = SkyCoord(ra=9.6055383, dec=-44.1895542, unit="deg")
filter_roman = 'H158' #F184, H158, J129, K213, and Y106 are available in the data preview
coadd_roman = get_roman_coadd(coord, filter_roman)
```

--------------------------------

### Select Specific SN Ia and Get Coordinates

Source: https://github.com/caltech-ipac/irsa-tutorials/blob/main/tutorials/openuniversesims/openuniverse2024_roman_simulated_timedomainsurvey.md

Selects a specific supernova by its ID (oid) from the filtered list and retrieves its Right Ascension (RA) and Declination (Dec). It then creates an Astropy SkyCoord object from these coordinates.

```ipython3
# Let's choose SN 20000808.
oid = 20000808
chosen_object = nearby_preview_sn1a[nearby_preview_sn1a['id'] == oid]
ra = chosen_object.get('ra')
dec = chosen_object.get('dec')
ra, dec = 9.619282, -44.313894
coord = SkyCoord(ra*u.deg, dec*u.deg)
```

--------------------------------

### Create Data Directory

Source: https://github.com/caltech-ipac/irsa-tutorials/blob/main/tutorials/euclid_access/4_Euclid_intro_PHZ_catalog.md

Checks if a 'data' directory exists and creates it if it doesn't, for saving output files.

```ipython3
download_path = "data"
if os.path.exists(download_path):
    print("Output directory already created.")
else:
    print("Creating data directory.")
    os.mkdir(download_path)
```

--------------------------------

### Define Roman Simulated Data Blocks

Source: https://github.com/caltech-ipac/irsa-tutorials/blob/main/tutorials/firefly/OpenUniverse2024Preview_Firefly.md

Defines the centers and size of simulated Roman coadd blocks, arranged in a grid. This setup is crucial for mapping sky coordinates to specific data files.

```ipython3
#Centers of roman data preview blocks. Do not alter.
ra_block_centers = np.array([9.76330352298415, 9.724522605135252, 9.68574158906671,
                        9.646960496603766, 9.608179349571955, 9.56939816979703,
                        9.530616979104877, 9.491835799321422, 9.453054652272561,
                        9.414273559784032, 9.375492543681393, 9.336711625789874]) * u.deg
dec_block_centers = np.array([-44.252584927082495, -44.22480733304182, -44.197029724175756,
                            -44.16925210374898, -44.14147447502621, -44.11369684127218,
                            -44.08591920575162, -44.05814157172923, -44.03036394246976,
                            -44.0025863212379, -43.974808711298394, -43.94703111591591]) * u.deg
block_size = 100 * u.arcsec # each block is 100 arcsec across
```

--------------------------------

### Creating Multiwavelength Cutouts

Source: https://github.com/caltech-ipac/irsa-tutorials/blob/main/tutorials/euclid_access/1_Euclid_intro_MER_images.md

Generates image cutouts from FITS files specified by URLs. It defines the cutout size and center coordinates, then iterates through each URL to create a cutout with updated WCS information. The cutouts are stored in a list and combined into a single HDUList.

```ipython3
######################## User defined section ############################
## How large do you want the image cutout to be?
im_cutout = 1.0 * u.arcmin

## What is the center of the cutout?
## For now choosing a random location on the image
## because the star itself is saturated
ra = 273.8667
dec =  64.525

## Bright star position
# ra = 273.474451
# dec = 64.397273

coords_cutout = SkyCoord(ra, dec, unit='deg', frame='icrs')

##########################################################################

## Iterate through each filter

cutout_list = []

for url in urls:
    ## Use fsspec to interact with the fits file without downloading the full file
    hdu = fits.open(url, use_fsspec=True)
    print(f"Opened {url}")

    ## Store the header
    header = hdu[0].header

    ## Read in the cutout of the image that you want
    cutout_data = Cutout2D(hdu[0].section, position=coords_cutout, size=im_cutout, wcs=WCS(hdu[0].header))

    ## Close the file
    # hdu.close()

    ## Define a new fits file based on this smaller cutout, with accurate WCS based on the cutout size
    new_hdu = fits.PrimaryHDU(data=cutout_data.data, header=header)
    new_hdu.header.update(cutout_data.wcs.to_header())

    ## Append the cutout to the list
    cutout_list.append(new_hdu)

## Combine all cutouts into a single HDUList and display information
final_hdulist = fits.HDUList(cutout_list)
final_hdulist.info()
```

--------------------------------

### Construct TAP Query for NEOWISE-R

Source: https://github.com/caltech-ipac/irsa-tutorials/blob/main/tutorials/firefly/NEOWISE_light_curve_demo.md

Builds a URL to query the NEOWISE-R catalog for data related to '558 Carmen' using the TAP protocol. It selects all columns from the 'neowiser_p1ba_mch' table where the object ID starts with '(558)'.

```ipython3
BASE_URL = "https://irsa.ipac.caltech.edu/TAP/sync"
QUERY = """
SELECT *
FROM neowiser_p1ba_mch AS n
WHERE n.objid LIKE '(558)%'
"""
table_url = f"{BASE_URL}?QUERY={urllib.parse.quote_plus(QUERY)}"
table_url
```

--------------------------------

### Import Libraries

Source: https://github.com/caltech-ipac/irsa-tutorials/blob/main/tutorials/euclid_access/1_Euclid_intro_MER_images.md

Imports essential Python libraries for astronomical data processing, including numpy, matplotlib, astropy components (coordinates, fits, nddata, utils, visualization, wcs, units), astroquery.ipac.irsa, and sep.

```python
import re

import numpy as np

import matplotlib.pyplot as plt
from matplotlib.patches import Ellipse

from astropy.coordinates import SkyCoord
from astropy.io import fits
from astropy.nddata import Cutout2D
from astropy.utils.data import download_file
from astropy.visualization import ImageNormalize, PercentileInterval, AsinhStretch,  ZScaleInterval, SquaredStretch
from astropy.wcs import WCS
from astropy import units as u

from astroquery.ipac.irsa import Irsa
import sep
```

--------------------------------

### Import Libraries

Source: https://github.com/caltech-ipac/irsa-tutorials/blob/main/tutorials/parquet-catalog-demos/wise-allwise-catalog-demo.md

Imports essential Python libraries for data manipulation, astronomical calculations, and file system operations. This includes sys, hpgeom, pandas, pyarrow, astropy, matplotlib, and pyarrow's S3FileSystem.

```ipython3
import sys

import hpgeom as hp
import pandas as pd
import pyarrow.compute as pc
import pyarrow.dataset as ds
from astropy import units as u
from astropy.coordinates import SkyCoord
from matplotlib import colors
from matplotlib import pyplot as plt
from pyarrow.fs import S3FileSystem
```

--------------------------------

### List and Describe Table Columns

Source: https://github.com/caltech-ipac/irsa-tutorials/blob/main/tutorials/cosmodc2/cosmoDC2_TAP_access.md

Retrieves the columns for the selected table and prints the name and description for each column. This helps in understanding the available data fields.

```ipython3
columns = tables[tablename].columns
print(len(columns))

for col in columns:
    print(f'{f"{col.name}":30s}  {col.description}')
```

--------------------------------

### Get Roman Coadd File Path

Source: https://github.com/caltech-ipac/irsa-tutorials/blob/main/tutorials/firefly/OpenUniverse2024Preview_Firefly.md

Constructs the S3 file path for a simulated Roman coadd based on a sky coordinate and a specified filter. It uses the `get_block_axis` function to determine the correct row and column.

```ipython3
def get_roman_coadd_fpath(coord, filter):
    col = get_block_axis(ra_block_centers, coord, 'ra')
    row = get_block_axis(dec_block_centers, coord, 'dec')
    
    # Construct the coadd filename from the chosen filter, row, and column.
    coadd_fname_root = f"prod_{filter[0]}_{col}_{row}_map.fits"
    coadd_fpath = f"{BUCKET_NAME}/{ROMAN_COADD_PATH}/{filter}/Row{row}/{coadd_fname_root}"
    return coadd_fpath
```

--------------------------------

### Read Spectrum Data from URI

Source: https://github.com/caltech-ipac/irsa-tutorials/blob/main/tutorials/euclid_access/5_Euclid_intro_SPE_catalog.md

Constructs a file URI from the query results and opens the FITS file to read the spectrum data and header from the specified extension.

```python
file_uri = urllib.parse.urljoin(Irsa.tap_url, result_table2['uri'][0])
file_uri
```

```python
with fits.open(file_uri) as hdul:
    spectrum = QTable.read(hdul[result_table2['hdu'][0]], format='fits')
    spec_header = hdul[result_table2['hdu'][0]].header
```

--------------------------------

### Get MJD of Pointing

Source: https://github.com/caltech-ipac/irsa-tutorials/blob/main/tutorials/openuniversesims/openuniverse2024_roman_simulated_timedomainsurvey.md

Defines a function to retrieve the Modified Julian Date (MJD) for a given pointing from a simulated observation sequence file. It opens a FITS file, reads the observation data into an Astropy Table, and extracts the date for the specified pointing.

```ipython3
def get_mjd(pointing,
            obseq_path=f's3://nasa-irsa-simulations/openuniverse2024/roman/preview/RomanTDS/Roman_TDS_obseq_11_6_23.fits'):

    """
    Retrieve MJD of a given pointing.

    :param pointing: Pointing ID.
    :type pointing: int
    :param obseq_path: Path to obseq file Roman_TDS_obseq_11_6_23.fits.
    :type obseq_path: str, optional
    :return: MJD of specified pointing.
    :rtype: float
    """

    with fits.open(obseq_path, fsspec_kwargs={"anon": True}) as obs:
        obseq = Table(obs[1].data)
    mjd = float(obseq['date'][int(pointing)])

    return mjd
```