### Install Datalab from Built Package

Source: https://github.com/astro-datalab/datalab/blob/master/README.md

Installs the 'astro-datalab' package from the locally built wheel file. Replace '<version>' with the actual version number. The '--user' flag can be used for private Python repository installations.

```bash
pip install dist/astro_datalab-<version>-py3-none-any.whl
pip install --user dist/astro_datalab-<version>-py3-none-any.whl

```

--------------------------------

### Upgrade Pip and Setuptools

Source: https://github.com/astro-datalab/datalab/blob/master/README.md

Ensures that pip and setuptools are updated to their latest versions. This is a prerequisite for building and installing Python packages from source to avoid potential compatibility issues.

```bash
python -m pip install --upgrade pip setuptools

```

--------------------------------

### Get Datalab Command Help

Source: https://github.com/astro-datalab/datalab/blob/master/README.md

Provides a list of all available datalab commands (tasks) and their basic usage. This helps users understand the capabilities of the command-line client.

```bash
datalab --help
```

--------------------------------

### Install Datalab Client via Pip

Source: https://github.com/astro-datalab/datalab/blob/master/README.md

Installs or upgrades the 'astro-datalab' Python package using pip. This is the recommended and simplest method for installing the 'datalab' command-line client and associated libraries. Ensure pip is up-to-date for the best experience.

```bash
pip install --upgrade astro-datalab

```

--------------------------------

### Build Datalab Package

Source: https://github.com/astro-datalab/datalab/blob/master/README.md

Builds the 'astro-datalab' package from source using the 'build' tool. This step generates distribution files (e.g., .whl) in the 'dist' directory, which are then used for installation.

```bash
python -m pip install build
python -m build

```

--------------------------------

### Clone Datalab Repository

Source: https://github.com/astro-datalab/datalab/blob/master/README.md

Clones the 'astro-datalab/datalab' repository from GitHub and changes the current directory to the cloned repository. This is the first step for installing the client from source.

```bash
git clone git@github.com:astro-datalab/datalab.git && \
   cd datalab

```

--------------------------------

### Access VOSpace Programmatically with Low-Level Client

Source: https://context7.com/astro-datalab/datalab/llms.txt

Provides examples for interacting with the Virtual Observatory Space (VOSpace) using a low-level client. Operations include getting node information, listing directory contents, reading and writing files, streaming large files, copying, moving, deleting files, and creating directories. Requires a VOSpace authentication token.

```python
import vos

token = 'your_datalab_token_here'
client = vos.Client(vospace_token=token)

# Get node information
node = client.get_node('vos://datalab.noirlab.edu!vospace/myfile.csv')
print(f"Node type: {node.type}")
print(f"Node URI: {node.uri}")

# List directory contents
file_list = client.listdir('vos://datalab.noirlab.edu!vospace/mydata/')
for filename in file_list:
    print(filename)

# Open and read file directly
with client.open('vos://datalab.noirlab.edu!vospace/data.txt', mode='r') as f:
    content = f.read()
    print(content)

# Stream large file
with client.open('vos://datalab.noirlab.edu!vospace/large.csv', mode='r') as f:
    for line in f:
        if 'important_value' in line:
            print(line.strip())

# Write file to VOSpace
data_to_write = "RA,Dec,Mag\n150.0,2.0,18.5\n150.1,2.1,19.2\n"
with client.open('vos://datalab.noirlab.edu!vospace/output.csv', mode='w') as f:
    f.write(data_to_write)

# Copy file
client.copy('vos://datalab.noirlab.edu!vospace/source.fits',
            'vos://datalab.noirlab.edu!vospace/backup/dest.fits')

# Move file
client.move('vos://datalab.noirlab.edu!vospace/old_name.csv',
            'vos://datalab.noirlab.edu!vospace/new_name.csv')

# Delete file
client.delete('vos://datalab.noirlab.edu!vospace/temp.dat')

# Create directory
client.mkdir('vos://datalab.noirlab.edu!vospace/new_folder')
```

--------------------------------

### Runner Capability Configuration Example

Source: https://github.com/astro-datalab/datalab/blob/master/caps/README.md

This configuration file defines a 'runner' capability that executes a shell command. It requires 'name', 'version', and 'cmd' parameters. The 'cmd' parameter supports variable substitution, as demonstrated with the 'message' variable.

```shell
name=runner
version=1
cmd={echo `date` $message >> logfile}
message="New file added"
```

--------------------------------

### Check Datalab Version

Source: https://github.com/astro-datalab/datalab/blob/master/README.md

Displays the currently installed version of the datalab command-line client. This is a simple command to verify the installation and version number.

```bash
datalab --version
```

--------------------------------

### Get Specific Datalab Task Help

Source: https://github.com/astro-datalab/datalab/blob/master/README.md

Retrieves detailed help and argument summaries for a specific datalab task, such as 'login'. This shows required and optional parameters for the task.

```bash
datalab login help
```

--------------------------------

### Install Pycurl using Conda

Source: https://github.com/astro-datalab/datalab/blob/master/README.md

Installs the 'pycurl' package from the 'conda-forge' channel using Conda. This is an alternative method for installing 'pycurl', particularly useful for macOS ARM users if pip-based installation fails to resolve import errors.

```bash
conda install -c conda-forge pycurl

```

--------------------------------

### Basic Python Query with Datalab

Source: https://github.com/astro-datalab/datalab/blob/master/README.md

Demonstrates how to perform a basic synchronous database query using the 'datalab' Python client. It imports the query client and executes a SQL query to fetch RA and Dec from the 'smash_dr1.object' table, printing the results. This requires the 'astro-datalab' package to be installed.

```python
>>> from dl import queryClient as qc
>>> result = qc.query(sql='SELECT TOP 5 ra,dec from smash_dr1.object')
>>> print(result)
ra,dec
296.0702105660565,-75.58008799398345
296.0689079309987,-75.57850708319104
296.0695746063349,-75.5771115243687
296.0734998386567,-75.57729189836104
296.074467291614,-75.57941799334213

```

--------------------------------

### Check Datalab Version on macOS ARM

Source: https://github.com/astro-datalab/datalab/blob/master/README.md

A diagnostic command to check the installed version of the 'datalab' package. This is useful for troubleshooting potential import errors, particularly on macOS ARM architectures where version mismatches can occur.

```python
python -c "import dl; print(dl.__version__)"

```

--------------------------------

### Reinstall Pycurl using Pip

Source: https://github.com/astro-datalab/datalab/blob/master/README.md

Uninstalls and then reinstalls the 'pycurl' package using pip, with caching disabled. This is a common step in resolving 'pycurl' import issues on macOS ARM by ensuring a fresh installation that is compatible with the system's libcurl.

```bash
pip uninstall pycurl
pip install --no-cache-dir pycurl

```

--------------------------------

### Resolve Astronomical Object Names to Coordinates

Source: https://context7.com/astro-datalab/datalab/llms.txt

Converts astronomical object names (e.g., 'M31', 'M51') into celestial coordinates (RA, Dec) using the Sesame name resolver via the `resolve` function from `dl.helpers.utils`. The resolved coordinates can be directly used in queries, for example, to find sources within a specific radial distance.

```python
from dl.helpers.utils import resolve
from dl import queryClient as qc

# Resolve single object
coords = resolve('M31')
print(f"RA: {coords.ra.deg} deg, Dec: {coords.dec.deg} deg")
print(f"RA (hms): {coords.ra.to_string('h')}, Dec: {coords.dec.to_string('d')}")

# Resolve and use in query
m51_coords = resolve('M51')
ra, dec = m51_coords.ra.deg, m51_coords.dec.deg

result = qc.query(sql=f'''
    SELECT ra, dec, gmag
    FROM gaia_dr3.gaia_source
    WHERE q3c_radial_query(ra, dec, {ra}, {dec}, 0.5)
    ORDER BY gmag
    LIMIT 100
''', fmt='pandas')
print(f"Found {len(result)} sources near M51")
```

--------------------------------

### List Available Services with queryClient

Source: https://github.com/astro-datalab/datalab/blob/master/RELEASE_NOTES.md

Demonstrates how to list available services using the `services()` function in `queryClient`. It shows how to filter by service type (e.g., 'sia') and by name using pattern matching (e.g., 'usno%').

```python
from noao.datalab import queryClient

# List the available SIA services
queryClient.services(svc_type="sia")

# List the available USNO services, note the '%' matching metacharacter
queryClient.services(name="usno%")
```

--------------------------------

### Execute SQL Queries Asynchronously with queryClient

Source: https://github.com/astro-datalab/datalab/blob/master/RELEASE_NOTES.md

Shows how to run SQL queries asynchronously using the `query()` function in `queryClient`. It demonstrates submitting a query, checking its status, and retrieving results, similar to ADQL queries but with standard SQL.

```python
from noao.datalab import queryClient

query = 'SELECT ra,dec,gmag,rmag,imag FROM smash_dr1.object LIMIT 10' # NEW: SQL
jobid = queryClient.query(query, async=True)
queryClient.status(jobid)
# Expected output: 'COMPLETED'
results = queryClient.results(jobid)
```

--------------------------------

### Login to Datalab

Source: https://github.com/astro-datalab/datalab/blob/master/README.md

Initiates the login process for the datalab service, prompting the user for credentials if not provided on the command line. A successful login generates a new configuration file if one is missing.

```bash
datalab login
```

--------------------------------

### Query Data Using Datalab Python Module

Source: https://github.com/astro-datalab/datalab/blob/master/README.md

Demonstrates how to use the 'dl' Python module to connect to Data Lab and execute SQL queries. It shows importing the query client and printing the results of a sample query.

```python
from dl import queryClient as qc
result = qc.query(sql='SELECT ra,dec from smash_dr1.object LIMIT 10')
print(result)
```

--------------------------------

### Index Tables with mydb_index() in queryClient

Source: https://github.com/astro-datalab/datalab/blob/master/RELEASE_NOTES.md

Demonstrates the usage of the `mydb_index()` method in `queryClient` for creating indexes on table columns. It covers basic column indexing and advanced indexing with Q3C clustering.

```python
from noao.datalab import queryClient

# Index the table's "id" column
queryClient.mydb_index('foo1', 'id')

# Index and cluster the table by position
queryClient.mydb_index('foo1', q3c='ra,dec', cluster=True)
```

--------------------------------

### Execute Data Lab Command-Line Operations

Source: https://context7.com/astro-datalab/datalab/llms.txt

Illustrates common operations performed using the Data Lab command-line interface (CLI). This includes logging in, querying databases, managing asynchronous jobs, uploading/downloading files to/from VOSpace, managing MyDB tables, and retrieving schema/service information.

```bash
# Login to Data Lab
datalab login
# Prompts for: username, password

# Query database and save to file
datalab query --sql="SELECT * FROM gaia_dr3.gaia_source WHERE parallax > 10 LIMIT 1000" \
  --out=high_parallax.csv --fmt=csv

# Asynchronous query for large dataset
datalab query --sql="SELECT * FROM ls_dr10.tractor WHERE z < 18" \
  --async=True --fmt=csv
# Returns: Job ID for checking status

# Check async query status
datalab qstatus --jobId=<job_id>

# Retrieve async query results
datalab qresults --jobId=<job_id> --out=results.csv

# Upload file to VOSpace
datalab put --fr=local_data.csv --to=vos://mydata/uploaded.csv

# Download file from VOSpace
datalab get --fr=vos://mydata/results.fits --to=local_results.fits

# List VOSpace directory
datalab ls --name=vos://mydata/ --format=table

# Create MyDB table from CSV
datalab mydb_import --table=my_catalog --data=sources.csv

# Query MyDB table
datalab query --sql="SELECT * FROM mydb://my_catalog WHERE mag < 20"

# Index MyDB table for performance
datalab mydb_index --table=my_catalog --column="ra,dec" --q3c=True

# Get schema information
datalab schema --value=gaia_dr3.gaia_source

# List available services
datalab services --svc_type=tap

# Check version
datalab version

# Show service URLs
datalab svc_urls
```

--------------------------------

### Manage Personal Database Tables (MyDB) (Python)

Source: https://context7.com/astro-datalab/datalab/llms.txt

Create, populate, and query personal database tables (MyDB) within Astro Data Lab. Supports automatic schema inference, indexing (including spatial indexing with Q3C), and joining user data with public catalogs. Includes functions to list, copy, and drop tables.

```python
from dl import queryClient as qc

# Create a new MyDB table with explicit schema
schema = 'id INT, ra FLOAT, dec FLOAT, gmag FLOAT, source VARCHAR(50)'
qc.mydb_create('my_sources', schema)

# Import CSV data into the table
qc.mydb_import('my_sources', 'local_catalog.csv')

# Create spatial index for fast coordinate queries
qc.mydb_index('my_sources', 'ra,dec', q3c=True, cluster=True)

# Query your personal table with joins to public catalogs
result = qc.query(sql='''
    SELECT m.id, m.ra, m.dec, g.parallax, g.pmra, g.pmdec
    FROM mydb://my_sources AS m
    JOIN gaia_dr3.gaia_source AS g
    ON q3c_join(m.ra, m.dec, g.ra, g.dec, 0.0003)
    WHERE m.gmag < 18
''', fmt='pandas')
print(f"Cross-matched {len(result)} sources")

# List all your MyDB tables
tables = qc.mydb_list(table=None, index=False)
print(tables)

# Copy table for backup
qc.mydb_copy('my_sources', 'my_sources_backup')

# Drop table when no longer needed
qc.mydb_drop('my_sources_backup')

```

--------------------------------

### Discover Database Schemas and Services

Source: https://context7.com/astro-datalab/datalab/llms.txt

Explores how to retrieve schema information for databases and tables, list available data services, and view query profiles. This functionality relies on the `queryClient` from the `dl` library. Outputs can be in text or JSON format, allowing for programmatic parsing of schema details and service endpoints.

```python
from dl import queryClient as qc
import json

# Get full schema information for a database
schema_info = qc.schema('gaia_dr3', format='text')
print(schema_info)

# Get schema for specific table
table_schema = qc.schema('gaia_dr3.gaia_source', format='text')
print(table_schema)

# Get schema in JSON format for parsing
schema_json = qc.schema('smash_dr1', format='json')
data = json.loads(schema_json)
for table in data['tables']:
    print(f"Table: {table['name']}, Rows: {table.get('nrows', 'unknown')}")

# List all available data services
services = qc.services(name=None, svc_type=None, format='text')
print(services)

# Filter services by type (e.g., TAP, SCS, SIA)
tap_services = qc.services(svc_type='tap', format='text')
print(tap_services)

# List query profiles (different backend configurations)
profiles = qc.list_profiles(format='text')
print(profiles)

# Use specific profile for query
result = qc.query(sql='SELECT * FROM ls_dr10.tractor LIMIT 10',
                  profile='high-memory')
```

--------------------------------

### Query Data and Manage Authentication

Source: https://context7.com/astro-datalab/datalab/llms.txt

Demonstrates how to query data using a demo token, log out, and check service availability. It requires the `dl` library and an authentication client (`ac`) and query client (`qc`). The output of the query is a result object, and the `isAlive` function returns a boolean.

```python
from dl import queryClient as qc
from dl import authClient as ac

# Use demo account for testing
result = qc.query(sql='SELECT * FROM gaia_dr3.gaia_source LIMIT 100',
                  token=ac.DEMO_TOKEN)

# Logout
ac.logout(token)

# Check service availability
alive = ac.isAlive()
print(f"Auth service alive: {alive}")
```

--------------------------------

### Create .netrc file for VOSpace

Source: https://github.com/astro-datalab/datalab/blob/master/README.md

Creates an empty '.netrc' file in the user's home directory. This file is necessary for authenticating with remote storage services like VOSpace when mounting it as a local filesystem.

```bash
touch ~/.netrc

```

--------------------------------

### Upload and Download Files to Cloud Storage

Source: https://context7.com/astro-datalab/datalab/llms.txt

Manage files in VOSpace cloud storage, including uploading, downloading, listing, creating directories, copying, moving, and deleting files and directories.

```APIDOC
## Upload and Download Files to Cloud Storage

### Description
Manage files in VOSpace cloud storage with operations for uploading, downloading, listing, directory management, copying, moving, tagging, and deleting files and directories.

### Method
`put`
`get`
`ls`
`mkdir`
`cp`
`mv`
`stat`
`tag`
`rm`
`rmdir`

### Endpoint
N/A (Client library functions)

### Parameters
#### Path Parameters
N/A

#### Query Parameters
N/A

#### Request Body
- **local_path** (string) - Required for `put`. The path to the local file.
- **vos_path** (string) - Required for `put`, `get`, `cp`, `mv`, `stat`, `tag`, `rm`. The VOSpace path (e.g., 'vos://mybucket/myfile.txt').
- **vos_path_dest** (string) - Required for `cp`, `mv`. The destination VOSpace path.
- **format** (string) - Optional for `ls`. The desired output format (e.g., 'csv').
- **tag_string** (string) - Required for `tag`. A comma-separated string of key=value tags.

### Request Example
```python
from dl import storeClient as sc

# Upload file
sc.put('local_results.csv', 'vos://mydata/results.csv', verbose=True)

# Download file
sc.get('vos://mydata/results.csv', 'downloaded_results.csv', verbose=True)

# List directory
files = sc.ls('vos://mydata/', format='csv')
print(files)

# Create directory
sc.mkdir('vos://mydata/archive/')

# Copy file
sc.cp('vos://mydata/results.csv', 'vos://mydata/archive/results_backup.csv')

# Move file
sc.mv('vos://mydata/temp.csv', 'vos://mydata/final.csv')

# Get metadata
info = sc.stat('vos://mydata/results.csv')
print(info)

# Tag file
sc.tag('vos://mydata/results.csv', 'processed=2024-01-15,quality=good')

# Delete file
sc.rm('vos://mydata/temp.csv')

# Delete directory
sc.rmdir('vos://mydata/old_archive/')
```

### Response
#### Success Response
- **put, get, mkdir, cp, mv, tag, rm, rmdir**: None (operations are performed in place).
- **ls**: List of files and directories in the specified path, format depends on the `format` parameter.
- **stat**: Dictionary containing file metadata.

#### Response Example
```json
# ls (CSV format) example:
# path,size,last_modified
# vos://mydata/results.csv,1024,2024-01-15T10:00:00Z

# stat example:
# {'size': 1024, 'last_modified': '2024-01-15T10:00:00Z', 'tags': {'processed': '2024-01-15', 'quality': 'good'}}
```
```

--------------------------------

### Manage Personal Database Tables (MyDB)

Source: https://context7.com/astro-datalab/datalab/llms.txt

Create, populate, and query personal database tables with automatic schema inference and indexing. Supports importing data from CSV and performing spatial queries.

```APIDOC
## Manage Personal Database Tables (MyDB)

### Description
Create, populate, and query personal database tables with automatic schema inference and indexing. Supports importing data from CSV and performing spatial queries.

### Method
`mydb_create`
`mydb_import`
`mydb_index`
`mydb_list`
`mydb_copy`
`mydb_drop`
`query` (for querying MyDB tables)

### Endpoint
N/A (Client library functions)

### Parameters
#### Path Parameters
N/A

#### Query Parameters
N/A

#### Request Body
- **schema** (string) - Required for `mydb_create`. Defines the table schema (e.g., 'id INT, ra FLOAT').
- **table** (string) - Required for `mydb_import`, `mydb_index`, `mydb_copy`, `mydb_drop`. The name of the MyDB table.
- **q3c** (boolean) - Optional for `mydb_index`. If True, creates a Q3C spatial index.
- **cluster** (boolean) - Optional for `mydb_index`. If True, clusters the index.
- **table_from** (string) - Required for `mydb_copy`. The name of the table to copy from.
- **table_to** (string) - Required for `mydb_copy`. The name of the new table.

### Request Example
```python
from dl import queryClient as qc

# Create table
schema = 'id INT, ra FLOAT, dec FLOAT, gmag FLOAT, source VARCHAR(50)'
qc.mydb_create('my_sources', schema)

# Import data
qc.mydb_import('my_sources', 'local_catalog.csv')

# Create index
qc.mydb_index('my_sources', 'ra,dec', q3c=True, cluster=True)

# Query table
result = qc.query(sql='SELECT m.id, m.ra, m.dec FROM mydb://my_sources AS m WHERE m.gmag < 18', fmt='pandas')
print(result)

# List tables
tables = qc.mydb_list()
print(tables)

# Copy table
qc.mydb_copy('my_sources', 'my_sources_backup')

# Drop table
qc.mydb_drop('my_sources_backup')
```

### Response
#### Success Response
- **mydb_create**: None
- **mydb_import**: None
- **mydb_index**: None
- **mydb_list**: List of table names (list of strings).
- **mydb_copy**: None
- **mydb_drop**: None
- **query**: Query results in the specified format.

#### Response Example
```json
# mydb_list example:
# ['my_sources', 'my_sources_backup']
```
```

--------------------------------

### Resolve Service URL with queryClient

Source: https://github.com/astro-datalab/datalab/blob/master/RELEASE_NOTES.md

Illustrates how to retrieve the specific service URL for a given service name using the `services()` function with the `mode='resolve'` argument in `queryClient`.

```python
from noao.datalab import queryClient

# Get the serviceURL of the USNO-A2 table
queryClient.services(name="usno/a2", mode="resolve")
```

--------------------------------

### Manage Files in VOSpace Cloud Storage (Python)

Source: https://context7.com/astro-datalab/datalab/llms.txt

Interact with VOSpace cloud storage for uploading, downloading, and managing data files. Supports directory operations like creation and listing, file operations like copy, move, and delete, and metadata tagging.

```python
from dl import storeClient as sc

# Upload local file to VOSpace
sc.put('local_results.csv', 'vos://mydata/results.csv', verbose=True)

# Download file from VOSpace
sc.get('vos://mydata/results.csv', 'downloaded_results.csv', verbose=True)

# List directory contents
files = sc.ls('vos://mydata/', format='csv')
print(files)

# Create directory
sc.mkdir('vos://mydata/archive/')

# Copy file within VOSpace
sc.cp('vos://mydata/results.csv', 'vos://mydata/archive/results_backup.csv')

# Move/rename file
sc.mv('vos://mydata/temp.csv', 'vos://mydata/final.csv')

# Get file metadata
info = sc.stat('vos://mydata/results.csv')
print(info)

# Tag/annotate a file
sc.tag('vos://mydata/results.csv', 'processed=2024-01-15,quality=good')

# Delete file
sc.rm('vos://mydata/temp.csv')

# Delete directory (must be empty)
sc.rmdir('vos://mydata/old_archive/')

```

--------------------------------

### Query Database Catalogs

Source: https://context7.com/astro-datalab/datalab/llms.txt

Execute SQL/ADQL queries against astronomical databases synchronously or asynchronously, with support for various output formats including CSV and Pandas DataFrames. Also provides functionality to manage asynchronous job status and retrieve results.

```APIDOC
## Query Database Catalogs

### Description
Execute SQL/ADQL queries against astronomical databases synchronously or asynchronously. Supports various output formats and asynchronous job management.

### Method
`query` (synchronous and asynchronous)
`status`
`wait`
`results`

### Endpoint
N/A (Client library functions)

### Parameters (for `query` function)
#### Query Parameters
- **sql** (string) - Required - The SQL or ADQL query to execute.
- **fmt** (string) - Optional - The desired output format (e.g., 'csv', 'pandas', 'json'). Defaults to CSV.
- **async_** (boolean) - Optional - If True, executes the query asynchronously and returns a job ID.

#### Request Body
N/A

### Request Example
```python
from dl import queryClient as qc

# Synchronous query
result = qc.query(sql='SELECT TOP 5 ra,dec,gmag FROM smash_dr1.object')
print(result)

# Asynchronous query
jobId = qc.query(sql='SELECT * FROM gaia_dr3.gaia_source WHERE parallax > 10', async_=True)
print(f"Job submitted: {jobId}")

# Check status
status_info = qc.status(jobId)
print(status_info)

# Wait and retrieve results
qc.wait(jobId, wait=5, verbose=True)
results = qc.results(jobId, delete=True)
```

### Response
#### Success Response
- **result** (string or pandas.DataFrame) - The query results in the specified format if synchronous.
- **jobId** (string) - The ID of the submitted job if asynchronous.
- **status_info** (dict) - Information about the job status.

#### Response Example
```json
# Synchronous CSV output example:
# ra,dec,gmag
# 296.0702105660565,-75.58008799398345,18.234

# Asynchronous job submission example:
# Job submitted: 12345-abcde

# Status info example:
# {'status': 'COMPLETED', 'result_url': '...'}
```
```

--------------------------------

### Remove Old Datalab Configuration File

Source: https://github.com/astro-datalab/datalab/blob/master/README.md

Removes the old datalab configuration file to force regeneration with updated domain information. This is necessary when upgrading from versions prior to v2.20.0. After removal, any datalab command will create a new configuration file.

```bash
rm $HOME/.datalab/dl.conf
```

--------------------------------

### Authenticate with Astro Data Lab Services (Python)

Source: https://context7.com/astro-datalab/datalab/llms.txt

Manage authentication for accessing Astro Data Lab services. Includes functions for logging in with credentials, logging out, checking current user status, validating tokens, and using anonymous access.

```python
from dl import authClient as ac

# Login with username and password
token = ac.login('myusername', 'mypassword')
print(f"Logged in with token: {token}")

# Check who is currently logged in
user = ac.whoAmI()
print(f"Current user: {user}")

# Validate a token
is_valid = ac.isValidToken(token)
print(f"Token valid: {is_valid}")

# Check if user is logged in
logged_in = ac.isUserLoggedIn('myusername')
print(f"User logged in: {logged_in}")

# Use anonymous access (no login required)
result = qc.query(sql='SELECT TOP 5 * FROM smash_dr1.object',
                  token=ac.ANON_TOKEN)

```

--------------------------------

### Authenticate with Data Lab Services

Source: https://context7.com/astro-datalab/datalab/llms.txt

Manage authentication for accessing Astro Data Lab services, including logging in, logging out, checking current user, validating tokens, and anonymous access.

```APIDOC
## Authenticate with Data Lab Services

### Description
Manage authentication for accessing Astro Data Lab services, including logging in with credentials, checking the current user, validating tokens, checking login status, and using anonymous access.

### Method
`login`
`logout`
`whoAmI`
`isValidToken`
`isUserLoggedIn`

### Endpoint
N/A (Client library functions)

### Parameters
#### Path Parameters
N/A

#### Query Parameters
N/A

#### Request Body
- **username** (string) - Required for `login`. The user's username.
- **password** (string) - Required for `login`. The user's password.
- **token** (string) - Required for `isValidToken`. The token to validate.
- **username** (string) - Required for `isUserLoggedIn`. The username to check.

### Request Example
```python
from dl import authClient as ac

# Login
token = ac.login('myusername', 'mypassword')
print(f"Logged in with token: {token}")

# Check current user
user = ac.whoAmI()
print(f"Current user: {user}")

# Validate token
is_valid = ac.isValidToken(token)
print(f"Token valid: {is_valid}")

# Check if user is logged in
logged_in = ac.isUserLoggedIn('myusername')
print(f"User logged in: {logged_in}")

# Use anonymous token (example with queryClient)
from dl import queryClient as qc
result = qc.query(sql='SELECT TOP 5 * FROM smash_dr1.object', token=ac.ANON_TOKEN)
```

### Response
#### Success Response
- **login**: The authentication token (string).
- **whoAmI**: The username of the current user (string).
- **isValidToken**: Boolean indicating if the token is valid.
- **isUserLoggedIn**: Boolean indicating if the user is logged in.

#### Response Example
```json
# login example:
# Logged in with token: eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9...

# whoAmI example:
# Current user: myusername

# isValidToken example:
# Token valid: true

# isUserLoggedIn example:
# User logged in: true
```
```

--------------------------------

### Convert Query Results to Different Formats

Source: https://context7.com/astro-datalab/datalab/llms.txt

Shows how to convert query results obtained as CSV strings into various data structures like Pandas DataFrames, NumPy arrays, and Astropy Tables using the `convert` helper function. It also demonstrates directly querying in specified formats (e.g., 'pandas', 'votable'). Dependencies include `dl.helpers.utils` and optionally `pandas` and `astropy.table`.

```python
from dl import queryClient as qc
from dl.helpers.utils import convert
import pandas as pd
from astropy.table import Table

# Get result as CSV string
csv_result = qc.query(sql='SELECT ra,dec,gmag,rmag FROM gaia_dr3.gaia_source LIMIT 1000')

# Convert to Pandas DataFrame
df = convert(csv_result, 'pandas')
print(df.describe())
print(f"DataFrame shape: {df.shape}")

# Convert to NumPy array
arr = convert(csv_result, 'array')
print(f"Array shape: {arr.shape}, dtype: {arr.dtype}")

# Convert to NumPy structured array with column names
struct_arr = convert(csv_result, 'structarray')
print(f"Column names: {struct_arr.dtype.names}")
print(f"RA values: {struct_arr['ra'][:5]}")

# Convert to Astropy Table
table = convert(csv_result, 'table')
print(table.info)
print(table['ra', 'dec'][:10])

# Direct query with format specification
df_direct = qc.query(sql='SELECT * FROM smash_dr1.object LIMIT 100', fmt='pandas')
# Returns Pandas DataFrame directly

# Query with VOTable format
votable_result = qc.query(sql='SELECT * FROM gaia_dr3.gaia_source LIMIT 50',
                          fmt='votable')
# Returns VOTable XML string
```

--------------------------------

### Find Clusters in 2D Coordinate Data using Machine Learning

Source: https://context7.com/astro-datalab/datalab/llms.txt

Applies machine learning clustering algorithms (K-means, DBSCAN) to 2D coordinate data, likely from astronomical sources like Gaia. It involves querying data, finding clusters, constructing outlines, and plotting the results. Dependencies include 'dl.helpers.cluster', 'numpy', and 'matplotlib.pyplot'.

```python
from dl.helpers.cluster import findClusters, constructOutlines
import numpy as np
import matplotlib.pyplot as plt
from dl import queryClient as qc

result = qc.query(sql='''
    SELECT ra, dec
    FROM gaia_dr3.gaia_source
    WHERE ra BETWEEN 150 AND 151
      AND dec BETWEEN 2 AND 3
      AND parallax > 5
''', fmt='pandas')

x, y = result['ra'].values, result['dec'].values

# Find clusters using K-means
labels = findClusters(x, y, method='MiniBatchKMeans', n_clusters=5)
print(f"Found {len(np.unique(labels))} clusters")

# Find clusters using DBSCAN
labels_db = findClusters(x, y, method='DBSCAN', eps=0.01, min_samples=10)
n_clusters = len(set(labels_db)) - (1 if -1 in labels_db else 0)
print(f"DBSCAN found {n_clusters} clusters, {sum(labels_db == -1)} noise points")

# Construct convex hull outlines
outlines = constructOutlines(x, y, labels)

# Plot results
plt.figure(figsize=(10, 8))
scatter = plt.scatter(x, y, c=labels, cmap='viridis', s=5, alpha=0.6)
plt.colorbar(scatter, label='Cluster ID')

for cluster_id, outline in outlines.items():
    if outline is not None:
        plt.plot(outline[:, 0], outline[:, 1], 'r-', linewidth=2)

plt.xlabel('RA (deg)')
plt.ylabel('Dec (deg)')
plt.title('Spatial Clustering of Gaia Sources')
plt.savefig('clusters.png', dpi=150)
```

--------------------------------

### Update Homebrew and Curl on macOS ARM

Source: https://github.com/astro-datalab/datalab/blob/master/README.md

Updates Homebrew and the 'curl' package on macOS. This is part of the troubleshooting steps for macOS ARM users experiencing pycurl import errors, aiming to resolve version mismatches with libcurl.

```bash
brew update
brew upgrade curl

```

--------------------------------

### Query Astronomical Databases with SQL/ADQL (Python)

Source: https://context7.com/astro-datalab/datalab/llms.txt

Execute synchronous and asynchronous SQL/ADQL queries against astronomical databases. Supports various output formats including CSV and Pandas DataFrames. Asynchronous queries allow for processing large datasets, with functions to check status, wait for completion, and retrieve results.

```python
from dl import queryClient as qc

# Simple synchronous query returning CSV
result = qc.query(sql='SELECT TOP 5 ra,dec,gmag FROM smash_dr1.object')
print(result)
# Output:
# ra,dec,gmag
# 296.0702105660565,-75.58008799398345,18.234
# 296.0689079309987,-75.57850708319104,19.123
# ...

# Query with Pandas DataFrame output
import pandas as pd
df = qc.query(sql='SELECT ra,dec,gmag FROM gaia_dr3.gaia_source WHERE gmag < 15 LIMIT 1000',
              fmt='pandas')
print(df.head())
print(f"Retrieved {len(df)} stars")

# Asynchronous query for large datasets
jobId = qc.query(sql='SELECT * FROM gaia_dr3.gaia_source WHERE parallax > 10',
                 async_=True)
print(f"Job submitted: {jobId}")

# Check query status
status_info = qc.status(jobId)
print(status_info)

# Wait for completion and retrieve results
qc.wait(jobId, wait=5, verbose=True)
results = qc.results(jobId, delete=True)  # Retrieve and delete from server

```

--------------------------------

### Cross-Match Coordinate Catalogs Locally

Source: https://context7.com/astro-datalab/datalab/llms.txt

Facilitates fast, local cross-matching between two sets of astronomical coordinates using the `xmatch` function from `dl.helpers.crossmatch`. It accepts NumPy arrays for RA and Dec, and allows specifying a maximum distance and units for matching. Can also be used to cross-match query results with external catalogs loaded from files.

```python
from dl.helpers.crossmatch import xmatch
import numpy as np
import pandas as pd
from dl import queryClient as qc

# Two coordinate catalogs
ra1 = np.array([150.12, 150.15, 150.18, 150.21])
dec1 = np.array([2.05, 2.08, 2.11, 2.14])

ra2 = np.array([150.125, 150.155, 150.175, 150.195, 150.215])
dec2 = np.array([2.055, 2.085, 2.105, 2.125, 2.145])

# Find matches within 1 arcsecond
idx, distances = xmatch(ra1, dec1, ra2, dec2,
                        maxdist=1.0, units='arcsec')

print(f"Found {len(idx)} matches")
print(f"Matched indices: {idx}")
print(f"Match distances: {distances}")

# Cross-match with larger radius
idx, dist = xmatch(ra1, dec1, ra2, dec2, maxdist=0.01, units='deg')

# Cross-match query results with external catalog
result = qc.query(sql='SELECT ra,dec FROM gaia_dr3.gaia_source LIMIT 1000',
                  fmt='pandas')

# Load external catalog
external = pd.read_csv('my_catalog.csv')

# Match with 2 arcsecond radius
idx, dist = xmatch(result['ra'].values, result['dec'].values,
                   external['ra'].values, external['dec'].values,
                   maxdist=2.0, units='arcsec')

# Create matched catalog
matched = result.iloc[idx].reset_index(drop=True)
matched['match_dist_arcsec'] = dist.value
matched['external_id'] = external['id'].iloc[idx].values
```

--------------------------------

### Resolve Astronomical Objects and Convert Coordinates

Source: https://context7.com/astro-datalab/datalab/llms.txt

Demonstrates how to resolve astronomical object names to their coordinates and convert these coordinates to different systems (e.g., ICRS, Galactic). It uses a 'resolve' function and accesses coordinate attributes like 'ra', 'dec', 'icrs', and 'galactic'.

```python
targets = ['NGC 1234', 'M42', 'Sirius']
for name in targets:
    try:
        c = resolve(name)
        print(f"{name}: RA={c.ra.deg:.4f}, Dec={c.dec.deg:.4f}")
    except Exception as e:
        print(f"Could not resolve {name}: {e}")

coords = resolve('Betelgeuse')
print(f"ICRS: RA={coords.icrs.ra.deg}, Dec={coords.icrs.dec.deg}")
print(f"Galactic: l={coords.galactic.l.deg}, b={coords.galactic.b.deg}")
```

=== COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.