### Start Dwarfium Development Server

Source: https://github.com/stevejcl/dwarfium/wiki/Home

Starts the development server for the Dwarfium application, allowing for live development and testing. Requires Node.js and npm to be installed.

```bash
npm run dev
```

--------------------------------

### Start Production Server for Dwarfium

Source: https://github.com/stevejcl/dwarfium/blob/apiV2/README.md

These commands launch the production server for the Dwarfium application on Windows and Linux, respectively. This is used after building the production-ready API.

```bash
npm run start:windows
```

```bash
npm run start:linux
```

--------------------------------

### Start Development Server for Dwarfium

Source: https://github.com/stevejcl/dwarfium/blob/apiV2/README.md

These commands initiate the development server for the Dwarfium application, catering to different operating systems. This allows for real-time development and testing.

```bash
npm run dev:windows
```

```bash
npm run dev:linux
```

--------------------------------

### Start Dwarfium API in Production

Source: https://github.com/stevejcl/dwarfium/wiki/Home

Starts the Dwarfium API after a production build has been created. This command is used to run the fully optimized application.

```bash
npm run start:api
```

--------------------------------

### Install Dependencies for Dwarfium Development

Source: https://github.com/stevejcl/dwarfium/wiki/Home

Installs the necessary libraries for developing and contributing to the Dwarfium project. This is a standard Node.js package installation command.

```bash
npm install
```

--------------------------------

### Web Scraping Target URL for Wikipedia Constellations

Source: https://github.com/stevejcl/dwarfium/blob/apiV2/notebooks/4_download_data.ipynb

This line specifies the Wikipedia URL from which constellation data will be scraped. It's a common starting point for gathering astronomical information.

```python
wikiurl="https://en.wikipedia.org/wiki/IAU_designated_constellations"
```

--------------------------------

### Python Project Initialization and Imports

Source: https://github.com/stevejcl/dwarfium/blob/apiV2/notebooks/6_dwarf_docs.ipynb

This Python code initializes the project path and imports necessary libraries such as sys, os, pathlib, pandas, BeautifulSoup, requests, re, and json. It also imports a custom logging utility function 'log_df'.

```python
import sys
import os 
from pathlib import Path
import pandas as pd
from bs4 import BeautifulSoup 
import requests
import re
import json

project_path = Path(os.path.dirname(os.path.realpath("__file__"))).parent

from scripts.utils import log_df
```

--------------------------------

### Build Desktop App with Tauri

Source: https://github.com/stevejcl/dwarfium/wiki/Home

Compiles the Dwarfium desktop application for the current operating system using the Tauri framework. Requires Rust to be installed on the system.

```bash
npm run tauri build
```

--------------------------------

### Install Dwarfium Certificate on Linux (Ubuntu)

Source: https://github.com/stevejcl/dwarfium/blob/apiV2/README.md

This sequence of commands installs the Dwarfium SSL certificate on Ubuntu systems. It involves copying the certificate file to the system's CA certificates directory, updating the certificate store, and verifying the installation. A browser or application restart may be necessary.

```bash
sudo cp CADwarfiumCert.pem /usr/local/share/ca-certificates/
sudo update-ca-certificates
sudo ls /etc/ssl/certs | grep CADwarfiumCert
```

--------------------------------

### Extract and Filter Table Data with Pandas

Source: https://github.com/stevejcl/dwarfium/blob/apiV2/notebooks/4_download_data.ipynb

This code fetches a web page, parses its HTML to find all tables, selects a specific table (the 6th one), converts it to a pandas DataFrame, fills missing values, and then filters rows where the 'Object' column starts with 'planet'. Dependencies: requests, beautifulsoup4, pandas.

```python
from bs4 import BeautifulSoup
import pandas as pd

url = 'https://en.wikipedia.org/wiki/Apparent_magnitude'
response = requests.get(url)

soup = BeautifulSoup(response.text, 'html.parser')
tables = soup.findAll('table')

df = pd.read_html(str(tables))
df = pd.DataFrame(df[5])
df = df.fillna('')

planets_df = df[df['Object'].str.startswith('planet')]
# Assuming log_df is a custom function for logging DataFrame info
# log_df(df)
```

--------------------------------

### JavaScript Path Definitions for Dwarfium Data

Source: https://github.com/stevejcl/dwarfium/blob/apiV2/notebooks/4_download_data.ipynb

This JavaScript code defines file paths for constellation and solar system data. It assumes a project structure where 'raw_data' and 'data' directories exist relative to a base path.

```javascript
constellation_path = project_path/'raw_data'/'constellations.csv'
constellation_js_path = project_path/'data'/'constellations.js'

solar_system_path = project_path/'raw_data'/'solar_system.csv'
```

--------------------------------

### Import Libraries and Project Path Setup (Python)

Source: https://github.com/stevejcl/dwarfium/blob/apiV2/notebooks/2_QA_catalogues.ipynb

This snippet sets up the necessary Python environment by importing core libraries such as sys, os, pathlib, and pandas. It also defines the project's root path using the current file's location, which is crucial for relative path resolution. The `log_df` utility function is imported for data frame logging.

```python
import sys
import os 
from pathlib import Path
import pandas as pd

project_path = Path(os.path.dirname(os.path.realpath("__file__"))).parent

from scripts.utils import log_df
```

--------------------------------

### Load and Process HTML File with Python

Source: https://github.com/stevejcl/dwarfium/blob/apiV2/notebooks/6_dwarf_docs.ipynb

Demonstrates how to load an HTML file, parse it using Beautiful Soup, and then process the parsed content with the `process_html` function. It also shows how to determine the number of records extracted. Assumes `raw_error_path` is defined and `BeautifulSoup` and `process_html` are available.

```python
with open(raw_error_path) as fp:
    soup = BeautifulSoup(fp, 'html.parser')

records = process_html(soup)
len(records)
```

--------------------------------

### Python Path Definitions for Raw Data

Source: https://github.com/stevejcl/dwarfium/blob/apiV2/notebooks/6_dwarf_docs.ipynb

This Python code defines the file paths for raw error codes and status codes HTML files within the Dwarfium project's raw data directory. It utilizes the 'project_path' variable established earlier.

```python
raw_error_path = project_path/'raw_data'/'dwarfii'/'error_codes.html'
raw_status_path = project_path/'raw_data'/'dwarfii'/'status_codes.html'
```

--------------------------------

### Filter DataFrame by String Prefix (Pandas)

Source: https://github.com/stevejcl/dwarfium/blob/apiV2/notebooks/4_download_data.ipynb

This code snippet filters a Pandas DataFrame (`planets_df`) to include only rows where the 'Notes' column starts with the string 'mean'. It assumes the DataFrame is already loaded and available. The output is a filtered DataFrame.

```python
planets_df = planets_df[planets_df['Notes'].str.startswith('mean')]

planets_df
```

--------------------------------

### Fetch and Print Wikipedia Page Status

Source: https://github.com/stevejcl/dwarfium/blob/apiV2/notebooks/4_download_data.ipynb

This snippet uses the requests library to fetch the content of a Wikipedia page and prints the HTTP status code of the response. It's a basic step to verify if the page is accessible. Dependencies: requests.

```python
import requests

wikiurl = 'https://en.wikipedia.org/wiki/List_of_proper_names_in_Unicode'
response = requests.get(wikiurl)
print(response.status_code)
```

--------------------------------

### Convert DataFrame to JSON and Save to File

Source: https://github.com/stevejcl/dwarfium/blob/apiV2/notebooks/4_download_data.ipynb

This snippet iterates through a pandas DataFrame, creates a dictionary mapping specific columns, converts this dictionary to a JSON string, and writes it to a file. Dependencies: pandas, json.

```python
import pandas as pd
import json

# Assuming df is defined and constellation_js_path is the desired file path
# object_dict = {}
# for index, row in df.iterrows():
#     object_dict[row['Abbreviations IAU']] = row['name']

# json_object = json.dumps(object_dict)

# with open(constellation_js_path, "w") as outfile:
#     outfile.write(json_object)
```

--------------------------------

### Python Imports and Path Configuration for Dwarfium

Source: https://github.com/stevejcl/dwarfium/blob/apiV2/notebooks/4_download_data.ipynb

This Python script imports essential libraries for data manipulation (pandas), web scraping (BeautifulSoup, requests, lxml), file system operations (os, pathlib), and logging. It also configures the base project path, assuming a standard directory structure.

```python
import sys
import os 
from pathlib import Path
import pandas as pd
from bs4 import BeautifulSoup 
import requests
import lxml
import json

project_path = Path(os.path.dirname(os.path.realpath("__file__"))).parent

from scripts.utils import log_df
```

--------------------------------

### Load and Process Status HTML with Python

Source: https://github.com/stevejcl/dwarfium/blob/apiV2/notebooks/6_dwarf_docs.ipynb

Loads an HTML file specified by `raw_status_path`, parses it using Beautiful Soup, and processes the content with the `process_html` function. This is similar to processing error logs but intended for status-related HTML content. It returns the number of records extracted.

```python
with open(raw_status_path) as fp:
    soup = BeautifulSoup(fp, 'html.parser')

records = process_html(soup)
len(records)
```

--------------------------------

### Filter DataFrame by String Prefix and Value (Pandas)

Source: https://github.com/stevejcl/dwarfium/blob/apiV2/notebooks/4_download_data.ipynb

This code filters a Pandas DataFrame (`df`) in two steps. First, it selects rows where the 'Object' column is 'Moon'. Second, it further filters these rows to include only those where the 'Seen from...' column starts with 'lit by earthlight,'. This is useful for isolating specific data points.

```python
moon_df = df[df['Object'] == 'Moon']
moon_df = moon_df[moon_df['Seen from...'].str.startswith('lit by earthlight,')]

moon_df
```

--------------------------------

### Process Plate Solving Descriptions in Python

Source: https://github.com/stevejcl/dwarfium/blob/apiV2/notebooks/6_dwarf_docs.ipynb

This Python snippet processes a DataFrame to extract and clean descriptions associated with plate solving actions. It uses regular expressions to standardize spacing and stores the results in a dictionary, mapping numerical values to cleaned description strings. Dependencies include the 'pandas' and 're' libraries.

```python
results = {}
for index, row in df.iterrows():
    results[str(row['value'])] = re.sub('\n +', ' ', row['description'])
```

--------------------------------

### Get First 50 Unique IC Values

Source: https://github.com/stevejcl/dwarfium/blob/apiV2/notebooks/2_QA_catalogues.ipynb

Retrieves and displays the first 50 unique values from the 'IC' column of the DataFrame. The 'IC' column likely refers to the Index Catalogue of deep-sky objects.

```python
df['IC'].unique()[0:50]
```

--------------------------------

### Transform DataFrame to Dictionary with Python

Source: https://github.com/stevejcl/dwarfium/blob/apiV2/notebooks/6_dwarf_docs.ipynb

Iterates through a Pandas DataFrame to create a dictionary. Each row's 'value' is used as the key, and the cleaned 'description' is used as the value. Descriptions are cleaned by removing excess newlines and spaces. Assumes `df` is a Pandas DataFrame with 'value' and 'description' columns.

```python
results = {}
for index, row in df.iterrows():
    results[str(row['value'])] = re.sub('\n +', ' ', row['description'])

results
```

--------------------------------

### Parse HTML Table with BeautifulSoup and Pandas

Source: https://github.com/stevejcl/dwarfium/blob/apiV2/notebooks/4_download_data.ipynb

This code snippet uses BeautifulSoup to parse the HTML content of a web page and extract a specific table identified by its class. The extracted table is then converted into a pandas DataFrame for further manipulation. Dependencies: requests, beautifulsoup4, pandas.

```python
from bs4 import BeautifulSoup
import pandas as pd

soup = BeautifulSoup(response.text, 'html.parser')
table = soup.find('table', {'class': "wikitable"})
df = pd.read_html(str(table))
df = pd.DataFrame(df[0])
df.columns = ['Constellation', 'Abbreviations IAU',
              'Abbreviations NASA', 'Genitive',
              'Origin', 'Meaning', 'Brightest star']
# Assuming log_df is a custom function for logging DataFrame info
# log_df(df)
```

--------------------------------

### Restructure DataFrame Columns

Source: https://github.com/stevejcl/dwarfium/blob/apiV2/notebooks/4_download_data.ipynb

This snippet demonstrates how to split a column in a pandas DataFrame into multiple new columns and then drop the temporary column. It's useful for cleaning and normalizing data. Dependencies: pandas.

```python
import pandas as pd

# Assuming df is already defined and populated
# df[['name', 'b']] = df['Constellation'].str.split(' /', expand=True)
# del df['b']
# log_df(df)
```

--------------------------------

### Clean DataFrame Duplicates and Log with Python

Source: https://github.com/stevejcl/dwarfium/blob/apiV2/notebooks/6_dwarf_docs.ipynb

Converts a list of records into a Pandas DataFrame, removes duplicate rows in place, and then logs the resulting DataFrame. It assumes the `records` list is populated and `log_df` function is defined. The output shows the shape of the DataFrame after duplicate removal.

```python
df = pd.DataFrame(records)
df.drop_duplicates(inplace=True)
log_df(df)
```

--------------------------------

### Get First 50 Unique M Values

Source: https://github.com/stevejcl/dwarfium/blob/apiV2/notebooks/2_QA_catalogues.ipynb

Retrieves and displays the first 50 unique values from the 'M' column of the DataFrame. The 'M' column might represent magnitudes or other classifications.

```python
df['M'].unique()[0:50]
```

--------------------------------

### Parse HTML Table Rows with Python

Source: https://github.com/stevejcl/dwarfium/blob/apiV2/notebooks/6_dwarf_docs.ipynb

Processes an HTML document to extract table row data. It specifically looks for `<tr>` elements and extracts text from `<span>` tags within them, categorizing the data into 'code', 'value', and 'description'. This function is designed to handle specific HTML structures found in the dwarfium project's documentation.

```python
import re
from bs4 import BeautifulSoup
import pandas as pd

def process_html(soup):
    regex = re.compile('.*author.*')
    records = []
    
    for index, row in enumerate(soup.find_all('tr')):
        if index == 0:
            continue
            
        data = {}
        for index, span in enumerate(row.find_all('span', regex)):
            if index == 0:
                key = 'code'
            elif index == 1:
                key = 'value'
            else:
                key = 'description'
                
            data[key] = span.text
        if 'value' in data:
            records.append(data)

    return records
         
```

--------------------------------

### Save DataFrame to CSV

Source: https://github.com/stevejcl/dwarfium/blob/apiV2/notebooks/4_download_data.ipynb

This code saves a pandas DataFrame to a CSV file. The `index=False` argument prevents pandas from writing the DataFrame index as a column in the CSV file. Dependencies: pandas.

```python
import pandas as pd

# Assuming df is defined and constellation_path is the desired file path
# df.to_csv(constellation_path, index=False)
```

--------------------------------

### Remove Quarantine Attribute on macOS

Source: https://github.com/stevejcl/dwarfium/wiki/Home

Removes the quarantine attribute from the Dwarfium macOS application, which can resolve installation issues on ARM Macs. This command is executed in the terminal.

```bash
xattr -d com.apple.quarantine /Applications/Dwarfium.app
```

--------------------------------

### Export DataFrame to CSV (Pandas)

Source: https://github.com/stevejcl/dwarfium/blob/apiV2/notebooks/4_download_data.ipynb

This code snippet exports a Pandas DataFrame (`combine_df`) to a CSV file. The `to_csv` method is used, with `index=False` preventing the DataFrame index from being written to the file. The `solar_system_path` variable is assumed to hold the desired file path.

```python
combine_df.to_csv(solar_system_path, index=False)
```

--------------------------------

### Get First 50 Unique NGC Values

Source: https://github.com/stevejcl/dwarfium/blob/apiV2/notebooks/2_QA_catalogues.ipynb

Retrieves and displays the first 50 unique values from the 'NGC' column of the DataFrame. The 'NGC' column likely refers to the New General Catalogue of deep-sky objects.

```python
df['NGC'].unique()[0:50]
```

--------------------------------

### Get Unique Magnitude Values (Python)

Source: https://github.com/stevejcl/dwarfium/blob/apiV2/notebooks/2_QA_catalogues.ipynb

This Python snippet extracts and returns all unique values present in the 'Mag' column of the 'mike_df' DataFrame. This is useful for identifying the different magnitude representations and potential inconsistencies or special values within the data.

```python
mike_df["Mag"].unique()
```

--------------------------------

### Launch Dwarfium Server and Proxy Components

Source: https://github.com/stevejcl/dwarfium/blob/apiV2/README.md

These commands launch the Dwarfium web server and the proxy with D3 video stream tools, respectively. Ensure you are in the correct directories after unzipping the provided files. These scripts are part of the Dwarfium distribution for Windows.

```cmd
cd Dwarfium
./launch-server.bat
```

```cmd
cd DwarfiumProxy
./launch-tools.bat
```

--------------------------------

### Build Dwarfium API for Production

Source: https://github.com/stevejcl/dwarfium/wiki/Home

Creates a production-ready build of the Dwarfium API. This command is used to prepare the application for deployment or further packaging.

```bash
npm run build:api
```

--------------------------------

### Launch Web Server and Tools on Linux

Source: https://github.com/stevejcl/dwarfium/blob/apiV2/README.md

This script launches a Python web server and necessary tools within the Dwarfium directory on Linux. It's used to serve the web browser version of the application.

```bash
cd Dwarfium
./launch-server&tools
```

--------------------------------

### Launch Web Server and Tools on Windows

Source: https://github.com/stevejcl/dwarfium/blob/apiV2/README.md

This batch script launches a Python web server and necessary tools within the Dwarfium directory on Windows. It's used to serve the web browser version of the application.

```cmd
cd Dwarfium
./launch-server&tools.bat
```

--------------------------------

### Launch Web Server on Linux

Source: https://github.com/stevejcl/dwarfium/wiki/Home

Launches a local Python web server and necessary tools for the web browser version of Dwarfium on Linux systems. This script is located within the unzipped Dwarfium directory.

```bash
cd Dwarfium
./launch-tools
```

--------------------------------

### Launch Web Server on Windows

Source: https://github.com/stevejcl/dwarfium/wiki/Home

Launches a local Python web server and necessary tools for the web browser version of Dwarfium on Windows systems. This batch script is located within the unzipped Dwarfium directory.

```cmd
cd Dwarfium
./launch-tools.bat
```

--------------------------------

### Import Libraries and Define Project Path (Python)

Source: https://github.com/stevejcl/dwarfium/blob/apiV2/notebooks/1_process_openngc.ipynb

This snippet imports necessary Python libraries such as sys, os, pathlib, pandas, and json. It also defines the project's root path, which is crucial for accessing data files. It relies on the 'scripts.utils' module for logging functionality.

```python
import sys
import os 
from pathlib import Path
import pandas as pd
import json

project_path = Path(os.path.dirname(os.path.realpath("__file__"))).parent

from scripts.utils import log_df
```

--------------------------------

### Concatenate DataFrames (Pandas)

Source: https://github.com/stevejcl/dwarfium/blob/apiV2/notebooks/4_download_data.ipynb

This code snippet concatenates two Pandas DataFrames, `moon_df` and `planets_df`, into a single DataFrame called `combine_df`. It uses `pd.concat` for this operation, effectively merging the rows from both DataFrames. The resulting DataFrame contains data from both moons and planets.

```python
combine_df = pd.concat([moon_df, planets_df])

combine_df
```

--------------------------------

### Define Input and Output Paths for OpenNGC Data (Python)

Source: https://github.com/stevejcl/dwarfium/blob/apiV2/notebooks/1_process_openngc.ipynb

This Python code defines the file paths for the original and merged OpenNGC database CSV files. It constructs these paths relative to the project root, ensuring consistent data access. The paths specify locations within a 'raw_data' directory.

```python
orginal_openngc_path = project_path / 'raw_data' / 'OpenNGC' / 'database_files' / 'NGC.csv'
orginal_openngc_addendum_path = project_path / 'raw_data' / 'OpenNGC' / 'database_files' / 'addendum.csv'

openngc_path = project_path / 'raw_data' / 'OpenNGC.csv'
```

--------------------------------

### Load and Log OpenNGC Data (Python)

Source: https://github.com/stevejcl/dwarfium/blob/apiV2/notebooks/2_QA_catalogues.ipynb

This Python snippet loads the 'OpenNGC.csv' file into a pandas DataFrame, ensuring the 'M' column is read as a string to preserve leading zeros. It then uses the `log_df` function to display the shape and a preview of the loaded DataFrame, which contains data about NGC objects.

```python
df = pd.read_csv(openngc_path, dtype = {'M': str})

log_df(df)
```

--------------------------------

### Import Libraries and Define Project Path

Source: https://github.com/stevejcl/dwarfium/blob/apiV2/notebooks/5_create_catalogues.ipynb

Imports necessary Python libraries for data manipulation and file path management. It defines the base project path using pathlib and os modules, which is crucial for locating data and script files within the project structure.

```python
import sys
import os 
from pathlib import Path
import pandas as pd
import json
import numpy as np

project_path = Path(os.path.dirname(os.path.realpath("__file__"))).parent

from scripts.utils import log_df

from scripts.catalog_utils import (
format_ngc_catalog,
format_best_500_catalog,
format_best_500_catalog_M, 
merge_type_constellation, 
catalog_columns,
decToHMS,
convert_to_hh_mm_ss,
convert_to_hh_mm_ss_d
)
```

--------------------------------

### Define Column Names for Different Catalogs

Source: https://github.com/stevejcl/dwarfium/blob/apiV2/notebooks/5_create_catalogues.ipynb

Specifies lists of column names used for parsing different astronomical data files, including OpenNGC, Mike's catalog, Best 500 DSO, and the HYG Database. These lists ensure correct data extraction and alignment.

```python
openngc_columns = [
    'Name', 'Type', 'RA', 'Dec', 'Const', 'MajAx', 'MinAx', 'V-Mag', 
    'M', 'NGC', 'IC',  'Common names'
]
mike_columns = [
    'Catalog', 
    'Name', 'Name (no leading zero)', 'Name (final)', 'Common Name', 'NGC/ID', 
    "Width ('')", "Height ('')", 'Mag', 'Type', 'RA', 'Dec'
]

best500_columns = [
    'CONST.', 'TYPE', 'NGC DESIGNATION', 'OTHER NAME(S)', 'R.A.', 'DEC.', 'MAG', 'MAX SZ', 'MIN SZ'
]

best500_columns_modif = [
    'CONST.', 'Types_OpenNGC', 'NGC DESIGNATION', 'OTHER NAME(S)', 'Name normalized', 'R.A.','DEC.','MAG','MAX SZ','MIN SZ','Name catalog','Name number','Best_500'
]
hyg_columns = [
    'hip',  'hd', 'hr', 
    'proper', 'ra', 'dec', 'mag', 'con'
]
```

--------------------------------

### Define File Paths for Astronomical Data

Source: https://github.com/stevejcl/dwarfium/blob/apiV2/notebooks/3_object_types.ipynb

Defines various file paths for raw data, processed data, and draft observing lists. These paths are constructed relative to the project's root directory.

```python
openngc_types_path = project_path / 'raw_data' / 'OpenNGC_types.csv'

stellarium_path = project_path / 'data' / 'draft'/'observingList_DwarfII.json' 
stellarium_types_path = project_path / 'raw_data' / 'stellarium_types.csv' 
moon_planets_path = project_path / 'raw_data' / 'moon_planets.csv' 
telescopius_types_path = project_path / 'raw_data' / 'telescopius_types.csv'
```

--------------------------------

### Load and Log Mike Camilleri's Data (Python)

Source: https://github.com/stevejcl/dwarfium/blob/apiV2/notebooks/2_QA_catalogues.ipynb

This Python snippet loads the 'mike_camilleri_list.csv' file into a pandas DataFrame, explicitly casting 'Width (')' and 'Height (')' columns to float. It then uses the imported `log_df` function to display the shape and a preview of the loaded DataFrame, aiding in initial data inspection.

```python
mike_df = pd.read_csv(mike_path, dtype={"Width ('"): float, "Height ('"): float})
log_df(mike_df)
```

--------------------------------

### Initialize Dictionaries for Type and Category Grouping

Source: https://github.com/stevejcl/dwarfium/blob/apiV2/notebooks/3_object_types.ipynb

Initializes empty dictionaries to store object types and their corresponding categories, as well as to group types by category. These will be populated by subsequent data processing steps.

```python
grouped_types = {}
types = {}
```

--------------------------------

### Define Data File Paths (Python)

Source: https://github.com/stevejcl/dwarfium/blob/apiV2/notebooks/2_QA_catalogues.ipynb

This Python code defines the absolute paths to various astronomical data CSV files, including OpenNGC, OpenNGC types, Mike Camilleri's list, and the HYG Database. These paths are constructed relative to the project's root directory, ensuring consistent data access across different environments.

```python
openngc_path = project_path / 'raw_data' / 'OpenNGC.csv'
openngc_types_path = project_path / 'raw_data' / 'OpenNGC_types.csv' 
mike_path = project_path / 'raw_data' / 'mike_camilleri_list.csv' 
hyg_path = project_path / 'raw_data' / 'HYG-Database' / 'hyg' / 'v3' / 'hyg_v35.csv'
```

--------------------------------

### Read and Log HYG Star Catalog

Source: https://github.com/stevejcl/dwarfium/blob/apiV2/notebooks/5_create_catalogues.ipynb

This Python code reads the HYG star catalog from a CSV file, specifying data types for certain columns and selecting a subset of columns. It then logs the initial rows of the loaded DataFrame. Dependencies include pandas.

```python
types = {'hip': pd.Int64Dtype(), 'hd': pd.Int64Dtype(), 'hr': pd.Int64Dtype()}
df = pd.read_csv(hyg_path, dtype=types, usecols=hyg_columns)
log_df(df)
```

--------------------------------

### Define File Paths for Astronomical Data

Source: https://github.com/stevejcl/dwarfium/blob/apiV2/notebooks/5_create_catalogues.ipynb

Defines various file paths for different astronomical catalogs and temporary files used during data processing. These paths are constructed relative to the project's root directory and cover raw data, draft data, and final catalog outputs.

```python
mike_draft_path = project_path/'data'/'draft'/'mike_catalog_draft.csv'

best_500dso_path_draft = project_path/'data'/'draft'/'Best_500_Deep_Sky_Objects-draft.csv'
# final best_500dso file
best_500dso_path = project_path/'data'/'draft'/'Best_500_Deep_Sky_Objects-util.csv'

# tmp file for control
# temp best_500dso files
best_500dso_path_tmp = project_path/'data'/'draft'/'Best_500_Deep_Sky_Objects-tmp.csv'
best_500dso_path_merge = project_path/'data'/'draft'/'Best_500_Deep_Sky_Objects-merge.csv'

# openngc file
openngc_path = project_path / 'raw_data' / 'OpenNGC.csv'

# temp openngc files
ngc_draft_path_tmp1 = project_path/'data'/'draft'/'openngc_catalog_draft_tmp1.csv'
ngc_draft_path_tmp2 = project_path/'data'/'draft'/'openngc_catalog_draft_tmp2.csv'

# final ngc_draft file
ngc_draft_path = project_path/'data'/'draft'/'openngc_catalog_draft.csv'


# type files
best_500dso_type_path = project_path/'raw_data'/'Best_500_types.csv'
openngc_types_path = project_path / 'raw_data' / 'OpenNGC_types.csv' 
constellation_path = project_path/'raw_data'/'constellations.csv'

# others files
constellation_path = project_path/'raw_data'/'constellations.csv'
solar_system_path = project_path/'raw_data'/'moon_planets.csv'
hyg_path = project_path / 'raw_data' / 'HYG-Database' / 'hyg' / 'v3' / 'hyg_v35.csv' 

# solar_system files
solar_system_catalog_path = project_path/'data'/'catalogs'/'moon_planets.csv'
solar_system_catalog_json_path = project_path/'data'/'catalogs'/'moon_planets.json'

# final dso_catalog file
dso_catalog_path = project_path/'data'/'catalogs'/'openngc_dso.csv'
dso_catalog_json_path = project_path/'data'/'catalogs'/'dso_catalog.json'
stars_catalog_path = project_path/'data'/'catalogs'/'hyg_stars.csv'

demo_path = project_path/'data'/'draft'/'demo_catalog.json'
```

--------------------------------

### Load and Log OpenNGC Object Types

Source: https://github.com/stevejcl/dwarfium/blob/apiV2/notebooks/3_object_types.ipynb

Loads object types from the OpenNGC catalog using a pandas DataFrame and logs the DataFrame's shape. This provides insight into the OpenNGC dataset's size and format.

```python
openngc_df = pd.read_csv(openngc_types_path)

log_df(openngc_df)
```

--------------------------------

### Format and Save Best 500 DSO Catalog Data

Source: https://github.com/stevejcl/dwarfium/blob/apiV2/notebooks/5_create_catalogues.ipynb

Reads the draft 'Best 500 DSO' catalog, formats it using utility functions, and saves processed data. It also extracts and saves unique catalog types and creates a modified version for merging with other datasets.

```python
best500_df = pd.read_csv(best_500dso_path_draft, usecols=best500_columns)
best500_df = format_best_500_catalog(best500_df)

# tmp file for control
#best500_df.to_csv(best_500dso_path_tmp, index=False)

# Extract non-duplicate values of the column
types_Best500 = best500_df['TYPE'].drop_duplicates()
# Convert non-duplicate values to a DataFrame
types_Best500_df = pd.DataFrame(types_Best500, columns=['TYPE'])

# Save  best_500dso TYPE to a CSV file
types_Best500_df.to_csv(best_500dso_type_path, index=False)

best500_2_df = format_best_500_catalog_M(best500_df)
best500_2_df.to_csv(best_500dso_path, index=False)
```

--------------------------------

### Process and Clean OpenNGC Catalog Data

Source: https://github.com/stevejcl/dwarfium/blob/apiV2/notebooks/5_create_catalogues.ipynb

Loads the OpenNGC catalog using pandas, selecting specific columns and ensuring correct data types. It then filters out duplicate entries and corrects a specific common name entry, logging the resulting DataFrame's dimensions.

```python
ngc_df = pd.read_csv(openngc_path, usecols=openngc_columns, dtype={'M': pd.Int64Dtype()})
ngc_df = ngc_df[ngc_df['Type'] != 'Dup']
ngc_df.loc[ngc_df['M'] == 65, 'Common names'] = 'Leo Triplet'

log_df(ngc_df)
# (13340, 12)
```

--------------------------------

### Load and Log Open NGC DataFrame

Source: https://github.com/stevejcl/dwarfium/blob/apiV2/notebooks/2_QA_catalogues.ipynb

Reads a CSV file into a pandas DataFrame named 'open_df', ensuring the 'M' column is treated as a string. It then logs the shape of the loaded DataFrame, which likely contains data from the Open NGC catalogue.

```python
open_df = pd.read_csv(openngc_path, dtype = {'M': str})

log_df(open_df)
```

--------------------------------

### Consolidate Object Types into a Single Dictionary

Source: https://github.com/stevejcl/dwarfium/blob/apiV2/notebooks/3_object_types.ipynb

Iterates through DataFrames from OpenNGC, Stellarium, Planets, and Telescopius, extracting object names/types and their categories. It populates a single 'types' dictionary with these mappings and then sorts it alphabetically by object name.

```python
for index, rows in openngc_df.iterrows():
    types[rows['name']] = rows['category']

for index, rows in stellarium_df.iterrows():
    types[rows['objtype']] = rows['category']

for index, rows in planets_df.iterrows():
    types[rows['type']] = rows['category']

for index, rows in telescopius_df.iterrows():
    types[rows['Type']] = rows['category']

dict(sorted(types.items()))
```

--------------------------------

### Format and Refine Star Catalog Entries - Python

Source: https://github.com/stevejcl/dwarfium/blob/apiV2/notebooks/5_create_catalogues.ipynb

This snippet prepares a DataFrame for a star catalog by creating new columns for catalogue entries, alternative identifiers, and familiar names. It also converts celestial coordinates (RA and Dec) using a `decToHMS` function and renames existing columns to a more descriptive format. Additional columns with `pd.NA` are added for other celestial object properties, and the DataFrame is filtered to include only specified columns before logging.

```python
cat_df = merge_df.copy()
cat_df['Catalogue Entry'] = 'HIP ' + cat_df['hip'].astype(str)
cat_df['Name catalog'] = 'HIP'
cat_df['Alternative Entries'] = ('HD ' + cat_df['hd'].astype(str) + ', HR ' + cat_df['hr'].astype(str))

cat_df['ra'] = cat_df['ra'].apply(decToHMS)
cat_df['dec'] = cat_df['dec'].apply(lambda row: decToHMS(row, True))

cat_df.rename(columns={
    'proper': 'Familiar Name',
    'ra': 'Right Ascension',
    'dec': 'Declination',
    'hip': 'Name number',
    'mag': 'Magnitude'
}, inplace=True)


cat_df['Major Axis'] = pd.NA
cat_df['Minor Axis'] = pd.NA
cat_df['Surface Brightness'] = pd.NA
cat_df["Width ('") = pd.NA
cat_df["Height ('") = pd.NA
cat_df['Surface Brightness'] = pd.NA
cat_df['Type'] = 'Star'
cat_df['Type Category'] = 'stars'
cat_df['Notes'] = 'bright_named_stars'

cat_df = cat_df[catalog_columns]

log_df(cat_df)
```

--------------------------------

### Read and Log Astronomical Catalog CSV Data (Python)

Source: https://github.com/stevejcl/dwarfium/blob/apiV2/notebooks/5_create_catalogues.ipynb

This snippet demonstrates reading a CSV file containing astronomical catalog data (e.g., Deep Sky Objects or Stars) into a pandas DataFrame and then logging its dimensions and column information. It assumes the existence of a `stars_catalog_path` variable and a `log_df` function.

```python
stars_df = pd.read_csv(stars_catalog_path)
log_df(stars_df)
# (118, 16)
```

--------------------------------

### Read and Inspect OpenNGC Addendum CSV Data (Python)

Source: https://github.com/stevejcl/dwarfium/blob/apiV2/notebooks/1_process_openngc.ipynb

This Python code reads the addendum CSV file for OpenNGC using pandas, similar to the main file. It uses a semicolon separator and reads all data as strings. The `.shape` attribute is called to show the number of rows and columns in the addendum data.

```python
add_df = pd.read_csv(orginal_openngc_addendum_path, sep=";", dtype=str)
add_df.shape
```

--------------------------------

### Create Stellarium Object Types CSV

Source: https://github.com/stevejcl/dwarfium/blob/apiV2/notebooks/3_object_types.ipynb

Reads Stellarium observation data from a JSON file, extracts unique object types, and creates a pandas DataFrame. This DataFrame is then logged and saved to a CSV file.

```python
with open(stellarium_path, 'r') as f:
    data = json.load(f)

objects = data['observingLists']['observingLists']['{5e727f81-e0a8-43f0-9258-3848aa2d9762}']['objects']

records = []
tmp = set()
for object in objects:
    if object['objtype'] not in tmp:
        records.append({'objtype': object['objtype'], 'type': object['type'], 'category': ''})
        tmp.add(object['objtype'])
        
df = pd.DataFrame(records)
log_df(df)

df.to_csv(stellarium_types_path, index=False)
```

--------------------------------

### Group Object Names by Category

Source: https://github.com/stevejcl/dwarfium/blob/apiV2/notebooks/3_object_types.ipynb

Populates the 'grouped_types' dictionary by iterating through the OpenNGC and Stellarium DataFrames. For each object, it appends its name (or objtype for Stellarium) to a list associated with its category. If a category does not yet exist in the dictionary, it is created.

```python
for index, rows in openngc_df.iterrows():
    category = rows['category']
    name = rows['name']
    if category not in grouped_types:
        grouped_types[category] = []

    grouped_types[category].append(name)

for index, rows in stellarium_df.iterrows():
    category = rows['category']
    name = rows['objtype']
    if category not in grouped_types:
        grouped_types[category] = []

    grouped_types[category].append(name)
    

grouped_types
```

--------------------------------

### Load and Log Moon/Planet Object Types

Source: https://github.com/stevejcl/dwarfium/blob/apiV2/notebooks/3_object_types.ipynb

Reads data for moons and planets from a specified CSV path into a pandas DataFrame and logs its dimensions. This prepares the data for integration into the main type dictionary.

```python
planets_df = pd.read_csv(moon_planets_path)

log_df(planets_df)
```

--------------------------------

### Consolidate Alternate and Common Names in DataFrame (Python)

Source: https://github.com/stevejcl/dwarfium/blob/apiV2/notebooks/5_create_catalogues.ipynb

This snippet creates a copy of the `draft_df`, drops rows with missing 'Name' values, and then iterates through the remaining rows. For each row, it consolidates various potential alternate names and common names into new columns named 'Alternate Names' and 'Common Names', respectively, by combining unique, non-null values from multiple source columns.

```python
tmp_df =  draft_df.copy()
tmp_df.dropna(subset=['Name'], inplace=True)

for index, row in tmp_df.iterrows():
    # create string with multiple names
    names = set([row['Name normalized'],
                 row['IC name'], row['M name'],
                 row['Name (no leading zero)'], row['IC name_MIKE'], row['NGC name']])
    names.remove(row['Name normalized'])
    names = [name for name in names if pd.notna(name)]
    names.sort()
    tmp_df.at[index, 'Alternate Names'] = ', '.join(names)

    # create string with multiple common names
    common_names = set([row['Common names'], row['Common Name']])
    common_names = [name for name in common_names if pd.notna(name)]
    common_names.sort()
    tmp_df.at[index, 'Common Names'] = ', '.join(common_names)

log_df(tmp_df)
```

--------------------------------

### Display DataFrame Columns in Python

Source: https://github.com/stevejcl/dwarfium/blob/apiV2/notebooks/2_QA_catalogues.ipynb

This snippet displays all column names present in a pandas DataFrame. It is useful for understanding the available data fields before performing any analysis or filtering.

```python
df.columns
```

--------------------------------

### Load and Log Type DataFrame

Source: https://github.com/stevejcl/dwarfium/blob/apiV2/notebooks/2_QA_catalogues.ipynb

Reads a CSV file into a pandas DataFrame named 'type_df' and then logs its shape. This DataFrame likely contains information about different types of celestial objects.

```python
type_df = pd.read_csv(openngc_types_path)

log_df(type_df)
```

--------------------------------

### Select and Display Specific DataFrame Columns

Source: https://github.com/stevejcl/dwarfium/blob/apiV2/notebooks/2_QA_catalogues.ipynb

This code snippet selects a subset of columns ('hip', 'hd', 'hr', 'proper', 'ra', 'dec', 'mag', 'con') from a filtered DataFrame and then logs the resulting smaller DataFrame and its dimensions.

```python
cols = ['hip', 'hd', 'hr', 'proper', 'ra', 'dec', 'mag', 'con']

tmp_df = filter_df[cols]
log_df(tmp_df)
```

--------------------------------

### Verify Column Name Consistency Between CSVs (Python)

Source: https://github.com/stevejcl/dwarfium/blob/apiV2/notebooks/1_process_openngc.ipynb

This Python snippet compares the column names of the main OpenNGC DataFrame (`ngc_df`) with the addendum DataFrame (`add_df`). It returns a boolean NumPy array indicating whether each corresponding column name is identical, ensuring data compatibility for merging.

```python
add_df.columns == ngc_df.columns
```

--------------------------------

### Display DataFrame Columns (Python)

Source: https://github.com/stevejcl/dwarfium/blob/apiV2/notebooks/2_QA_catalogues.ipynb

This Python code retrieves and displays the column names of the 'mike_df' pandas DataFrame. This is useful for understanding the available data fields and for referencing them in subsequent data manipulation or analysis steps.

```python
mike_df.columns
```

--------------------------------

### Load and Log Stellarium Object Types

Source: https://github.com/stevejcl/dwarfium/blob/apiV2/notebooks/3_object_types.ipynb

Reads the Stellarium object types from a CSV file into a pandas DataFrame and logs its dimensions. This step is crucial for understanding the structure and volume of the Stellarium data.

```python
stellarium_df = pd.read_csv(stellarium_types_path)

log_df(stellarium_df)
```

--------------------------------

### Read and Rename Constellation Data

Source: https://github.com/stevejcl/dwarfium/blob/apiV2/notebooks/5_create_catalogues.ipynb

This Python snippet reads constellation data from a CSV file, selecting specific columns and renaming one for consistency. It then logs the resulting DataFrame. Dependencies include pandas.

```python
cons_df = pd.read_csv(constellation_path, usecols=['Abbreviations IAU', 'name'])
cons_df.rename(columns={'name': 'Constellation'}, inplace=True)
log_df(cons_df)
```

--------------------------------

### Format and Log Processed NGC Catalog

Source: https://github.com/stevejcl/dwarfium/blob/apiV2/notebooks/5_create_catalogues.ipynb

Applies a formatting function to the cleaned NGC catalog DataFrame and then logs the dimensions of the formatted DataFrame using the `log_df` utility function. This step prepares the data for further analysis or output.

```python
df = format_ngc_catalog(ngc_df)

log_df(df)
```

--------------------------------

### Read Solar System Data - Python

Source: https://github.com/stevejcl/dwarfium/blob/apiV2/notebooks/5_create_catalogues.ipynb

This code snippet reads solar system data from a CSV file into a Pandas DataFrame and then logs the DataFrame. It requires the Pandas library and a specified path to the solar system CSV file.

```python
df = pd.read_csv(solar_system_path)

log_df(df)
```

--------------------------------

### Filter DataFrame for Specific Catalog Entries (Python)

Source: https://github.com/stevejcl/dwarfium/blob/apiV2/notebooks/5_create_catalogues.ipynb

Creates a copy of the 'dso_df' DataFrame and filters it based on multiple conditions: 'Name catalog' must be 'M', 'Familiar Name' must not be null, and 'Major Axis' must be less than 17. The filtered DataFrame is then logged.

```python
df = dso_df.copy()
df = df[(df['Name catalog']=='M') & (df['Familiar Name'].notna()) & (df['Major Axis'] < 17)]
log_df(df)
```

--------------------------------

### Read and Inspect OpenNGC CSV Data (Python)

Source: https://github.com/stevejcl/dwarfium/blob/apiV2/notebooks/1_process_openngc.ipynb

This Python snippet utilizes the pandas library to read the main OpenNGC CSV file into a DataFrame. It specifies the separator as ';' and reads all columns as strings to prevent potential type inference issues. The `.shape` attribute is then used to display the dimensions of the loaded data.

```python
ngc_df = pd.read_csv(orginal_openngc_path, sep=";", dtype=str)

ngc_df.shape
```

--------------------------------

### Read CSV File into DataFrame - Python

Source: https://github.com/stevejcl/dwarfium/blob/apiV2/notebooks/2_QA_catalogues.ipynb

Reads a CSV file from a specified path ('hyg_path') into a pandas DataFrame and then logs the DataFrame, displaying its dimensions and a preview of the data.

```python
df = pd.read_csv(hyg_path)
log_df(df)
```

--------------------------------

### Merge and Log NGC Catalog Data

Source: https://github.com/stevejcl/dwarfium/blob/apiV2/notebooks/5_create_catalogues.ipynb

This Python code merges the NGC catalog DataFrame with constellation and openNGC type data, then logs the resulting merged DataFrame. It helps in enriching the catalog with additional astronomical information. Dependencies include pandas and custom functions `merge_type_constellation` and `log_df`.

```python
merge_df = merge_type_constellation(ngc_catalog_df, openngc_types_path, constellation_path)

log_df(merge_df)
# (228, 16)
```

--------------------------------

### Read JSON into DataFrame (Python)

Source: https://github.com/stevejcl/dwarfium/blob/apiV2/notebooks/5_create_catalogues.ipynb

This code snippet shows how to load data from a JSON file into a pandas DataFrame. It utilizes the `read_json` function, specifying the 'records' orientation to match the structure created by `to_json(orient='records')`. The `inplace=True` argument modifies the DataFrame directly, and `log_df(df)` is assumed to be a function for logging or displaying the DataFrame. Ensure `dso_catalog_json_path` points to the correct JSON file.

```python
df = pd.read_json(dso_catalog_json_path, orient='records', inplace=True)
log_df(df)
```

--------------------------------

### Count Bright and Tiny Astronomical Objects

Source: https://github.com/stevejcl/dwarfium/blob/apiV2/notebooks/5_create_catalogues.ipynb

This Python snippet counts astronomical objects that are 'bright' (Magnitude <= 10) and 'tiny' according to size criteria. It utilizes the pre-defined 'tiny_sizes' boolean mask and requires a pandas DataFrame 'df' with a 'Magnitude' column.

```python
# bright and  tiny
df[tiny_sizes].shape
```

--------------------------------

### Count Astronomical Objects with 'small_dso' Note

Source: https://github.com/stevejcl/dwarfium/blob/apiV2/notebooks/5_create_catalogues.ipynb

This Python snippet counts astronomical objects that have the 'small_dso' note. It requires a pandas DataFrame 'df' with a 'Notes' column.

```python
df[(df['Notes'] == 'small_dso')].shape
```

--------------------------------

### Filter Brightest Stars and Identify Missing Constellations

Source: https://github.com/stevejcl/dwarfium/blob/apiV2/notebooks/5_create_catalogues.ipynb

This Python code filters a star catalog DataFrame to find the brightest stars (proper name exists and magnitude <= 2.1). It then identifies which constellations are represented by these bright stars and which are not present in the filtered data. Dependencies include pandas and assumes the DataFrame `df` is already loaded.

```python
# Make a copy of the DataFrame
df_copy = df.copy()

# Filter out rows where 'proper' is not NaN and 'mag' is less than or equal to 2.1
brightest_stars_df = df_copy[df_copy['proper'].notna() & (df_copy['mag'] <= 2.1)]

# Get the list of constellations that are represented in the filtered DataFrame
represented_constellations = brightest_stars_df['con'].unique()

# Get the list of constellations in the original DataFrame
all_constellations = df_copy['con'].unique()

# Find constellations that are not represented in the filtered DataFrame
missing_constellations = set(all_constellations) - set(represented_constellations)
```

--------------------------------

### Sum of Bright/Large and Unknown Magnitude/Large Objects

Source: https://github.com/stevejcl/dwarfium/blob/apiV2/notebooks/5_create_catalogues.ipynb

This Python snippet calculates the sum of counts for objects that are 'bright and large' and those with 'unknown Magnitude and large'. This is a manual summation of previously derived counts.

```python
# bright + large, unknown mag and large

119 + 84
```

--------------------------------

### JavaScript for Toggling Icon Codes

Source: https://github.com/stevejcl/dwarfium/blob/apiV2/src/fontello/demo.html

This JavaScript function, toggleCodes, controls the visibility of icon code elements. When called with 'on' as true, it adds the 'codesOn' class to the target element, which in turn displays the icon codes. When 'on' is false, it removes the class, hiding the codes.

```javascript
function toggleCodes(on) {
  var obj = document.getElementById('icons');
  if (on) {
    obj.className += ' codesOn';
  } else {
    obj.className = obj.className.replace(' codesOn', '');
  }
}
```

--------------------------------

### Concatenate and Save Merged OpenNGC Data (Python)

Source: https://github.com/stevejcl/dwarfium/blob/apiV2/notebooks/1_process_openngc.ipynb

This Python code concatenates the `ngc_df` and `add_df` DataFrames vertically into a single DataFrame `df`. It then saves this merged DataFrame to a new CSV file named 'OpenNGC.csv' in the 'raw_data' directory, excluding the DataFrame index from the output file.

```python
df = pd.concat([ngc_df, add_df])
df.to_csv(openngc_path, index=False)
```

--------------------------------

### Count Astronomical Objects by Type

Source: https://github.com/stevejcl/dwarfium/blob/apiV2/notebooks/5_create_catalogues.ipynb

This Python snippet uses the pandas library to group astronomical data by object 'Type' and count the occurrences of each type. It requires a pandas DataFrame named 'df' as input.

```python
df.groupby(['Type']).size()
```