### Setup Environment and Load Data

Source: https://github.com/man-group/arcticdb/blob/master/docs/mkdocs/docs/notebooks/ArcticDB_pythagorean_won_loss_formula_notebook.ipynb

Installs necessary packages and imports libraries. It includes conditional logic to handle different environments (like Google Colab) and sets up the data file path.

```python
try:
    import google.colab
    import requests
    exec(requests.get(f"https://raw.githubusercontent.com/man-group/ArcticDB/master/docs/mkdocs/docs/notebooks/styling.py").text)
    dataFile = f'https://raw.githubusercontent.com/man-group/ArcticDB/master/docs/mkdocs/docs/notebooks/data/2025-08-26-sports.csv'
except ImportError:
    from styling import *
    dataFile = "data/2025-08-26-sports.csv"

```

```python
%pip -q install arcticdb pandas numpy statsmodels plotly scikit-learn
import arcticdb as adb
adb.__version__
```

```python
import numpy as np
import pandas as pd
import statsmodels.api as sm
import statsmodels.formula.api as smf

colors = setup_plotly_theme()

```

--------------------------------

### Install ArcticDB

Source: https://github.com/man-group/arcticdb/blob/master/docs/mkdocs/docs/notebooks/ArcticDB_billion_row_challenge.ipynb

Installs the arcticdb library. Run this command in your environment before importing.

```python
!pip install arcticdb
```

--------------------------------

### Basic Usage Example

Source: https://github.com/man-group/arcticdb/blob/master/docs/claude/python/README.md

A simple example demonstrating how to connect to ArcticDB, create a library, write data, and read data.

```APIDOC
## Basic Usage

### Example
```python
from arcticdb import Arctic
import pandas as pd

# Connect to storage
ac = Arctic("lmdb://./my_db")

# Create library
lib = ac.create_library("my_lib")

# Write data
df = pd.DataFrame({"col": [1, 2, 3]})
lib.write("symbol", df)

# Read data
result = lib.read("symbol")
print(result.data)
```
```

--------------------------------

### PowerShell Commands for ArcticDB Setup

Source: https://github.com/man-group/arcticdb/wiki/Dev:-Building

This sequence of PowerShell commands covers essential steps for setting up the ArcticDB development environment, including SSH key generation, repository cloning, submodule initialization, virtual environment creation, and dependency installation.

```powershell
# SSH key
ssh-keygen.exe
>>> copy ~/.ssh/id_rsa.pub key to your Github account

# Clone
mkdir ~/source && cd ~/source
git clone git@github.com:man-group/ArcticDB.git
cd ~/source/ArcticDB
git submodule update --init --recursive

# Create venv - Python version can be changed in the obvious way. This step requires PowerShell to be running with admin privileges.
# For some reason using non-symlinked python environment in Visual Studio will not load arcticdb symbol during debug
C:\Users\<username>\AppData\Local\Programs\Python\Python310\python.exe -m venv --symlinks ..\venvs\310

# This step requires PowerShell to be running with admin privileges
cd .\cpp\third_party\lmdbcxx\ & .\symlink_srcs.ps1

# Activate the venv
..\venvs\310\Scripts\Activate.ps1

# Get all Python dependencies - very hacky, should be improved
# Remove `dataclasses` from `setup.cfg`
pip install wheel
python.exe setup.py egg_info
# Remove `[Testing]` line and other such lines including `[]` from .\python\arcticdb.egg-info\requires.txt
pip install -r .\python\arcticdb.egg-info\requires.txt

# Generate the Python protobuf files
python.exe setup.py protoc --build-lib python

```

--------------------------------

### Installation

Source: https://github.com/man-group/arcticdb/blob/master/docs/claude/python/README.md

Instructions for installing the arcticdb Python package.

```APIDOC
## Installation

### Command
```bash
pip install arcticdb
```
```

--------------------------------

### Install Build Prerequisites on Man Linux

Source: https://github.com/man-group/arcticdb/blob/master/CLAUDE.md

Installs necessary system packages for the vcpkg-based build. `ccache` is optional but recommended for faster rebuilds.

```bash
sudo apt install pkg-config flex bison libsasl2-dev ccache -y
```

--------------------------------

### Setup Library and Symbol

Source: https://github.com/man-group/arcticdb/blob/master/docs/mkdocs/docs/notebooks/ArcticDB_demo_compact_data.ipynb

Delete existing library and create a new one to ensure a clean state for the demonstration.

```python
# So that each run of the notebook starts from an empty library
arctic.delete_library("compact_data")
# library - rows_per_segment not specified, so will default to 100,000
lib = arctic.get_library("compact_data", create_if_missing=True)
# symbol
sym = "OHLCV_minutely"
```

--------------------------------

### Initialize ArcticDB Session

Source: https://github.com/man-group/arcticdb/blob/master/docs/mkdocs/docs/notebooks/ArcticDB_demo_equity_options.ipynb

Sets up a connection to the ArcticDB library. Ensure ArcticDB is installed and configured.

```python
from arctic import Arctic
from arctic.auth import Authenticator

# Assuming Arctic is running locally or accessible via connection string
# For authenticated access, provide credentials
# auth = Authenticator('username', 'password')
# arctic = Arctic('localhost:5000', authenticator=auth)

arctic = Arctic('localhost:5000')
```

--------------------------------

### Install Azurite

Source: https://github.com/man-group/arcticdb/wiki/Dev:-Building

Install Azurite, a local Azure Storage emulator, using nvm to manage Node.js versions and npm for global package installation.

```bash
# Install azurite
# 1. Install nvm
curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.7/install.sh | bash
source ~/.bashrc
# 2. Install npm using nvm
nvm install 16
# 3. Install azurite using npm
npm install -g azurite
# Optional: use this as your system installation - can be useful if you launch tests from an IDE and do not see the path modifications that nvm made
# sudo mv /usr/bin/node /usr/bin/node.bak
# sudo ln -s $(which node) /usr/bin/node

```

--------------------------------

### Install ArcticDB Extension

Source: https://github.com/man-group/arcticdb/blob/master/cpp/arcticdb/CMakeLists.txt

Defines the installation destination and components for the built module.

```cmake
install(TARGETS arcticdb_ext
        LIBRARY DESTINATION . COMPONENT Python_Lib
        RUNTIME DESTINATION . COMPONENT Python_Lib)
```

--------------------------------

### Install C++ Toolchain

Source: https://github.com/man-group/arcticdb/wiki/Dev:-Building

Install essential C++ build tools, including GCC and CMake, using apt-get.

```bash
# Install C++ toolchain
sudo apt update
sudo apt-get install build-essential gcc-11 cmake gdb

```

--------------------------------

### Install ArcticDB using pip

Source: https://github.com/man-group/arcticdb/blob/master/README.md

Install the ArcticDB library using pip. This command installs the latest version available on PyPI.

```bash
pip install arcticdb
```

--------------------------------

### Build ArcticDB Wheel and Install

Source: https://github.com/man-group/arcticdb/blob/master/docs/README.md

Builds a Python wheel for ArcticDB from source and installs it. This is required before generating API documentation with MkDocs.

```bash
cd $PROJECT_ROOT
python setup.py bdist_wheel
pip install <generated wheel>
```

--------------------------------

### Set up ArcticDB Library

Source: https://github.com/man-group/arcticdb/blob/master/docs/mkdocs/docs/notebooks/ArcticDB_demo_snapshots.ipynb

Initializes ArcticDB, deletes an existing library if present to ensure a clean state, and gets or creates the specified library.

```python
lib_name = 'demo'
arctic = adb.Arctic("lmdb://arcticdb_snapshot_demo")
if lib_name in arctic.list_libraries():
    arctic.delete_library(lib_name)
lib = arctic.get_library('demo', create_if_missing=True)
```

--------------------------------

### Install to Custom Prefix

Source: https://github.com/man-group/arcticdb/blob/master/cpp/third_party/msgpack-c/README.md

If you lack superuser permissions or prefer not to install the library system-wide, use the CMAKE_INSTALL_PREFIX option to specify a custom installation directory.

```bash
cmake -DCMAKE_INSTALL_PREFIX=/your/custom/prefix .
```

--------------------------------

### Create Makefile.local

Source: https://github.com/man-group/arcticdb/blob/master/CLAUDE.md

Copies the example Makefile.local to Makefile.local for Man-specific settings. This file is git-ignored.

```bash
cp Makefile.local.example Makefile.local
```

--------------------------------

### Initialize ArcticDB and Library

Source: https://github.com/man-group/arcticdb/blob/master/docs/mkdocs/docs/notebooks/ArcticDB_merge.ipynb

Create an ArcticDB instance pointing to a local LMDB storage and get or create a library named 'prices'.

```python
ac = arcticdb.Arctic("lmdb://merge_example")
lib = ac.get_library("prices", create_if_missing=True)
```

--------------------------------

### Clone and Install msgpack-c (C++03)

Source: https://github.com/man-group/arcticdb/blob/master/cpp/third_party/msgpack-c/README.md

Use these commands to clone the repository, checkout the C++ master branch, configure the build with CMake, and install the library system-wide. Requires gcc and cmake.

```bash
git clone https://github.com/msgpack/msgpack-c.git
cd msgpack-c
git checkout cpp_master
cmake .
sudo cmake --build . --target install
```

--------------------------------

### Install Build Dependencies

Source: https://github.com/man-group/arcticdb/wiki/Dev:-ASV-Benchmarks

Installs essential build tools and libraries required for compiling and running benchmarks on a Linux environment.

```bash
sudo apt update
sudo apt-get install build-essential gcc-11 cmake gdb
sudo apt-get install zip pkg-config flex bison libkrb5-dev libsasl2-dev libcurl4-openssl-dev
```

--------------------------------

### Serve MkDocs Development Server

Source: https://github.com/man-group/arcticdb/blob/master/docs/README.md

Starts a local development server for MkDocs, accessible at 0.0.0.0:8000. Changes to the documentation source will trigger live reloads.

```bash
mkdocs serve -a 0.0.0.0:8000
```

--------------------------------

### Install ArcticDB and yfinance

Source: https://github.com/man-group/arcticdb/blob/master/docs/mkdocs/docs/notebooks/ArcticDB_demo_equity_analytics.ipynb

Installs the necessary Python packages for ArcticDB and yfinance. Run this command in your environment before proceeding.

```python
!pip install arcticdb yfinance
```

--------------------------------

### Install Required Libraries

Source: https://github.com/man-group/arcticdb/wiki/Dev:-Building

Install necessary shared libraries and tools for building ArcticDB, such as zip, pkg-config, and Kerberos client libraries.

```bash
# Required shared libraries and tools
sudo apt-get install zip pkg-config flex bison libkrb5-dev libsasl2-dev libcurl4-openssl-dev

```

--------------------------------

### Install X11 Libraries and Browser

Source: https://github.com/man-group/arcticdb/wiki/Dev:-Building

Install X11 development libraries and Firefox, which may be needed for certain development tools like CLion to function correctly within WSL.

```bash
# Install x-11 libs, needed for clion UI
sudo apt-get install libxi-dev libxtst6 libxrender1

# Install a browser (useful for clion to open links)
sudo apt-get install firefox

```

--------------------------------

### Install ArcticDB and S3 Libraries

Source: https://github.com/man-group/arcticdb/blob/master/docs/mkdocs/docs/notebooks/ArcticDB_aws_public_blockchain.ipynb

Installs necessary Python libraries for ArcticDB, AWS S3 interaction, and data handling. Ensure these are installed before proceeding.

```python
# s3fs is used by pandas.read_parquet('s3://...')
%pip install arcticdb boto3 tqdm s3fs fastparquet
```

--------------------------------

### Configure Azure Blob Storage in Python

Source: https://github.com/man-group/arcticdb/blob/master/docs/claude/cpp/STORAGE_BACKENDS.md

Examples for initializing Arctic connections with various Azure authentication and configuration parameters.

```python
# With account key
ac = Arctic("azure://Container=mycontainer;AccountName=myaccount;AccountKey=BASE64KEY")

# With SAS token
ac = Arctic("azure://Container=mycontainer;AccountName=myaccount;SharedAccessSignature=TOKEN")

# With path prefix
ac = Arctic("azure://Container=mycontainer;AccountName=myaccount;AccountKey=KEY;Path_prefix=arcticdb/data")

# With CA certificate (Linux only)
ac = Arctic("azure://Container=mycontainer;AccountName=myaccount;AccountKey=KEY;CA_cert_path=/etc/ssl/certs/ca-certificates.crt")
```

--------------------------------

### Install Built Package Locally

Source: https://github.com/man-group/arcticdb/wiki/Dev:-Building-From-Feedstock

Install the ArcticDB package from the local build using mamba. If ArcticDB is already installed, you may need to uninstall it first using 'mamba remove arcticdb'.

```bash
mamba install -c local arcticdb
```

--------------------------------

### Install clang with apt-get

Source: https://github.com/man-group/arcticdb/wiki/Dev:-Building

Installs clang and optional LLVM debugging tools using the apt-get package manager. Useful for Debian-based Linux distributions.

```bash
sudo apt-get install clang
# Optional install LLVM's debugger and clang-tidy
sudo apt-get install lldb clang-tidy
```

--------------------------------

### Pushdown Query Example

Source: https://github.com/man-group/arcticdb/blob/master/docs/claude/ARCHITECTURE.md

Demonstrates reading specific columns with a filter applied at the storage level. Ensure `col` and `q` are imported from `arcticdb.version_store.processing`.

```python
# Example: Only reads required columns and filters at storage level
lib.read("symbol", columns=["price", "volume"], query_builder=q.filter(col("price") > 100))
```

--------------------------------

### Getting ArcticDB Version

Source: https://github.com/man-group/arcticdb/blob/master/docs/mkdocs/docs/notebooks/ArcticDB_demo_equity_options.ipynb

Retrieves the installed version of the ArcticDB library. This can be useful for compatibility checks or reporting.

```python
from arctic import __version__

print(f'ArcticDB version: {__version__}')
```

--------------------------------

### Build Flow Diagram

Source: https://github.com/man-group/arcticdb/blob/master/docs/claude/ARCHITECTURE.md

Illustrates the build process for ArcticDB, showing the sequence from Python setup to Protobuf compilation and CMake-based C++ compilation.

```text
setup.py
    │
    ├── CompileProto ────► Generate *_pb2.py from .proto files
    │                      (for protobuf 4, 5, 6 compatibility)
    │
    └── CMakeBuild ──────► CMake configure + build
                               │
                               ├── vcpkg ────► Download/build C++ dependencies
                               │
                               └── ninja ────► Compile arcticdb_ext.so
```

--------------------------------

### Configure ArcticDB S3 Connection

Source: https://github.com/man-group/arcticdb/blob/master/docs/claude/cpp/STORAGE_BACKENDS.md

Examples for initializing an ArcticDB connection using different S3 authentication and configuration methods.

```python
from arcticdb import Arctic

# AWS S3 with default credentials (uses AWS credential provider chain)
ac = Arctic("s3://s3.us-east-1.amazonaws.com:my-bucket?aws_auth=true")

# With explicit credentials
ac = Arctic("s3://s3.us-east-1.amazonaws.com:my-bucket?access=AKID&secret=SECRET")

# With path prefix (data stored under prefix in bucket)
ac = Arctic("s3://s3.us-east-1.amazonaws.com:my-bucket?path_prefix=arcticdb/data")

# S3-compatible (MinIO, etc.)
ac = Arctic("s3://localhost:9000:my-bucket?access=minioadmin&secret=minioadmin")
```

--------------------------------

### Setup Development Environment

Source: https://github.com/man-group/arcticdb/blob/master/CLAUDE.md

Sets up the development environment, including submodules, virtual environment, protoc, and building the project. Use `CLEAN=1` to replace an existing virtual environment.

```bash
make setup NAME=x
```

```bash
make setup NAME=x CLEAN=1
```

--------------------------------

### AWS C++ SDK Assertion Error Example

Source: https://github.com/man-group/arcticdb/blob/master/docs/mkdocs/docs/aws.md

This error occurs when ArcticDB cannot obtain a temporary token by assuming a role, often due to incorrect IAM setup or credentials files.

```text
virtual void Aws::Auth::STSProfileCredentialsProvider::Reload(): Assertion `!profileIt->second.GetCredentials().IsEmpty()' failed.
```

--------------------------------

### Setup and Helper Functions for ArcticDB Staging Demo

Source: https://github.com/man-group/arcticdb/blob/master/docs/mkdocs/docs/notebooks/ArcticDB_staged_data_with_tokens.ipynb

Initializes the ArcticDB library with specific options for demonstration purposes and defines helper functions for creating DataFrames and managing the library.

```python
import pandas as pd
import numpy as np
import time
from multiprocessing import Process, Queue

from arcticdb.exceptions import NoSuchVersionException, UnsortedDataException
from arcticdb import Arctic, LibraryOptions

def make_df(rows: int, cols: int, start_date):
    index = pd.date_range(start=pd.to_datetime(start_date), periods=rows, freq="D")
    data = np.arange(1, rows * cols + 1).reshape(rows, cols, order="F")
    return pd.DataFrame(data, index=index, columns=[f"col_{i}" for i in range(cols)])

# Below library is configured to have only 2 rows per segment. This is to make the example more readable as the default is 100 000 rows per segment.    
def get_lib():
    return Arctic("lmdb://tmp/stage_tokens_demo").get_library("stage_tokens_demo", library_options=LibraryOptions(rows_per_segment=2), create_if_missing=True)

def clear_lib():
    get_lib()._nvs.version_store.clear()

sym = "sym"
clear_lib()
lib = get_lib()
```

--------------------------------

### Load and Prepare Options Data

Source: https://github.com/man-group/arcticdb/blob/master/docs/mkdocs/docs/notebooks/ArcticDB_demo_equity_options.ipynb

Loads options data from ArcticDB and prepares it for analysis. Ensure you have the necessary libraries installed and ArcticDB is accessible.

```python
from arctic import Arctic
from arctic.auth import Auth
import pandas as pd
import numpy as np
import plotly.graph_objects as go
from arctic.decorators import arctic_read

# Connect to ArcticDB
# Replace with your ArcticDB connection details
# auth = Auth("user", "password")
# arctic = Arctic("host:port", auth=auth)

# For local testing, you might use:
arctic = Arctic("localhost:5555")

# Access the library
lib = arctic['options']

# Load data for a specific date and symbol
date = pd.to_datetime('2023-10-26')
symbol = 'AAPL'

# Fetch data using arctic_read decorator
@arctic_read('options', 'AAPL')
def get_options_data(start_date, end_date):
    return pd.DataFrame()

# Example of fetching data (replace with actual fetch logic if needed)
# For demonstration, we'll create dummy data if the library is empty
if len(lib.read('AAPL').fetch_all()) == 0:
    print("No data found, creating dummy data.")
    # Create dummy data for demonstration
    dates = pd.to_datetime(['2023-10-26', '2023-10-27'])
    symbols = ['AAPL', 'GOOG']
    
    all_data = []
    for d in dates:
        for s in symbols:
            strikes = np.linspace(90, 150, 10)
            expiries = [d + pd.Timedelta(days=x) for x in [30, 60, 90]]
            
            for expiry in expiries:
                for strike in strikes:
                    # Simulate some volatility and price
                    vol = np.random.uniform(0.15, 0.35)
                    price = strike * (1 + np.random.normal(0, 0.05))
                    
                    row = {
                        'date': d,
                        'symbol': s,
                        'expiry': expiry,
                        'strike': strike,
                        'mid_price': price,
                        'implied_volatility': vol,
                        'option_type': np.random.choice(['call', 'put'])
                    }
                    all_data.append(row)
    
    df = pd.DataFrame(all_data)
    df.set_index(['date', 'symbol', 'expiry', 'strike', 'option_type'], inplace=True)
    
    # Write dummy data to ArcticDB
    for s in symbols:
        lib.write(s, df[df.index.get_level_values('symbol') == s])
    print("Dummy data created.")

# Fetch data for a specific symbol and date
# Assuming 'AAPL' and '2023-10-26' exist in your library

symbol_data = lib.read(symbol).fetch(start_date=date, end_date=date)

# Display the first few rows of the loaded data
print(f"Loaded data for {symbol} on {date.date()}:")
print(s y mbol_data.head())

```

--------------------------------

### Get or Create a Library

Source: https://github.com/man-group/arcticdb/blob/master/docs/mkdocs/docs/notebooks/ArcticDB_demo_lmdb.ipynb

Retrieves an existing library named 'sample' or creates it if it does not exist. Libraries are used to organize symbols (data tables).

```python
lib = arctic.get_library('sample', create_if_missing=True)
```

--------------------------------

### Download and Install Mambaforge

Source: https://github.com/man-group/arcticdb/wiki/Dev:-Building-From-Feedstock

Download the Mambaforge installer script and execute it to install Mambaforge, a package manager that includes conda and mamba.

```bash
wget https://github.com/conda-forge/miniforge/releases/latest/download/Mambaforge-Linux-x86_64.sh
bash ./Mambaforge-Linux-x86_64.sh
```

--------------------------------

### Install Python Dependencies for ArcticDB

Source: https://github.com/man-group/arcticdb/wiki/Dev:-Building

Install all necessary Python dependencies for ArcticDB. This involves installing wheel, generating egg info, and then installing requirements from the generated file. Note that some manual modifications to configuration files might be required.

```bash
# Get all Python dependencies - very hacky, should be improved
# Remove `dataclasses` from `setup.cfg`
pip install wheel
python.exe setup.py egg_info
# Remove `[Testing]` line from .\python\arcticdb.egg_info\requires.txt
pip install -r .\python\arcticdb.egg_info\requires.txt
```

--------------------------------

### Install ArcticDB Python Dependencies

Source: https://github.com/man-group/arcticdb/wiki/Dev:-Building

Install ArcticDB's Python dependencies using pip. This involves installing wheel, setuptools, and then installing from a requirements file generated by egg_info. Note: Requires manual modification of setup.cfg and requires.txt.

```bash
# Get all Python dependencies - very hacky, should be improved
# Remove `dataclasses` from `setup.cfg`
pip install wheel
pip install setuptools
python setup.py egg_info
# Remove `[Testing]` line and '[:python_version < "3.7"]' line from ./python/arcticdb.egg-info/requires.txt
pip install -r ./python/arcticdb.egg-info/requires.txt

```

--------------------------------

### Install ArcticDB using conda-forge

Source: https://github.com/man-group/arcticdb/blob/master/README.md

Install the ArcticDB library using conda-forge. This is an alternative installation method for users who prefer conda environments.

```bash
conda install -c conda-forge arcticdb
```

--------------------------------

### Python Example: AWS Permissions with ArcticDB

Source: https://github.com/man-group/arcticdb/blob/master/docs/mkdocs/docs/aws_permissions.md

Demonstrates creating and accessing ArcticDB libraries with AWS credentials, showing how permissions are enforced based on the library path which includes user and team information. Unauthorized access attempts will result in a PermissionException.

```python
import numpy as np
import arcticdb as adb

# jane@acme team=data
access = '<REDACTED>'
secret = '<REDACTED>'
bucket='acme-arcticdb'
region='eu-west-2'

arctic = adb.Arctic(f's3://s3.{region}.amazonaws.com:{bucket}?access={access}&secret={secret}')

# Create library as me
arctic.create_library('jane@acme/weather')
lib = arctic.get_library('jane@acme/weather')
lib.write('test', np.arange(100))

# Create library as data team
arctic.create_library('data/forecast')
lib = arctic.get_library('data/forecast')
lib.write('test', np.arange(100))

# See all libraries
arctic.list_libraries()
# ['alan@acme/bonds', 'data/forecast', 'jane@acme/weather', 'quant/stocks']

# Can't use or delete Alan or Quant team data
arctic.get_library('alan@acme/bonds')
arctic.get_library('quant/stocks')
arctic.delete_libraru('alan@acme/bonds')
# All raise:
# PermissionException: E_PERMISSION Permission error: S3Error#15 : No response body.
```

--------------------------------

### Initialize Arctic with Azure

Source: https://github.com/man-group/arcticdb/blob/master/docs/claude/python/ADAPTERS.md

Provides examples of initializing the Arctic library with Azure storage URIs. Use either an account key or a SAS token for authentication.

```python
# With account key
ac = Arctic("azure://Container=mycontainer;AccountName=myaccount;AccountKey=BASE64KEY==")

# With SAS token
ac = Arctic("azure://Container=mycontainer;AccountName=myaccount;SharedAccessSignature=sv=2021...")
```

--------------------------------

### Initialize ArcticDB Library

Source: https://github.com/man-group/arcticdb/blob/master/docs/mkdocs/docs/notebooks/ArcticDB_demo_equity_analytics.ipynb

Connects to a local ArcticDB instance using LMDB storage and retrieves or creates a library named 'demo'.

```python
arctic = adb.Arctic("lmdb://arcticdb_equity")
lib = arctic.get_library('demo', create_if_missing=True)
```

--------------------------------

### Initialize ArcticDB instance and library

Source: https://github.com/man-group/arcticdb/blob/master/docs/mkdocs/docs/notebooks/ArcticDB_demo_recursive_normalizers.ipynb

Creates an ArcticDB instance using the LMDB backend and retrieves or creates a library named 'demo'.

```python
# Create Arctic instance with LMDB backend
arctic = adb.Arctic("lmdb://recursive_normalizers_demo")

lib = arctic.get_library("demo", create_if_missing=True)
```

--------------------------------

### Initialize Arctic with S3

Source: https://github.com/man-group/arcticdb/blob/master/docs/claude/python/ADAPTERS.md

Shows how to initialize the Arctic library using S3 URIs. Supports default AWS credentials, explicit credentials, and S3-compatible services like MinIO.

```python
from arcticdb import Arctic

# AWS S3 with default credentials
ac = Arctic("s3://s3.amazonaws.com:my-bucket?region=us-east-1")

# With explicit credentials
ac = Arctic("s3://s3.amazonaws.com:my-bucket?access=AKID&secret=SECRET&region=us-east-1")

# MinIO or S3-compatible
ac = Arctic("s3://localhost:9000:my-bucket?region=us-east-1")
```

--------------------------------

### Initialize NativeVersionStore

Source: https://github.com/man-group/arcticdb/blob/master/docs/claude/python/NATIVE_VERSION_STORE.md

Access the NativeVersionStore instance from an existing Library or create it directly using configuration and environment objects.

```python
from arcticdb.version_store._store import NativeVersionStore

# Usually obtained via Library._nvs or for migration
nvs = lib._nvs

# Or construct directly (advanced)
nvs = NativeVersionStore.create_store_from_config(config, env)
```

--------------------------------

### Install MkDocs Dependencies with Mamba

Source: https://github.com/man-group/arcticdb/blob/master/docs/README.md

Installs necessary packages for MkDocs, including themes and plugins, using mamba.

```bash
mamba install mkdocs-material mkdocs-jupyter mkdocstrings-python black pybind11-stubgen mike wheel
```

--------------------------------

### Python Example for Creating a Snapshot

Source: https://github.com/man-group/arcticdb/blob/master/docs/claude/cpp/VERSIONING.md

Illustrates creating a snapshot of all symbols at their current versions. Also shows how to create a snapshot for specific symbols at specified versions.

```python
lib.snapshot("my_snapshot")  # Snapshot all symbols at current versions
lib.snapshot("my_snapshot", versions={"sym_a": 3, "sym_b": 2})  # Specific versions
```

--------------------------------

### Install MkDocs Dependencies with Pip

Source: https://github.com/man-group/arcticdb/blob/master/docs/README.md

Installs necessary packages for MkDocs, including themes and plugins, using pip.

```bash
pip3 install mkdocs-material mkdocs-jupyter 'mkdocstrings[python]' black pybind11-stubgen mike wheel
```

--------------------------------

### Initialize Arctic with LMDB

Source: https://github.com/man-group/arcticdb/blob/master/docs/claude/python/ADAPTERS.md

Demonstrates initializing the Arctic library with LMDB storage. Supports relative and absolute paths, and allows specifying a custom map size for the database.

```python
# Relative path
ac = Arctic("lmdb://./my_arctic_db")

# Absolute path
ac = Arctic("lmdb:///home/user/arcticdb")

# With custom map size (10GB)
ac = Arctic("lmdb://./my_db?map_size=10737418240")
```

--------------------------------

### Install Build Tools

Source: https://github.com/man-group/arcticdb/wiki/Dev:-Building-From-Feedstock

Install necessary build tools, including conda-build and git, within the activated conda environment.

```bash
mamba install conda-build git
```

--------------------------------

### Initialize ArcticDB with Local Storage

Source: https://github.com/man-group/arcticdb/blob/master/docs/mkdocs/docs/index.md

Import the ArcticDB library and instantiate it using a local LMDB path for storage. This sets up the database engine for local use.

```python
import arcticdb as adb
# this will set up the storage using the local file system
uri = "lmdb://tmp/arcticdb_intro"
ac = adb.Arctic(uri)
```

--------------------------------

### Install Py-Spy

Source: https://github.com/man-group/arcticdb/wiki/ArcticDB-Performance-Profiling

Install Py-spy using pip. This tool is useful for profiling Python code that relies on C++ extensions.

```bash
pip install py-spy
```

--------------------------------

### Initialize ArcticDB Connection

Source: https://github.com/man-group/arcticdb/blob/master/docs/mkdocs/docs/notebooks/ArcticDB_demo_equity_options.ipynb

Establishes a connection to the ArcticDB instance using LMDB and retrieves or creates the 'demo_options' library.

```python
arctic = adb.Arctic("lmdb://arcticdb_equity_options")
lib = arctic.get_library('demo_options', create_if_missing=True)
```

--------------------------------

### Migrate NativeVersionStore from V1 to V2

Source: https://github.com/man-group/arcticdb/blob/master/docs/claude/python/NATIVE_VERSION_STORE.md

Demonstrates the transition from direct NativeVersionStore usage to the recommended Arctic library interface, including how to access the underlying store if necessary.

```python
# V1 code
from arcticdb.version_store._store import NativeVersionStore
nvs = ...
nvs.write("symbol", df)

# V2 code (recommended)
from arcticdb import Arctic
ac = Arctic("lmdb://./db")
lib = ac.create_library("my_lib")
lib.write("symbol", df)

# If you need NativeVersionStore from Library
nvs = lib._nvs
```

--------------------------------

### Install ArcticDB in Editable Mode

Source: https://github.com/man-group/arcticdb/blob/master/CLAUDE.md

Installs ArcticDB in editable mode, allowing Python code changes without requiring a C++ rebuild.

```bash
make install-editable
```

--------------------------------

### Serve Versioned MkDocs Locally

Source: https://github.com/man-group/arcticdb/blob/master/docs/README.md

Serves the versioned documentation locally using the 'mike' tool. Ensure you are in the 'docs/mkdocs' directory.

```bash
mike serve
```

--------------------------------

### Install ArcticDB Testing Dependencies

Source: https://github.com/man-group/arcticdb/blob/master/docs/mkdocs/docs/technical/contributing.md

Install the ArcticDB package with testing extras using pip. This command ensures all necessary packages for running tests are available.

```bash
python -m pip install arcticdb[Testing]
```

--------------------------------

### Install Python dev headers on MacOS

Source: https://github.com/man-group/arcticdb/wiki/Dev:-Building

Installs necessary Python development headers and tools on MacOS using Homebrew. This is a prerequisite for building Python wheels.

```bash
brew update
# This will install the dev headers as well, no need to install python-devel
brew install python bison

# Assuming python3.13, which is the current latest
python3 -m venv ~/venvs/313

# Install the required packages in the venv (repeat the steps from above)
```

--------------------------------

### XSS via href with Start of Heading and JavaScript URI

Source: https://github.com/man-group/arcticdb/blob/master/python/tests/util/blns.txt

Tests XSS using a Start of Heading character (\x03) within a JavaScript URI in an 'href' attribute.

```html
<a href="\x03javascript:javascript:alert(72)" id="fuzzelement1">test</a>
```

--------------------------------

### Benchmark Class Setup for Multiple Storages

Source: https://github.com/man-group/arcticdb/wiki/Dev:-ASV-Benchmarks

This Python code demonstrates how to structure an ASV benchmark class to run against multiple storage backends. It defines storage parameters, sets up libraries for each enabled storage, and includes logic to skip benchmarks if a storage is not available.

```python
from asv_runner.benchmarks.mark import SkipNotImplemented
from benchmarks.environment_setup import Storage, create_libraries_across_storages

class ModificationFunctions:
    # 1. Define the storage parameter list
    storages = [Storage.LMDB, Storage.AMAZON]

    # 2. Define all parameters and their names
    rows_and_cols = [(1_000_000, 2), (10_000_000, 2)]
    params = [rows_and_cols, storages]
    param_names = ["rows_and_cols", "storage"]

    # 3. setup_cache runs ONCE per benchmark class (shared across all param combos)
    def setup_cache(self):
        # Create libraries for all enabled storages
        lib_for_storage = create_libraries_across_storages(ModificationFunctions.storages)

        # Optionally pre-populate data
        for storage in ModificationFunctions.storages:
            lib = lib_for_storage[storage]
            if lib is None:
                continue  # Storage not enabled
            lib.write("sym", some_dataframe)

        return lib_for_storage  # Pickled and passed to setup by ASV

    # 4. setup runs BEFORE each benchmark method for each parameter combination
    def setup(self, libs_for_storage, rows_and_cols, storage):
        self.lib = libs_for_storage[storage]
        if self.lib is None:
            raise SkipNotImplemented  # Crucial: Skip the benchmark if the storage is not enabled

        # Prepare test data...

    # 5. Write a benchmark
    def time_write(self, *args):
        self.lib.write("sym", self.df)
```

--------------------------------

### Azure URI Examples

Source: https://github.com/man-group/arcticdb/blob/master/docs/claude/python/ADAPTERS.md

Demonstrates the URI formats for Azure storage, using semicolon-separated key-value pairs for container, account name, and authentication. Supports account keys, SAS tokens, and custom endpoints.

```text
azure://Container=container;AccountName=account;AccountKey=key
azure://Container=container;AccountName=account;SharedAccessSignature=token
azure://BlobEndpoint=endpoint;Container=container;AccountName=account;AccountKey=key
azure://Container=mycontainer;AccountName=myaccount;AccountKey=BASE64KEY==
azure://Container=mycontainer;AccountName=myaccount;SharedAccessSignature=sv=2021-06-08&ss=b...
azure://Container=mycontainer;AccountName=myaccount;AccountKey=KEY;Path_prefix=arcticdb/data
azure://BlobEndpoint=https://myaccount.blob.core.windows.net;Container=mycontainer;AccountName=myaccount;AccountKey=KEY
```

--------------------------------

### Install Python 3.11 and Create venv

Source: https://github.com/man-group/arcticdb/wiki/Dev:-Building

Commands to install Python 3.11 and create a virtual environment for building Python wheels. This is a prerequisite for building wheels for Python 3.11.

```bash
# Install python 3.11
sudo apt-get install python3.11-dev python3.11 python3.11-venv
python3.11 -m venv ~/venvs/311

# Install the required packages in the venv (repeat the steps from above)
```

--------------------------------

### Run MinIO Server with Docker

Source: https://github.com/man-group/arcticdb/wiki/Dev:-SSL-Behaviour-and-Testing

Launches a MinIO server instance using Docker, mapping ports and volumes, and setting root credentials. Ensure MinIO is not hosted on Windows for ArcticDB compatibility.

```bash
sudo docker run \
   -p 9000:9000 \
   -p 9001:9001 \
   --name minio \
   -v /home/aseaton/minio/data:/data \
   -v /home/aseaton/.minio/:/root/.minio/ \
   -e "MINIO_ROOT_USER=ROOTNAME" \
   -e "MINIO_ROOT_PASSWORD=CHANGEME123" \
   quay.io/minio/minio server /data --console-address ":9001"
```

--------------------------------

### Install Python 3.10 and venv

Source: https://github.com/man-group/arcticdb/wiki/Dev:-Building

Install Python 3.10 and its development headers, along with the venv module, using a PPA. This is a workaround for the default Python version in Ubuntu 24.04.

```bash
# Use following commands:
# based on : https://docs.vultr.com/how-to-install-python-and-pip-on-ubuntu-24-04
sudo add-apt-repository ppa:deadsnakes/ppa
sudo apt update
sudo apt install python3.10
sudo apt install python3.10-dev
sudo apt install python3.10-venv

```

--------------------------------

### Editable Install Python Package

Source: https://github.com/man-group/arcticdb/blob/master/docs/mkdocs/docs/technical/contributing.md

Build and install the ArcticDB Python package in editable mode within the 'arcticdb' environment. This is recommended for development. Ensure ARCTICDB_USING_CONDA and ARCTIC_CMAKE_PRESET are set appropriately.

```bash
export ARCTICDB_USING_CONDA=1
# Adapt the CMake preset to your setup.
export ARCTIC_CMAKE_PRESET=linux-conda-debug
python -m pip install --no-build-isolation --no-deps --verbose --editable .
```

--------------------------------

### Install ArcticDB in Editable Mode

Source: https://github.com/man-group/arcticdb/blob/master/docs/mkdocs/docs/technical/contributing.md

Installs ArcticDB in editable mode using pip, which compiles changed files. This is an alternative to setting PYTHONPATH and must be rerun after C++ file changes.

```bash
$MY_PYTHON -m pip install -ve .
```

--------------------------------

### Set up AWS S3 Bucket for ArcticDB

Source: https://github.com/man-group/arcticdb/blob/master/docs/mkdocs/docs/notebooks/ArcticDB_aws_public_blockchain.ipynb

Initializes an S3 resource and checks for existing buckets starting with 'arcticdb-data-'. If none are found, a new bucket is created in the specified region.

```python
s3 = boto3.resource('s3')
region = boto3.session.Session().region_name

bucket = [b for b in s3.buckets.all() if b.name.startswith('arcticdb-data-')]

if bucket:
    bucket_name = bucket[0].name
    print('Bucket found:', bucket_name)
else:
    bucket_name = f'arcticdb-data-{uuid4()}'
    s3.create_bucket(Bucket=bucket_name, CreateBucketConfiguration={'LocationConstraint':region})
    print('Bucket created:', bucket_name)
```

--------------------------------

### Configure C++ Breakpoints in MSVS

Source: https://github.com/man-group/arcticdb/wiki/Dev:-Building

Set the 'arcticdb_ext' project as the startup project and configure debugging properties to attach to a Python process for hitting C++ breakpoints.

```text
1. Go to `View -> Solution Explorer`. Right click on `arcticdb_ext` project and select `Set as startup project`.
2. Right click on `arcticdb_ext`. `Properties -> Debugging`
    1. `Command` set the absolute path to python from your virtual environment
    2. `Command Arguments` set to `-m pytest <absolute_path_to_tests>`
```

--------------------------------

### Install Build Tools and Ninja using Chocolatey

Source: https://github.com/man-group/arcticdb/wiki/Dev:-Building

Install Visual Studio 2022 build tools and Ninja using Chocolatey. This command specifies the exact version and components for the build tools.

```powershell
choco install -y -f visualstudio2022buildtools --version=117.11.4 --params "--add Microsoft.VisualStudio.Component.VC.Tools.x86.x64 --installChannelUri https://aka.ms/vs/17/release/390666095_1317821361/channel"
choco install -y ninja
```

--------------------------------

### Define Executable Source Files

Source: https://github.com/man-group/arcticdb/blob/master/cpp/third_party/msgpack-c/example/boost/CMakeLists.txt

Lists the C++ source files that will be compiled into executables. This includes general variant examples and optionally Asio-based examples if C++11 or later is supported and ZLIB is found.

```cmake
LIST (APPEND exec_PROGRAMS
    msgpack_variant_capitalize.cpp
    msgpack_variant_mapbased.cpp
)

IF (MSGPACK_CXX11 OR MSGPACK_CXX14 OR MSGPACK_CXX17 OR MSGPACK_CXX20)
    LIST (APPEND exec_PROGRAMS asio_send_recv.cpp)
    IF (ZLIB_FOUND)
        LIST (APPEND exec_PROGRAMS asio_send_recv_zlib.cpp)
    ENDIF ()
ENDIF ()
```

--------------------------------

### Activate Virtual Environment

Source: https://github.com/man-group/arcticdb/blob/master/CLAUDE.md

Prints the path to activate the virtual environment. Use `source $(make activate NAME=x)` to activate.

```bash
make activate NAME=x
```

--------------------------------

### Inspect row-slice boundaries

Source: https://github.com/man-group/arcticdb/blob/master/docs/mkdocs/docs/notebooks/ArcticDB_demo_compact_data.ipynb

Retrieves the start and end segments of row-slices before compaction.

```python
compact_data_info.row_slices_before[:5]
```

```python
compact_data_info.row_slices_before[-5:]
```

--------------------------------

### Generate Sample Data for Snapshots

Source: https://github.com/man-group/arcticdb/blob/master/docs/mkdocs/docs/tutorials/snapshots.md

This script generates sample pricing and factor CSV files to be used with the snapshotting example. It creates a specified number of files and symbols, with factor files generated periodically.

```python
import argparse
import numpy as np
import pandas as pd
from datetime import datetime, timedelta

def run(num_files, num_symbols):
    starting_date = datetime.today() - timedelta(weeks=num_files)
    starting_date = datetime(starting_date.year, starting_date.month, starting_date.day)

    for file_num in range(num_files):
        index_size = 7 * 24
        this_file_starting_date = starting_date + timedelta(weeks=file_num)

        df = pd.DataFrame(np.random.randint(0,index_size,size=(index_size, num_symbols)), columns=['SYM_%d' % i for i in range(num_symbols)])

        df.index = pd.date_range(this_file_starting_date, periods=index_size, freq="H")

        df.to_csv(f"PRICING_{this_file_starting_date}.csv")
        if file_num % 3 == 0:
            df = pd.DataFrame(np.random.randint(0, 5,size=(5, num_symbols)), columns=['SYM_%d' % i for i in range(num_symbols)])
            df['FACTORS'] = ['FACTOR_1', 'FACTOR_2', 'FACTOR_3', 'FACTOR_4', 'FACTOR_5']
            df.to_csv(f"FACTORS_{this_file_starting_date}.csv")

        print(f"Written {file_num + 1} / {num_files}")


if __name__ == "__main__":
    parser = argparse.ArgumentParser()

    parser.add_argument('--num-files', type=int, default=15)
    parser.add_argument('--symbols-per-file', type=int, default=500)

    args = parser.parse_args()
    
    run(args.num_files, args.symbols_per_file)
```

--------------------------------

### Initialize Git Submodules

Source: https://github.com/man-group/arcticdb/blob/master/CLAUDE.md

Initializes git submodules, which is a requirement for vcpkg.

```bash
git submodule update --init --recursive
```

--------------------------------

### ArcticDB Exception Hierarchy

Source: https://github.com/man-group/arcticdb/blob/master/docs/mkdocs/docs/error_messages.md

Illustrates the inheritance structure of ArcticDB exceptions, starting from RuntimeError.

```text
RuntimeError
└-- ArcticException
    |-- ArcticDbNotYetImplemented
    |-- MissingDataException
    |-- NoDataFoundException
    |-- NoSuchVersionException
    |-- NormalizationException
    |-- SchemaException
    |-- SortingException
    |   └-- UnsortedDataException
    |-- StorageException
    |   └-- LmdbMapFullError
    |   └-- PermissionException
    |   └-- DuplicateKeyException
    |-- StreamDescriptorMismatch
    └-- InternalException
```

--------------------------------

### Setup LMDB Dependency

Source: https://github.com/man-group/arcticdb/wiki/Dev:-Building

Run this PowerShell script once to set up the LMDB dependency for the project.

```bat
cd <arctic_repo_location>/ArcticDB/cpp/third_party/lmdbcxx/ & ./symlinks_srcs.ps1
```

--------------------------------

### Build MkDocs Site

Source: https://github.com/man-group/arcticdb/blob/master/docs/README.md

Builds the static MkDocs website into the 'docs/mkdocs/site' directory.

```bash
cd docs/mkdocs
mkdocs build
```

--------------------------------

### Index Handling in ArcticDB

Source: https://github.com/man-group/arcticdb/blob/master/docs/claude/python/NORMALIZATION.md

Examples demonstrating how different pandas index types are normalized and stored within ArcticDB.

```python
# RangeIndex (default)
df = pd.DataFrame({"a": [1, 2, 3]})
# Normalized to row numbers

# DatetimeIndex
df = pd.DataFrame(
    {"a": [1, 2, 3]},
    index=pd.date_range("2024-01-01", periods=3)
)
# Normalized to nanosecond timestamps

# Named index
df = pd.DataFrame({"a": [1, 2, 3]})
df.index.name = "my_index"
# Index stored with name preserved

# MultiIndex
df = pd.DataFrame(
    {"a": [1, 2, 3, 4]},
    index=pd.MultiIndex.from_tuples([
        ("A", 1), ("A", 2), ("B", 1), ("B", 2)
    ])
)
# Each level stored as separate column
```

--------------------------------

### Initialize Project Dependencies

Source: https://github.com/man-group/arcticdb/blob/master/cpp/CMakeLists.txt

Sets library paths and includes Python utilities before adding subdirectories.

```cmake
set(FIND_LIBRARY_USE_LIB64_PATHS ON)
include(PythonUtils) # Must be called before Pybind (third_party) to override its finding mechanism

add_subdirectory(third_party)
add_subdirectory(proto)


python_utils_dump_vars_if_enabled("After Pybind")
python_utils_check_include_dirs("accepted by pybind")
python_utils_check_version_is_as_expected()
```

--------------------------------

### Initialize LibraryTool for ArcticDB

Source: https://github.com/man-group/arcticdb/wiki/Using-the-LibraryTool-to-look-at-a-library's-internal-state

Instantiate the LibraryTool by passing the ArcticDB library object to it. Ensure ArcticDB and the library are properly initialized.

```python
from arcticdb.toolbox.library_tool import KeyType

ac = Arctic(...)
lib = ac[...]

lib_tool = lib._nvs.library_tool()
```

--------------------------------

### Read Data from ArcticDB

Source: https://github.com/man-group/arcticdb/blob/master/docs/mkdocs/docs/notebooks/ArcticDB_demo_equity_options.ipynb

Reads data from an ArcticDB table. Specify start and end dates for time-series slicing.

```python
data = table.read(start_date=start, end_date=end, columns=columns)

```

--------------------------------

### Build ArcticDB with CMake

Source: https://github.com/man-group/arcticdb/blob/master/docs/mkdocs/docs/technical/contributing.md

Sets up the Python executable path, installs build dependencies, and initiates the ArcticDB build process using CMake presets. Adjust MY_PYTHON if not running in the Docker container.

```bash
MY_PYTHON=/opt/python/cp39-cp39/bin/python3  # change if outside docker container/building against a different python
$MY_PYTHON -m pip install -U pip setuptools wheel grpcio-tools
ARCTIC_CMAKE_PRESET=skip $MY_PYTHON setup.py develop
# Change the below Python_EXECUTABLE value to build against a different Python version
cmake -DPython_EXECUTABLE=$MY_PYTHON -DTEST=off --preset linux-debug cpp
pushd cpp
cmake --build --preset linux-debug
popd
```

--------------------------------

### Incomplete Write Lifecycle

Source: https://github.com/man-group/arcticdb/blob/master/docs/claude/cpp/STREAM.md

Outlines the lifecycle of an incomplete append, from starting and adding data to finalization and crash recovery.

```text
1. Start append
   ── Create APPEND_REF pointing to first incomplete segment

2. Add more data
   ── Chain additional segments to APPEND_REF

3. Finalize append
   ── Create new VERSION with all data
   ── Delete APPEND_REF

4. On crash recovery
   ── APPEND_REF points to recoverable data
   ── Can discard or complete the append
```

--------------------------------

### Initialize ArcticDB Library

Source: https://github.com/man-group/arcticdb/blob/master/docs/mkdocs/docs/notebooks/ArcticDB_demo_equity_options.ipynb

Initializes the ArcticDB library. This is a common first step before interacting with the database.

```python
import arctic
from arctic import Arctic
from arctic.auth import ArcticAuth

# Connect to ArcticDB (replace with your connection details)
Arctic.initialize_uri("arctic://user:password@host:port/database")

# Or use authentication object
# auth = ArcticAuth(username='user', password='password')
# Arctic.initialize_uri("arctic://host:port/database", auth=auth)

# Access the ArcticDB library
lib = Arctic('my_library')

```

--------------------------------

### Get or Create Library

Source: https://github.com/man-group/arcticdb/blob/master/docs/mkdocs/docs/notebooks/ArcticDB_demo_concat.ipynb

Retrieves a library named 'concat' from the ArcticDB object store. If the library does not exist, it will be created.

```python
# library
lib = arctic.get_library('concat', create_if_missing=True)
```