### Initial Server Setup Script - Bash

Source: https://github.com/schlagonia/auctions-data-collection/blob/master/DEPLOYMENT.md

Executes a setup script on the server to prepare the environment for the Dutch Auction Analytics system. This script handles package updates, dependency installation (Python, Nginx), user creation, firewall configuration, and directory setup.

```bash
ssh root@YOUR_DROPLET_IP
cd /opt/auctions-data-collection
chmod +x deployment/scripts/setup.sh
./deployment/scripts/setup.sh
```

--------------------------------

### Run Deployment Script - Bash

Source: https://github.com/schlagonia/auctions-data-collection/blob/master/DEPLOYMENT.md

Executes the main deployment script to finalize the setup of the Dutch Auction Analytics system. This script manages the Python virtual environment, installs dependencies, configures systemd services and Nginx, initializes the database, and starts all necessary services.

```bash
su - auction
cd /opt/auctions-data-collection
chmod +x deployment/scripts/deploy.sh
./deployment/scripts/deploy.sh
```

--------------------------------

### Install Python Dependencies and Setup Directories

Source: https://github.com/schlagonia/auctions-data-collection/blob/master/README.md

Commands to clone the project, install Python dependencies using pip, and create necessary directories for logs and data. Assumes a bash environment.

```bash
# Clone or download the project
cd auctions-data-collection

# Install dependencies
pip install -r requirements.txt

# Create required directories
mkdir -p logs data
```

--------------------------------

### Start Web App (Python)

Source: https://github.com/schlagonia/auctions-data-collection/blob/master/web/README.md

Provides instructions on how to start the Dutch Auction Analytics Web App. It offers two options: starting the API and web app together, or starting them separately.

```python
# Option 1: Start both API and web app together
python start_web.py

# Option 2: Start components separately
# Terminal 1 - API Server
python src/main.py --api

# Terminal 2 - Web App  
cd web/
python app.py
```

--------------------------------

### API Usage

Source: https://github.com/schlagonia/auctions-data-collection/blob/master/README.md

This section outlines how to start the API server and provides examples for querying various endpoints.

```APIDOC
## API Usage

### Start API Server

To start the API server, run the following command:

```bash
python src/main.py --api
```

### Query Endpoints

Once the server is running, you can query the following endpoints:

- **Get Statistics**: `curl http://localhost:8000/statistics`
- **Get Factories**: `curl http://localhost:8000/factories`
- **Get Auctions by Factory**: `curl "http://localhost:8000/auctions?factory=0x..."`
- **Get Events by Auction**: `curl "http://localhost:8000/events?auction=0x..."`
- **Get Recent Data**: `curl "http://localhost:8000/recent?limit=50"`
```

--------------------------------

### Install Dependencies (Bash)

Source: https://github.com/schlagonia/auctions-data-collection/blob/master/web/README.md

Installs the necessary Python dependencies for the web application. Assumes you are in the 'web/' directory and have Python 3.8+ installed.

```bash
cd web/
pip install -r requirements.txt
```

--------------------------------

### Install Fail2ban for Intrusion Prevention

Source: https://github.com/schlagonia/auctions-data-collection/blob/master/DEPLOYMENT.md

Bash command to install Fail2ban, a service that scans log files and bans IPs showing malicious signs. Helps protect against brute-force attacks.

```bash
sudo apt install fail2ban
```

--------------------------------

### Start API Server and Export Data using Python

Source: https://context7.com/schlagonia/auctions-data-collection/llms.txt

Scripts to start the API server and export collected auction data. The export functionality supports JSON and CSV formats. Requires Python and associated libraries.

```bash
python src/main.py --api

python src/main.py --export output.json --export-format json
python src/main.py --export output.csv --export-format csv
```

--------------------------------

### Start API Server (Python)

Source: https://context7.com/schlagonia/auctions-data-collection/llms.txt

Initializes and starts a local API server for querying auction data. It depends on the DataStorage module and Web3 providers for blockchain interaction. The server runs in a background thread and can be stopped by calling server.stop().

```python
from modules import start_api_server, DataStorage

# Initialize storage
storage = DataStorage('data/auctions.db')

# Create Web3 providers for token resolution
from web3 import Web3
web3_providers = {
    1: Web3(Web3.HTTPProvider('https://eth.llamarpc.com')),
    860084: Web3(Web3.HTTPProvider('https://berachain-rpc.publicnode.com'))
}

# Start server
server = start_api_server(
    storage=storage,
    port=8000,
    web3_providers=web3_providers
)

print(f"API server running on http://localhost:8000")
# Server runs in background thread
# Call server.stop() to shutdown
```

--------------------------------

### Install Certbot and Get SSL Certificate - Bash

Source: https://github.com/schlagonia/auctions-data-collection/blob/master/DEPLOYMENT.md

Installs Certbot and its Nginx plugin to obtain and manage SSL certificates for the deployed application, enabling HTTPS. This command ensures secure communication with your analytics system.

```bash
sudo apt-get install certbot python3-certbot-nginx

sudo certbot --nginx -d your-domain.com
# Follow the prompts to configure SSL
```

--------------------------------

### Configuration File Structure for Auction Data Collection

Source: https://github.com/schlagonia/auctions-data-collection/blob/master/README.md

Example JSON structure for the `config.json` file, detailing settings for blockchain connection, factory contracts, and database path. Includes example data for chain settings and factory details.

```json
{
  "chain": {
    "name": "ethereum",
    "rpc_url": "https://eth.llamarpc.com",
    "chain_id": 1,
    "max_blocks_per_request": 1000,
    "request_delay": 0.1
  },
  "factories": [
    {
      "address": "0xYourFactoryAddress",
      "deployment_block": 18000000,
      "name": "Your Factory Name"
    }
  ],
  "database_path": "data/auctions.db",
  "api_port": 8000
}

/*{
{
      "address": "0xE6aB098E8582178A76DC80d55ca304d1Dec11AD8",
      "deployment_block": 19486096,
      "name": "Original",
      "chains": [1],
      "chain_deployments": {
        "1": 19486096
      },
      "legacy": true
    }
*/
```

--------------------------------

### Update and Upgrade System Packages

Source: https://github.com/schlagonia/auctions-data-collection/blob/master/DEPLOYMENT.md

Bash commands to update the package list and upgrade installed packages on a Debian-based system. Essential for security and stability.

```bash
sudo apt update && sudo apt upgrade
```

--------------------------------

### Optimize SQLite Database

Source: https://github.com/schlagonia/auctions-data-collection/blob/master/DEPLOYMENT.md

SQLite commands to optimize the database file. 'VACUUM' rebuilds the database, and 'ANALYZE' updates statistics for query optimization.

```bash
sqlite3 /opt/auctions-data-collection/data/auctions.db "VACUUM;"
```

```bash
sqlite3 /opt/auctions-data-collection/data/auctions.db "ANALYZE;"
```

--------------------------------

### Start Web Dashboard and API Server

Source: https://context7.com/schlagonia/auctions-data-collection/llms.txt

Scripts to launch the web dashboard and API server. The dashboard is accessible at http://localhost:5000 and the API at http://localhost:8000. An option to use a remote production API is available.

```bash
# Start both API server and web app together
python start_web.py

# Access dashboard at http://localhost:5000
# API available at http://localhost:8000

# Use remote production API instead of local
python start_web.py --use-remote-api
```

--------------------------------

### Enable Unattended Upgrades

Source: https://github.com/schlagonia/auctions-data-collection/blob/master/DEPLOYMENT.md

Command to configure unattended upgrades, allowing the system to automatically install security updates. Ensures the system remains protected.

```bash
sudo dpkg-reconfigure --priority=low unattended-upgrades
```

--------------------------------

### GET /

Source: https://github.com/schlagonia/auctions-data-collection/blob/master/README.md

Provides API documentation and links to available endpoints.

```APIDOC
## GET /

### Description
Provides API documentation and links to available endpoints.

### Method
GET

### Endpoint
/

### Parameters
None

### Request Example
None

### Response
#### Success Response (200)
- **message** (string) - A welcome message and links to other endpoints.

#### Response Example
```json
{
  "message": "Welcome to the Auctions Data Collection API!",
  "documentation": "GET /",
  "statistics": "GET /statistics",
  "factories": "GET /factories",
  "auctions": "GET /auctions?factory=<address>",
  "events": "GET /events?auction=<address>",
  "recent": "GET /recent?limit=100"
}
```
```

--------------------------------

### Check System Resource Usage

Source: https://github.com/schlagonia/auctions-data-collection/blob/master/DEPLOYMENT.md

Bash commands to check system resource usage. 'htop' provides an interactive process viewer, 'df -h' shows disk usage, and 'free -h' displays memory usage.

```bash
htop  # Interactive process viewer
```

```bash
df -h  # Disk usage
```

```bash
free -h  # Memory usage
```

--------------------------------

### Run API Server and Query Endpoints

Source: https://github.com/schlagonia/auctions-data-collection/blob/master/README.md

Commands to start the API server and query various endpoints for auction statistics and event data. Uses `curl` for HTTP requests. The API server must be running before querying.

```bash
# Start API server
python src/main.py --api

# Query endpoints
curl http://localhost:8000/statistics
curl http://localhost:8000/factories
curl "http://localhost:8000/auctions?factory=0x..."
curl "http://localhost:8000/events?auction=0x..."
curl "http://localhost:8000/recent?limit=50"
```

--------------------------------

### Restore Database from Backup

Source: https://github.com/schlagonia/auctions-data-collection/blob/master/DEPLOYMENT.md

Bash commands to restore the SQLite database from a gzipped backup file. Involves copying, unzipping, and moving the backup to the data directory.

```bash
cp /opt/auctions-data-collection/backups/latest_backup.gz /tmp/
gunzip /tmp/latest_backup.gz
mv /tmp/latest_backup /opt/auctions-data-collection/data/auctions.db
```

--------------------------------

### Monitor Data Synchronization Service

Source: https://github.com/schlagonia/auctions-data-collection/blob/master/DEPLOYMENT.md

Bash command to follow the logs of the auction data synchronization service ('auction-sync'). Useful for real-time monitoring of the syncing process.

```bash
sudo journalctl -u auction-sync -f
```

--------------------------------

### API Response Example (JSON)

Source: https://context7.com/schlagonia/auctions-data-collection/llms.txt

An example of the JSON response structure returned by the API endpoints, likely for statistics or summary data. It includes metrics related to auctions, events, and success rates.

```json
{
  "total_auctions": 1543,
  "kick_events": 8732,
  "take_events": 6891,
  "auctions_with_kicks": 1234,
  "auctions_with_takes": 987,
  "success_rate": 0.7891,
  "cowswap_takes": 3421,
  "cowswap_percentage": 49.6,
  "avg_time_to_take_seconds": 342,
  "median_time_to_take_seconds": 187
}
```

--------------------------------

### Clone Code using Git - Bash

Source: https://github.com/schlagonia/auctions-data-collection/blob/master/DEPLOYMENT.md

Clones the project repository directly onto the Digital Ocean droplet using git. This method is suitable if you prefer managing code updates through git pull operations.

```bash
ssh root@YOUR_DROPLET_IP
cd /opt/auctions-data-collection
git clone https://github.com/YOUR_USERNAME/auctions-data-collection.git .
```

--------------------------------

### Upload Code using rsync - Bash

Source: https://github.com/schlagonia/auctions-data-collection/blob/master/DEPLOYMENT.md

Uploads the project code to the Digital Ocean droplet using rsync, excluding specific directories and files like virtual environments, local databases, and gitignore. This is a recommended method for efficient file transfer.

```bash
rsync -avz --exclude 'venv' --exclude 'data/*.db' --exclude '.env' --exclude .git/ \
  /path/to/auctions-data-collection/ \
  root@YOUR_DROPLET_IP:/opt/auctions-data-collection/
```

--------------------------------

### Check Service Logs

Source: https://github.com/schlagonia/auctions-data-collection/blob/master/DEPLOYMENT.md

Bash commands to view recent logs for the auction API and web services using journalctl. Useful for troubleshooting startup issues.

```bash
sudo journalctl -u auction-api -n 50
```

```bash
sudo journalctl -u auction-web -n 50
```

--------------------------------

### Query SQLite Database

Source: https://github.com/schlagonia/auctions-data-collection/blob/master/DEPLOYMENT.md

Command to access the SQLite database for the auctions data collection project. Used for direct data inspection or manipulation.

```bash
sqlite3 /opt/auctions-data-collection/data/auctions.db
```

--------------------------------

### View Application Logs - Bash

Source: https://github.com/schlagonia/auctions-data-collection/blob/master/DEPLOYMENT.md

Displays logs generated by the auction API, web application, and synchronization process. This command is used for detailed log analysis and troubleshooting specific application events.

```bash
# Service logs
sudo journalctl -u auction-api --since "1 hour ago"
sudo journalctl -u auction-web --since "1 hour ago"

# Application logs
tail -f /var/log/auction-api.log
tail -f /var/log/auction-web.log
tail -f /var/log/auction-sync.log
```

--------------------------------

### Check Database Size - Bash

Source: https://github.com/schlagonia/auctions-data-collection/blob/master/DEPLOYMENT.md

Displays the disk space usage of the auction database file. This command helps in monitoring the database size and planning for storage needs.

```bash
du -h /opt/auctions-data-collection/data/auctions.db
```

--------------------------------

### Load Top Auctions and Tokens - JavaScript

Source: https://github.com/schlagonia/auctions-data-collection/blob/master/web/templates/volumes.html

Fetches and prepares data for top auctions and top tokens. `loadTopAuctions` fetches volume statistics to get top auctions and sets up filters. `loadTopTokens` fetches a list of top tokens and prepares them for filtering.

```javascript
async function loadTopAuctions() {
  try {
    const volumeStats = await fetchData('/volume-stats');
    allAuctionData = volumeStats.top_auctions_by_volume || [];
    applyAuctionFilters();
    setupAuctionFilterListeners();
  } catch (error) {
    console.error('Error loading top auctions:', error);
  }
}

async function loadTopTokens() {
  try {
    const topTokensData = await fetchData('/top-tokens?limit=50');
    allTokenData = topTokensData;
    applyTokenFilters();
    setupTokenFilterListeners();
  } catch (error) {
    console.error('Error loading top tokens:', error);
  }
}
```

--------------------------------

### Restart Services - Bash

Source: https://github.com/schlagonia/auctions-data-collection/blob/master/DEPLOYMENT.md

Restarts the auction API and web services. This command is typically used after configuration changes or to resolve temporary service disruptions.

```bash
sudo systemctl restart auction-api
sudo systemctl restart auction-web
```

--------------------------------

### Setup Token Filter Event Listeners

Source: https://github.com/schlagonia/auctions-data-collection/blob/master/web/templates/volumes.html

Sets up event listeners for token filtering controls (chain, min volume, symbol) to call `applyTokenFilters` upon interaction. It also handles a clear filters button. A check for a data attribute prevents redundant listener attachments.

```javascript
function setupTokenFilterListeners() {
  if (document.getElementById('token-chain-filter').hasAttribute('data-listeners-added')) return;

  document.getElementById('token-chain-filter').addEventListener('change', applyTokenFilters);
  document.getElementById('token-min-volume-filter').addEventListener('input', applyTokenFilters);
  document.getElementById('token-symbol-filter').addEventListener('input', applyTokenFilters);

  document.getElementById('clear-token-volume-filters').addEventListener('click', () => {
    document.getElementById('token-chain-filter').value = 'all';
    document.getElementById('token-min-volume-filter').value = '';
    document.getElementById('token-symbol-filter').value = '';
    applyTokenFilters();
  });

  document.getElementById('token-chain-filter').setAttribute('data-listeners-added', 'true');
}
```

--------------------------------

### Manual Database Backup - Bash

Source: https://github.com/schlagonia/auctions-data-collection/blob/master/DEPLOYMENT.md

Manually triggers the database backup script. This command can be used for on-demand backups outside of the scheduled cron job.

```bash
/opt/auctions-data-collection/deployment/scripts/backup.sh
```

--------------------------------

### Environment Configuration - .env

Source: https://github.com/schlagonia/auctions-data-collection/blob/master/DEPLOYMENT.md

Defines environment-specific variables for the Dutch Auction Analytics application. This includes settings for application environment, logging level, API and web ports, database path, RPC endpoint URLs for different blockchains, and a secret key.

```env
APP_ENV=production
LOG_LEVEL=INFO
API_PORT=8000
WEB_PORT=5000

DATABASE_PATH=data/auctions.db

# Replace with your actual RPC endpoints
ETHEREUM_RPC_URL=https://mainnet.gateway.tenderly.co/YOUR_KEY
BASE_RPC_URL=https://base.gateway.tenderly.co/YOUR_KEY
BERA_RPC_URL=https://berachain.gateway.tenderly.co/YOUR_KEY

SECRET_KEY=your-random-secret-key-here
```

--------------------------------

### Update Code and Redeploy - Bash

Source: https://github.com/schlagonia/auctions-data-collection/blob/master/DEPLOYMENT.md

Updates the project's codebase from the main branch of the git repository and then re-runs the deployment script. This process ensures that the latest version of the application is running on the server.

```bash
cd /opt/auctions-data-collection
git pull origin main
./deployment/scripts/deploy.sh
```

--------------------------------

### View Service Logs - Bash

Source: https://github.com/schlagonia/auctions-data-collection/blob/master/DEPLOYMENT.md

Retrieves real-time logs for the Dutch Auction Analytics services (API, web, sync) using `journalctl`. This command is useful for debugging and monitoring the application's behavior and performance.

```bash
sudo journalctl -u auction-api -f
sudo journalctl -u auction-web -f
sudo journalctl -u auction-sync -f
```

--------------------------------

### Schedule Database Backups - Cron

Source: https://github.com/schlagonia/auctions-data-collection/blob/master/DEPLOYMENT.md

Sets up a cron job to automatically execute a database backup script daily at 2 AM. This ensures regular backups of the auction data for disaster recovery purposes.

```cron
crontab -e
# Add this line:
0 2 * * * /opt/auctions-data-collection/deployment/scripts/backup.sh
```

--------------------------------

### Manual Data Sync - Python

Source: https://github.com/schlagonia/auctions-data-collection/blob/master/DEPLOYMENT.md

Performs manual synchronization of auction data using Python scripts. This includes options for a full data synchronization or an incremental update, executed within the project's virtual environment.

```python
cd /opt/auctions-data-collection
source venv/bin/activate
python src/main.py --full-sync  # Full sync
python src/main.py               # Incremental sync
```

--------------------------------

### Query API Endpoints (Bash)

Source: https://context7.com/schlagonia/auctions-data-collection/llms.txt

Provides a set of curl commands to interact with the local API server. These commands allow users to retrieve API documentation, check server health, get various statistics, and query specific auction and event data.

```bash
# Get API documentation
curl http://localhost:8000/

# Health check
curl http://localhost:8000/health

# Overall statistics
curl http://localhost:8000/statistics

# Cross-chain statistics
curl http://localhost:8000/cross-chain-stats

# Factory statistics
curl http://localhost:8000/factories

# List auctions for a factory
curl "http://localhost:8000/auctions?factory=0xCfA510188884F199fcC6e750764FAAbE6e56ec40&chain_id=1&limit=50"

# Get auction details
curl "http://localhost:8000/auction/0x1234...?chain=1"

# Get auction events
curl "http://localhost:8000/events?auction=0x1234...&chain=1"

# Recent events
curl "http://localhost:8000/recent?limit=100"

# Volume statistics
curl "http://localhost:8000/volume-stats?chain_id=1"

# Volume by chain
curl http://localhost:8000/volume-by-chain

# Timeline data for charts
curl "http://localhost:8000/timeline-stats?days=30"

# Top tokens by volume
curl "http://localhost:8000/top-tokens?limit=20&chain_id=1"

# CowSwap timing analysis
curl "http://localhost:8000/cowswap-analysis?chain_id=1&limit=100"
```

--------------------------------

### Create and Execute Service Health Check Script

Source: https://github.com/schlagonia/auctions-data-collection/blob/master/DEPLOYMENT.md

Bash script to check the health of the API and web application by querying their health endpoints. Includes instructions to make it executable and add it to cron for regular monitoring.

```bash
# Create monitoring script
cat > /opt/auctions-data-collection/check_health.sh << 'EOF'
#!/bin/bash
API_STATUS=$(curl -s -o /dev/null -w "%{http_code}" http://localhost:8000/health)
WEB_STATUS=$(curl -s -o /dev/null -w "%{http_code}" http://localhost:5000/health)

if [ $API_STATUS -ne 200 ]; then
    echo "API is down! Status: $API_STATUS"
    # Send alert (email, Slack, etc.)
fi

if [ $WEB_STATUS -ne 200 ]; then
    echo "Web app is down! Status: $WEB_STATUS"
    # Send alert
fi
EOF

chmod +x /opt/auctions-data-collection/check_health.sh

# Add to crontab for monitoring every 5 minutes
crontab -e
# Add: */5 * * * * /opt/auctions-data-collection/check_health.sh
```

--------------------------------

### JavaScript: Page Initialization and Event Handling

Source: https://github.com/schlagonia/auctions-data-collection/blob/master/web/templates/volumes.html

The `initializePage` function orchestrates the loading of various data components including volume statistics, charts, top auctions, top tokens, and chain volume stats. It uses `Promise.all` for concurrent loading and sets up event listeners. Once data is loaded, it hides the loading indicator and displays the main content.

```javascript
// Initialize page
async function initializePage() {
  try {
    await Promise.all([
      loadVolumeStatistics(),
      loadVolumeCharts(),
      loadTopAuctions(),
      loadTopTokens(),
      loadChainVolumeStats()
    ]);
    setupEventListeners();
    document.getElementById('loading').style.display = 'none';
    document.getElementById('volumes-content').style.display = 'block';
  } catch (error) {
    console.error('Error initializing page:', error);
    // Handle error, e.g., display an error message to the user
  }
}
```

--------------------------------

### Test RPC Connectivity

Source: https://github.com/schlagonia/auctions-data-collection/blob/master/DEPLOYMENT.md

cURL command to test connectivity to an RPC endpoint, specifically for Ethereum-like blockchains. Sends a JSON-RPC request to get the latest block number.

```bash
curl -X POST -H "Content-Type: application/json" \
  --data '{"jsonrpc":"2.0","method":"eth_blockNumber","params":[],"id":1}' \
  YOUR_RPC_URL
```

--------------------------------

### Run Full Data Collection (Bash)

Source: https://context7.com/schlagonia/auctions-data-collection/llms.txt

Command-line interface commands for running the data collection process. Options include initial full synchronization, incremental sync, syncing specific chains, and selective discovery or event collection.

```bash
# Initial full sync from deployment blocks
python src/main.py --full-sync

# Incremental sync (resumes from last checkpoint)
python src/main.py

# Sync specific chain only
python src/main.py --chain-id 1 --full-sync

# Discovery only (no event collection)
python src/main.py --discover-only --chain-id 1

# Events only (assumes auctions already discovered)
python src/main.py --events-only --chain-id 860084

# Calculate volumes with rate limiting
python src/main.py --calculate-volumes --chain-id 1 --batch-size 5 --delay 2.0
```

--------------------------------

### GET /recent

Source: https://github.com/schlagonia/auctions-data-collection/blob/master/README.md

Retrieves a list of the most recent auction events.

```APIDOC
## GET /recent

### Description

Retrieves a list of the most recent auction-related events collected by the system. You can specify a limit for the number of results.

### Method

GET

### Endpoint

`/recent`

### Parameters

#### Query Parameters

- **limit** (integer) - Optional - The maximum number of recent events to return. Defaults to 50 if not specified.

### Request Example

```bash
curl "http://localhost:8000/recent?limit=10"
```

### Response

#### Success Response (200)

- **recent_events** (array of objects) - A list of the most recent event objects.
  - **event_type** (string) - The type of event.
  - **timestamp** (string) - The time the event occurred (ISO 8601 format).
  - **data** (object) - Specific data associated with the event.

#### Response Example

```json
{
  "recent_events": [
    {
      "event_type": "NewAuction",
      "timestamp": "2023-10-27T11:00:00Z",
      "data": {
        "auction_address": "0xnew...auction"
      }
    },
    {
      "event_type": "Bid",
      "timestamp": "2023-10-27T11:01:00Z",
      "data": {
        "bidder": "0xanother...bidder",
        "amount": "0.5 ETH"
      }
    }
  ]
}
```
```

--------------------------------

### GET /statistics

Source: https://github.com/schlagonia/auctions-data-collection/blob/master/README.md

Retrieves overall statistics for all collected auctions.

```APIDOC
## GET /statistics

### Description
Retrieves overall statistics for all collected auctions, such as total auctions, total events, and unique factories.

### Method
GET

### Endpoint
/statistics

### Parameters
None

### Request Example
None

### Response
#### Success Response (200)
- **total_auctions** (integer) - The total number of auctions collected.
- **total_events** (integer) - The total number of kick/take events collected.
- **unique_factories** (integer) - The number of unique factory contracts found.

#### Response Example
```json
{
  "total_auctions": 1500,
  "total_events": 30000,
  "unique_factories": 5
}
```
```

--------------------------------

### Load Multi-Chain Configuration in Python

Source: https://context7.com/schlagonia/auctions-data-collection/llms.txt

Loads the application configuration from a JSON file using the Config class from the 'config' module. It demonstrates accessing chain-specific settings like RPC URLs and maximum blocks per request, and retrieving factory details for a given chain. Dependencies include the 'config' module.

```python
from config import Config

# Load configuration from JSON file
config = Config('config.json')

# Access chain configurations
for chain_id, chain_config in config.chains.items():
    print(f"Chain: {chain_config.name}")
    print(f"  RPC: {chain_config.rpc_url}")
    print(f"  Chain ID: {chain_config.chain_id}")
    print(f"  Max blocks per request: {chain_config.max_blocks_per_request}")

# Get specific chain by ID
ethereum = config.get_chain_by_id(1)
berachain = config.get_chain_by_id(860084)

# List factories for a specific chain
factories = config.get_factories_for_chain(1)
for factory in factories:
    print(f"Factory: {factory.name} at {factory.address}")
    deployment = factory.get_deployment_block(1)
    print(f"  Deployed at block: {deployment}")
```

--------------------------------

### GET /events

Source: https://github.com/schlagonia/auctions-data-collection/blob/master/README.md

Retrieves a list of events associated with a specific auction.

```APIDOC
## GET /events

### Description

Retrieves a list of specific events that occurred during a particular auction. You must provide the auction's address.

### Method

GET

### Endpoint

`/events`

### Parameters

#### Query Parameters

- **auction** (string) - Required - The contract address of the auction.

### Request Example

```bash
curl "http://localhost:8000/events?auction=0xabcd...1234"
```

### Response

#### Success Response (200)

- **events** (array of objects) - A list of event objects, each detailing an occurrence during the auction.
  - **event_type** (string) - The type of event (e.g., 'Bid', 'Sale').
  - **timestamp** (string) - The time the event occurred (ISO 8601 format).
  - **data** (object) - Specific data associated with the event.

#### Response Example

```json
{
  "events": [
    {
      "event_type": "Bid",
      "timestamp": "2023-10-27T10:00:00Z",
      "data": {
        "bidder": "0xbidder...",
        "amount": "1.5 ETH"
      }
    },
    {
      "event_type": "Sale",
      "timestamp": "2023-10-27T10:05:00Z",
      "data": {
        "buyer": "0xbuyer...",
        "final_price": "2.0 ETH"
      }
    }
  ]
}
```
```

--------------------------------

### GET /factories

Source: https://github.com/schlagonia/auctions-data-collection/blob/master/README.md

Retrieves a list of all unique auction factory addresses.

```APIDOC
## GET /factories

### Description

Retrieves a list of all unique auction factory addresses that have been registered and processed by the collection system.

### Method

GET

### Endpoint

`/factories`

### Parameters

This endpoint does not accept any parameters.

### Request Example

```bash
curl http://localhost:8000/factories
```

### Response

#### Success Response (200)

- **factories** (array of strings) - A list of hexadecimal strings representing auction factory addresses.

#### Response Example

```json
{
  "factories": [
    "0x123...abc",
    "0xdef...456",
    "0x789...ghi"
  ]
}
```
```

--------------------------------

### JavaScript API Data Fetching and Page Initialization

Source: https://github.com/schlagonia/auctions-data-collection/blob/master/web/templates/auctions.html

Contains functions for fetching data from a backend API and initializing the web page. The `fetchData` function handles GET requests with error handling, while `initializePage` orchestrates the loading of initial data (factories, auctions, statistics), populates filter dropdowns, sets up event listeners, and manages the display of loading and content states. It assumes a global `apiBaseUrl` variable.

```javascript
// Global state - use /api/ proxy when in production
{% if api_url %} // Use API URL provided from server let apiBaseUrl = '{{ api_url }}'; {% else %} // Default behavior let apiBaseUrl = window.location.hostname === 'localhost' ? 'http://localhost:8000' : '/api'; {% endif %}
let currentPage = 1;
let pageSize = 50;
let allFilteredAuctions = []; // Store filtered auctions for pagination
let totalResults = 0;
let allFactories = [];
let currentAuctions = [];
let sortBy = null;
let sortDirection = 'desc';

// API functions
async function fetchData(endpoint) {
  try {
    const response = await fetch(`${apiBaseUrl}${endpoint}`);
    if (!response.ok) {
      throw new Error(`HTTP error! status: ${response.status}`);
    }
    return await response.json();
  } catch (error) {
    console.error(`Error fetching ${endpoint}:`, error);
    throw error;
  }
}

// Initialize page
async function initializePage() {
  try {
    // Load factories for filter
    const factories = await fetchData('/factories');
    allFactories = factories;
    populateFactoryFilter(factories);
    populateChainFilter();

    // Load initial auctions and statistics
    await Promise.all([
      loadAuctions(),
      loadStatistics()
    ]);

    // Set up event listeners
    setupEventListeners();

    // Show content
    document.getElementById('loading').style.display = 'none';
    document.getElementById('auctions-content').style.display = 'block';
  } catch (error) {
    console.error('Error initializing page:', error);
    showError('Failed to load auctions data. Please ensure the API server is running.');
  }
}

function populateFactoryFilter(factories) {
  const select = document.getElementById('factory-filter');
  factories.forEach(factory => {
    const option = document.createElement('option');
    option.value = factory.address;
    option.textContent = factory.name || formatAddress(factory.address);
    select.appendChild(option);
  });
}

function populateChainFilter() {
  const select = document.getElementById('chain-filter');
  const chains = [
    {id: 1, name: 'Ethereum'},
    // {id: 137, name: 'Polygon'},
    // {id: 42161, name: 'Arbitrum'},
    // {id: 10, name: 'Optimism'},
    {id: 8453, name: 'Base'},
    // {id: 100, name: 'Gnosis'},
    {id: 860084, name: 'Berachain'},
    {id: 747474, name: 'Katana'}
  ];
  chains.forEach(chain => {
    const option = document.createElement('option');
    option.value = chain.id;
    option.textContent = chain.name;
    select.appendChild(option);
  });
}

function applyAuctionFilters(auctions) {
  const statusFilter = document.getElementById('status-filter').value;
  const minVolumeFilter = parseFloat(document.getElementById('min-volume-filter').value) || 0;
  const minKicksFilter = parseInt(document.getElementById('min-kicks-filter').value) || 0;

  return auctions.filter(auction => {
    // Status filter
    if (statusFilter) {
      const kickCount = auction.kick_count || 0;
      const take
```

--------------------------------

### GET /statistics

Source: https://github.com/schlagonia/auctions-data-collection/blob/master/README.md

Retrieves general statistics about the collected auction data.

```APIDOC
## GET /statistics

### Description

Retrieves general statistics about the collected auction data, such as the total number of factories, auctions, and events.

### Method

GET

### Endpoint

`/statistics`

### Parameters

This endpoint does not accept any parameters.

### Request Example

```bash
curl http://localhost:8000/statistics
```

### Response

#### Success Response (200)

- **total_factories** (integer) - The total number of unique auction factories found.
- **total_auctions** (integer) - The total number of auctions recorded.
- **total_events** (integer) - The total number of auction-related events processed.

#### Response Example

```json
{
  "total_factories": 150,
  "total_auctions": 1200,
  "total_events": 50000
}
```
```

--------------------------------

### GET /recent

Source: https://github.com/schlagonia/auctions-data-collection/blob/master/README.md

Retrieves a list of the most recent events across all auctions.

```APIDOC
## GET /recent

### Description
Retrieves a list of the most recent kick and take events across all auctions, ordered by timestamp.

### Method
GET

### Endpoint
/recent

### Parameters
#### Query Parameters
- **limit** (integer) - Optional - The maximum number of recent events to return (default is 100).

### Request Example
```bash
GET /recent?limit=50
```

### Response
#### Success Response (200)
- **events** (array) - An array of recent event objects, similar to the `/events` endpoint but includes auction and factory information.
  - **transaction_hash** (string) - The hash of the transaction containing the event.
  - **log_index** (integer) - The index of the log within the transaction.
  - **event_type** (string) - The type of event ('kick' or 'take').
  - **timestamp** (string) - The timestamp of the event (ISO 8601 format).
  - **block_number** (integer) - The block number in which the event occurred.
  - **auction_address** (string) - The address of the auction contract.
  - **factory_address** (string) - The address of the factory that deployed the auction.
  - **data** (object) - The specific data associated with the event.

#### Response Example
```json
{
  "events": [
    {
      "transaction_hash": "0xghi...",
      "log_index": 1,
      "event_type": "take",
      "timestamp": "2023-10-27T11:00:00Z",
      "block_number": 18500100,
      "auction_address": "0xAuctionAddress3",
      "factory_address": "0xYourFactoryAddress1",
      "data": {
        "taker": "0xAnotherTaker",
        "amount": "950000000000000000"
      }
    },
    {
      "transaction_hash": "0xjkl...",
      "log_index": 3,
      "event_type": "kick",
      "timestamp": "2023-10-27T10:58:00Z",
      "block_number": 18500098,
      "auction_address": "0xAuctionAddress4",
      "factory_address": "0xYourFactoryAddress2",
      "data": {
        "bidder": "0xAnotherBidder",
        "amount": "1050000000000000000"
      }
    }
  ]
}
```
```

--------------------------------

### Initialize Page Load with JavaScript

Source: https://github.com/schlagonia/auctions-data-collection/blob/master/web/templates/auction.html

Sets up the initial page load sequence. It waits for the DOM to be fully loaded and then calls `loadAuctionData` to fetch and display the initial set of auction events. This ensures that all necessary HTML elements are available before interacting with them.

```javascript
document.addEventListener('DOMContentLoaded', function() {
  loadAuctionData();
});
```

--------------------------------

### GET /auctions

Source: https://github.com/schlagonia/auctions-data-collection/blob/master/README.md

Retrieves a list of auctions associated with a specific factory address.

```APIDOC
## GET /auctions

### Description

Retrieves a list of auctions created by a specific auction factory. You must provide the factory's address.

### Method

GET

### Endpoint

`/auctions`

### Parameters

#### Query Parameters

- **factory** (string) - Required - The hexadecimal address of the auction factory.

### Request Example

```bash
curl "http://localhost:8000/auctions?factory=0x123...abc"
```

### Response

#### Success Response (200)

- **auctions** (array of objects) - A list of auction objects, each containing details like auction ID and address.
  - **auction_id** (string) - The unique identifier for the auction.
  - **auction_address** (string) - The contract address of the auction.

#### Response Example

```json
{
  "auctions": [
    {
      "auction_id": "auction_1",
      "auction_address": "0xabcd...1234"
    },
    {
      "auction_id": "auction_2",
      "auction_address": "0xefgh...5678"
    }
  ]
}
```
```

--------------------------------

### Basic Data Collection Commands in Python

Source: https://github.com/schlagonia/auctions-data-collection/blob/master/README.md

Python commands to run the data collection script for initial full synchronization and subsequent incremental updates. Includes an option to set the log level.

```python
# Initial full sync
python src/main.py --full-sync --log-level DEBUG

# Regular incremental updates
python src/main.py
```

--------------------------------

### GET /auctions

Source: https://github.com/schlagonia/auctions-data-collection/blob/master/README.md

Retrieves a list of auctions deployed by a specific factory contract.

```APIDOC
## GET /auctions

### Description
Retrieves a list of auctions deployed by a specific factory contract. Supports pagination.

### Method
GET

### Endpoint
/auctions

### Parameters
#### Query Parameters
- **factory** (string) - Required - The address of the factory contract to filter by.
- **page** (integer) - Optional - The page number for pagination (default is 1).
- **limit** (integer) - Optional - The number of auctions per page (default is 100).

### Request Example
```bash
GET /auctions?factory=0xYourFactoryAddress&page=2&limit=50
```

### Response
#### Success Response (200)
- **auctions** (array) - An array of auction objects.
  - **address** (string) - The address of the auction contract.
  - **factory_address** (string) - The address of the factory that deployed this auction.
  - **creation_block** (integer) - The block number in which the auction was created.
- **total_pages** (integer) - The total number of pages available.
- **current_page** (integer) - The current page number.

#### Response Example
```json
{
  "auctions": [
    {
      "address": "0xAuctionAddress1",
      "factory_address": "0xYourFactoryAddress",
      "creation_block": 18500000
    },
    {
      "address": "0xAuctionAddress2",
      "factory_address": "0xYourFactoryAddress",
      "creation_block": 18500010
    }
  ],
  "total_pages": 15,
  "current_page": 2
}
```
```

--------------------------------

### GET /factories

Source: https://github.com/schlagonia/auctions-data-collection/blob/master/README.md

Lists all discovered factory contract addresses and their associated statistics.

```APIDOC
## GET /factories

### Description
Lists all discovered factory contract addresses and their associated statistics, including the number of auctions deployed by each.

### Method
GET

### Endpoint
/factories

### Parameters
None

### Request Example
None

### Response
#### Success Response (200)
- **factories** (array) - An array of factory objects.
  - **address** (string) - The address of the factory contract.
  - **name** (string) - The human-readable name of the factory (if provided).
  - **auctions_count** (integer) - The number of auctions deployed by this factory.

#### Response Example
```json
{
  "factories": [
    {
      "address": "0xYourFactoryAddress1",
      "name": "Factory One",
      "auctions_count": 750
    },
    {
      "address": "0xYourFactoryAddress2",
      "name": "Factory Two",
      "auctions_count": 750
    }
  ]
}
```
```

--------------------------------

### Initialize Page and Event Listeners for Auction Data

Source: https://github.com/schlagonia/auctions-data-collection/blob/master/web/templates/auctions.html

Initializes the page by setting up event listeners for pagination controls (min-kicks filter, previous button, next button) and the export button. It also includes the logic for handling page load and initial data fetching.

```javascript
document.getElementById('min-kicks-filter').addEventListener('input', () => { currentPage = 1; loadAuctions(); }); // Pagination
document.getElementById('prev-btn').addEventListener('click', () => { if (currentPage > 1) { currentPage--; loadAuctions(); } });
document.getElementById('next-btn').addEventListener('click', () => { const totalPages = Math.ceil(allFilteredAuctions.length / pageSize); if (currentPage < totalPages) { currentPage++; loadAuctions(); } }); // Export button
document.getElementById('export-btn').addEventListener('click', exportData);

// Initialize when page loads
document.addEventListener('DOMContentLoaded', initializePage);
```

--------------------------------

### GET /events

Source: https://github.com/schlagonia/auctions-data-collection/blob/master/README.md

Fetches all kick and take events associated with a specific auction contract.

```APIDOC
## GET /events

### Description
Fetches all kick and take events associated with a specific auction contract. Supports pagination.

### Method
GET

### Endpoint
/events

### Parameters
#### Query Parameters
- **auction** (string) - Required - The address of the auction contract to filter by.
- **page** (integer) - Optional - The page number for pagination (default is 1).
- **limit** (integer) - Optional - The number of events per page (default is 100).

### Request Example
```bash
GET /events?auction=0xAuctionAddress1&page=1&limit=50
```

### Response
#### Success Response (200)
- **events** (array) - An array of event objects.
  - **transaction_hash** (string) - The hash of the transaction containing the event.
  - **log_index** (integer) - The index of the log within the transaction.
  - **event_type** (string) - The type of event ('kick' or 'take').
  - **timestamp** (string) - The timestamp of the event (ISO 8601 format).
  - **block_number** (integer) - The block number in which the event occurred.
  - **data** (object) - The specific data associated with the event (e.g., bidder address, amount).
- **total_pages** (integer) - The total number of pages available.
- **current_page** (integer) - The current page number.

#### Response Example
```json
{
  "events": [
    {
      "transaction_hash": "0xabc...",
      "log_index": 1,
      "event_type": "kick",
      "timestamp": "2023-10-27T10:30:00Z",
      "block_number": 18500005,
      "data": {
        "bidder": "0xBidderAddress",
        "amount": "1000000000000000000"
      }
    },
    {
      "transaction_hash": "0xdef...",
      "log_index": 2,
      "event_type": "take",
      "timestamp": "2023-10-27T10:35:00Z",
      "block_number": 18500006,
      "data": {
        "taker": "0xTakerAddress",
        "amount": "1100000000000000000"
      }
    }
  ],
  "total_pages": 5,
  "current_page": 1
}
```
```

--------------------------------

### Check Database Integrity

Source: https://github.com/schlagonia/auctions-data-collection/blob/master/DEPLOYMENT.md

SQLite command to perform an integrity check on the database file. Helps identify corruption or data inconsistencies.

```bash
sqlite3 /opt/auctions-data-collection/data/auctions.db "PRAGMA integrity_check;"
```

--------------------------------

### Run Auction Data Collection Script

Source: https://github.com/schlagonia/auctions-data-collection/blob/master/README.md

Bash commands to execute the main Python script for data collection. Includes options for full historical sync, incremental updates, showing statistics, calculating volumes, and running the API server.

```bash
# Full historical sync (first run)
PYTHONPATH=src python src/main.py --full-sync

# Incremental sync (subsequent runs)
python src/main.py

# Show current statistics
python src/main.py --stats

python src/main.py --calculate-volumes --chain-id 1 --batch-size 5 --delay 2.0

# Start the REST API
python src/main.py --api
```

--------------------------------

### Set File/Directory Ownership

Source: https://github.com/schlagonia/auctions-data-collection/blob/master/DEPLOYMENT.md

Bash commands to check and set ownership for the auctions data collection directory. Ensures the correct user ('auction') has permissions.

```bash
ls -la /opt/auctions-data-collection/
```

```bash
chown -R auction:auction /opt/auctions-data-collection
```

--------------------------------

### Discover Auction Contracts from Factory in Python

Source: https://context7.com/schlagonia/auctions-data-collection/llms.txt

Discovers auction contracts deployed by a factory contract on an EVM-compatible blockchain using Python. It initializes Web3.py, sets up the configuration, and then uses the FactoryDiscovery module to find auctions, printing details like auction address, want token, and block number. Discovered auctions are then stored using the DataStorage module. Dependencies include web3.py, config, and modules.

```python
from web3 import Web3
from web3.middleware import geth_poa_middleware
from config import Config
from modules import FactoryDiscovery, DataStorage

# Initialize configuration
config = Config('config.json')

# Create Web3 connection
chain_config = config.get_chain_by_id(1)
w3 = Web3(Web3.HTTPProvider(chain_config.rpc_url))

# Add PoA middleware for non-Ethereum chains
if chain_config.chain_id != 1:
    w3.middleware_onion.inject(geth_poa_middleware, layer=0)

# Initialize factory discovery
discovery = FactoryDiscovery(w3, config)

# Discover auctions from a specific factory
factory = config.factories[0]
auctions = discovery.discover_auctions_for_factory(factory)

# Process discovered auctions
for auction in auctions:
    print(f"Auction: {auction['auction_address']}")
    print(f"  Want Token: {auction['want_address']}")
    print(f"  Factory: {auction['factory_address']}")
    print(f"  Block: {auction['block_number']}")
    print(f"  Timestamp: {auction['deployment_timestamp']}")
    print(f"  Receiver: {auction.get('receiver', 'N/A')}")

# Store discovered auctions
storage = DataStorage(config.database_path)
storage.store_chain(chain_config.chain_id, chain_config.name)
storage.store_factory(
    chain_config.chain_id,
    factory.address,
    factory.name,
    factory.deployment_block,
    factory.legacy
)
count = storage.store_auctions(chain_config.chain_id, auctions)
print(f"Stored {count} new auctions")
```