### Complete EC2 Setup Script for Trading Bot

Source: https://github.com/faxulous/supersecretbetcodehistoryformcp/blob/master/mcp-ultimate/infrastructure-complete.md

This script automates the setup of an EC2 instance for a trading bot. It installs necessary software, configures services, and sets up monitoring and security.

```bash
#!/bin/bash
# Complete EC2 setup for trading bot

# Update system
sudo yum update -y

# Install Python 3.11
sudo yum install -y gcc openssl-devel bzip2-devel libffi-devel
cd /opt
sudo wget https://www.python.org/ftp/python/3.11.0/Python-3.11.0.tgz
sudo tar xzf Python-3.11.0.tgz
cd Python-3.11.0
sudo ./configure --enable-optimizations
sudo make altinstall

# Install system dependencies
sudo yum install -y git docker htop iotop sysstat

# Install Redis
sudo amazon-linux-extras install redis6 -y
sudo systemctl enable redis
sudo systemctl start redis

# Setup trading user
sudo useradd -m -s /bin/bash trading
sudo usermod -aG wheel trading
sudo usermod -aG docker trading

# Setup directories
sudo mkdir -p /home/trading/{app,logs,data,config,backups}
sudo chown -R trading:trading /home/trading

# Install Python packages as trading user
sudo -u trading pip3.11 install --user \
    betfairlightweight \
    flumine \
    pandas \
    numpy \
    redis \
    prometheus-client \
    psutil \
    python-json-logger

# Setup systemd service
cat <<'EOF' | sudo tee /etc/systemd/system/trading-bot.service
[Unit]
Description=Trading Bot Service
After=network-online.target redis.service
Wants=network-online.target

[Service]
Type=simple
User=trading
Group=trading
WorkingDirectory=/home/trading/app

Environment="PYTHONUNBUFFERED=1"
Environment="PYTHONOPTIMIZE=2"
Environment="PYTHONPATH=/home/trading/app"

ExecStartPre=/usr/local/bin/python3.11 -m pip install --user -r requirements.txt
ExecStart=/usr/local/bin/python3.11 main.py
ExecReload=/bin/kill -HUP $MAINPID

Restart=always
RestartSec=10
StartLimitInterval=60
StartLimitBurst=3

# Resource limits
LimitNOFILE=65536
MemoryLimit=2G
CPUQuota=80%

# Security
PrivateTmp=true
NoNewPrivileges=true

StandardOutput=append:/home/trading/logs/stdout.log
StandardError=append:/home/trading/logs/stderr.log

[Install]
WantedBy=multi-user.target
EOF

# Setup log rotation
cat <<'EOF' | sudo tee /etc/logrotate.d/trading-bot
/home/trading/logs/*.log {
    daily
    rotate 14
    compress
    delaycompress
    missingok
    notifempty
    create 0644 trading trading
    sharedscripts
    postrotate
        systemctl reload trading-bot > /dev/null 2>&1 || true
    endscript
}
EOF

# Setup monitoring
cat <<'EOF' | sudo tee /home/trading/monitor.sh
#!/bin/bash
# Health monitoring script

check_service() {
    if systemctl is-active --quiet trading-bot; then
        echo "Service: Running"
    else
        echo "Service: DOWN - Restarting"
        systemctl restart trading-bot
    fi
}

check_memory() {
    mem_usage=$(free | grep Mem | awk '{print int($3/$2 * 100)}')
    if [ $mem_usage -gt 90 ]; then
        echo "Memory: CRITICAL ($mem_usage%)"
        # Restart if memory critical
        systemctl restart trading-bot
    else
        echo "Memory: OK ($mem_usage%)"
    fi
}

check_disk() {
    disk_usage=$(df -h / | tail -1 | awk '{print int($5)}')
    if [ $disk_usage -gt 80 ]; then
        echo "Disk: WARNING ($disk_usage%)"
        # Clean old logs
        find /home/trading/logs -name "*.gz" -mtime +30 -delete
    else
        echo "Disk: OK ($disk_usage%)"
    fi
}

# Run checks
check_service
check_memory
check_disk
EOF

sudo chmod +x /home/trading/monitor.sh

# Add monitoring to crontab
(crontab -l 2>/dev/null; echo "*/5 * * * * /home/trading/monitor.sh >> /home/trading/logs/monitor.log 2>&1") | crontab -

# Setup firewall
sudo yum install -y firewalld
sudo systemctl enable firewalld
sudo systemctl start firewalld
sudo firewall-cmd --permanent --add-port=8080/tcp  # Health check
sudo firewall-cmd --permanent --add-port=9090/tcp  # Prometheus
sudo firewall-cmd --reload

# Enable and start services
sudo systemctl daemon-reload
sudo systemctl enable trading-bot
sudo systemctl start trading-bot

echo "Setup complete! Check status with: systemctl status trading-bot"

```

--------------------------------

### Setting Up and Activating a Virtual Environment

Source: https://github.com/faxulous/supersecretbetcodehistoryformcp/blob/master/mcp-ultimate/version-compatibility.md

This bash snippet provides commands for creating, activating, and installing dependencies within a virtual environment. It covers commands for both Windows and Linux/Mac systems, promoting isolated project setups.

```bash
# Create isolated environment
python -m venv betting_env

# Activate (Windows)
betting_env\Scripts\activate

# Activate (Linux/Mac)
source betting_env/bin/activate

# Install requirements
pip install -r requirements.txt
```

--------------------------------

### Install BetfairLightweight

Source: https://github.com/faxulous/supersecretbetcodehistoryformcp/blob/master/mcp-ultimate/performance-secrets.md

Upgrade to betfairlightweight version 2.12.0b0 or later to benefit from significant performance improvements in streaming.

```bash
pip install betfairlightweight>=2.12.0b0
```

--------------------------------

### Configure Basic Logging

Source: https://github.com/faxulous/supersecretbetcodehistoryformcp/blob/master/mcp-ultimate/flumine-mastery.md

Set up basic logging configuration with a specified level and format. This example shows how to log order placement and performance metrics.

```python
import logging

logging.basicConfig(
    level=logging.INFO,
    format='%(asctime)s - %(name)s - %(levelname)s - %(message)s'
)

# Log order placement
logger.info(f"Placing order: market={market.market_id}, price={price}, size={size}")

# Log performance metrics
logger.info(f"Processing time: {duration:.3f}s, Orders: {order_count}")
```

--------------------------------

### EC2 User Data Script for Trading Bot Setup

Source: https://github.com/faxulous/supersecretbetcodehistoryformcp/blob/master/mcp-ultimate/production-patterns.md

A bash script to automate the setup of a trading bot on an EC2 instance. It updates the system, installs Python 3.11, necessary tools, creates a dedicated user, sets up directories, installs Python packages, configures a systemd service, and sets up log rotation. Ensure the `main.py` script exists in `/home/trading/`.

```bash
#!/bin/bash
# User data for automatic setup

# Update system
yum update -y

# Install Python 3.11
yum install -y python3.11 python3.11-pip

# Install monitoring tools
yum install -y htop iotop sysstat

# Create trading user
useradd -m trading
usermod -aG wheel trading

# Setup directories
mkdir -p /home/trading/{logs,data,config}
chown -R trading:trading /home/trading

# Install Python packages
su - trading -c "pip3.11 install --user betfairlightweight flumine pandas redis prometheus-client"

# Setup systemd service
cat > /etc/systemd/system/trading-bot.service << 'EOF'
[Unit]
Description=Trading Bot Service
After=network.target

[Service]
Type=simple
User=trading
WorkingDirectory=/home/trading
ExecStart=/usr/bin/python3.11 /home/trading/main.py
Restart=always
RestartSec=10
StandardOutput=append:/home/trading/logs/stdout.log
StandardError=append:/home/trading/logs/stderr.log

[Install]
WantedBy=multi-user.target
EOF

systemctl daemon-reload
systemctl enable trading-bot.service

# Setup log rotation
cat > /etc/logrotate.d/trading << 'EOF'
/home/trading/logs/*.log {
    daily
    rotate 7
    compress
    delaycompress
    missingok
    notifempty
    create 0644 trading trading
}
EOF
```

--------------------------------

### Placing a Starting Price (SP) Order

Source: https://github.com/faxulous/supersecretbetcodehistoryformcp/blob/master/mcp-ultimate/api-complete-guide.md

Use `persistence_type='PERSIST'` to ensure your order participates in the Starting Price (SP) market. Orders with other persistence types will not participate in SP.

```python
# Understanding SP participation
def place_sp_order(client, market_id, selection_id, size):
    """Place order that participates in SP"""
    
    # Will participate in SP
    sp_order = PlaceInstruction(
        selection_id=selection_id,
        order_type=LimitOrder(
            price=1000,  # Max price
            size=size,
            persistence_type="PERSIST"  # KEY: Must be PERSIST for SP
        ),
        side="BACK"
    )
    
    # Will NOT participate in SP (default)
    regular_order = PlaceInstruction(
        selection_id=selection_id,
        order_type=LimitOrder(
            price=2.5,
            size=size,
            persistence_type="LAPSE"  # Default - won't go to SP
        ),
        side="BACK"
    )
    
    # WARNING: SP orders match at whatever SP is determined!
```

--------------------------------

### Optimized Production Dockerfile

Source: https://github.com/faxulous/supersecretbetcodehistoryformcp/blob/master/mcp-ultimate/infrastructure-complete.md

This Dockerfile uses a multi-stage build to create a minimal production image. It installs build and runtime dependencies separately, sets up a virtual environment, configures SSL for Betfair, creates a non-root user, and includes a health check.

```dockerfile
# Multi-stage build for minimal image size
FROM python:3.11-slim-bookworm as builder

# Install build dependencies
RUN apt-get update && apt-get install -y \
    gcc \
    g++ \
    libpq-dev \
    && rm -rf /var/lib/apt/lists/*

# Create virtual environment
RUN python -m venv /opt/venv
ENV PATH="/opt/venv/bin:$PATH"

# Install Python dependencies
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

# Final stage
FROM python:3.11-slim-bookworm

# Install runtime dependencies only
RUN apt-get update && apt-get install -y \
    libpq5 \
    curl \
    && rm -rf /var/lib/apt/lists/*

# Copy virtual environment from builder
COPY --from=builder /opt/venv /opt/venv
ENV PATH="/opt/venv/bin:$PATH"

# Fix SSL for Betfair
RUN echo "[ ssl_client ]\n\nbasicConstraints = CA:FALSE\nnsCertType = client\nkeyUsage = digitalSignature, keyEncipherment\nextendedKeyUsage = clientAuth" >> /etc/ssl/openssl.cnf

# Create non-root user
RUN useradd -m -u 1000 -s /bin/bash trading
WORKDIR /app

# Copy application
COPY --chown=trading:trading . .

# Switch to non-root user
USER trading

# Health check
HEALTHCHECK --interval=30s --timeout=10s --start-period=60s --retries=3 \
    CMD curl -f http://localhost:8080/health || exit 1

# Run application
CMD ["python", "-u", "main.py"]

```

--------------------------------

### Dockerfile for Production Trading Bot

Source: https://github.com/faxulous/supersecretbetcodehistoryformcp/blob/master/mcp-ultimate/production-patterns.md

Uses a Debian-based Python image for better compatibility than Alpine. Includes system dependency installation, SSL configuration for Betfair, creation of a non-root user, and application setup with health checks.

```dockerfile
# RECOMMENDED: Debian-based for compatibility
FROM python:3.11-bookworm

# NOT RECOMMENDED: Alpine (compatibility issues)
# FROM python:3.11-alpine

# Install system dependencies
RUN apt-get update && apt-get install -y \
    gcc \
    g++ \
    curl \
    libssl-dev \
    && rm -rf /var/lib/apt/lists/*

# Fix SSL for Betfair authentication
RUN echo "[ ssl_client ]\n\
basicConstraints = CA:FALSE\n\
nsCertType = client\n\
keyUsage = digitalSignature, keyEncipherment\n\
extendedKeyUsage = clientAuth" >> /etc/ssl/openssl.cnf

# Create non-root user
RUN useradd -m -u 1000 trading
WORKDIR /home/trading

# Install Python dependencies
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

# Copy application
COPY --chown=trading:trading . .

USER trading

# Health check
HEALTHCHECK --interval=30s --timeout=10s --start-period=60s \
  CMD curl -f http://localhost:8080/health || exit 1

CMD ["python", "-u", "main.py"]
```

--------------------------------

### PostgreSQL Connection Pooling Setup

Source: https://github.com/faxulous/supersecretbetcodehistoryformcp/blob/master/mcp-ultimate/infrastructure-complete.md

Initializes a threaded connection pool for PostgreSQL. Configures minimum and maximum connections, host, and other connection parameters. Requires `psycopg2` and `psycopg2.pool`.

```python
import psycopg2
from psycopg2.pool import ThreadedConnectionPool
from contextlib import contextmanager
import logging

logger = logging.getLogger(__name__)

class TradingDatabase:
    """Production PostgreSQL setup with connection pooling"""
    
    def __init__(self, config):
        self.pool = ThreadedConnectionPool(
            minconn=2,
            maxconn=20,
            host=config['host'],
```

--------------------------------

### Recommended Dependency Versioning Strategy

Source: https://github.com/faxulous/supersecretbetcodehistoryformcp/blob/master/mcp-ultimate/version-compatibility.md

This text-based snippet illustrates recommended practices for pinning dependency versions. It shows good examples that allow minor updates for bug fixes while preventing breaking changes, and contrasts them with overly restrictive or permissive approaches.

```text
# Good - allows bug fixes, prevents breaking changes
betfairlightweight>=2.20.0,<3.0.0
flumine>=2.6.0,<3.0.0
pandas>=1.5.0,<2.0.0

# Avoid - too restrictive, misses important fixes
betfairlightweight==2.20.1

# Avoid - too permissive, allows breaking changes
betfairlightweight>=2.0.0
```

--------------------------------

### Install Specific Flumine Version

Source: https://github.com/faxulous/supersecretbetcodehistoryformcp/blob/master/mcp-ultimate/flumine-mastery.md

Use this command to install a specific version of Flumine, particularly useful for resolving version compatibility issues.

```bash
pip install flumine==2.6.2
```

--------------------------------

### Docker Compose Production Setup

Source: https://github.com/faxulous/supersecretbetcodehistoryformcp/blob/master/mcp-ultimate/production-patterns.md

Defines services for a trading bot, Redis, and Prometheus. Configures environment variables, volumes for persistent data, resource limits, logging, and restart policies for production environments.

```yaml
version: '3.8'

services:
  trading-bot:
    build: .
    restart: unless-stopped
    environment:
      - BETFAIR_USERNAME=${BETFAIR_USERNAME}
      - BETFAIR_PASSWORD=${BETFAIR_PASSWORD}
      - BETFAIR_APP_KEY=${BETFAIR_APP_KEY}
      - PYTHONUNBUFFERED=1
    volumes:
      - ./data:/home/trading/data
      - ./logs:/home/trading/logs
      - ./config:/home/trading/config:ro
    deploy:
      resources:
        limits:
          memory: 2G
        reservations:
          memory: 1G
    logging:
      driver: "json-file"
      options:
        max-size: "10m"
        max-file: "3"
    
  redis:
    image: redis:7-alpine
    restart: unless-stopped
    volumes:
      - redis_data:/data
    command: redis-server --appendonly yes
    
  prometheus:
    image: prom/prometheus
    restart: unless-stopped
    volumes:
      - ./prometheus.yml:/etc/prometheus/prometheus.yml
      - prometheus_data:/prometheus
    ports:
      - "9090:9090"

volumes:
  redis_data:
  prometheus_data:
```

--------------------------------

### Filter Markets by Start Time

Source: https://github.com/faxulous/supersecretbetcodehistoryformcp/blob/master/mcp-ultimate/api-complete-guide.md

Construct market filters with the correct date format for 'marketStartTime' to accurately retrieve markets within a specified time range. Ensure all required fields are included to avoid DSC-0018 errors.

```python
from datetime import datetime, timedelta

# Parse input date
game_time = datetime.strptime("15 March 2024 15:30", "%d %B %Y %H:%M")

# Create filter with proper formatting
market_filter = {
    'eventTypeIds': ['1'],  # Soccer
    'marketTypeCodes': ['MATCH_ODDS'],
    'marketStartTime': {
        'from': (game_time - timedelta(hours=2)).strftime("%Y-%m-%dT%H:%M:%SZ"),
        'to': (game_time + timedelta(hours=2)).strftime("%Y-%m-%dT%H:%M:%SZ")
    }
}

# Avoid DSC-0018 error by including all required fields

```

--------------------------------

### Configure Systemd Service for Trading Bot

Source: https://github.com/faxulous/supersecretbetcodehistoryformcp/blob/master/mcp-ultimate/production-patterns.md

Defines a systemd service unit for the trading bot. Ensure correct paths and user/group permissions are set for your environment. This configuration includes environment variables, execution commands, restart policies, resource limits, and logging setup.

```ini
# /etc/systemd/system/trading-bot.service
[Unit]
Description=Betfair Trading Bot
After=network-online.target
Wants=network-online.target

[Service]
Type=simple
User=trading
Group=trading
WorkingDirectory=/home/trading

# Environment
Environment="PYTHONUNBUFFERED=1"
Environment="PYTHONOPTIMIZE=2"

# Execution
ExecStartPre=/usr/bin/python3 -m pip install -r requirements.txt
ExecStart=/usr/bin/python3 main.py
ExecReload=/bin/kill -HUP $MAINPID

# Restart policy
Restart=always
RestartSec=10
StartLimitInterval=60
StartLimitBurst=3

# Resource limits
LimitNOFILE=65536
MemoryLimit=2G

# Logging
StandardOutput=journal
StandardError=journal

[Install]
WantedBy=multi-user.target

```

--------------------------------

### Create and Restore Pip Lock File

Source: https://github.com/faxulous/supersecretbetcodehistoryformcp/blob/master/mcp-ultimate/version-compatibility.md

Illustrates how to generate a file with exact working package versions and how to use it to restore the environment.

```bash
# Create lock file when everything works
pip freeze > working_versions.txt

# Can restore later if needed
pip install -r working_versions.txt
```

--------------------------------

### Detect Race Start with GPS/TPD Data

Source: https://github.com/faxulous/supersecretbetcodehistoryformcp/blob/master/mcp-ultimate/flumine-mastery.md

Process GPS/TPD stream data to assist with in-play start detection. This function checks for a 'RUNNING' race status and records the actual start time.

```python
def process_sports_data(self, market, sports_data):
    """Process GPS/TPD data for race start detection"""
    
    if sports_data.get('raceStatus') == 'RUNNING':
        self.race_started = True
        self.start_time = sports_data.get('actualStart')
```

--------------------------------

### Place Orders with Async

Source: https://github.com/faxulous/supersecretbetcodehistoryformcp/blob/master/mcp-ultimate/api-complete-guide.md

Shows how to place orders asynchronously for improved performance using the Betfair API.

```APIDOC
## Place Orders with Async

### Description
This function demonstrates how to place orders asynchronously for better performance. It utilizes `async_=True` to enable asynchronous execution of the `place_orders` call.

### Method
`client.betting.place_orders`

### Parameters
- `market_id` (string) - Required - The ID of the market.
- `instructions` (list of PlaceInstruction) - Required - A list containing place instructions, specifying order details like selection ID, price, size, and side.
- `async_` (boolean) - Required - Set to `True` to enable asynchronous order placement.

### Request Example (Python)
```python
from betfairlightweight.filters import LimitOrder, PlaceInstruction

instruction = PlaceInstruction(
    selection_id=selection_id,
    order_type=LimitOrder(
        price=price,
        size=size,
        persistence_type="LAPSE"  # Default
    ),
    side="BACK"
)

result = client.betting.place_orders(
    market_id=market_id,
    instructions=[instruction],
    async_=True  # Key parameter!
)
```
```

--------------------------------

### Get All Exposures for a Strategy

Source: https://github.com/faxulous/supersecretbetcodehistoryformcp/blob/master/mcp-ultimate/flumine-mastery.md

Retrieve all exposure calculations for a given strategy and market, including selection and market-level exposures. The `handicap` and `selection_id` tuple is required.

```python
# Full exposure calculation
exposures = market.blotter.get_exposures(
    strategy,
    (handicap, selection_id)
)
```

--------------------------------

### Define Core and Optional Dependencies

Source: https://github.com/faxulous/supersecretbetcodehistoryformcp/blob/master/mcp-ultimate/version-compatibility.md

Shows how to specify version constraints for core and optional Python dependencies in a requirements file.

```txt
# Core requirements only
betfairlightweight>=2.19.0,<3.0.0
flumine>=2.6.0,<3.0.0

# Optional/development dependencies separate
pandas>=1.5.0,<2.0.0  # If you use it
numpy>=1.20.0,<2.0.0  # If you use it
```

--------------------------------

### Cancel Orders Correct Format

Source: https://github.com/faxulous/supersecretbetcodehistoryformcp/blob/master/mcp-ultimate/api-complete-guide.md

Demonstrates how to correctly cancel orders using the Betfair API, including a Python implementation and a raw JSON example.

```APIDOC
## Cancel Orders Correct Format

### Description
This function demonstrates how to cancel orders using the Betfair API. It takes a client object, market ID, and bet ID as input and returns the result of the cancellation request.

### Method
`client.betting.cancel_orders`

### Parameters
- `market_id` (string) - Required - The ID of the market.
- `instructions` (list of CancelInstruction) - Required - A list containing cancel instructions, each specifying a `bet_id`.

### Request Example (Python)
```python
from betfairlightweight.filters import CancelInstruction

instruction = CancelInstruction(bet_id=bet_id)
result = client.betting.cancel_orders(
    market_id=market_id,
    instructions=[instruction]
)
```

### Request Example (Raw JSON)
```json
{
    "params": {
        "marketId": "1.219815567",
        "instructions": [
            {"betId": "325032312929"}
        ]
    },
    "id": 1
}
```
```

--------------------------------

### Handle Synchronous and Asynchronous Client Login

Source: https://github.com/faxulous/supersecretbetcodehistoryformcp/blob/master/mcp-ultimate/version-compatibility.md

This pattern shows how to create a client that can handle both synchronous and asynchronous login methods, adapting to whether async support is available.

```python
# Pattern: Libraries add async support over time
# Be prepared for both patterns

class FlexibleClient:
    def __init__(self):
        self.is_async = hasattr(APIClient, 'login_async')
    
    def login(self):
        if self.is_async:
            # Use async if available
            import asyncio
            return asyncio.run(self.client.login_async())
        else:
            # Fall back to sync
            return self.client.login()
```

--------------------------------

### Efficient Time-Series Database Class

Source: https://github.com/faxulous/supersecretbetcodehistoryformcp/blob/master/mcp-ultimate/data-management.md

A Python class for optimized time-series market data storage using SQLite. Includes table setup, batch insertion, and data retrieval methods.

```python
import sqlite3
from datetime import datetime

class TimeSeriesDB:
    """Optimized database for market data"""
    
    def __init__(self, db_path):
        self.conn = sqlite3.connect(db_path)
        self.setup_tables()
        
    def setup_tables(self):
        """Create optimized table structure"""
        
        # Main market data table
        self.conn.execute("""
            CREATE TABLE IF NOT EXISTS market_data (
                market_id TEXT,
                timestamp INTEGER,
                runner_id INTEGER,
                back_price REAL,
                back_size REAL,
                lay_price REAL,
                lay_size REAL,
                total_matched REAL,
                PRIMARY KEY (market_id, timestamp, runner_id)
            ) WITHOUT ROWID
        """)
        
        # Indexes for common queries
        self.conn.execute("""
            CREATE INDEX IF NOT EXISTS idx_market_time 
            ON market_data(market_id, timestamp)
        """)
        
        self.conn.execute("""
            CREATE INDEX IF NOT EXISTS idx_time 
            ON market_data(timestamp)
        """)
        
        # Summary table for quick access
        self.conn.execute("""
            CREATE TABLE IF NOT EXISTS market_summary (
                market_id TEXT PRIMARY KEY,
                start_time INTEGER,
                end_time INTEGER,
                total_matched_final REAL,
                winner_id INTEGER,
                num_runners INTEGER
            )
        """)
        
    def insert_batch(self, data):
        """Batch insert for performance"""
        
        with self.conn:
            self.conn.executemany(
                """INSERT OR REPLACE INTO market_data 
                   VALUES (?, ?, ?, ?, ?, ?, ?, ?)""",
                data
            )
    
    def get_market_data(self, market_id, start_time=None, end_time=None):
        """Retrieve market data efficiently"""
        
        query = """
            SELECT * FROM market_data 
            WHERE market_id = ?
        """
        params = [market_id]
        
        if start_time:
            query += " AND timestamp >= ?"
            params.append(start_time)
        
        if end_time:
            query += " AND timestamp <= ?"
            params.append(end_time)
            
        query += " ORDER BY timestamp"
        
        cursor = self.conn.execute(query, params)
        return cursor.fetchall()

```

--------------------------------

### Migrate Market Filters from Flat to Structured Parameters

Source: https://github.com/faxulous/supersecretbetcodehistoryformcp/blob/master/mcp-ultimate/version-compatibility.md

Demonstrates how to adapt market filter creation from older flat parameter styles to newer structured object-based approaches. Includes an adaptive function to handle both.

```python
# Pattern: Libraries move from flat params to structured objects
# Example: Market filters

# Old way (might be deprecated)
def old_market_filter(event_types, countries):
    return {
        'event_type_ids': event_types,
        'market_countries': countries
    }

# New way (future-proof)
def new_market_filter(event_types, countries):
    from betfairlightweight.filters import market_filter
    return market_filter(
        event_type_ids=event_types,
        market_countries=countries
    )

# Adaptive approach
def adaptive_market_filter(event_types, countries):
    try:
        # Try new method first
        return new_market_filter(event_types, countries)
    except (ImportError, AttributeError):
        # Fall back to old method
        return old_market_filter(event_types, countries)
```

--------------------------------

### EC2 Instance Recommendations for Trading Bots

Source: https://github.com/faxulous/supersecretbetcodehistoryformcp/blob/master/mcp-ultimate/production-patterns.md

Dictionary mapping environment types to recommended EC2 instance specifications, including vCPU, memory, cost, and use case. Useful for selecting appropriate instance types based on trading strategy complexity and frequency.

```python
# Based on community experience
INSTANCE_RECOMMENDATIONS = {
    'development': {
        'type': 't3.small',
        'vcpu': 2,
        'memory': '2GB',
        'cost': '$15/month',
        'use_case': 'Testing and development'
    },
    'production_basic': {
        'type': 't3.medium',
        'vcpu': 2,
        'memory': '4GB',
        'cost': '$30/month',
        'use_case': 'Single strategy, low frequency'
    },
    'production_standard': {
        'type': 'c5.large',
        'vcpu': 2,
        'memory': '4GB',
        'cost': '$62/month',
        'use_case': 'Multiple strategies, backtesting'
    },
    'production_performance': {
        'type': 'c5.xlarge',
        'vcpu': 4,
        'memory': '8GB',
        'cost': '$123/month',
        'use_case': 'High frequency, multiple markets'
    }
}
```

--------------------------------

### Process Race Stream Data with Python

Source: https://github.com/faxulous/supersecretbetcodehistoryformcp/blob/master/mcp-ultimate/data-management.md

Use this class to process incoming sports data, storing GPS information and detecting race start events. It handles race position changes and can cancel unmatched orders when a race begins.

```python
class RaceStreamProcessor:
    """Process GPS/TPD data for racing"""
    
    def __init__(self):
        self.race_data = {}
        
    def process_sports_data(self, market, sports_data):
        """Process incoming sports data"""
        
        market_id = market.market_id
        
        # Store GPS data
        if 'rpc' in sports_data:  # Race position change
            for runner_change in sports_data['rpc']:
                runner_id = runner_change['id']
                position = runner_change.get('p')  # Position
                distance = runner_change.get('d')  # Distance
                
                if runner_id not in self.race_data:
                    self.race_data[runner_id] = []
                    
                self.race_data[runner_id].append({
                    'timestamp': sports_data['pt'],  # Publish time
                    'position': position,
                    'distance': distance
                })
        
        # Detect race start
        if sports_data.get('raceStatus') == 'RUNNING':
            logger.info(f"Race {market_id} has started")
            self.on_race_start(market)
            
    def on_race_start(self, market):
        """Handle race start event"""
        
        # Cancel unmatched orders if needed
        for order in market.orders:
            if order.status == "EXECUTABLE":
                logger.info(f"Cancelling order {order.id} - race started")
                order.cancel()

```

--------------------------------

### Place Order Asynchronously

Source: https://github.com/faxulous/supersecretbetcodehistoryformcp/blob/master/mcp-ultimate/api-complete-guide.md

Shows how to place an order asynchronously for improved performance. The `async_=True` parameter is key for enabling this feature.

```python
from betfairlightweight.filters import LimitOrder, PlaceInstruction

def place_order_async(client, market_id, selection_id, price, size):
    """Place order with async for better performance"""
    
    instruction = PlaceInstruction(
        selection_id=selection_id,
        order_type=LimitOrder(
            price=price,
            size=size,
            persistence_type="LAPSE"  # Default
        ),
        side="BACK"
    )
    
    # Enable async for this transaction
    result = client.betting.place_orders(
        market_id=market_id,
        instructions=[instruction],
        async_=True  # Key parameter!
    )
    
    return result
```

--------------------------------

### Prometheus Metrics for Trading Bot

Source: https://github.com/faxulous/supersecretbetcodehistoryformcp/blob/master/mcp-ultimate/production-patterns.md

Define and expose Prometheus metrics for trading activities like orders, latency, positions, exposure, and P&L. Requires the 'prometheus_client' library. Starts an HTTP server on a specified port to serve metrics.

```python
from prometheus_client import Counter, Histogram, Gauge, start_http_server
import time

# Define metrics
orders_placed = Counter('trading_orders_placed', 'Total orders placed', ['strategy', 'side'])
order_latency = Histogram('trading_order_latency_seconds', 'Order placement latency')
active_positions = Gauge('trading_active_positions', 'Number of active positions')
total_exposure = Gauge('trading_total_exposure', 'Total market exposure')
profit_loss = Gauge('trading_profit_loss', 'Current P&L', ['strategy'])

class MetricsCollector:
    def __init__(self, port=8000):
        # Start metrics server
        start_http_server(port)
        
    def record_order(self, strategy, side, latency):
        orders_placed.labels(strategy=strategy, side=side).inc()
        order_latency.observe(latency)
        
    def update_positions(self, count):
        active_positions.set(count)
        
    def update_exposure(self, exposure):
        total_exposure.set(exposure)
        
    def update_pnl(self, strategy, pnl):
        profit_loss.labels(strategy=strategy).set(pnl)

# Prometheus configuration
# prometheus.yml
prometheus_config = """
global:
  scrape_interval: 15s

scrape_configs:
  - job_name: 'trading-bot'
    static_configs:
      - targets: ['localhost:8000']
"""
```

--------------------------------

### Create Test Market Book (Python)

Source: https://github.com/faxulous/supersecretbetcodehistoryformcp/blob/master/mcp-ultimate/expert-snippets.md

Generates a mock MarketBook object for testing purposes. Useful for simulating market data without live API calls. Requires 'betfairlightweight' and 'unittest.mock'.

```python
def create_test_market_book(num_runners=3, total_matched=10000):
    """Create realistic market book for testing"""
    
    from betfairlightweight.resources import MarketBook
    from unittest.mock import Mock
    
    market_book = Mock(spec=MarketBook)
    market_book.market_id = "1.123456"
    market_book.status = "OPEN"
    market_book.inplay = False
    market_book.total_matched = total_matched
    market_book.total_available = total_matched * 0.1
    
    # Create runners
    runners = []
    for i in range(num_runners):
        runner = Mock()
        runner.selection_id = 12345 + i
        runner.status = "ACTIVE"
        runner.total_matched = total_matched / num_runners
        runner.last_price_traded = 2.0 + (i * 0.5)
        
        # Add order book
        runner.ex = Mock()
        runner.ex.available_to_back = [
            {'price': 2.0 + (i * 0.5), 'size': 100},
            {'price': 1.98 + (i * 0.5), 'size': 200},
            {'price': 1.96 + (i * 0.5), 'size': 300}
        ]
        runner.ex.available_to_lay = [
            {'price': 2.02 + (i * 0.5), 'size': 100},
            {'price': 2.04 + (i * 0.5), 'size': 200},
            {'price': 2.06 + (i * 0.5), 'size': 300}
        ]
        
        runners.append(runner)
        
    market_book.runners = runners
    
    return market_book
```

--------------------------------

### Minimal Data Requirements for Streaming

Source: https://github.com/faxulous/supersecretbetcodehistoryformcp/blob/master/mcp-ultimate/performance-secrets.md

Configure market and price filters to request only necessary data, reducing bandwidth and processing load.

```python
# Only request what you need
market_filter = {
    'marketIds': market_ids,
    'ladderLevels': 3  # Only top 3 price levels
}

price_filter = {
    'priceData': ['EX_BEST_OFFERS'],  # Not full depth
    'virtualise': False,  # Reduce processing
    'rollupModel': 'STAKE'  # Minimal rollup
}
```

--------------------------------

### Upload Market Data with Compression

Source: https://github.com/faxulous/supersecretbetcodehistoryformcp/blob/master/mcp-ultimate/infrastructure-complete.md

Uploads market data to S3, optionally compressing it. Uses smart_open for direct S3 streaming and gzip for compression. Ensure `smart_open` and `gzip` are imported.

```python
def upload_market_data(self, market_id, data, compress=True):
    """Upload market data with compression"""
    
    timestamp = datetime.now().strftime('%Y%m%d_%H%M%S')
    date_prefix = datetime.now().strftime('%Y/%m/%d')
    
    if compress:
        # Compress data
        filename = f"{market_id}_{timestamp}.json.gz"
        s3_key = f"market_data/{date_prefix}/{filename}"
        
        # Use smart_open for direct S3 streaming
        with smart_open(f"s3://{self.bucket}/{s3_key}", 'wb') as f:
            compressed = gzip.compress(json.dumps(data).encode())
            f.write(compressed)
    else:
        filename = f"{market_id}_{timestamp}.json"
        s3_key = f"market_data/{date_prefix}/{filename}"
        
        with smart_open(f"s3://{self.bucket}/{s3_key}", 'w') as f:
            json.dump(data, f)
            
    logger.debug(f"Uploaded {s3_key}")
    return s3_key
```

--------------------------------

### Place Smart Order with Python

Source: https://github.com/faxulous/supersecretbetcodehistoryformcp/blob/master/mcp-ultimate/expert-snippets.md

Place orders with intelligent pricing based on predefined strategies like AGGRESSIVE, PASSIVE, or MIDPOINT. Ensure the market object and runner details are correctly initialized.

```python
def place_smart_order(market, selection_id, side, target_price, size, strategy="PASSIVE"):
    """Place order with smart pricing"""
    
    runner = next((r for r in market.market_book.runners 
                   if r.selection_id == selection_id), None)
    
    if not runner:
        logger.error(f"Runner {selection_id} not found")
        return None
        
    # Determine actual price based on strategy
    if strategy == "AGGRESSIVE":
        # Cross the spread
        if side == "BACK" and runner.ex.available_to_lay:
            price = runner.ex.available_to_lay[0]['price']
        elif side == "LAY" and runner.ex.available_to_back:
            price = runner.ex.available_to_back[0]['price']
        else:
            price = target_price
            
    elif strategy == "PASSIVE":
        # Join the queue
        if side == "BACK" and runner.ex.available_to_back:
            price = runner.ex.available_to_back[0]['price']
        elif side == "LAY" and runner.ex.available_to_lay:
            price = runner.ex.available_to_lay[0]['price']
        else:
            price = target_price
            
    elif strategy == "MIDPOINT":
        # Place at midpoint
        if runner.ex.available_to_back and runner.ex.available_to_lay:
            back_price = runner.ex.available_to_back[0]['price']
            lay_price = runner.ex.available_to_lay[0]['price']
            price = round((back_price + lay_price) / 2, 2)
        else:
            price = target_price
    else:
        price = target_price
        
    # Create and place order
    with market.transaction() as t:
        order = market.create_order(
            side=side,
            selection_id=selection_id,
            price=price,
            size=size
        )
        t.place_order(order)
        
    logger.info(f"Placed {side} order: {selection_id} @ {price} x {size}")
    return order
```

--------------------------------

### Minimal Data Subscription

Source: https://github.com/faxulous/supersecretbetcodehistoryformcp/blob/master/mcp-ultimate/api-complete-guide.md

Explains how to create a streaming subscription with minimal data to optimize performance.

```APIDOC
## Minimal Data Subscription

### Description
This function demonstrates how to create a streaming subscription with a minimal data filter to enhance performance. It configures `conflate_ms` for frequency control and specifies a `marketDataFilter` to limit the fields received.

### Method
`client.streaming.create_stream`

### Parameters
- `conflate_ms` (integer) - Required - The conflation rate in milliseconds (e.g., 500 for medium frequency).
- `market_filter` (dict) - Required - A dictionary specifying filters for the market stream, including `marketIds`, `ladderLevels`, and `marketDataFilter`.
  - `marketDataFilter.fields` (list of strings) - Specifies the data fields to subscribe to (e.g., `EX_BEST_OFFERS_DISP`, `EX_TRADED`).
  - `marketDataFilter.ladderLevels` (integer) - Specifies the number of ladder levels to include.

### Request Example (Python)
```python
market_filter = {
    'marketIds': ['1.123456'],
    'ladderLevels': 3,  # Only top 3 levels
    'marketDataFilter': {
        'fields': ['EX_BEST_OFFERS_DISP', 'EX_TRADED'],
        'ladderLevels': 3
    }
}

stream = client.streaming.create_stream(
    conflate_ms=500,  # 500ms for medium frequency
    market_filter=market_filter
)
```
```

--------------------------------

### BetConnect API Client Initialization and Request Filtering

Source: https://github.com/faxulous/supersecretbetcodehistoryformcp/blob/master/mcp-ultimate/api-complete-guide.md

Demonstrates initializing the BetConnect API client and constructing a request filter for fetching bets. Note that 'accept_each_way' must be set to 1.

```python
from betconnect import APIClient as BCClient
from betconnect import resources

bc_client = BCClient(username, password)

# Correct parameter usage
request_filter = resources.GetBetRequestFilter(
    sport_id=14,
    accept_each_way=1  # Must be 1, not 0!
)

bets = bc_client.betting.get_bet_request(
    request_filter=request_filter
)
```

--------------------------------

### Upgrade Dependencies Incrementally with Pip

Source: https://github.com/faxulous/supersecretbetcodehistoryformcp/blob/master/mcp-ultimate/version-compatibility.md

Demonstrates a strategy for updating Python packages one by one to isolate and test compatibility issues.

```bash
# Don't update everything at once
pip install --upgrade betfairlightweight
# Test thoroughly
pip install --upgrade flumine  
# Test again
```

--------------------------------

### Create PostgreSQL Trading Tables

Source: https://github.com/faxulous/supersecretbetcodehistoryformcp/blob/master/mcp-ultimate/infrastructure-complete.md

Defines and creates essential tables for trading operations: market_data, orders, trades, and performance_metrics. Includes partitioning for market_data and indexes for efficient querying.

```sql
CREATE TABLE IF NOT EXISTS market_data (
                        market_id VARCHAR(20),
                        timestamp TIMESTAMPTZ,
                        runner_id BIGINT,
                        back_price DECIMAL(10,2),
                        back_size DECIMAL(12,2),
                        lay_price DECIMAL(10,2),
                        lay_size DECIMAL(12,2),
                        total_matched DECIMAL(14,2),
                        last_price_traded DECIMAL(10,2),
                        PRIMARY KEY (market_id, timestamp, runner_id)
                    ) PARTITION BY RANGE (timestamp);
                    
                    -- Create partitions for each month
                    CREATE TABLE IF NOT EXISTS market_data_2024_01
                    PARTITION OF market_data
                    FOR VALUES FROM ('2024-01-01') TO ('2024-02-01');
                    
                    -- Orders table
                    CREATE TABLE IF NOT EXISTS orders (
                        order_id VARCHAR(50) PRIMARY KEY,
                        market_id VARCHAR(20),
                        selection_id BIGINT,
                        strategy VARCHAR(50),
                        side VARCHAR(4),
                        price DECIMAL(10,2),
                        size DECIMAL(12,2),
                        status VARCHAR(20),
                        placed_at TIMESTAMPTZ,
                        matched_at TIMESTAMPTZ,
                        cancelled_at TIMESTAMPTZ,
                        size_matched DECIMAL(12,2),
                        size_remaining DECIMAL(12,2),
                        size_cancelled DECIMAL(12,2),
                        average_price_matched DECIMAL(10,2)
                    );
                    
                    CREATE INDEX IF NOT EXISTS idx_orders_market 
                    ON orders(market_id, placed_at);
                    
                    CREATE INDEX IF NOT EXISTS idx_orders_strategy 
                    ON orders(strategy, placed_at);
                    
                    -- Trades table
                    CREATE TABLE IF NOT EXISTS trades (
                        trade_id SERIAL PRIMARY KEY,
                        order_id VARCHAR(50),
                        market_id VARCHAR(20),
                        selection_id BIGINT,
                        strategy VARCHAR(50),
                        side VARCHAR(4),
                        price DECIMAL(10,2),
                        size DECIMAL(12,2),
                        executed_at TIMESTAMPTZ,
                        commission DECIMAL(10,2),
                        profit_loss DECIMAL(12,2),
                        running_total DECIMAL(14,2)
                    );
                    
                    CREATE INDEX IF NOT EXISTS idx_trades_strategy_time 
                    ON trades(strategy, executed_at);
                    
                    -- Performance metrics table
                    CREATE TABLE IF NOT EXISTS performance_metrics (
                        metric_date DATE,
                        strategy VARCHAR(50),
                        total_trades INTEGER,
                        winning_trades INTEGER,
                        total_pnl DECIMAL(14,2),
                        max_drawdown DECIMAL(10,4),
                        sharpe_ratio DECIMAL(10,4),
                        win_rate DECIMAL(5,4),
                        avg_win DECIMAL(12,2),
                        avg_loss DECIMAL(12,2),
                        PRIMARY KEY (metric_date, strategy)
                    );

```

--------------------------------

### Configure S3 Bucket Lifecycle Policy

Source: https://github.com/faxulous/supersecretbetcodehistoryformcp/blob/master/mcp-ultimate/infrastructure-complete.md

Sets up lifecycle rules for an S3 bucket, including expiration and storage class transitions. Ensure the S3 client is initialized and the bucket name is available.

```python
lifecycle_config = {
            'Rules': [
                {
                    'ID': 'MoveOldDataToGlacier', 
                    'Status': 'Enabled',
                    'Filter': {
                        'Prefix': 'market_data/'
                    },
                    'Transitions': [
                        {
                            'Days': 30,
                            'StorageClass': 'STANDARD_IA'
                        },
                        {
                            'Days': 90,
                            'StorageClass': 'GLACIER'
                        },
                        {
                            'Days': 365,
                            'StorageClass': 'DEEP_ARCHIVE'
                        }
                    ],
                    'Expiration': {
                        'Days': 730  # Delete after 2 years
                    }
                },
                {
                    'ID': 'DeleteTempFiles',
                    'Status': 'Enabled',
                    'Prefix': 'temp/',
                    'Expiration': {
                        'Days': 7
                    }
                },
                {
                    'ID': 'AbortIncompleteMultipart',
                    'Status': 'Enabled',
                    'AbortIncompleteMultipartUpload': {
                        'DaysAfterInitiation': 7
                    }
                }
            ]
        }
        
try:
    self.s3_client.put_bucket_lifecycle_configuration(
        Bucket=self.bucket,
        LifecycleConfiguration=lifecycle_config
    )
    logger.info(f"Lifecycle policy configured for {self.bucket}")
except Exception as e:
    logger.error(f"Failed to setup lifecycle: {e}")
```

--------------------------------

### S3 Storage Manager Initialization

Source: https://github.com/faxulous/supersecretbetcodehistoryformcp/blob/master/mcp-ultimate/infrastructure-complete.md

Initializes an S3 storage manager with a specified bucket and region, setting up a local cache and configuring lifecycle policies for cost optimization.

```python
import boto3
from smart_open import open as smart_open
import json
import gzip
from datetime import datetime, timedelta
from pathlib import Path
import logging

logger = logging.getLogger(__name__)

class S3StorageManager:
    """Production S3 storage with caching and lifecycle"""
    
    def __init__(self, bucket_name, region='eu-west-1'):
        self.bucket = bucket_name
        self.s3_client = boto3.client('s3', region_name=region)
        self.local_cache = Path('/tmp/s3_cache')
        self.local_cache.mkdir(exist_ok=True)
        
        # Setup lifecycle rules
        self.setup_lifecycle_policy()
        
    def setup_lifecycle_policy(self):
        """Configure S3 lifecycle for cost optimization"""
        
        lifecycle_config = {
            'Rules': [
                {
                    'ID': 'ArchiveOldMarketData',
                    'Status': 'Enabled',
                    'Prefix': 'market_data/',
                    'Transitions': [
                        {

```

--------------------------------

### Configure S3 Lifecycle for Cost Optimization

Source: https://github.com/faxulous/supersecretbetcodehistoryformcp/blob/master/mcp-ultimate/data-management.md

Set up S3 lifecycle rules to automatically transition objects to cheaper storage classes like Standard-IA and Glacier based on age. This helps manage storage costs effectively.

```python
def setup_s3_lifecycle(bucket_name):
    """Configure S3 lifecycle for cost optimization"""
    
    s3_client = boto3.client('s3')
    
    lifecycle_config = {
        'Rules': [
            {
                'ID': 'ArchiveOldData',
                'Status': 'Enabled',
                'Transitions': [
                    {
                        'Days': 30,
                        'StorageClass': 'STANDARD_IA'  # Infrequent Access
                    },
                    {
                        'Days': 90,
                        'StorageClass': 'GLACIER'  # Archive
                    }
                ],
                'NoncurrentVersionTransitions': [
                    {
                        'NoncurrentDays': 7,
                        'StorageClass': 'GLACIER'
                    }
                ]
            }
        ]
    }
    
    s3_client.put_bucket_lifecycle_configuration(
        Bucket=bucket_name,
        LifecycleConfiguration=lifecycle_config
    )
    
    logger.info(f"Lifecycle policy configured for {bucket_name}")
```

--------------------------------

### Enable Detailed Logging in Python

Source: https://github.com/faxulous/supersecretbetcodehistoryformcp/blob/master/mcp-ultimate/error-solutions-complete.md

Configure Python's logging module to capture detailed debug information, specifically for the 'betfairlightweight' library. This is useful for diagnosing issues by seeing all API calls.

```python
import logging

logging.basicConfig(
    level=logging.DEBUG,
    format='%(asctime)s - %(name)s - %(levelname)s - %(message)s'
)

# Log all API calls
logging.getLogger('betfairlightweight').setLevel(logging.DEBUG)
```