### Console Output on Startup Source: https://github.com/krish567366/submicro-execution-engine/blob/main/docs/MONITORING_SUMMARY.md Example console output displayed when the HFT system starts up, indicating the dashboard server is initializing and ready. ```text [INIT] Real-Time Dashboard Server (http://localhost:8080) Dashboard server started on port 8080 Open http://localhost:8080 in your browser ``` -------------------------------- ### Install and Start PTP Daemon Source: https://github.com/krish567366/submicro-execution-engine/blob/main/docs/ARCHITECTURE.md Installs the linuxptp package and starts the PTP daemon to synchronize system time with a PTP grandmaster, essential for accurate timestamping. ```bash # Install linuxptp sudo apt-get install linuxptp # Start PTP daemon sudo ptp4l -i eth0 -m -s # Check sync status pmc -u -b 0 'GET TIME_STATUS_NP' ``` -------------------------------- ### Install QuickFIX Library Source: https://github.com/krish567366/submicro-execution-engine/blob/main/docs/BROKER_INTEGRATION.md Instructions for installing the QuickFIX C++ library on macOS, Ubuntu, or from source. Ensure the library is correctly installed before proceeding with FIX integration. ```bash # macOS brew install quickfix # Ubuntu sudo apt-get install libquickfix-dev # From source git clone https://github.com/quickfix/quickfix.git cd quickfix ./configure make sudo make install ``` -------------------------------- ### Install Dependencies for Binance Integration Source: https://github.com/krish567366/submicro-execution-engine/blob/main/docs/BROKER_INTEGRATION.md Installs necessary system dependencies for broker integration. Ensure these are installed before proceeding. ```bash brew install boost openssl curl ``` -------------------------------- ### Install Dependencies Source: https://github.com/krish567366/submicro-execution-engine/blob/main/docs/QUICKSTART_INTEGRATED.md Installs necessary build tools and libraries for compiling the HFT system. Ensure you have CMake, g++, and libnuma-dev installed. ```bash sudo apt-get update sudo apt-get install -y cmake g++ libnuma-dev # Optional: For hardware features sudo apt-get install -y linux-tools-generic # perf tools ``` -------------------------------- ### Deploy with Netlify CLI Source: https://github.com/krish567366/submicro-execution-engine/blob/main/web/DEPLOYMENT_GUIDE.md Install the Netlify CLI and use it for production deployments. ```bash cd web npm install -g netlify-cli netlify deploy --prod ``` -------------------------------- ### Clone and Build the Repository Source: https://github.com/krish567366/submicro-execution-engine/blob/main/README.md Clone the repository and build the system with automatic optimization flags. This is the first step to get the system running. ```bash # 1. Clone the repository git clone https://github.com/krish567366/submicro-execution-engine.git cd submicro-execution-engine # 2. Build the system (automatic optimization flags) ./scripts/build_all.sh ``` -------------------------------- ### Initialize and Start Dashboard Server Source: https://github.com/krish567366/submicro-execution-engine/blob/main/docs/MONITORING_SUMMARY.md Initializes the MetricsCollector and DashboardServer before the trading loop begins. Starts the dashboard server to broadcast metrics. ```cpp // Step 1: Initialize (before trading loop) MetricsCollector metrics_collector(10000); DashboardServer dashboard(metrics_collector, 8080); dashboard.start(); ``` -------------------------------- ### Deploy with Vercel CLI Source: https://github.com/krish567366/submicro-execution-engine/blob/main/web/DEPLOYMENT_GUIDE.md Install the Vercel CLI and use it to deploy the project. This is presented as the easiest option. ```bash cd web npm install -g vercel vercel ``` -------------------------------- ### Instant Setup for HFT System Source: https://github.com/krish567366/submicro-execution-engine/blob/main/dashboard/README.md Build and run the HFT system to access the real-time dashboard. Ensure the system is running and accessible via the specified URL. ```bash # 1. Build the system (dashboard auto-included) ./build.sh # 2. Run the HFT system ./build/hft_system # 3. Open browser to: http://localhost:8080 ``` -------------------------------- ### Multi-Process Setup for Viewers Source: https://github.com/krish567366/submicro-execution-engine/blob/main/docs/MONITORING_SUMMARY.md Demonstrates setting up multiple viewer terminals that connect to the running trading engine. Each viewer establishes an independent WebSocket connection to the dashboard. ```bash # Terminal 1: Trading Engine ./build/hft_system # Terminal 2+: Multiple Viewers # Open dashboard in multiple browsers # Each gets independent WebSocket connection ``` -------------------------------- ### Build and Run C++ System Source: https://github.com/krish567366/submicro-execution-engine/blob/main/docs/SYSTEM_FEATURES.md Execute the full C++ build process and start the trading system. ```bash ./build.sh ./run.sh ``` -------------------------------- ### Start HFT System and Dashboard Source: https://github.com/krish567366/submicro-execution-engine/blob/main/docs/DASHBOARD_GUIDE.md Execute the HFT system binary. The dashboard will automatically start on port 8080, providing a web interface for monitoring. ```bash ./build/hft_system ``` -------------------------------- ### Log Bundle Structure Example Source: https://github.com/krish567366/submicro-execution-engine/blob/main/docs/PUBLICATION_READINESS.md Illustrates the directory structure and file names for the logging system, including hardware timestamps, strategy events, exchange acknowledgments, and PTP synchronization logs. ```text logs/ ├── nic_rx_tx_hw_ts.log # Layer 1: Hardware timestamps ├── strategy_trace.log # Layer 2: TSC events ├── exchange_ack.log # Layer 3: External truth ├── ptp_sync.log # Layer 4: Clock sync ├── order_gateway.log # Layer 5: Order boundary ├── MANIFEST.sha256 # Cryptographic integrity └── README.md # Verification guide ``` -------------------------------- ### Build and Run HFT System for Monitoring Source: https://github.com/krish567366/submicro-execution-engine/blob/main/docs/QUICKSTART_MONITORING.md These commands are used to build the system and start the HFT application, which enables live monitoring. Access the monitoring dashboard via http://localhost:8080. ```bash ./build.sh ``` ```bash ./build/hft_system ``` -------------------------------- ### Initialize ModelStore with JSON Configuration Source: https://github.com/krish567366/submicro-execution-engine/blob/main/docs/PRODUCTION_READINESS.md Initializes the ModelStore using a local JSON file for development environments. This is a simple setup for local testing and configuration. ```cpp ModelStore model_store("./config/parameters.json"); ``` -------------------------------- ### Install Linux Headers for Kernel Modules Source: https://github.com/krish567366/submicro-execution-engine/blob/main/docs/BENCHMARK_GUIDE.md Install the appropriate Linux kernel headers if build errors indicate missing headers required for kernel module compilation. ```bash # Missing headers sudo apt-get install linux-headers-$(uname -r) ``` -------------------------------- ### Hardware-in-the-Loop Bridge Production Path Source: https://github.com/krish567366/submicro-execution-engine/blob/main/docs/SYSTEM_FEATURES.md Example of initializing and using the HardwareInTheLoopBridge in production mode, routing predictions to the FPGA. ```cpp HardwareInTheLoopBridge bridge(AcceleratorMode::HARDWARE_FPGA); auto predictions = bridge.predict(features); // Routes to FPGA automatically ``` -------------------------------- ### Decision Latency Example Flow Source: https://github.com/krish567366/submicro-execution-engine/blob/main/docs/LATENCY_ANALYSIS.md Illustrates the timeline from market data arrival to a trading decision, highlighting the target of under 500 ns for internal computation. ```text t=0 ns → Packet arrives in DMA buffer t=50 ns → Parse market data (bid: $100.00, ask: $100.02) t=150 ns → Update order book with AVX-512 SIMD t=250 ns → Calculate Kalman filter alpha signal: +0.35 t=400 ns → Evaluate 64 strategies in parallel (superposition) t=450 ns → Check risk limits (TSX lock-free) t=500 ns → DECISION: BUY 100 shares @ $100.02 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ DECISION LATENCY = 500 ns ``` -------------------------------- ### Tick-to-Tick Latency Example Source: https://github.com/krish567366/submicro-execution-engine/blob/main/docs/LATENCY_ANALYSIS.md Demonstrates the processing time for consecutive market data updates, illustrating the target of under 1,000 ns for high throughput capacity. ```text Tick 1: t=0 ns → Process AAPL bid update t=500 ns → Decision complete t=600 ns → State reset, ready for next tick Tick 2: t=600 ns → Process AAPL ask update t=1,100 ns → Decision complete ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ TICK-TO-TICK LATENCY = 600 ns ``` -------------------------------- ### Run Backtest Script Source: https://github.com/krish567366/submicro-execution-engine/blob/main/README.md Execute the backtesting script to run simulations. This is a common starting point for testing strategy performance. ```bash scripts/run_backtest.py ``` -------------------------------- ### Zero-Copy Protocol Decoder Example Source: https://github.com/krish567366/submicro-execution-engine/blob/main/docs/SYSTEM_FEATURES.md Demonstrates direct memory access for parsing order book updates using the ZeroCopyDecoder, bypassing intermediate data copies. ```cpp const auto* msg = ZeroCopyDecoder::parse_order_book_update(ring_buffer_ptr); double price = msg->price; // Direct memory access, no copy uint64_t order_id = msg->order_id; ``` -------------------------------- ### Run Multiple Dashboard Instances Source: https://github.com/krish567366/submicro-execution-engine/blob/main/docs/DASHBOARD_GUIDE.md Instantiate and start multiple DashboardServer objects on different ports to run concurrent dashboard instances. ```cpp DashboardServer dashboard_main(metrics_collector, 8080); DashboardServer dashboard_backup(metrics_collector, 8081); dashboard_main.start(); dashboard_backup.start(); ``` -------------------------------- ### Live Trading Output Example Source: https://github.com/krish567366/submicro-execution-engine/blob/main/docs/QUICKSTART_INTEGRATED.md Displays key trading metrics updated every 1000 cycles, including mid-price, position, spread, advanced financial indicators like Lyapunov Exponent and Reynolds Number, and system performance metrics such as cycle latency and NIC utilization. ```text --- Cycle: 1000 --- Mid Price: $100.05 Position: 0 Active Quotes: Bid=100.04 Ask=100.06 Spread=2.00 bps Hawkes: Buy=10.250 Sell=10.180 Imbalance=0.035 Lyapunov Exponent: 0.0023 (NEUTRAL) Reynolds Number: 45.3 (LAMINAR - Smooth Execution) Nash L1 Probability: 67.45% Toxicity Score: 0.234 Micro Price: $100.0485 (vs Mid: $100.05) Kalman Alpha: 0.0145 ±0.0231 Spiking Network Firing: 12 / 256 neurons Superposition Strategies Triggered: 38 / 64 Grid Ladder Active Levels: 20 Regime: NORMAL (multiplier=1.00) Last Cycle Latency: 850 ns (0.85 µs) NIC Queue Utilization: 12.5% ``` -------------------------------- ### Initialize and Connect to Binance WebSocket and REST API Source: https://github.com/krish567366/submicro-execution-engine/blob/main/docs/BROKER_INTEGRATION.md This C++ snippet demonstrates initializing a Binance WebSocket feed for market data and a REST Order Gateway for placing orders. It starts the market data feed in a separate thread and includes a basic trading loop for order submission. ```cpp #include "ws_market_data.hpp" #include "rest_order_gateway.hpp" int main() { // Initialize broker connections BinanceWebSocketFeed market_feed("btcusdt", market_data_queue); market_feed.connect(); BinanceOrderGateway order_gateway( std::getenv("BINANCE_API_KEY"), std::getenv("BINANCE_API_SECRET") ); // Start market data feed in separate thread std::thread feed_thread([&market_feed]() { market_feed.start_reading(); }); // Main trading loop while (running) { MarketTick tick; if (market_data_queue.try_pop(tick)) { // [Existing HFT logic: Hawkes, FPGA, Quotes, Risk] // Send orders via broker if (should_send_order) { Order order = construct_order(quotes); bool success = order_gateway.send_order(order); if (!success) { std::cerr << "Order rejected by broker!" << std::endl; } } } } feed_thread.join(); return 0; } ``` -------------------------------- ### Linux Production Build Dependencies and Configuration Source: https://github.com/krish567366/submicro-execution-engine/blob/main/BUILD_REQUIREMENTS.md Installs necessary build tools and libraries, then configures the build using CMake with Release type and C++11 compiler. Ensure you are in the project's root directory. ```bash # 1. Install dependencies sudo apt-get update sudo apt-get install -y \ build-essential \ cmake \ g++-11 \ libboost-all-dev \ libnlohmann-json3-dev \ libssl-dev # 2. Configure build cd /path/to/new-trading-system mkdir -p build && cd build cmake .. \ -DCMAKE_BUILD_TYPE=Release \ -DCMAKE_CXX_COMPILER=g++-11 \ -DCMAKE_INTERPROCEDURAL_OPTIMIZATION=ON ``` -------------------------------- ### FPGA Hardware Validation Setup Source: https://github.com/krish567366/submicro-execution-engine/blob/main/docs/PRODUCTION_READINESS.md Configures and validates FPGA hardware by opening device files, memory-mapping I/O registers and DMA buffers, and asserting latency constraints. ```cpp // Connect to FPGA test card int fpga_fd = open("/dev/xdma0_user", O_RDWR); void* control_regs = mmap(BAR0_ADDRESS, ...); void* dma_buffer = mmap(BAR2_ADDRESS, ...); // Verify latency matches software stub assert(hardware_latency < 400ns); ``` -------------------------------- ### Probabilistic Fill Model Example Source: https://github.com/krish567366/submicro-execution-engine/blob/main/TECHNICAL_FEEDBACK.md Illustrates the current system's approach using a probabilistic fill model, where order execution depends on a calculated probability rather than precise simulation. This is less common in HFT development. ```cpp // Probabilistic fill model double fill_probability = calculate_fill_probability(order, market_impact); if (random() < fill_probability) { execute_order(order); } ``` -------------------------------- ### Core Affinity Configuration Source: https://github.com/krish567366/submicro-execution-engine/blob/main/docs/ARCHITECTURE.md Illustrates a typical core affinity setup for the execution engine, dedicating specific cores for trading, logging, and background tasks to ensure isolation and performance. ```text Core 0-5: OS, background tasks Core 6: Trading thread (isolated) Core 7: Logging thread Core 8-27: Available for expansion ``` -------------------------------- ### Complete Latency Timeline Example Source: https://github.com/krish567366/submicro-execution-engine/blob/main/docs/LATENCY_ANALYSIS.md Illustrates the end-to-end latency of a trading scenario, from order placement to fill acknowledgment, detailing events across exchange, network, and server components. ```text TIMESTAMP EVENT CUMULATIVE ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ EXCHANGE SIDE: t=0 μs Order placed on exchange 0 μs t=5 μs Matching engine processes 5 μs t=10 μs Market data update generated 10 μs t=15 μs Packet leaves exchange NIC 15 μs NETWORK: t=15-40 μs Packet travels through fiber (co-located) 25 μs YOUR SERVER: t=40.000 μs Packet arrives at your NIC 40.000 μs t=40.050 μs DMA to memory (NIC → RAM) 40.050 μs ┌────────────────────────────────────┐ │ DECISION LATENCY STARTS HERE │ └────────────────────────────────────┘ t=40.100 μs Parse market data packet 40.100 μs t=40.200 μs Update order book (AVX-512) 40.200 μs t=40.350 μs Calculate alpha signals 40.350 μs t=40.400 μs Evaluate 64 strategies parallel 40.400 μs t=40.450 μs Risk check (TSX lock-free) 40.450 μs t=40.500 μs DECISION: BUY 100 @ $100.02 40.500 μs ┌────────────────────────────────────┐ │ DECISION LATENCY = 500 ns │ └────────────────────────────────────┘ t=40.505 μs Serialize order (preserialized) 40.505 μs t=40.540 μs DMA to NIC TX ring 40.540 μs t=40.580 μs First byte on wire 40.580 μs ┌────────────────────────────────────┐ │ TICK-TO-TRADE = 580 ns │ └────────────────────────────────────┘ NETWORK: t=40.58-65 μs Packet travels back to exchange 25 μs EXCHANGE SIDE: t=65 μs Order arrives at exchange 65 μs t=70 μs Matching engine processes 70 μs t=75 μs Order filled (if liquidity available) 75 μs t=80 μs Fill acknowledgment sent 80 μs YOUR SERVER: t=105 μs Fill acknowledgment received 105 μs ┌────────────────────────────────────┐ │ ROUND-TRIP TIME = 105 μs │ └────────────────────────────────────┘ ``` -------------------------------- ### Binance Order Gateway Implementation Source: https://github.com/krish567366/submicro-execution-engine/blob/main/docs/BROKER_INTEGRATION.md Implements a C++ class for sending orders to the Binance API. It handles API key management, request signing using HMAC-SHA256, and HTTP POST requests via libcurl. Includes setup and cleanup for curl. ```cpp #ifndef REST_ORDER_GATEWAY_HPP #define REST_ORDER_GATEWAY_HPP #include "common_types.hpp" #include #include #include class BinanceOrderGateway { public: BinanceOrderGateway( const std::string& api_key, const std::string& api_secret ) : api_key_(api_key), api_secret_(api_secret) { curl_global_init(CURL_GLOBAL_DEFAULT); } ~BinanceOrderGateway() { curl_global_cleanup(); } bool send_order(const Order& order) { // Build order parameters std::string params = build_order_params(order); // Sign request std::string signature = hmac_sha256(params, api_secret_); params += "&signature=" + signature; // Send HTTP POST CURL* curl = curl_easy_init(); if (!curl) return false; std::string url = "https://api.binance.com/api/v3/order?" + params; struct curl_slist* headers = nullptr; headers = curl_slist_append(headers, ("X-MBX-APIKEY: " + api_key_).c_str()); curl_easy_setopt(curl, CURLOPT_URL, url.c_str()); curl_easy_setopt(curl, CURLOPT_HTTPHEADER, headers); curl_easy_setopt(curl, CURLOPT_POST, 1L); std::string response; curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, write_callback); curl_easy_setopt(curl, CURLOPT_WRITEDATA, &response); CURLcode res = curl_easy_perform(curl); curl_easy_cleanup(curl); curl_slist_free_all(headers); if (res != CURLE_OK) { std::cerr << "Order send failed: " << curl_easy_strerror(res) << std::endl; return false; } // Parse response auto j = json::parse(response); if (j.contains("orderId")) { std::cout << "Order placed: " << j["orderId"] << std::endl; return true; } return false; } private: std::string build_order_params(const Order& order) { auto now_ms = std::chrono::duration_cast( std::chrono::system_clock::now().time_since_epoch() ).count(); std::stringstream ss; ss << "symbol=" << get_symbol_string(order.symbol_id) << "&side=" << (order.side == Side::BUY ? "BUY" : "SELL") << "&type=LIMIT" << "&timeInForce=GTC" << "&quantity=" << std::fixed << std::setprecision(8) << (order.quantity / 1000.0) << "&price=" << std::fixed << std::setprecision(2) << order.price << "×tamp=" << now_ms; return ss.str(); } std::string hmac_sha256(const std::string& data, const std::string& key) { unsigned char hash[32]; HMAC(EVP_sha256(), key.c_str(), key.length(), reinterpret_cast(data.c_str()), data.length(), hash, nullptr); std::stringstream ss; for (int i = 0; i < 32; ++i) { ss << std::hex << std::setw(2) << std::setfill('0') << (int)hash[i]; } return ss.str(); } static size_t write_callback(void* contents, size_t size, size_t nmemb, std::string* s) { size_t new_length = size * nmemb; s->append((char*)contents, new_length); return new_length; } std::string get_symbol_string(int symbol_id) { // Map internal ID to exchange symbol if (symbol_id == 1) return "BTCUSDT"; if (symbol_id == 2) return "ETHUSDT"; return "BTCUSDT"; } std::string api_key_; std::string api_secret_; }; #endif // REST_ORDER_GATEWAY_HPP ``` -------------------------------- ### Configure ESLint with React and React DOM Plugins Source: https://github.com/krish567366/submicro-execution-engine/blob/main/web/README.md Integrate eslint-plugin-react-x and eslint-plugin-react-dom for React-specific linting rules. This setup requires the plugins to be installed. ```javascript // eslint.config.js import reactX from 'eslint-plugin-react-x' import reactDom from 'eslint-plugin-react-dom' export default defineConfig([ globalIgnores(['dist']), { files: ['**/*.{ts,tsx}'] extends: [ // Other configs... // Enable lint rules for React reactX.configs['recommended-typescript'], // Enable lint rules for React DOM reactDom.configs.recommended, ], languageOptions: { parserOptions: { project: ['./tsconfig.node.json', './tsconfig.app.json'], tsconfigRootDir: import.meta.dirname, }, // other options... }, }, ]) ``` -------------------------------- ### FPGA Hardware Initialization Steps Source: https://github.com/krish567366/submicro-execution-engine/blob/main/docs/CODE_VERIFICATION.md Outlines the essential steps for initializing the FPGA hardware in a production environment, including detection, memory mapping, bitstream loading, and configuration. ```cpp // PRODUCTION INTEGRATION STEPS: // 1. DETECT FPGA CARD (PCIe scan, vendor ID verification) // 2. MAP MEMORY REGIONS (mmap PCIe BARs, allocate DMA buffers) // 3. LOAD BITSTREAM (program FPGA with inference accelerator) // 4. CONFIGURE INFERENCE ENGINE (load weights, set parameters) // 5. HEALTH CHECK (run dummy inference, validate latency) // Example production code structure: // int fpga_fd = open("/dev/xdma0_user", O_RDWR | O_SYNC); // void* control_regs = mmap(BAR0_ADDRESS, ...); // void* dma_buffer = mmap(BAR2_ADDRESS, ...); ``` -------------------------------- ### Create New Repository and Clone Source: https://github.com/krish567366/submicro-execution-engine/blob/main/docs/NIC_DRIVER_REPO_PLAN.md Steps to create a new GitHub repository for the NIC drivers and clone it locally. ```bash # On GitHub: Create new repo "ultra-low-latency-nic-drivers" git clone https://github.com/krish567366/ultra-low-latency-nic-drivers cd ultra-low-latency-nic-drivers ``` -------------------------------- ### Manual Deployment Steps Source: https://github.com/krish567366/submicro-execution-engine/blob/main/web/DEPLOYMENT_GUIDE.md Build the project for production and manually upload the generated files to your subdomain using SCP. ```bash cd web npm run build # The dist/ folder contains production-ready files # Upload to your subdomain: scp -r dist/* user@server:/var/www/your-subdomain.com/ ``` -------------------------------- ### CSV Data Format Example Source: https://github.com/krish567366/submicro-execution-engine/blob/main/docs/DASHBOARD_GUIDE.md This is an example of the CSV format used for exporting trading metrics. ```csv timestamp_ns,mid_price,spread_bps,pnl,position,buy_intensity,sell_intensity,latency_us,orders_sent,orders_filled,regime,position_limit_usage 1733788800000000000,100.50,2.5,125.50,250,12.3,10.8,0.847,1234,1189,0,25.0 ... ``` -------------------------------- ### Configure and Run Backtest Engine Source: https://github.com/krish567366/submicro-execution-engine/blob/main/docs/BACKTESTING_SUMMARY.md Demonstrates how to configure the backtesting engine with specific parameters, load historical data, and execute a backtest. It also shows how to run latency sensitivity analysis. ```cpp // Configure backtest BacktestingEngine::Config config; config.simulated_latency_ns = 500; config.initial_capital = 100000.0; config.commission_per_share = 0.0005; config.enable_adverse_selection = true; config.random_seed = 42; // Reproducibility // Create engine BacktestingEngine engine(config); // Load historical data engine.load_historical_data("market_data.csv"); // Run backtest auto metrics = engine.run_backtest(); // Print results metrics.print_summary(); // Run latency sensitivity auto latency_results = engine.run_latency_sensitivity_analysis(); ``` -------------------------------- ### Install Real-Time Kernel Source: https://github.com/krish567366/submicro-execution-engine/blob/main/docs/ARCHITECTURE.md Installs a real-time or low-latency kernel for improved scheduling and reduced jitter. Choose the appropriate package manager command for your distribution. ```bash # RHEL/CentOS sudo yum install kernel-rt ``` ```bash # Ubuntu sudo apt-get install linux-lowlatency ``` -------------------------------- ### Chart Update Method Source: https://github.com/krish567366/submicro-execution-engine/blob/main/docs/DASHBOARD_VISUAL.md JavaScript example for updating charts without animation for performance. ```javascript chart.update('none'); // No animation, instant update // Smooth at 60 FPS due to requestAnimationFrame ``` -------------------------------- ### Dashboard UI Example Source: https://github.com/krish567366/submicro-execution-engine/blob/main/docs/QUICKSTART_MONITORING.md A visual representation of the dashboard's key metrics and status indicators. ```text ┌─────────────────────────────────────────┐ │ P&L: $245 Position: 450 │ │ Price: $100 Latency: 847μs │ │ [Green Charts Updating Smoothly] │ │ ● Connected NORMAL (1.0×) │ └─────────────────────────────────────────┘ ``` -------------------------------- ### Build for Production Source: https://github.com/krish567366/submicro-execution-engine/blob/main/web/DEPLOYMENT_GUIDE.md Run this command to create an optimized build of your landing page for deployment. ```bash npm run build ``` -------------------------------- ### README.md Content Source: https://github.com/krish567366/submicro-execution-engine/blob/main/docs/NIC_DRIVER_REPO_PLAN.md Content for the main README.md file of the new repository, outlining features and quick start. ```markdown # Ultra-Low-Latency NIC Drivers Zero-abstraction network drivers for high-frequency trading achieving 20-50ns packet receive latency. ## Features - Direct memory-mapped hardware access - Zero-copy packet processing - Lock-free ring buffers - NUMA-aware memory allocation - Hardware timestamp support - Multi-queue RSS support ## Quick Start [Installation, Basic Usage, Examples] ## Performance [Benchmarks, Comparisons, Tuning Guide] ## Supported Hardware [List of tested NICs] ``` -------------------------------- ### Automated Daily Benchmarks with Cron Source: https://github.com/krish567366/submicro-execution-engine/blob/main/docs/BENCHMARK_GUIDE.md Set up a cron job to automatically run daily benchmarks at a specified time. Output is logged to a file. ```bash # Edit crontab crontab -e # Add daily benchmark at 6 AM 0 6 * * * cd ~/trading-system && sudo ./build/hft_benchmark \ --samples 10000000 --output daily_$(date +%Y%m%d) \ >> benchmark_log.txt 2>&1 ``` -------------------------------- ### Determinism Verification Output Source: https://github.com/krish567366/submicro-execution-engine/blob/main/docs/BACKTESTING_SUMMARY.md Example output demonstrating the verification of deterministic results across multiple backtesting runs. ```text Run #1 P&L: $0.000000 | Sharpe: 0.000000 Run #2 P&L: $0.000000 | Sharpe: 0.000000 Run #3 P&L: $0.000000 | Sharpe: 0.000000 DETERMINISM VERIFIED: All runs produced identical results! ``` -------------------------------- ### Deploy Using Script Source: https://github.com/krish567366/submicro-execution-engine/blob/main/web/DEPLOYMENT_GUIDE.md Execute a provided deployment script to deploy the application to a specified subdomain. ```bash cd web ./deploy.sh your-subdomain.yourcompany.com ``` -------------------------------- ### C++ Minimal Overhead Timing Source: https://github.com/krish567366/submicro-execution-engine/blob/main/docs/ARCHITECTURE.md Provides an example of minimal overhead timing instrumentation for performance monitoring within critical sections. ```cpp // Minimal overhead timing const Timestamp t0 = now(); // ... critical section ... const int64_t latency_ns = to_nanos(now()) - to_nanos(t0); ``` -------------------------------- ### WebSocket Server Update Payload Source: https://github.com/krish567366/submicro-execution-engine/blob/main/docs/DASHBOARD_GUIDE.md Example JSON payload sent from the WebSocket server to clients, containing real-time trading metrics. ```json { "type": "update", "timestamp": 1733788800000000000, "mid_price": 100.50, "spread": 2.5, "pnl": 245.80, "position": 250, "buy_intensity": 12.3, "sell_intensity": 10.8, "latency": 847.2, "orders_sent": 1234, "orders_filled": 1189, "regime": 0, "position_usage": 25.0 } ``` -------------------------------- ### C++ Pre-allocation and Stack Usage Source: https://github.com/krish567366/submicro-execution-engine/blob/main/docs/ARCHITECTURE.md Demonstrates pre-allocating buffers at startup and using stack-allocated variables for temporaries in performance-critical paths to avoid dynamic allocation overhead. ```cpp // Pre-allocate all buffers at startup std::vector buffer_; buffer_.reserve(MAX_EVENTS); // One-time allocation // Use stack for temporaries in hot path QuotePair quotes; // Stack-allocated ``` -------------------------------- ### Run Quick Benchmark Test Source: https://github.com/krish567366/submicro-execution-engine/blob/main/docs/BENCHMARK_GUIDE.md Executes a quick benchmark test with 1 million samples, expected to take around 10 seconds. Results are saved to a file. ```bash sudo ./build/hft_benchmark --samples 1000000 --output test_results ``` -------------------------------- ### Build HFT System with Dashboard Support Source: https://github.com/krish567366/submicro-execution-engine/blob/main/docs/DASHBOARD_GUIDE.md Navigate to your HFT system directory and rebuild the project to include dashboard support. This step ensures the monitoring system is integrated. ```bash cd "/Users/krishnabajpai/code/research codes/new-trading-system" ./build.sh ``` -------------------------------- ### Tick-to-Trade Latency Example Flow Source: https://github.com/krish567366/submicro-execution-engine/blob/main/docs/LATENCY_ANALYSIS.md Details the end-to-end process from receiving a market tick to submitting an order to the network, aiming for under 1,000 ns. ```text t=0 ns → Market tick arrives (bid $100.00 → $100.01) t=500 ns → Decision made: BUY 100 @ $100.02 t=505 ns → Serialize order (preserialized template) t=540 ns → DMA to NIC TX ring t=580 ns → NIC transmits first byte to wire ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ TICK-TO-TRADE LATENCY = 580 ns ``` -------------------------------- ### Build and Run Local Development Server Source: https://github.com/krish567366/submicro-execution-engine/blob/main/docs/MONITORING_SUMMARY.md Builds the HFT system locally and runs the executable. The dashboard can then be accessed via a web browser. ```bash ./build.sh ./build/hft_system # Open http://localhost:8080 ``` -------------------------------- ### Tick-to-Trade Latency Queue Position Example Source: https://github.com/krish567366/submicro-execution-engine/blob/main/docs/LATENCY_ANALYSIS.md Demonstrates how tick-to-trade latency affects an order's position in the exchange queue, influencing fill probability. ```text Exchange Queue at Price $100.02: Position 1: Order arrived at t=40.58 μs (your system) Position 2: Order arrived at t=40.85 μs (competitor, 3 μs slower) Position 3: Order arrived at t=45.20 μs (slow competitor) Available liquidity: 100 shares Position 1 (you): Gets filled immediately ✓ Position 2: Gets filled if 200+ shares available Position 3: Unlikely to fill at this price ``` -------------------------------- ### Comparison Benchmark Goals Source: https://github.com/krish567366/submicro-execution-engine/blob/main/docs/NIC_DRIVER_REPO_PLAN.md Outlines the goals for comparison benchmarks against DPDK and OpenOnload, focusing on latency distribution. ```cpp // Side-by-side with DPDK, OpenOnload // Show latency distribution (p50, p99, p99.9, max) ``` -------------------------------- ### Round-Trip Time (RTT) Example Source: https://github.com/krish567366/submicro-execution-engine/blob/main/docs/LATENCY_ANALYSIS.md Calculates the total Round-Trip Time (RTT) for a co-located system, including one-way latency and exchange processing time. ```text Your system → Exchange: 25 μs (one-way) Exchange processing: 50 μs (matching engine) Exchange → Your system: 25 μs (one-way) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ Total RTT: 100 μs ``` -------------------------------- ### Initialize ModelStore with Redis Backend Source: https://github.com/krish567366/submicro-execution-engine/blob/main/docs/PRODUCTION_READINESS.md Initializes the ModelStore using Redis as a fast key-value store with pub/sub capabilities for live updates. This backend offers microsecond latency for cache hits. ```cpp // Fast key-value store with pub/sub for live updates RedisModelStore model_store("redis://prod-cache:6379"); // Cache hit: <1 microsecond // Cache miss: <100 microseconds (network fetch) ``` -------------------------------- ### GitHub README Badges Source: https://github.com/krish567366/submicro-execution-engine/blob/main/docs/NIC_DRIVER_REPO_PLAN.md Example Markdown for GitHub README badges showcasing key metrics like latency, performance, C++ version, and license. ```markdown [![Latency](https://img.shields.io/badge/latency-20--50ns-brightgreen)]() [![Performance](https://img.shields.io/badge/performance-14.88_Mpps-blue)]() [![C++](https://img.shields.io/badge/C%2B%2B-17-orange)]() [![License](https://img.shields.io/badge/license-MIT-green)]() ``` -------------------------------- ### Get Symbol String from Internal ID Source: https://github.com/krish567366/submicro-execution-engine/blob/main/docs/BROKER_INTEGRATION.md Retrieves the FIX symbol string corresponding to an internal integer ID. Defaults to 'EUR/USD' for unrecognized IDs. ```cpp if (symbol_id == 1) return "EUR/USD"; if (symbol_id == 2) return "GBP/USD"; return "EUR/USD"; ``` -------------------------------- ### Basic Custom Driver Usage Source: https://github.com/krish567366/submicro-execution-engine/blob/main/docs/NIC_DRIVER_REPO_PLAN.md Demonstrates initializing a custom NIC driver and continuously receiving and processing packets. ```cpp #include int main() { // Initialize custom driver ull_nic::CustomDriver nic("0000:01:00.0"); nic.init(); // Receive packets while (true) { auto pkt = nic.receive(); if (pkt) { process_packet(*pkt); } } } ``` -------------------------------- ### Initialize Binance WebSocket and Order Gateway Source: https://github.com/krish567366/submicro-execution-engine/blob/main/docs/BROKER_INTEGRATION.md Includes necessary headers and initializes the Binance WebSocket feed and Order Gateway using API credentials. Replace simulated NIC with a real WebSocket feed. ```cpp #include "ws_market_data.hpp" #include "rest_order_gateway.hpp" // Replace simulated NIC with real WebSocket feed BinanceWebSocketFeed feed("btcusdt", market_queue); BinanceOrderGateway gateway(getenv("BINANCE_API_KEY"), getenv("BINANCE_API_SECRET")); ``` -------------------------------- ### Configure ESLint for Type-Checked Linting Source: https://github.com/krish567366/submicro-execution-engine/blob/main/web/README.md Use this configuration to enable type-aware lint rules in your ESLint setup. Ensure 'tsconfig.node.json' and 'tsconfig.app.json' are correctly specified. ```javascript export default defineConfig([ globalIgnores(['dist']), { files: ['**/*.{ts,tsx}'] extends: [ // Other configs... // Remove tseslint.configs.recommended and replace with this tseslint.configs.recommendedTypeChecked, // Alternatively, use this for stricter rules tseslint.configs.strictTypeChecked, // Optionally, add this for stylistic rules tseslint.configs.stylisticTypeChecked, // Other configs... ], languageOptions: { parserOptions: { project: ['./tsconfig.node.json', './tsconfig.app.json'], tsconfigRootDir: import.meta.dirname, }, // other options... }, }, ]) ``` -------------------------------- ### Find Google Test and OpenSSL Source: https://github.com/krish567366/submicro-execution-engine/blob/main/tests/unit/CMakeLists.txt Locates the Google Test and OpenSSL libraries, which are required for compiling and running unit tests. Ensure these libraries are installed on your system. ```cmake find_package(GTest REQUIRED) # Find OpenSSL find_package(OpenSSL REQUIRED) ``` -------------------------------- ### Initialize ModelStore with PostgreSQL Backend Source: https://github.com/krish567366/submicro-execution-engine/blob/main/docs/PRODUCTION_READINESS.md Initializes the ModelStore using PostgreSQL with TimescaleDB for full ACID compliance and time-series support. This backend is suitable for complex queries and auditing. ```cpp // Full ACID compliance with time-series support PostgreSQLModelStore model_store("postgresql://prod-db:5432/params"); // Supports complex queries: // - "Show me all parameter versions from last month" // - "Audit trail: who changed risk parameters?" // - "Rollback to version 42" ``` -------------------------------- ### One-Way Network Latency Example Source: https://github.com/krish567366/submicro-execution-engine/blob/main/docs/LATENCY_ANALYSIS.md Calculates the estimated one-way network latency for a trade from New York to a NASDAQ datacenter, considering distance and network hops. ```text Distance: 40 km (25 miles) Speed in fiber: 200,000 km/s (2/3 speed of light) Physical: 200 μs (200,000 ns) Router hops (5×): 50 μs Total: ~250 μs (250,000 ns) ``` -------------------------------- ### FPGA Hardware Initialization Source: https://github.com/krish567366/submicro-execution-engine/blob/main/docs/PRODUCTION_READINESS.md Outlines the steps required to initialize the FPGA hardware for inference. This includes detection, memory mapping, bitstream loading, configuration, and health checks. ```cpp bool initialize_fpga_hardware() { // 1. Detect FPGA card via PCIe // 2. Map memory regions (BARs) // 3. Load bitstream // 4. Configure inference engine // 5. Health check } ``` -------------------------------- ### Build Benchmark Executable Source: https://github.com/krish567366/submicro-execution-engine/blob/main/docs/BENCHMARK_GUIDE.md Builds the HFT benchmark executable. First, makes the build script executable, then runs it. Supports both release and debug builds. ```bash # Make build script executable chmod +x build_benchmark.sh # Build (Release mode) ./build_benchmark.sh # Or build with debug symbols ./build_benchmark.sh Debug ``` -------------------------------- ### Create Test Executables and Configure Source: https://github.com/krish567366/submicro-execution-engine/blob/main/tests/unit/CMakeLists.txt Iterates through each test source file to create a corresponding executable. It sets include directories, C++ standard, enables exceptions, and links required libraries. ```cmake foreach(TEST_SOURCE ${TEST_SOURCES}) get_filename_component(TEST_NAME ${TEST_SOURCE} NAME_WE) add_executable(${TEST_NAME} ${TEST_SOURCE}) target_include_directories(${TEST_NAME} PRIVATE ${CMAKE_SOURCE_DIR}/include /opt/homebrew/include ) target_compile_features(${TEST_NAME} PRIVATE cxx_std_17) # Enable exceptions for tests (since they use try/catch) target_compile_options(${TEST_NAME} PRIVATE -fexceptions) # Link Google Test and OpenSSL target_link_libraries(${TEST_NAME} PRIVATE GTest::gtest GTest::gtest_main OpenSSL::SSL OpenSSL::Crypto) # Register as test add_test(NAME ${TEST_NAME} COMMAND ${TEST_NAME}) endforeach() ``` -------------------------------- ### Create Binance WebSocket Adapter Source: https://github.com/krish567366/submicro-execution-engine/blob/main/docs/BROKER_INTEGRATION.md Implements a C++ class to connect to Binance's WebSocket API, subscribe to depth data, parse incoming JSON messages, and push market tick data into a lock-free queue. Requires Boost.Beast and nlohmann/json libraries. ```cpp #ifndef WS_MARKET_DATA_HPP #define WS_MARKET_DATA_HPP #include "common_types.hpp" #include "lockfree_queue.hpp" #include #include #include #include namespace beast = boost::beast; namespace websocket = beast::websocket; namespace net = boost::asio; using tcp = net::ip::tcp; using json = nlohmann::json; class BinanceWebSocketFeed { public: BinanceWebSocketFeed( const std::string& symbol, LockFreeSPSC& output_queue ) : symbol_(symbol), output_queue_(output_queue), ioc_(), ws_(ioc_) {} void connect() { // Binance WebSocket endpoint const std::string host = "stream.binance.com"; const std::string port = "9443"; const std::string path = "/ws/" + symbol_ + "@depth@100ms"; // Resolve and connect tcp::resolver resolver{ioc_}; auto results = resolver.resolve(host, port); auto ep = net::connect(ws_.next_layer(), results); // SSL handshake ws_.next_layer().handshake(ssl::stream_base::client); // WebSocket handshake ws_.handshake(host, path); std::cout << "Connected to Binance: " << symbol_ << std::endl; } void start_reading() { read_loop(); } private: void read_loop() { ws_.async_read(buffer_, [this](beast::error_code ec, std::size_t bytes) { if (ec) { std::cerr << "WebSocket read error: " << ec.message() << std::endl; return; } // Parse JSON message std::string msg = beast::buffers_to_string(buffer_.data()); buffer_.consume(buffer_.size()); auto j = json::parse(msg); // Convert to MarketTick MarketTick tick = parse_binance_depth(j); // Push to lock-free queue while (!output_queue_.try_push(tick)) { // Busy-wait if queue full (should never happen with proper sizing) std::this_thread::yield(); } // Continue reading read_loop(); }); } MarketTick parse_binance_depth(const json& j) { MarketTick tick; tick.timestamp = now(); tick.symbol_id = 1; // Map symbol to ID // Parse bids (up to 10 levels) auto bids = j["bids"]; tick.depth_levels = std::min(10, (int)bids.size()); for (int i = 0; i < tick.depth_levels; ++i) { tick.bid_prices[i] = std::stod(bids[i][0].get()); tick.bid_sizes[i] = static_cast( std::stod(bids[i][1].get()) * 1000 ); } // Parse asks auto asks = j["asks"]; for (int i = 0; i < tick.depth_levels; ++i) { tick.ask_prices[i] = std::stod(asks[i][0].get()); tick.ask_sizes[i] = static_cast( std::stod(asks[i][1].get()) * 1000 ); } return tick; } std::string symbol_; LockFreeSPSC& output_queue_; net::io_context ioc_; websocket::stream ws_; beast::flat_buffer buffer_; }; #endif // WS_MARKET_DATA_HPP ``` -------------------------------- ### Set CPU Governor to Performance Source: https://github.com/krish567366/submicro-execution-engine/blob/main/docs/BENCHMARK_GUIDE.md Configures the CPU frequency scaling governor to 'performance' mode to ensure maximum CPU speed. Requires installation of cpupower tools. ```bash # Install cpupower (if not present) sudo apt-get install linux-tools-common linux-tools-$(uname -r) # Ubuntu # OR sudo yum install kernel-tools # RHEL # Set performance governor sudo cpupower frequency-set -g performance # Verify cat /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor ``` -------------------------------- ### Run Production Benchmark Source: https://github.com/krish567366/submicro-execution-engine/blob/main/docs/BENCHMARK_GUIDE.md Executes the full production benchmark with 100 million samples, which may take 10-20 minutes. Results are saved to a specified output file. Progress is monitored. ```bash # Full benchmark with 100M samples sudo ./build/hft_benchmark --samples 100000000 --output prod_results # Monitor progress # You'll see: Progress: 0%...100% ``` -------------------------------- ### Run Full System Benchmark Source: https://github.com/krish567366/submicro-execution-engine/blob/main/docs/BENCHMARK_GUIDE.md Executes a full system benchmark with 50 million samples, focusing on end-to-end performance. Results are saved to a specified output file. ```bash sudo ./build/hft_benchmark --full --samples 50000000 --output full_system ``` -------------------------------- ### Tick-to-Tick Latency Throughput Example Source: https://github.com/krish567366/submicro-execution-engine/blob/main/docs/LATENCY_ANALYSIS.md Compares the processing capacity of a fast (600 ns) vs. slow (25,000 ns) tick-to-tick system against a high market data rate. ```text AAPL Market Data Rate: 50,000 updates/second Time between ticks: 20 μs average Your System (600 ns tick-to-tick): - Can process each update with 19.4 μs idle time - 97% headroom for bursts - Never falls behind Slow System (25,000 ns tick-to-tick): - Can only handle 40,000 updates/sec - Falls behind during bursts - Misses critical updates ``` -------------------------------- ### Decision Latency Impact Example Source: https://github.com/krish567366/submicro-execution-engine/blob/main/docs/LATENCY_ANALYSIS.md Compares the profit potential of a fast (500 ns) vs. slow (5,000 ns) decision latency in response to a market price change. ```text Market Scenario: AAPL jumps $100.00 → $100.10 (10 cent move) Your System (500 ns decision): - See $100.00 → $100.10 at t=0 - Decide to buy at t=500 ns - Submit order at $100.10 - Get filled (first in queue) - Profit potential: $0.02-0.05/share Slow System (5,000 ns decision): - See $100.00 → $100.10 at t=0 - Decide to buy at t=5,000 ns - Submit order at $100.10 - Price already moved to $100.15 (you're late) - Profit lost or reduced to $0.00-0.01/share ```