### Verify QAMA Installation Source: https://context7.com/dxee-e/qama/llms.txt Verifies the successful installation of the QAMA library by importing a specific module and printing a success message. This is a quick check to ensure the library is accessible. ```python python -c "from QAMA.QAMultiheadAttention import QAMultiheadAttention; print('QAMA installed successfully')" ``` -------------------------------- ### Install Python Dependencies Source: https://context7.com/dxee-e/qama/llms.txt Installs essential Python packages required for QAMA and its associated tools, including Jupyter for interactive environments, testing frameworks like pytest, and performance analysis tools. ```shell pip install jupyter ipykernel icecream pytest black isort line_profiler tqdm torchmetrics pip install pyyaml einops tensorboard ``` -------------------------------- ### Install QAMA Package and Dependencies via Conda and Pip Source: https://context7.com/dxee-e/qama/llms.txt Provides shell commands to set up the QAMA development environment. This includes creating and activating a Conda environment with a specific Python version, and installing PyTorch with CUDA support, followed by optimization solvers like Gurobi. ```bash # Create conda environment conda create -n QAMA python==3.8.20 conda activate QAMA # Install PyTorch with CUDA support pip install torch==2.4.1 torchvision==0.19.1 torchaudio==2.4.1 --index-url https://download.pytorch.org/whl/cu121 # Install optimization solvers pip install gurobipy # For exact QUBO optimization ``` -------------------------------- ### Install QAMA in Development Mode Source: https://context7.com/dxee-e/qama/llms.txt Installs the QAMA package in editable/development mode, allowing for direct modifications to the source code without needing to reinstall the package. ```shell cd QAMA pip install -e . ``` -------------------------------- ### End-to-End QUBO Solving Pipeline Source: https://context7.com/dxee-e/qama/llms.txt This snippet outlines the complete pipeline for solving QUBO problems, starting from attention matrices and proceeding to obtain binary solutions using a configured QUBO solver. It implies integration of coefficient preparation and solver execution. ```python import torch from QAMA.QUBO import QUBO ``` -------------------------------- ### Compile C++ Simulated Annealing Solver Source: https://context7.com/dxee-e/qama/llms.txt Compiles the C++ simulated annealing solver for QAMA to enhance performance. This process requires CMake and a C++ compiler to be installed on the system. ```shell cd QAMA/backend_c_sa mkdir build && cd build cmake .. make ``` -------------------------------- ### Untitled No description -------------------------------- ### Untitled No description -------------------------------- ### Untitled No description -------------------------------- ### Untitled No description -------------------------------- ### Untitled No description -------------------------------- ### Untitled No description -------------------------------- ### Untitled No description -------------------------------- ### Untitled No description -------------------------------- ### Configure QAMA Model and Solver with PyTorch Source: https://context7.com/dxee-e/qama/llms.txt Demonstrates how to configure a QAMA model with specific quadratic and linear coefficients, define solver arguments for simulated annealing and Monte Carlo greedy, and then solve a QUBO problem using PyTorch tensors for attention matrices. It shows how to instantiate the QUBO object and use its solve method with different backends. ```python import torch from qama import QUBO # Configure model and solver model_args = { "quadratic_coeffecient": 1.0, "linear_fix_coeffecient": 0.16, "penalty_multi_head_fix_coeffecient": 0.8, } solver_args = { 'c_sa': { "initial_temperature": 1000.0, "alpha": 0.99, "cutoff_temperature": 0.001, "iterations_per_t": 5, "patience": 50, "is_local_search": True, }, 'montecarlo_greedy': { "sample_num": 200, "seed": 0, "device": "cuda", "iterations": 3, } } qubo = QUBO( model_args=model_args, solver_args=solver_args, enable_solvers=['c_sa', 'montecarlo_greedy'] ) # Input attention matrices J = torch.rand((2, 8, 49, 49), dtype=torch.float32) J = (J + J.transpose(-1, -2)) / 2 J[:, :, torch.eye(49, dtype=torch.bool), :] = 0 h = torch.rand((2, 8, 49), dtype=torch.float32) # Solve with different backends x_csa = qubo.solve(J, h, solver_name='c_sa') x_mc = qubo.solve(J, h, solver_name='montecarlo_greedy') print(f"C_SA solution shape: {x_csa.shape}") # [2, 8, 49] print(f"Monte Carlo solution shape: {x_mc.shape}") # [2, 8, 49] print(f"Solutions differ: {not torch.equal(x_csa, x_mc)}") ``` -------------------------------- ### Untitled No description -------------------------------- ### Compare Attention Backends via Command Line Source: https://context7.com/dxee-e/qama/llms.txt Provides bash commands to train models using different attention mechanisms, including standard attention, QAMA with C++ simulated annealing, and various efficient transformer variants like Performer, Linformer, and BigBird. It also specifies the expected output log file locations. ```bash # Standard attention baseline python experiments/train.py experiments/configs/CIFAR10-SimpleViT.yaml # QAMA with C++ simulated annealing python experiments/train.py experiments/configs/CIFAR10-AttnMLP-QAMA-c_sa.yaml # QAMA with different efficient attention variants python experiments/train.py experiments/configs/CIFAR10-AttnMLP-QAMA-Performer.yaml python experiments/train.py experiments/configs/CIFAR10-AttnMLP-QAMA-Linformer.yaml python experiments/train.py experiments/configs/CIFAR10-AttnMLP-QAMA-BigBird.yaml # Results are automatically logged to: # - ./results/{DATASET}-{MODEL}-QAMA-{SOLVER}/train_record.json # - ./results/{DATASET}-{MODEL}-QAMA-{SOLVER}/val_record.json # - ./results/{DATASET}-{MODEL}-QAMA-{SOLVER}/train.log ``` -------------------------------- ### Untitled No description -------------------------------- ### Configure Multi-Solver Backend for QAMA Attention (Python) Source: https://context7.com/dxee-e/qama/llms.txt Demonstrates how to configure and utilize multiple solver backends ('c_sa', 'gurobi', 'montecarlo_greedy') within the QAMultiheadAttention layer. This allows switching between fast approximate solutions and exact optimization at inference time. ```python from QAMA.QAMultiheadAttention import QAMultiheadAttention import torch # Configure multiple solvers args_solver = { 'c_sa': { "initial_temperature": 1000.0, "alpha": 0.99, "cutoff_temperature": 0.001, "iterations_per_t": 5, "patience": 50, "is_local_search": True, }, 'gurobi': { "batch_num_process": 8, "timeout": 10, "mipgap": 0.01, "gurobi_num_threads": 1, }, 'montecarlo_greedy': { "sample_num": 200, "seed": 0, "device": "cuda", "iterations": 3, } } # Initialize with all solvers enabled model = QAMultiheadAttention( d_model=256, embed_dim=32, num_heads=8, args_solver=args_solver, enable_solvers=['c_sa', 'gurobi', 'montecarlo_greedy'] ).cuda() # Switch between solvers at inference time Q = torch.randn(128, 49, 256).cuda() # Fast approximate solution output_mc = model(Q, solver_name='montecarlo_greedy') # Exact optimization (slower) output_gurobi = model(Q, solver_name='gurobi') # Compiled C++ simulated annealing output_csa = model(Q, solver_name='c_sa') ``` -------------------------------- ### Untitled No description -------------------------------- ### Train Vision Transformer with QAMA using YAML Config Source: https://context7.com/dxee-e/qama/llms.txt Shows how to configure and train a Vision Transformer model with QAMA attention for image classification using a YAML configuration file. It details the structure of the configuration file, including dataset, model, QAMA, backend solver, model settings, and training parameters. It also demonstrates programmatic loading, modification, and saving of the configuration using PyYAML. ```yaml # Configuration file: configs/CIFAR10-SimpleViT-QAMA-c_sa.yaml """ base: dataset: CIFAR10 model: SimpleViT QAMA: True backend_solver: solver: c_sa c_sa: initial_temperature: 1000.0 alpha: 0.99 cutoff_temperature: 0.001 iterations_per_t: 5 patience: 50 is_local_search: True model_setting: image_size: [32, 32] patch_size: [4, 4] num_classes: 10 dim: 64 depth: 1 heads: 8 mlp_dim: 64 dim_head: 8 channels: 3 train_setting: batch_size: 128 num_epochs: 50 learning_rate: 0.001 device: 'cuda:0' seed: 42 """ ``` ```python # Train from command line: # python experiments/train.py experiments/configs/CIFAR10-SimpleViT-QAMA-c_sa.yaml # Programmatic usage: import yaml from experiments.train import Trainer # Load and customize config with open('experiments/configs/CIFAR10-SimpleViT-QAMA-c_sa.yaml', 'r') as f: config = yaml.safe_load(f) # Modify if needed config['train_setting']['batch_size'] = 256 config['train_setting']['learning_rate'] = 0.0005 # Save modified config with open('/tmp/custom_config.yaml', 'w') as f: yaml.dump(config, f) # Results saved to: ./results/CIFAR10-SimpleViT-QAMA-c_sa/ # Contains: train_record.json, val_record.json, config.yaml, saved_model/ ``` -------------------------------- ### Untitled No description -------------------------------- ### Solve QUBO with C++ Simulated Annealing Source: https://context7.com/dxee-e/qama/llms.txt This snippet demonstrates how to use the C++ simulated annealing backend for high-performance QUBO problem solving. It requires configuring simulated annealing parameters and preparing a symmetric QUBO matrix. ```python import torch from QAMA.backend_c_sa.solver import Solver # Configure simulated annealing parameters solver_config = { "initial_temperature": 1000.0, "alpha": 0.99, # Temperature decay rate "cutoff_temperature": 0.001, # Stopping temperature "iterations_per_t": 5, # Iterations per temperature "patience": 50, # Early stopping patience "is_local_search": True, # Enable local search refinement "flag_evolution_history": False, "history_file_path": "", "rand_seed": None, "is_input_check": False, } # Create solver instance solver = Solver(args_solver=solver_config) # Prepare QUBO matrix (must be symmetric) batch_size = 1 qubit_num = 392 # 8 heads * 49 sequence length Q = torch.randn(batch_size, qubit_num, qubit_num, device='cuda', dtype=torch.float32) Q = (Q + Q.transpose(-1, -2)) / 2 # Make symmetric # Solve QUBO problem x = solver.solve(Q) print(f"Solution shape: {x.shape}") # [1, 392] print(f"Binary values: ones={int((x == 1).sum())}, zeros={int((x == 0).sum())}") ``` -------------------------------- ### Initialize QAMA Multi-Head Attention Layer (Python) Source: https://context7.com/dxee-e/qama/llms.txt Initializes a quantum annealing-based multi-head attention layer using the 'c_sa' (compiled C++ simulated annealing) solver. This layer replaces standard softmax attention with QUBO optimization. It takes model and solver configurations as input and performs a forward and backward pass. ```python import torch from QAMA.QAMultiheadAttention import QAMultiheadAttention # Configuration parameters d_model = 256 # Input embedding dimension embed_dim = 32 # Head dimension num_heads = 8 # Number of attention heads batch_size = 128 seq_length = 49 # Model hyperparameters args_model = { "quadratic_coeffecient": 1.0, "linear_fix_coeffecient": 0.16, "penalty_multi_head_fix_coeffecient": 0.8, } # Solver configuration (c_sa backend) args_solver = { 'c_sa': { "initial_temperature": 1000.0, "alpha": 0.99, "cutoff_temperature": 0.001, "iterations_per_t": 5, "patience": 50, "is_local_search": True, } } # Create QAMA attention layer qama_attn = QAMultiheadAttention( d_model=d_model, embed_dim=embed_dim, num_heads=num_heads, args_model=args_model, args_solver=args_solver, enable_solvers=['c_sa'] ).cuda() # Forward pass with self-attention Q = torch.randn(batch_size, seq_length, d_model).cuda() output = qama_attn(Q, solver_name='c_sa') print(f"Output shape: {output.shape}") # [128, 49, 256] # Backward pass loss = torch.mean(output) loss.backward() ``` -------------------------------- ### CMake Build Configuration for C++ Project with PyTorch and Pybind11 Source: https://github.com/dxee-e/qama/blob/main/QAMA/QAMA/backend_c_sa/CMakeLists.txt This CMake script configures the build process for a C++ project. It finds and links dependencies such as Python, PyTorch, and Pybind11, sets C++ standards, and defines output properties for the resulting shared library. It's designed for a project that creates Python bindings using Pybind11. ```cmake cmake_minimum_required(VERSION 3.18) project(simulated_annealing_qubo) set(CMAKE_CXX_STANDARD 17) find_package(Python 3 REQUIRED COMPONENTS Interpreter Development) find_package(pybind11 CONFIG REQUIRED) # set(Torch_DIR "$ENV{LIBTORCH_HOME}/share/cmake/Torch") execute_process( COMMAND ${Python_EXECUTABLE} -c "import torch; print(torch.utils.cmake_prefix_path)" OUTPUT_VARIABLE Torch_DIR OUTPUT_STRIP_TRAILING_WHITESPACE ) set(Torch_DIR "${Torch_DIR}/Torch") message(STATUS "LibTorch directory: ${Torch_DIR}") find_package(Torch REQUIRED) find_library(TORCH_PYTHON_LIBRARY torch_python PATH "${TORCH_INSTALL_PREFIX}/lib") pybind11_add_module(simulated_annealing_qubo simulated_annealing_qubo.cpp) target_link_libraries(simulated_annealing_qubo PRIVATE ${TORCH_LIBRARIES} ${TORCH_PYTHON_LIBRARY} Python::Python) set_property(TARGET simulated_annealing_qubo PROPERTY CXX_STANDARD 17) set_target_properties(simulated_annealing_qubo PROPERTIES LIBRARY_OUTPUT_DIRECTORY ${PROJECT_SOURCE_DIR} PREFIX "" SUFFIX ".so" ) set(CMAKE_BUILD_TYPE "Release" CACHE STRING "Choose the build type.") ``` -------------------------------- ### Prepare QUBO Coefficients from Attention Matrices Source: https://context7.com/dxee-e/qama/llms.txt This snippet illustrates how to prepare the QUBO coefficient matrix Q from attention matrices J and h. It involves initializing the QUBO object with specific coefficients and ensuring the resulting matrix adheres to multi-head constraints and has a symmetric structure with a zero diagonal. ```python import torch from QAMA.QUBO import QUBO # Initialize QUBO with custom coefficients model_args = { "quadratic_coeffecient": 1.0, "linear_fix_coeffecient": 0.16, "penalty_multi_head_fix_coeffecient": 0.8, } qubo = QUBO(model_args=model_args, enable_solvers=['c_sa']) # Attention matrices (symmetric J, zero diagonal) b, t, n = 2, 8, 49 # batch, heads, sequence length J = torch.rand((b, t, n, n), dtype=torch.float32) J = (J + J.transpose(-1, -2)) / 2 # Make symmetric J[:, :, torch.eye(n, dtype=bool), :] = 0 # Zero diagonal h = torch.rand((b, t, n), dtype=torch.float32) # Convert to QUBO matrix Q = qubo.prepare_coefficients(J, h) print(f"QUBO matrix shape: {Q.shape}") # [2, 392, 392] (t*n x t*n) # Verify constraints assert Q.shape == (b, t*n, t*n), "QUBO matrix dimension mismatch" print("QUBO coefficients prepared successfully") ``` -------------------------------- ### Untitled No description -------------------------------- ### Untitled No description -------------------------------- ### Untitled No description -------------------------------- ### Train Text Transformer with QAMA for Sentiment Analysis Source: https://context7.com/dxee-e/qama/llms.txt Details the configuration for training a Text Transformer model with QAMA attention on sentiment analysis tasks, specifically for the IMDB and SST2 datasets. It outlines the YAML configuration structure and provides command-line instructions for initiating the training process. The output includes accuracy, loss curves, and model checkpoints. ```yaml # Configuration: experiments/configs/IMDB-TextTransformer-QAMA-c_sa.yaml """ base: dataset: IMDB model: TextTransformer QAMA: True backend_solver: solver: c_sa c_sa: initial_temperature: 1000.0 alpha: 0.99 cutoff_temperature: 0.001 iterations_per_t: 5 model_setting: vocab_size: 10000 max_seq_length: 256 embedding_dim: 128 num_heads: 8 num_layers: 4 ff_dim: 512 num_classes: 2 train_setting: batch_size: 64 num_epochs: 20 learning_rate: 0.0003 device: 'cuda:0' seed: 42 """ ``` ```python # Train IMDB sentiment classifier # python experiments/train.py experiments/configs/IMDB-TextTransformer-QAMA-c_sa.yaml # Train SST2 binary classification # python experiments/train.py experiments/configs/SST2-TextTransformer-QAMA-c_sa.yaml # Results include accuracy, loss curves, and model checkpoints ``` -------------------------------- ### Solve QUBO with Monte Carlo Greedy Sampling Source: https://context7.com/dxee-e/qama/llms.txt This snippet details the use of a GPU-accelerated Monte Carlo greedy sampling approach for approximate QUBO solving. It allows configuration of sample number, random seed, device, and refinement iterations. The output is an approximate binary solution. ```python import torch from QAMA.backend_montecarlo_greedy.solver import Solver # Configure Monte Carlo greedy solver mc_config = { "sample_num": 200, # Number of random samples per iteration "seed": 0, # Random seed for reproducibility "device": "cuda", # Computation device "iterations": 3, # Number of refinement iterations } solver = Solver(args_solver=mc_config) # QUBO problem batch_size = 128 qubit_num = 392 Q = torch.randn(batch_size, qubit_num, qubit_num, device='cuda', dtype=torch.float32) Q = (Q + Q.transpose(-1, -2)) / 2 # Fast approximate solution x = solver.solve(Q) print(f"Solution shape: {x.shape}") # [128, 392] # Verify energy minimization energy = torch.einsum("bq,bqr,br->b", x, Q, x) print(f"Average energy: {energy.mean().item():.4f}") ``` -------------------------------- ### Solve QUBO with Gurobi Optimizer Source: https://context7.com/dxee-e/qama/llms.txt This snippet shows how to leverage Gurobi for exact QUBO optimization. It supports parallel batch processing and allows configuration of optimality gap tolerance and timeout settings. The input is a batch of QUBO matrices. ```python import torch from QAMA.backend_gurobi.solver import Solver # Configure Gurobi solver with parallelization gurobi_config = { "batch_num_process": 16, # Number of parallel processes "timeout": 10, # Timeout per chunk (seconds) "mipgap": 0.01, # 1% optimality gap tolerance "gurobi_num_threads": 1, # Threads per Gurobi instance } solver = Solver(args_solver=gurobi_config) # Large batch QUBO problem batch_size = 128 qubit_num = 392 Q = torch.randn(batch_size, qubit_num, qubit_num, device='cuda', dtype=torch.float32) Q = (Q + Q.transpose(-1, -2)) / 2 # Solve with automatic parallelization x = solver.solve(Q) print(f"Solved {batch_size} problems") print(f"Solution shape: {x.shape}") # [128, 392] print(f"Device: {x.device}") # Returns to original device ``` === COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.