### Install and Run Examples

Source: https://github.com/takara-ai/go-attention/blob/main/README.md

Instructions to get the go-attention module and run the provided examples.

```bash
# Get the module
go get github.com/takara-ai/go-attention

# Run the examples
go run api_examples.go
```

--------------------------------

### Basic Dot-Product Attention Example

Source: https://github.com/takara-ai/go-attention/blob/main/README.md

Demonstrates the usage of the DotProductAttention function for a basic sequence processing task, showing how to set up query, keys, and values, and compute the attention output and weights.

```go
import "github.com/takara-ai/go-attention/attention"
import "log"

// Create query-key-value setup
query := attention.Vector{1.0, 0.0, 1.0, 0.0}  // Pattern to search for
keys := attention.Matrix{
    {1.0, 0.0, 1.0, 0.0},  // Similar to query
    {0.0, 1.0, 0.0, 1.0},  // Different from query
    {0.5, 0.5, 0.5, 0.5},  // Neutral pattern
}
values := attention.Matrix{
    {1.0, 2.0},  // Value for similar key
    {3.0, 4.0},  // Value for different key
    {5.0, 6.0},  // Value for neutral key
}

// Compute attention
output, weights, err := attention.DotProductAttention(query, keys, values)
if err != nil {
    log.Fatal(err)
}

// Output will be a weighted combination of values based on query-key similarity
// Weights will show how much attention each key received
```

--------------------------------

### Matrix Pool Management

Source: https://github.com/takara-ai/go-attention/blob/main/API.md

Manages a pool of reusable Matrix objects to optimize memory allocation and deallocation. Includes methods for getting and returning matrices from the pool.

```APIDOC
MatrixPool:
  A pool for managing reusable Matrix objects.

  NewMatrixPool(rows, cols int) *MatrixPool
    Creates a new matrix pool for matrices of the specified dimensions.
    Parameters:
      - rows: The number of rows for matrices in the pool.
      - cols: The number of columns for matrices in the pool.
    Returns:
      - *MatrixPool: A pointer to the newly created MatrixPool.

  Get() Matrix
    Returns a matrix from the pool. If the pool is empty, a new matrix is allocated.
    Returns:
      - Matrix: A matrix object from the pool.

  Put(m Matrix)
    Returns a matrix to the pool for reuse. Matrices returned should not be used after being put back.
    Parameters:
      - m: The Matrix object to return to the pool.
```

```go
type MatrixPool struct {
    pool sync.Pool
    rows int
    cols int
}
```

```go
func NewMatrixPool(rows, cols int) *MatrixPool
```

```go
func (mp *MatrixPool) Get() Matrix
```

```go
func (mp *MatrixPool) Put(m Matrix)
```

--------------------------------

### Vector Pool Management

Source: https://github.com/takara-ai/go-attention/blob/main/API.md

Manages a pool of reusable Vector objects to optimize memory allocation and deallocation. Includes methods for getting and returning vectors from the pool.

```APIDOC
VectorPool:
  A pool for managing reusable Vector objects.

  NewVectorPool(size int) *VectorPool
    Creates a new vector pool for vectors of the specified size.
    Parameters:
      - size: The size of vectors to be managed by the pool.
    Returns:
      - *VectorPool: A pointer to the newly created VectorPool.

  Get() Vector
    Returns a vector from the pool. If the pool is empty, a new vector is allocated.
    Returns:
      - Vector: A vector object from the pool.

  Put(v Vector)
    Returns a vector to the pool for reuse. Vectors returned should not be used after being put back.
    Parameters:
      - v: The Vector object to return to the pool.
```

```go
type VectorPool struct {
    pool sync.Pool
    size int
}
```

```go
func NewVectorPool(size int) *VectorPool
```

```go
func (vp *VectorPool) Get() Vector
```

```go
func (vp *VectorPool) Put(v Vector)
```

--------------------------------

### MultiHeadAttention Forward Pass and String Representation

Source: https://github.com/takara-ai/go-attention/blob/main/API.md

Implements the forward pass for the Multi-Head Attention module, processing input matrices through multiple attention heads. It also provides a method to get a string representation of the module's configuration.

```go
func (mha *MultiHeadAttention) Forward(query, key, value Matrix) (Matrix, error)
// Processes input through multi-head attention.
// Parameters:
// - query, key, value Matrix: Input matrices [batch_size*seq_len, d_model]
// Returns:
// - Matrix: Output matrix [batch_size*seq_len, d_model]
// - error: Error for dimension mismatches

func (mha *MultiHeadAttention) String() string
// Returns a string representation of the multi-head attention configuration.
```

--------------------------------

### Configure and Use Multi-Head Attention in Go

Source: https://github.com/takara-ai/go-attention/blob/main/README.md

Demonstrates how to configure and utilize the Multi-Head Attention mechanism. It covers setting up the configuration struct, initializing the attention module, and processing batched input sequences through the Forward method. Requires the 'go-attention/attention' package.

```go
import "github.com/takara-ai/go-attention/attention"
import "log"

// Configure multi-head attention
config := attention.MultiHeadConfig{
    NumHeads:    4,        // Number of parallel attention heads
    DModel:      64,       // Size of input/output embeddings
    DKey:        16,       // Size per head (DModel/NumHeads)
    DValue:      16,       // Size per head (DModel/NumHeads)
    DropoutRate: 0.1,      // For regularization
}

// Create the attention module
mha, err := attention.NewMultiHeadAttention(config)
if err != nil {
    log.Fatal(err)
}

// Process sequences (batched input)
batchSize, seqLen := 2, 3  // Process 2 sequences, each with 3 tokens

// Create input matrices [batchSize × seqLen × DModel]
queries := make(attention.Matrix, batchSize*seqLen)
keys := make(attention.Matrix, batchSize*seqLen)
values := make(attention.Matrix, batchSize*seqLen)

// Initialize your matrices with actual data...

// Process through multi-head attention
output, err := mha.Forward(queries, keys, values)
if err != nil {
    log.Fatal(err)
}
```

--------------------------------

### Configure and Use Transformer Layer in Go

Source: https://github.com/takara-ai/go-attention/blob/main/README.md

Shows how to set up and employ a full Transformer Layer, including self-attention and a feed-forward network. It details the configuration process, module instantiation, and input processing via the Forward method. Depends on 'go-attention/transformer' and 'go-attention/attention' packages.

```go
import (
    "github.com/takara-ai/go-attention/transformer"
    "github.com/takara-ai/go-attention/attention"
    "log"
)

// Configure transformer layer
config := transformer.TransformerConfig{
    DModel:      64,       // Size of token embeddings
    NumHeads:    4,        // Number of attention heads
    DHidden:     256,      // Size of feed-forward hidden layer
    DropoutRate: 0.1,      // For regularization
}

// Create transformer layer
layer, err := transformer.NewTransformerLayer(config)
if err != nil {
    log.Fatal(err)
}

// Create input sequence [seq_len × d_model]
seqLen := 3
input := make(attention.Matrix, seqLen)
for i := range input {
    input[i] = make(attention.Vector, config.DModel)
    // Fill with your embedding data...
}

// Process through transformer
output, err := layer.Forward(input)
if err != nil {
    log.Fatal(err)
}
```

--------------------------------

### Performance Monitoring API

Source: https://github.com/takara-ai/go-attention/blob/main/API.md

API documentation for performance monitoring utilities, including configuration, statistics retrieval, and resetting.

```APIDOC
Performance Monitoring:
  Struct Definitions:
    type PerformanceStats struct {
        OperationCount    int64         // Number of operations performed
        TotalTime         time.Duration // Total time spent on operations
        AverageTime       time.Duration // Average time per operation
        MinTime           time.Duration // Minimum time for an operation
        MaxTime           time.Duration // Maximum time for an operation
        LastOperationTime time.Time     // Timestamp of the last operation
        MemoryAllocs      int64         // Number of memory allocations
        MemoryBytes       int64         // Total bytes allocated
    }

    type PerformanceConfig struct {
        EnableMonitoring bool // Flag to enable/disable monitoring
        EnableAutoTuning bool // Flag to enable automatic performance tuning
        MinVectorSize    int // Minimum vector size for parallel operations
        MinMatrixSize    int // Minimum matrix size for parallel operations
        MaxWorkers       int // Maximum number of worker goroutines
    }

  DefaultPerformanceConfig() PerformanceConfig
    Returns a struct with sensible default values for performance configuration.
    Returns:
      - PerformanceConfig: Default configuration settings.

  SetPerformanceConfig(config PerformanceConfig)
    Updates the global performance configuration settings.
    Parameters:
      - config: The new PerformanceConfig to apply.

  GetPerformanceStats(operation string) (*PerformanceStats, bool)
    Retrieves performance statistics for a specific named operation.
    Parameters:
      - operation: The name of the operation to query.
    Returns:
      - *PerformanceStats: A pointer to the statistics if found.
      - bool: True if statistics were found, false otherwise.

  GetAllPerformanceStats() map[string]*PerformanceStats
    Retrieves all collected performance statistics for all operations.
    Returns:
      - map[string]*PerformanceStats: A map where keys are operation names and values are their statistics.

  ResetPerformanceStats()
    Clears all collected performance statistics, resetting counters and timers.

  PerformanceWrappedDotProduct(v1, v2 Vector) (float64, error)
    Computes the dot product of two vectors with performance monitoring enabled.
    Parameters:
      - v1: The first input vector.
      - v2: The second input vector.
    Returns:
      - float64: The result of the dot product.
      - error: An error if the operation fails (e.g., dimension mismatch).
```

--------------------------------

### Performance Monitoring and Tuning Configuration

Source: https://github.com/takara-ai/go-attention/blob/main/README.md

Details how to configure performance monitoring and auto-tuning for the attention library. This includes enabling specific features, managing memory pools for reduced allocation overhead, and retrieving performance statistics. These functions are part of the core 'attention' package.

```APIDOC
attention.DefaultPerformanceConfig()
  - Returns a default configuration struct for performance settings.

attention.SetPerformanceConfig(config PerformanceConfig)
  - Applies the specified performance configuration globally.
  - Parameters:
    - config: A PerformanceConfig struct containing settings like EnableMonitoring and EnableAutoTuning.

attention.GetVectorFromPool(size int) Vector
  - Retrieves a pre-allocated vector of the specified size from the memory pool.
  - Parameters:
    - size: The desired size of the vector.
  - Returns: A Vector object.

attention.PutVectorToPool(v Vector)
  - Returns a vector to the memory pool for reuse.
  - Parameters:
    - v: The Vector object to return.

attention.GetAllPerformanceStats() PerformanceStats
  - Retrieves comprehensive performance statistics for the library.
  - Returns: A PerformanceStats struct containing metrics like allocation counts and timings.
```

--------------------------------

### MultiHeadAttention Configuration and Creation

Source: https://github.com/takara-ai/go-attention/blob/main/API.md

Defines the configuration structure for Multi-Head Attention and provides a constructor function to create a new, configured module. The configuration specifies the number of heads, embedding dimensions, and dropout rate.

```go
type MultiHeadConfig struct {
    NumHeads    int     // Number of parallel attention heads
    DModel      int     // Size of input/output embeddings
    DKey        int     // Size per head (DModel/NumHeads)
    DValue      int     // Size per head (DModel/NumHeads)
    DropoutRate float64 // For regularization
}

func NewMultiHeadAttention(config MultiHeadConfig) (*MultiHeadAttention, error)
// Creates a new multi-head attention module.
// Parameters:
// - config MultiHeadConfig: Configuration parameters
// Returns:
// - *MultiHeadAttention: Configured attention module
// - error: Error for invalid configuration
```

--------------------------------

### TransformerLayer API

Source: https://github.com/takara-ai/go-attention/blob/main/API.md

Documentation for the TransformerLayer component, including its configuration, constructor, and methods for forward passes and string representation.

```APIDOC
TransformerLayer:
  Struct Definition:
    type TransformerConfig struct {
        DModel      int     // Size of token embeddings
        NumHeads    int     // Number of attention heads
        DHidden     int     // Size of feed-forward hidden layer
        DropoutRate float64 // For regularization
    }

  NewTransformerLayer(config TransformerConfig) (*TransformerLayer, error)
    Creates a complete transformer layer, integrating self-attention and feed-forward networks.
    Parameters:
      - config: A TransformerConfig struct containing all necessary parameters.
    Returns:
      - *TransformerLayer: A pointer to the configured transformer layer.
      - error: An error if the provided configuration is invalid.

  Forward(input attention.Matrix) (attention.Matrix, error)
    Processes the input matrix through the complete transformer layer.
    Parameters:
      - input: The input matrix with shape [seq_len, d_model].
    Returns:
      - attention.Matrix: The output matrix after passing through the layer.
      - error: An error if dimension mismatches occur.

  String() string
    Returns a string representation of the transformer layer configuration.
```

--------------------------------

### FeedForward API

Source: https://github.com/takara-ai/go-attention/blob/main/API.md

Documentation for the FeedForward component, including its structure definition, constructor, and methods for processing data and string representation.

```APIDOC
FeedForward:
  Struct Definition:
    type FeedForward struct {
        DModel  int                // Input/output dimension
        DHidden int                // Hidden layer dimension
        W1      attention.Matrix   // First weight matrix
        B1      attention.Vector   // First bias vector
        W2      attention.Matrix   // Second weight matrix
        B2      attention.Vector   // Second bias vector
    }

  NewFeedForward(dModel, dHidden int) *FeedForward
    Creates a new feed-forward network module.
    Parameters:
      - dModel: The input and output dimension of the network.
      - dHidden: The dimension of the hidden layer.
    Returns:
      - *FeedForward: A pointer to the initialized FeedForward module.

  Forward(input attention.Matrix) (attention.Matrix, error)
    Processes the input matrix through the feed-forward network.
    Parameters:
      - input: The input matrix with shape [seq_len, d_model].
    Returns:
      - attention.Matrix: The output matrix after processing.
      - error: An error if dimension mismatches occur.

  String() string
    Returns a string representation of the feed-forward network configuration.
```

--------------------------------

### Parallel Processing Configuration

Source: https://github.com/takara-ai/go-attention/blob/main/README.md

Explains how to configure parallel processing capabilities within the attention library. This involves setting the number of worker goroutines to leverage multi-core processors for enhanced throughput. Uses default configurations from the 'attention' package.

```APIDOC
attention.DefaultParallelConfig()
  - Returns a default configuration struct for parallel processing settings.

attention.SetParallelConfig(config ParallelConfig)
  - Applies the specified parallel processing configuration globally.
  - Parameters:
    - config: A ParallelConfig struct containing settings like NumWorkers.
  - Example:
    config := attention.DefaultParallelConfig()
    config.NumWorkers = runtime.NumCPU() // Use all available CPU cores
```

--------------------------------

### LayerNorm API

Source: https://github.com/takara-ai/go-attention/blob/main/API.md

Documentation for the LayerNorm component, including its constructor and methods for applying normalization and retrieving string representations.

```APIDOC
LayerNorm:
  NewLayerNorm(dim int, eps float64) *LayerNorm
    Creates a new layer normalization module.
    Parameters:
      - dim: Dimension size of the input.
      - eps: Epsilon value for numerical stability during division.
    Returns:
      - *LayerNorm: A pointer to the initialized LayerNorm module.

  Forward(input attention.Matrix) (attention.Matrix, error)
    Applies layer normalization to the input matrix.
    Parameters:
      - input: The input matrix with shape [seq_len, dim].
    Returns:
      - attention.Matrix: The normalized output matrix.
      - error: An error if dimension mismatches occur.

  String() string
    Returns a string representation of the layer normalization configuration.
```

--------------------------------

### Performance Characteristics

Source: https://github.com/takara-ai/go-attention/blob/main/API.md

Provides an overview of the time and memory complexity for key operations within the library.

```APIDOC
Performance Characteristics:

  Time Complexity:
    - DotProduct: O(d) where d = vector dimension.
    - Softmax: O(n) where n = vector length.
    - DotProductAttention: O(n*d_k + n*d_v) where n = number of keys.
    - MultiHeadAttention: O(seq_len² * d_model).
    - LayerNorm: O(seq_len * dim).
    - FeedForward: O(seq_len * d_model * d_hidden).

  Memory Usage:
    - DotProduct: O(1) additional memory.
    - Softmax: O(n) for storing exponential values.
    - Attention: O(n) for attention weights.
    - MultiHead: O(seq_len * d_model) for intermediate results.
    - Transformer: O(seq_len * d_model) for residual connections.
```

--------------------------------

### NEON Optimized Dot Product

Source: https://github.com/takara-ai/go-attention/blob/main/API.md

An assembly-optimized function for calculating the dot product of two vectors using NEON instructions. This is a placeholder implementation.

```go
func dotProductNEON(a, b unsafe.Pointer, n int) float64
```

--------------------------------

### AVX2 Optimized Dot Product

Source: https://github.com/takara-ai/go-attention/blob/main/API.md

An assembly-optimized function for calculating the dot product of two vectors using AVX2 instructions. This is a placeholder implementation.

```go
func dotProductAVX2(a, b unsafe.Pointer, n int) float64
```

--------------------------------

### Error Handling Overview

Source: https://github.com/takara-ai/go-attention/blob/main/API.md

Details common error types returned by functions in the library, aiding in robust application development.

```APIDOC
Error Handling:
  All functions that can fail return an `error` value.

  Common Error Types:
    - Dimension mismatches: Occur when input vectors or matrices have incompatible dimensions for an operation.
    - Invalid configuration: Raised when configuration parameters are set to invalid values (e.g., negative dimensions).
    - Empty inputs: Signifies that required inputs are empty or nil, preventing operation execution.
```

--------------------------------

### DotProduct and BestDotProduct Functions

Source: https://github.com/takara-ai/go-attention/blob/main/API.md

Computes the dot product of two vectors. The `DotProduct` function is an optimized implementation that automatically selects the best algorithm based on input size and hardware. `BestDotProduct` is an alias for API clarity.

```go
func DotProduct(v1, v2 Vector) (float64, error)
// Computes the dot product of two vectors.
// Parameters:
// - v1, v2 Vector: Input vectors of equal length
// Returns:
// - float64: Dot product result
// - error: Error if vectors have different lengths
// Performance: O(d) where d = len(v1)

func BestDotProduct(v1, v2 Vector) (float64, error)
// Alias for DotProduct for API clarity. Provides the same functionality.
```

--------------------------------

### Core Data Types

Source: https://github.com/takara-ai/go-attention/blob/main/README.md

Defines the fundamental data structures used within the go-attention library for representing numerical data.

```APIDOC
Core Types:

Vector: Represents a 1D vector of float64 values.
  Type: []float64

Matrix: Represents a 2D matrix of float64 values.
  Type: []Vector
```

--------------------------------

### Global Pool Access for Vectors

Source: https://github.com/takara-ai/go-attention/blob/main/API.md

Provides functions to access vectors from globally managed pools based on their size, simplifying resource management across the application.

```APIDOC
Global Vector Pool Access:

  GetVectorFromPool(size int) Vector
    Retrieves a vector from an appropriate global pool based on the specified size.
    Parameters:
      - size: The desired size of the vector.
    Returns:
      - Vector: A vector object obtained from a global pool.

  PutVectorToPool(v Vector)
    Returns a vector to its appropriate global pool for reuse.
    Parameters:
      - v: The Vector object to return to the global pool.
```

```go
func GetVectorFromPool(size int) Vector
```

```go
func PutVectorToPool(v Vector)
```

--------------------------------

### Performance Wrapped Dot Product Parallel

Source: https://github.com/takara-ai/go-attention/blob/main/API.md

Provides a Go function for calculating the dot product of two vectors in parallel with performance monitoring enabled. It returns the computed dot product and any potential errors.

```go
func PerformanceWrappedDotProductParallel(v1, v2 Vector) (float64, error)
```

--------------------------------

### Dot Product Attention Mechanism

Source: https://github.com/takara-ai/go-attention/blob/main/README.md

Implements the dot product attention mechanism, a fundamental component in transformer models. It calculates attention weights based on the similarity between a query vector and a set of key vectors, then produces an output by applying these weights to corresponding value vectors.

```APIDOC
attention.DotProductAttention(query Vector, keys Matrix, values Matrix) (Vector, []float64, error)

  Description:
    Computes the dot product attention between a query, keys, and values.
    This function calculates the similarity scores between the query and each key,
    normalizes these scores into attention weights using a softmax function,
    and then computes a weighted sum of the values based on these weights.

  Parameters:
    query: The query vector (Vector).
    keys: A matrix where each row is a key vector (Matrix).
    values: A matrix where each row is a value vector (Matrix).

  Returns:
    Vector: The resulting context vector, a weighted sum of the value vectors.
    []float64: The attention weights assigned to each key.
    error: An error if the input dimensions are incompatible or other issues occur.

  Dependencies:
    Requires the 'attention' package from github.com/takara-ai/go-attention.

  Example:
    See 'Basic Dot-Product Attention Example' snippet.
```

--------------------------------

### Parallel Configuration

Source: https://github.com/takara-ai/go-attention/blob/main/API.md

Defines configuration parameters for parallel operations, including the number of worker goroutines and the size of work chunks.

```APIDOC
ParallelConfig:
  Configuration structure for parallel operations.

  Fields:
    - NumWorkers int: The number of worker goroutines to use. If 0, it defaults to auto-detection.
    - ChunkSize int: The size of work chunks to divide tasks into.

  DefaultParallelConfig() ParallelConfig
    Returns a reasonable default parallel configuration.
    Returns:
      - ParallelConfig: A default configuration object.
```

```go
type ParallelConfig struct {
    NumWorkers int // Number of worker goroutines (0 = auto-detect)
    ChunkSize  int // Size of work chunks
}
```

```go
func DefaultParallelConfig() ParallelConfig
```

--------------------------------

### Deprecated Dot Product Functions

Source: https://github.com/takara-ai/go-attention/blob/main/API.md

A collection of previously used dot product functions that are now deprecated. Users are advised to use the canonical `DotProduct` function instead.

```APIDOC
Deprecated Dot Product Functions:

  DotProductUnsafe(v1, v2 Vector) float64
    Deprecated: Use `DotProduct` instead. This function assumes equal vector lengths without validation.

  DotProductPooled(v1, v2 Vector) (float64, error)
    Deprecated: Use `DotProduct` instead. This function is now an alias for the canonical implementation.

  DotProductParallel(v1, v2 Vector, config interface{}) (float64, error)
    Deprecated: Use `DotProduct` instead. This function is now an alias for the canonical implementation.

  DotProductOptimized(v1, v2 Vector) (float64, error)
    Deprecated: Use `DotProduct` instead. This function is now an alias for the canonical implementation.
```

```go
func DotProductUnsafe(v1, v2 Vector) float64
```

```go
func DotProductPooled(v1, v2 Vector) (float64, error)
```

```go
func DotProductParallel(v1, v2 Vector, config interface{}) (float64, error)
```

```go
func DotProductOptimized(v1, v2 Vector) (float64, error)
```

--------------------------------

### Optimized Matrix Multiplication

Source: https://github.com/takara-ai/go-attention/blob/main/API.md

Performs matrix multiplication using optimized techniques such as blocking and cache-friendly access patterns for improved performance.

```APIDOC
MatrixMultiplyOptimized(a, b Matrix) (Matrix, error)
  Performs optimized matrix multiplication with blocking and cache-friendly access patterns.
  Parameters:
    - a Matrix: The first input matrix.
    - b Matrix: The second input matrix.
  Returns:
    - Matrix: The resulting matrix from the multiplication.
    - error: An error if matrix dimensions are incompatible.
```

```go
func MatrixMultiplyOptimized(a, b Matrix) (Matrix, error)
```

--------------------------------

### DotProductAttention Function

Source: https://github.com/takara-ai/go-attention/blob/main/API.md

Computes scaled dot-product attention, a core mechanism in transformer models. It calculates attended output and attention weights based on query, key, and value matrices. This is the canonical, optimized implementation.

```go
func DotProductAttention(query Vector, keys, values Matrix) (Vector, AttentionWeights, error)
// Computes scaled dot-product attention. This is the canonical, optimized implementation.
// Parameters:
// - query Vector: Query vector [d_k]
// - keys Matrix: Key matrix [n, d_k]
// - values Matrix: Value matrix [n, d_v]
// Returns:
// - Vector: Attended output [d_v]
// - AttentionWeights: Attention weights [n]
// - error: Error for dimension mismatches
// Performance: O(n*d_k + n*d_v) where n = len(keys)
```

```go
func BestDotProductAttention(query Vector, keys, values Matrix) (Vector, AttentionWeights, error)
// Alias for DotProductAttention for API clarity.
```

--------------------------------

### LayerNorm Struct and Fields

Source: https://github.com/takara-ai/go-attention/blob/main/API.md

Defines the structure for Layer Normalization, including its dimension size, epsilon for numerical stability, and learnable scale (Gamma) and shift (Beta) parameters, which are represented as Vectors.

```go
type LayerNorm struct {
    Dim   int           // Dimension size
    Eps   float64       // Epsilon for numerical stability
    Gamma attention.Vector // Scale parameter
    Beta  attention.Vector // Shift parameter
}
```

--------------------------------

### Parallel Matrix Multiplication

Source: https://github.com/takara-ai/go-attention/blob/main/API.md

Performs matrix multiplication using parallel processing to improve performance. It takes two matrices and a parallel configuration as input.

```APIDOC
MatrixMultiplyParallel(a, b Matrix, config ParallelConfig) (Matrix, error)
  Performs parallel matrix multiplication.
  Parameters:
    - a Matrix: The first input matrix.
    - b Matrix: The second input matrix.
    - config ParallelConfig: Configuration for parallel execution, including worker count and chunk size.
  Returns:
    - Matrix: The resulting matrix from the multiplication.
    - error: An error if matrix dimensions are incompatible or other issues occur.
```

```go
func MatrixMultiplyParallel(a, b Matrix, config ParallelConfig) (Matrix, error)
```

--------------------------------

### Parallel Softmax Computation

Source: https://github.com/takara-ai/go-attention/blob/main/API.md

Computes the softmax function for a given vector in parallel, leveraging multiple workers for faster execution.

```APIDOC
SoftmaxParallel(x Vector, config ParallelConfig) Vector
  Computes softmax in parallel.
  Parameters:
    - x Vector: The input vector for which to compute the softmax.
    - config ParallelConfig: Configuration for parallel execution, including worker count and chunk size.
  Returns:
    - Vector: A new vector containing the softmax probabilities.
```

```go
func SoftmaxParallel(x Vector, config ParallelConfig) Vector
```

--------------------------------

### Default Epsilon Constant

Source: https://github.com/takara-ai/go-attention/blob/main/API.md

Defines the default epsilon value used for numerical stability in operations like softmax and normalization.

```go
const DefaultEpsilon = 1e-5
```

--------------------------------

### Matrix Type Definition

Source: https://github.com/takara-ai/go-attention/blob/main/API.md

Defines a 2D matrix of float64 values. This type is used for batched operations, key-value pairs, and multi-dimensional data structures within the library.

```go
type Matrix []Vector
```

--------------------------------

### Softmax Function

Source: https://github.com/takara-ai/go-attention/blob/main/API.md

Applies the softmax function to a vector, normalizing its elements into a probability distribution where the sum of elements is 1.0. This is commonly used for attention weight normalization.

```go
func Softmax(x Vector) Vector
// Applies the softmax function to a vector for attention weight normalization.
// Parameters:
// - x Vector: Input vector
// Returns:
// - Vector: Softmax probabilities (sum to 1.0)
// Performance: O(n) where n = len(x)
```

--------------------------------

### AttentionWeights Type Alias

Source: https://github.com/takara-ai/go-attention/blob/main/API.md

A type alias for Vector, specifically designated for attention weights. This enhances semantic clarity in function signatures and code readability.

```go
type AttentionWeights = Vector
```

--------------------------------

### Vector Type Definition

Source: https://github.com/takara-ai/go-attention/blob/main/API.md

Defines a 1D vector of float64 values. This type is fundamental and used throughout the library for representing embeddings, attention weights, and other numerical data.

```go
type Vector []float64
```

--------------------------------

### AddVectors Function

Source: https://github.com/takara-ai/go-attention/blob/main/API.md

Performs element-wise addition of two vectors. Both input vectors must have the same length for the operation to be valid.

```go
func AddVectors(v1, v2 Vector) (Vector, error)
// Adds two vectors element-wise.
// Parameters:
// - v1, v2 Vector: Input vectors of equal length
// Returns:
// - Vector: Element-wise sum
// - error: Error if vectors have different lengths
```

--------------------------------

### validateMatrixDimensions Function

Source: https://github.com/takara-ai/go-attention/blob/main/API.md

A utility function to validate that a variable number of matrices share consistent dimensions. It returns an error if any dimension mismatch is found, ensuring data integrity for matrix operations.

```go
func validateMatrixDimensions(matrices ...Matrix) error
// Validates that all matrices have matching dimensions.
// Parameters:
// - matrices ...Matrix: Variable number of matrices to validate
// Returns:
// - error: Error if dimensions don't match
```

--------------------------------

### ScaleVector Function

Source: https://github.com/takara-ai/go-attention/blob/main/API.md

Multiplies each element of a vector by a given scalar value. This is a common operation for adjusting magnitudes or applying scaling factors.

```go
func ScaleVector(v Vector, scale float64) Vector
// Multiplies a vector by a scalar value.
// Parameters:
// - v Vector: Input vector
// - scale float64: Scaling factor
// Returns:
// - Vector: Scaled vector
```

=== COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.