### Initialize Tensors for Examples Source: https://github.com/under-peter/omeinsum.jl/blob/master/docs/src/basic.md Sets up various tensors, including a scalar, vectors, matrices, and a 3D tensor, which are used in the subsequent einsum operation examples. ```julia using OMEinsum s = fill(1) w, v = [1, 2], [4, 5] A, B = [1 2; 3 4], [5 6; 7 8] T1, T2 = reshape(1:8, 2, 2, 2), reshape(9:16, 2, 2, 2) ``` -------------------------------- ### Install OMEinsum Package Source: https://github.com/under-peter/omeinsum.jl/blob/master/README.md To install OMEinsum, use the Julia package manager. Ensure you are using Julia version 1.5 or later. ```julia pkg> add OMEinsum ``` -------------------------------- ### Binary Einsum Examples Source: https://github.com/under-peter/omeinsum.jl/blob/master/docs/src/basic.md Shows binary einsum operations including matrix multiplication, batch matrix multiplication, element-wise multiplication, and element-wise multiplication followed by summation. ```julia ein"ij, jk -> ik" (A, B) ein"ijb,jkb->ikb" (T1, T2) ein"ij,ij->ij" (A, B) ein"ij,ij->" (A, B) ein"ij,->ij" (A, s) ``` -------------------------------- ### Star Contraction Example Source: https://github.com/under-peter/omeinsum.jl/blob/master/docs/src/background.md Presents a more complex tensor contraction example known as star contraction, showing the input tensors, output tensor, and the summation over the shared variable. ```math \texttt{contract}(\{i,j,k,l\}, \\{\\text{A\}\{i, l\}}, B\{\{j, l\}}, C\{\{k, l\}\\}\\}, \\{\\text{i,j,k\}\\\}) \\\ = \sum_{l}A\{\{i,l\}\} B\{\{j,l\}\} C\{\{k,l\}\}. ``` -------------------------------- ### Unary Einsum Examples Source: https://github.com/under-peter/omeinsum.jl/blob/master/docs/src/basic.md Demonstrates various unary einsum operations such as summing elements, summing rows, calculating the trace (diagonal sum), and creating diagonal matrices or repeating vectors. ```julia ein"i->" (w) ein"ij->i" (A) ein"ii->" (A) ein"ij->" (A) ein"i->ii" (w) ein"i->ij" (w; size_info=Dict('j'=>2)) ein"ijk->ikj" (T1) ``` -------------------------------- ### Nary Einsum Examples Source: https://github.com/under-peter/omeinsum.jl/blob/master/docs/src/basic.md Presents n-ary einsum operations, specifically demonstrating a star contraction and a tensor train contraction involving multiple tensors. ```julia optein"ai,aj,ak->ijk" (A, A, B) optein"ia,ajb,bkc,cld,dm->ijklm" (A, T1, T2, T1, A) ``` -------------------------------- ### Specifying Contraction Order with Brackets Source: https://github.com/under-peter/omeinsum.jl/blob/master/README.md Illustrates how to specify the order of operations in Einstein summation using brackets. This can be for performance optimization or to guide the computer in selecting appropriate kernels. Examples show both macro and string style. ```julia julia> @ein Z[o,s] := x[i,s] * (W[o,i,j] * y[j,s]); # macro style julia> Z = ein"is, (oij, js) -> os"(x, W, y); # string style ``` -------------------------------- ### Check cuTENSOR Detection Source: https://github.com/under-peter/omeinsum.jl/blob/master/docs/src/cuda.md Prints detailed information about the CUDA installation, including whether cuTENSOR is detected and available. ```julia using CUDA CUDA.versioninfo() ``` -------------------------------- ### Check cuTENSOR Availability Source: https://github.com/under-peter/omeinsum.jl/blob/master/docs/src/cuda.md Verifies if the cuTENSOR library is installed and available for use with OMEinsum. This check is crucial before attempting to use the cuTENSOR backend. ```julia using CUDA using cuTENSOR # Must be loaded to enable the cuTENSOR backend using OMEinsum # Check if cuTENSOR is available if cuTENSOR.has_cutensor() println("cuTENSOR is available!") else println("cuTENSOR is not available") end ``` -------------------------------- ### Check CUDA.jl Version Source: https://github.com/under-peter/omeinsum.jl/blob/master/docs/src/cuda.md Verifies the installed version of the CUDA.jl package, which is a prerequisite for using cuTENSOR. ```julia using Pkg Pkg.status("CUDA") ``` -------------------------------- ### Check and Set Einsum Backend Source: https://github.com/under-peter/omeinsum.jl/blob/master/docs/src/basic.md Demonstrates how to retrieve the current einsum computation backend and how to set it to the default backend. This is relevant for managing computation resources, especially between CPU and GPU. ```julia get_einsum_backend() set_einsum_backend!(DefaultBackend()) ``` -------------------------------- ### Enable cuTENSOR Backend Source: https://github.com/under-peter/omeinsum.jl/blob/master/docs/src/cuda.md Demonstrates how to set the cuTENSOR backend globally for OMEinsum operations. Ensure cuTENSOR.jl is loaded first to enable its extension. ```julia using CUDA using cuTENSOR # ← Required! This loads the CuTENSORExt extension using OMEinsum # Set the backend globally set_einsum_backend!(CuTensorBackend()) # Now all GPU einsum operations will use cuTENSOR A = CUDA.rand(Float32, 100, 200, 300) B = CUDA.rand(Float32, 200, 300, 400) C = ein"ijk,jkl->il"(A, B) # Uses cuTENSOR # Reset to default backend set_einsum_backend!(DefaultBackend()) ``` -------------------------------- ### Basic CUDA Usage and Benchmarking Source: https://github.com/under-peter/omeinsum.jl/blob/master/docs/src/cuda.md Demonstrates uploading data to the GPU and benchmarking tensor contraction performance using OMEinsum with CUDA.jl. It compares CPU and GPU execution times. ```julia julia> using CUDA, OMEinsum julia> code = ein"ij,jk,kl,li->" # the einsum notation ij, jk, kl, li -> julia> A, B, C, D = rand(1000, 1000), rand(1000, 300), rand(300, 800), rand(800, 1000); julia> size_dict = OMEinsum.get_size_dict(getixsv(code), (A, B, C, D)) # get the size of the labels Dict{Char, Int64} with 4 entries: 'j' => 1000 'i' => 1000 'k' => 300 'l' => 800 julia> optcode = optimize_code(code, size_dict, TreeSA()) # optimize the contraction order SlicedEinsum{Char, DynamicNestedEinsum{Char}}(Char[], kl, kl -> ├─ ki, li -> kl │ ├─ jk, ij -> ki │ │ ├─ jk │ │ └─ ij │ └─ li └─ kl ) ``` ```julia julia> using BenchmarkTools julia> @btime optcode($A, $B, $C, $D) # the contraction on CPU 6.053 ms (308 allocations: 20.16 MiB) 0-dimensional Array{Float64, 0}: 1.4984046443610943e10 ``` ```julia julia> cuA, cuB, cuC, cuD = CuArray(A), CuArray(B), CuArray(C), CuArray(D); julia> @btime CUDA.@sync optcode($cuA, $cuB, $cuC, $cuD) # the contraction on GPU 243.888 μs (763 allocations: 28.56 KiB) 0-dimensional CuArray{Float64, 0, CUDA.DeviceMemory}: 1.4984046443610939e10 ``` -------------------------------- ### Benchmark CuTensorBackend with GEMM Source: https://github.com/under-peter/omeinsum.jl/blob/master/docs/src/cuda.md Benchmarks the CuTensorBackend for a GEMM-like contraction (ijk,jkl->il) using CUDA.@sync for synchronization. ```julia set_einsum_backend!(CuTensorBackend()) @btime CUDA.@sync ein"ijk,jkl->il"($A, $B) ``` -------------------------------- ### Einsum Construction Approach Source: https://github.com/under-peter/omeinsum.jl/blob/master/README.md Demonstrates an alternative way to specify Einstein summation contractions using a construction approach, which is useful when the contraction code is determined at runtime. ```julia julia> einsum(EinCode((('i','k'),('k','j')),('i','j')),(a,b)) ``` -------------------------------- ### Matrix Contraction with Default GPU Backend (CUBLAS) Source: https://github.com/under-peter/omeinsum.jl/blob/master/README.md Use the default backend for matrix-like contractions on the GPU. Ensure CUDA and OMEinsum are imported. ```julia using CUDA using OMEinsum A = CUDA.rand(Float32, 100, 200) B = CUDA.rand(Float32, 200, 300) # Default backend (CUBLAS) - good for matrix-like operations C = ein"ij,jk->ik"(A, B) ``` -------------------------------- ### Construct Optimized Einsum with @optein_str Source: https://github.com/under-peter/omeinsum.jl/blob/master/docs/src/contractionorder.md Conveniently constructs an optimized einsum contraction using the `@optein_str` macro. Assumes each index has a size of 2, which may not always yield the optimal order. ```julia optein"ij,jk,kl,li->" ``` -------------------------------- ### Optimized vs. Manual Contraction Order Source: https://github.com/under-peter/omeinsum.jl/blob/master/docs/src/basic.md Illustrates optimized contraction order using `@optein_str` for multiple tensors and contrasts it with a manually specified contraction order using `ein`. Optimized contraction is preferred when the order is unknown. ```julia tensors = [randn(100, 100) for _ in 1:4] optein"ij,jk,kl,lm->im" (tensors...) ein"(ij,jk),(kl,lm)->im" (tensors...) ``` -------------------------------- ### Perform Matrix Multiplication and Related Operations Source: https://github.com/under-peter/omeinsum.jl/blob/master/docs/src/basic.md Demonstrates using the defined einsum notation for matrix multiplication, retrieving size information, and performing lower-level and in-place operations. The `@ein` macro offers a convenient all-in-one approach. ```julia A, B = randn(2, 3), randn(3, 4) code1(A, B) size_dict = OMEinsum.get_size_dict(getixsv(code1), (A, B)) einsum(code1, (A, B), size_dict) einsum!(code1, (A, B), zeros(2, 4), true, false, size_dict) @ein C[i,k] := A[i,j] * B[j,k] ``` -------------------------------- ### Einsum Notation for Matrix Multiplication Source: https://github.com/under-peter/omeinsum.jl/blob/master/docs/src/background.md Demonstrates the concise representation of matrix multiplication using einsum notation with a string specifying input and output indices. ```einsum "ij,jk->ik" ``` -------------------------------- ### Nested Contractions with Optimized Order using CuTensorBackend Source: https://github.com/under-peter/omeinsum.jl/blob/master/docs/src/cuda.md Shows how to use the CuTensorBackend for nested tensor contractions by first optimizing the contraction order and then executing it on GPU arrays. ```julia using CUDA, cuTENSOR, OMEinsum set_einsum_backend!(CuTensorBackend()) # Define a tensor network code = ein"ij,jk,kl,lm->im" # Optimize contraction order size_dict = Dict('i'=>100, 'j'=>100, 'k'=>100, 'l'=>100, 'm'=>100) optcode = optimize_code(code, size_dict, TreeSA()) # Execute with cuTENSOR (each pairwise contraction uses cuTENSOR) A, B, C, D = [CUDA.rand(Float32, 100, 100) for _ in 1:4] result = optcode(A, B, C, D) ``` -------------------------------- ### Performance Comparison: DefaultBackend vs cuTENSOR Source: https://github.com/under-peter/omeinsum.jl/blob/master/docs/src/cuda.md Compares the performance of tensor contractions using the DefaultBackend (CUBLAS) and cuTENSOR backend for a non-GEMM pattern. Requires cuTENSOR.jl to be loaded. ```julia using CUDA using cuTENSOR # Required for cuTENSOR backend using OMEinsum, BenchmarkTools # Create test tensors (non-GEMM pattern) A = CUDA.rand(Float32, 64, 64, 64) B = CUDA.rand(Float32, 64, 64, 64) # Benchmark with DefaultBackend (CUBLAS) set_einsum_backend!(DefaultBackend()) @btime CUDA.@sync ein"ijk,jkl->il"($A, $B) ``` -------------------------------- ### Keep Data on GPU for Contractions Source: https://github.com/under-peter/omeinsum.jl/blob/master/docs/src/cuda.md Demonstrates the recommended practice of keeping tensor data on the GPU throughout a series of contractions to avoid costly CPU-GPU transfers. ```julia # Good: Keep data on GPU throughout cuA, cuB, cuC = CuArray.((A, B, C)) result = ein"(ij,jk),kl->il"(cuA, cuB, cuC) # Avoid: Repeated transfers result = ein"ij,jk->ik"(CuArray(A), CuArray(B)) # Transfer every call ``` -------------------------------- ### Einsum Macro Interface Source: https://github.com/under-peter/omeinsum.jl/blob/master/README.md Shows the @ein macro interface, which is closer to the standard way of writing Einstein summation operations in physics. This allows for a more declarative syntax. ```julia julia> @ein c[i,j] := a[i,k] * b[k,j]; ``` -------------------------------- ### Disabling Loop Einsum for Optimization Source: https://github.com/under-peter/omeinsum.jl/blob/master/README.md Demonstrates how to use `allow_loops(false)` to prevent the use of `loop_einsum` and catch cases where a more optimized einsum evaluation could be used. An error is printed if `loop_einsum` is invoked. ```julia julia> @ein Zl[o,s] := x[i,s] * W[o,i,j] * y[j,s]; julia> Z ≈ Zl true julia> allow_loops(false); julia> Zl = ein"is, oij, js -> os"(x, W, y); ┌ Error: using `loop_einsum` to evaluate │ code = is, oij, js -> os │ size.(xs) = ((10, 50), (20, 10, 10), (10, 50)) │ size(y) = (20, 50) └ @ OMEinsum ~/.julia/dev/OMEinsum/src/loop_einsum.jl:26 ``` -------------------------------- ### Basic Einsum Operations with @ein_str Source: https://github.com/under-peter/omeinsum.jl/blob/master/README.md Illustrates various Einstein summation operations using the non-standard string literal @ein_str. This includes tensor contraction, matrix multiplication, summing a matrix, and generating an identity matrix. ```julia julia> using OMEinsum, SymEngine julia> catty = fill(Basic(:🐱), 2, 2) 2×2 Array{Basic,2}: 🐱 🐱 🐱 🐱 julia> fish = fill(Basic(:🐟), 2, 3, 2) 2×3×2 Array{Basic,3}: [:, :, 1] = 🐟 🐟 🐟 🐟 🐟 🐟 [:, :, 2] = 🐟 🐟 🐟 🐟 🐟 🐟 julia> snake = fill(Basic(:🐍), 3, 3) 3×3 Array{Basic,2}: 🐍 🐍 🐍 🐍 🐍 🐍 🐍 🐍 🐍 julia> medicine = ein"ij,jki,kk->k"(catty, fish, snake) 3-element Array{Basic,1}: 4*🐱*🐍*🐟 4*🐱*🐍*🐟 4*🐱*🐍*🐟 julia> ein"ik,kj -> ij"(catty, catty) # multiply two matrices `a` and `b` 2×2 Array{Basic,2}: 2*🐱^2 2*🐱^2 2*🐱^2 2*🐱^2 julia> ein"ij -> "(catty)[] # sum a matrix, output 0-dimensional array 4*🐱 julia> ein"->ii"(asarray(snake[1,1]), size_info=Dict('i'=>5)) # get 5 x 5 identity matrix 5×5 Array{Basic,2}: 🐍 0 0 0 0 0 🐍 0 0 0 0 0 🐍 0 0 0 0 0 🐍 0 0 0 0 0 🐍 ``` -------------------------------- ### General Tensor Contraction with cuTENSOR Backend Source: https://github.com/under-peter/omeinsum.jl/blob/master/README.md For non-GEMM patterns and better performance on general tensor networks, switch to the cuTENSOR backend. This requires importing cuTENSOR and setting the backend. ```julia using CUDA using cuTENSOR # Pkg.add("cuTENSOR") - loads the CuTENSORExt extension using OMEinsum set_einsum_backend!(CuTensorBackend()) C = ein"ijk,jkl->il"(CUDA.rand(Float32, 64, 64, 64), CUDA.rand(Float32, 64, 64, 64)) ``` -------------------------------- ### Compute Gradient with Zygote Source: https://github.com/under-peter/omeinsum.jl/blob/master/docs/src/autodiff.md Leverage the Zygote package for general-purpose automatic differentiation of einsum expressions. The backward rule for einsum is compatible with ChainRulesCore, which Zygote uses. Ensure Zygote is imported. ```julia using Zygote # the 2nd way Zygote.gradient((A, B, C)->ein"(ij, jk), ki->"(A, B, C)[], A, B, C) ``` -------------------------------- ### Compute Cost and Gradient with OMEinsum Source: https://github.com/under-peter/omeinsum.jl/blob/master/docs/src/autodiff.md Use the built-in `cost_and_gradient` function from OMEinsum to compute the cost and gradient of an einsum expression. This method is efficient for tensor contractions. Ensure OMEinsum is imported. ```julia using OMEinsum # the 1st way A, B, C = randn(2, 3), randn(3, 4), randn(4, 2); y, g = cost_and_gradient(ein"(ij, jk), ki->", (A, B, C)) ``` -------------------------------- ### Define and Optimize Einsum Operation Source: https://github.com/under-peter/omeinsum.jl/blob/master/docs/src/index.md Define an einsum operation using the `ein` macro and then optimize the contraction order using `optimize_code` with a specified algorithm like `TreeSA`. ```julia using OMEinsum code = ein"ij,jk,kl,lm->im" # define the einsum operation ``` ```julia optcode = optimize_code(code, uniformsize(code, 100), TreeSA()) # optimize the contraction order ``` ```julia optcode(randn(100, 100), randn(100, 100), randn(100, 100), randn(100, 100)) # compute the result ``` -------------------------------- ### Optimize Einsum Code with TreeSA Source: https://github.com/under-peter/omeinsum.jl/blob/master/docs/src/contractionorder.md Optimizes the contraction order of an einsum code using the TreeSA algorithm. This significantly reduces time and read-write complexities. ```julia optcode = optimize_code(code, size_dict, TreeSA(ntrials=1)) ``` -------------------------------- ### Calculate Complexity of Optimized Code Source: https://github.com/under-peter/omeinsum.jl/blob/master/docs/src/contractionorder.md Calculates the time and read-write complexities of an already optimized einsum code. This demonstrates the performance improvement achieved through optimization. ```julia contraction_complexity(optcode, size_dict) ``` -------------------------------- ### Matrix Multiplication as Tensor Contraction Source: https://github.com/under-peter/omeinsum.jl/blob/master/docs/src/background.md Illustrates matrix multiplication as a specific instance of tensor contraction, defining the input tensors, output tensor, and the summation over contracted indices. ```math (AB)\{i, k\} = \texttt{contract}\left(\{i,j,k\}, \\{\\text{A\}\{i, j\}}, B\{\{j, k\}\\}\\}, \\{\\text{i, k\}\\}\right), where matrices A and B are input tensors containing the variable sets \\{\\text{i, j\}\\}, \\{\\text{j, k\}\\}, respectively, which are subsets of \(\Lambda = \\{\\text{i, j, k\}\\\}. The output tensor is comprised of variables \\{\\text{i, k\}\\} and the summation runs over variables \(\Lambda \setminus \\{\\text{i, k\}\\} = \\{\\text{j\}\\\}. The contraction corresponds to (A B)\\{i, k\} = \sum_j A\{\{i,j\}\}B\{\{j, k\}\}. ``` -------------------------------- ### Optimize Peterson graph 3-coloring contraction Source: https://github.com/under-peter/omeinsum.jl/blob/master/docs/src/applications.md Optimizes the tensor contraction order for the Peterson graph 3-coloring problem using `optimize_code` and `TreeSA`. This significantly reduces the computational complexity. ```julia optcode = optimize_code(code, uniformsize(code, 3), TreeSA()) contraction_complexity(optcode, uniformsize(optcode, 3)) optcode(fill(s, 10)...)[] ``` -------------------------------- ### Count 3-colorings for two vertices Source: https://github.com/under-peter/omeinsum.jl/blob/master/docs/src/applications.md Calculates the number of valid 3-colorings for a graph with two vertices by contracting the edge tensor 's' with itself. This demonstrates a basic application of OMEinsum for counting problems. ```julia ein"ijk,ijk->"(s,s)[] ``` -------------------------------- ### Construct DynamicEinCode from Graph Source: https://github.com/under-peter/omeinsum.jl/blob/master/docs/src/contractionorder.md Constructs a DynamicEinCode object from a graph to represent a contraction without an optimized order. This is useful for demonstrating the process before optimization. ```julia using OMEinsum, OMEinsumContractionOrders, OMEinsumContractionOrders.Graphs graph = random_regular_graph(20, 3; seed=42) code = EinCode([[e.src, e.dst] for e in edges(graph)], Int[]) ``` -------------------------------- ### Calculate gradient for relaxed 3-coloring Source: https://github.com/under-peter/omeinsum.jl/blob/master/docs/src/applications.md Uses Zygote's gradient function to calculate the sum of gradients for a relaxed 3-coloring problem on the Peterson graph. This helps determine if 3-colorings exist even when allowing duplicate colors at one vertex. ```julia using Zygote: gradient gradient(x->optcode(x,s,s,s,s,s,s,s,s,s)[], s)[1] |> sum ``` -------------------------------- ### Define Matrix Multiplication with String Literal and EinCode Source: https://github.com/under-peter/omeinsum.jl/blob/master/docs/src/basic.md Shows two equivalent ways to define a matrix multiplication operation: using the `@ein_str` literal and the `EinCode` object. The `EinCode` object is useful for reusing the notation. ```julia using OMEinsum code1 = ein"ij,jk -> ik" ixs = [[1, 2], [2, 3]] iy = [1, 3] code2 = EinCode(ixs, iy) ``` -------------------------------- ### Check GPU Compute Capability Source: https://github.com/under-peter/omeinsum.jl/blob/master/docs/src/cuda.md Retrieves the compute capability of the currently selected CUDA device, ensuring it meets the minimum requirement for cuTENSOR (≥ 6.0). ```julia using CUDA CUDA.capability(CUDA.device()) ``` -------------------------------- ### Manually Specify Contraction Order with @ein_str Source: https://github.com/under-peter/omeinsum.jl/blob/master/docs/src/contractionorder.md Manually specifies the contraction order for an einsum operation using the `@ein_str` string literal. This allows for explicit control over the computation path. ```julia ein"((ij,jk),kl),li->ik" ``` -------------------------------- ### Slice Einsum Code for Memory Reduction Source: https://github.com/under-peter/omeinsum.jl/blob/master/docs/src/contractionorder.md Reduces memory usage of a contraction by slicing the code using TreeSASlicer. This trades increased time complexity for reduced space complexity. ```julia slicer = TreeSASlicer(score=ScoreFunction(sc_target=2)) scode = slice_code(optcode, size_dict, slicer) contraction_complexity(scode, size_dict) scode.slicing ``` -------------------------------- ### Define contraction for Peterson graph 3-coloring Source: https://github.com/under-peter/omeinsum.jl/blob/master/docs/src/applications.md Defines the tensor contraction expression for determining the number of 3-colorings on the Peterson graph. This involves contracting multiple 's' tensors, one for each edge of the graph. ```julia code = ein"afl,bhn,cjf,dlh,enj,ago,big,cki,dmk,eom->" afl, bhn, cjf, dlh, enj, ago, big, cki, dmk, eom code(fill(s, 10)...)[] ``` -------------------------------- ### Calculate Contraction Complexity Source: https://github.com/under-peter/omeinsum.jl/blob/master/docs/src/contractionorder.md Calculates the time and space complexity of a given einsum code. The complexity is returned as log2 values of iterations, largest tensor elements, and elementwise operations. ```julia size_dict = uniformsize(code, 2) # size of the labels are set to 2 contraction_complexity(code, size_dict) # time and space complexity ``` -------------------------------- ### Define a 3-coloring tensor Source: https://github.com/under-peter/omeinsum.jl/blob/master/docs/src/applications.md Defines a tensor 's' representing the possible colorings for a single edge in a 3-coloring problem. This is used as a building block for larger graph coloring problems. ```julia using OMEinsum s = map(x->Int(length(unique(x.I)) == 3), CartesianIndices((3,3,3))) ``` -------------------------------- ### Flatten Optimized Einsum Code Source: https://github.com/under-peter/omeinsum.jl/blob/master/docs/src/contractionorder.md Flattens an optimized einsum code (e.g., a `SlicedEinsum` or `NestedEinsum`) back into a basic `EinCode` object that does not contain contraction order information. ```julia OMEinsum.flatten(optcode) ``` === COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.