### Install OpenCV and Open3D Source: https://github.com/zbw001/tapip3d/blob/main/third_party/megasam/base/thirdparty/lietorch/README.md Install OpenCV and Open3D for running the examples. Depending on the OS, these may need to be built from source. ```bash pip install opencv-python open3d ``` -------------------------------- ### Install from source Source: https://github.com/zbw001/tapip3d/blob/main/third_party/megasam/base/thirdparty/lietorch/README.md Install LieTorch from source. The build may take up to 10 minutes. Run the test script after installation. ```bash git clone --recursive https://github.com/princeton-vl/lietorch.git python setup.py install ./run_tests.sh ``` -------------------------------- ### Comma-initialization example and output Source: https://github.com/zbw001/tapip3d/blob/main/third_party/megasam/base/thirdparty/lietorch/eigen/doc/TutorialMatrixClass.dox Example and output for comma-initialization syntax. The example is included from an external file, and the output is shown separately. ```cpp Tutorial_commainit_01.cpp ``` ```text Tutorial_commainit_01.out ``` -------------------------------- ### Resizing example and output Source: https://github.com/zbw001/tapip3d/blob/main/third_party/megasam/base/thirdparty/lietorch/eigen/doc/TutorialMatrixClass.dox Example and output for resizing dynamic-size matrices using the resize() method. The example is included from an external file, and the output is shown separately. ```cpp tut_matrix_resize.cpp ``` ```text tut_matrix_resize.out ``` -------------------------------- ### Assignment and resizing example and output Source: https://github.com/zbw001/tapip3d/blob/main/third_party/megasam/base/thirdparty/lietorch/eigen/doc/TutorialMatrixClass.dox Example and output for assignment with automatic resizing. The example is included from an external file, and the output is shown separately. ```cpp tut_matrix_assignment_resizing.cpp ``` ```text tut_matrix_assignment_resizing.out ``` -------------------------------- ### Fixed-size resize example and output Source: https://github.com/zbw001/tapip3d/blob/main/third_party/megasam/base/thirdparty/lietorch/eigen/doc/TutorialMatrixClass.dox Example and output for resizing fixed-size matrices. The example is included from an external file, and the output is shown separately. ```cpp tut_matrix_resize_fixed_size.cpp ``` ```text tut_matrix_resize_fixed_size.out ``` -------------------------------- ### Eigen example: TopicStorageOrders_example.cpp and its output Source: https://github.com/zbw001/tapip3d/blob/main/third_party/megasam/base/thirdparty/eigen/doc/StorageOrders.dox The example code (TopicStorageOrders_example.cpp) and its output (TopicStorageOrders_example.out) illustrate the two storage orders using the PlainObjectBase::data() function, which returns a pointer to the first entry of the matrix. The example initializes a column-major matrix and assigns it to a row-major matrix, showing how Eigen reorders entries automatically. ```cpp \include TopicStorageOrders_example.cpp ``` ```text \verbinclude TopicStorageOrders_example.out ``` -------------------------------- ### Output of first Eigen example Source: https://github.com/zbw001/tapip3d/blob/main/third_party/megasam/base/thirdparty/eigen/doc/QuickStartGuide.dox Output produced by the first example program (QuickStart_example.cpp). It shows the 2x2 matrix m after the assignments, with the bottom-right entry computed as m(1,0) + m(0,1). ```text 3 -1 2.5 1.5 ``` -------------------------------- ### Install notebook dependencies Source: https://github.com/zbw001/tapip3d/blob/main/third_party/megasam/base/thirdparty/tartanair_tools/TartanAir_Sample.ipynb Install the required Python packages for the notebook: numpy, azure-storage-blob, and opencv-python. ```bash pip install numpy ``` ```bash pip install azure-storage-blob ``` ```bash pip install opencv-python ``` -------------------------------- ### Simple Eigen MatrixXd program Source: https://github.com/zbw001/tapip3d/blob/main/third_party/megasam/base/thirdparty/eigen/doc/QuickStartGuide.dox A minimal Eigen program that creates a 2x2 MatrixXd, sets its entries, and prints the matrix. It is the first example in the Getting Started guide and is used to demonstrate basic matrix creation and output. ```cpp #include #include using Eigen::MatrixXd; int main() { MatrixXd m(2,2); m(0,0) = 3; m(1,0) = 2.5; m(0,1) = -1; m(1,1) = m(1,0) + m(0,1); std::cout << m << std::endl; } ``` -------------------------------- ### Install via pip Source: https://github.com/zbw001/tapip3d/blob/main/third_party/megasam/base/thirdparty/lietorch/README.md Install LieTorch directly via pip from the GitHub repository. ```bash pip install git+https://github.com/princeton-vl/lietorch.git ``` -------------------------------- ### Install xformers for UniDepth Source: https://github.com/zbw001/tapip3d/blob/main/third_party/megasam/README.md Downloads and installs xformers 0.0.22.post7 from a prebuilt file for Python 3.10, CUDA 11.8, and PyTorch 2.0.1. This is needed for the UniDepth model. ```bash wget https://anaconda.org/xformers/xformers/0.0.22.post7/download/linux-64/xformers-0.0.22.post7-py310_cu11.8.0_pyt2.0.1.tar.bz2 ``` ```bash conda install xformers-0.0.22.post7-py310_cu11.8.0_pyt2.0.1.tar.bz2 ``` -------------------------------- ### Install dependencies Source: https://github.com/zbw001/tapip3d/blob/main/third_party/megasam/MoGe/README.md Install the required Python packages. MoGe is compatible with most versions of the requirements. ```bash pip install -r requirements.txt ``` -------------------------------- ### Coefficient accessors example and output Source: https://github.com/zbw001/tapip3d/blob/main/third_party/megasam/base/thirdparty/lietorch/eigen/doc/TutorialMatrixClass.dox Example and output for coefficient accessors using the overloaded parenthesis operators. The example is included from an external file, and the output is shown separately. ```cpp tut_matrix_coefficient_accessors.cpp ``` ```text tut_matrix_coefficient_accessors.out ``` -------------------------------- ### Output of second Eigen example Source: https://github.com/zbw001/tapip3d/blob/main/third_party/megasam/base/thirdparty/eigen/doc/QuickStartGuide.dox Output produced by the second example program (QuickStart_example2_dynamic.cpp). It shows the scaled random matrix m and the result of multiplying m by the vector v. ```text m = 94.0188 89.844 43.5223 89.018 97.4804 88.6024 49.3967 88.5646 82.6404 m * v = 404.274 512.237 441.273 ``` -------------------------------- ### EIGEN_INITIALIZE_MATRICES_BY_ZERO: 1x1 constructor example Source: https://github.com/zbw001/tapip3d/blob/main/third_party/megasam/base/thirdparty/eigen/doc/PreprocessorDirectives.dox Shows the unary constructor of a 1x1 fixed-size matrix being interpreted as an initialization constructor, where the argument is the coefficient value, not the size. This example creates a vector with coefficients [2,1]. ```cpp Vector2d v(2,1); ``` -------------------------------- ### Simple vector addition example Source: https://github.com/zbw001/tapip3d/blob/main/third_party/megasam/base/thirdparty/eigen/doc/InsideEigenExample.dox This is the main example program that the page analyzes. It creates three dynamic-size float vectors and adds two of them, storing the result in the third. ```cpp #include int main() { int size = 50; // VectorXf is a vector of floats, with dynamic size. Eigen::VectorXf u(size), v(size), w(size); u = v + w; } ``` -------------------------------- ### Example program: Eigen::VectorXf addition Source: https://github.com/zbw001/tapip3d/blob/main/third_party/megasam/base/thirdparty/lietorch/eigen/doc/InsideEigenExample.dox The main example program that demonstrates Eigen's compilation process. It creates three dynamic-size vectors of floats and adds two of them. The page analyzes how Eigen compiles this code with SSE2 vectorization enabled. ```cpp #include int main() { int size = 50; // VectorXf is a vector of floats, with dynamic size. Eigen::VectorXf u(size), v(size), w(size); u = v + w; } ``` -------------------------------- ### Coefficient accessors example Source: https://github.com/zbw001/tapip3d/blob/main/third_party/megasam/base/thirdparty/eigen/doc/TutorialMatrixClass.dox Example of using the parenthesis operator to access and mutate matrix coefficients. The row index is passed first for matrices, and a single index for vectors. Indexing starts at 0. ```cpp #include #include using namespace Eigen; int main() { MatrixXd m(2,2); m(0,0) = 3; m(1,0) = 2.5; m(0,1) = -1; m(1,1) = m(1,0) + m(0,1); std::cout << "Here is the matrix m:\n" << m << std::endl; VectorXd v(2); v(0) = 4; v(1) = v(0) - 1; std::cout << "Here is the vector v:\n" << v << std::endl; } ``` ```text Here is the matrix m: 3 -1 2.5 1.5 Here is the vector v: 4 3 ``` -------------------------------- ### Run MoGe inference with scripts/infer.py Source: https://github.com/zbw001/tapip3d/blob/main/third_party/megasam/MoGe/README.md Two example invocations of scripts/infer.py. The first saves maps, GLB, and PLY outputs; the second displays the result in a window and requires pyglet < 2.0 (e.g., pip install pyglet==1.5.29). ```bash python scripts/infer.py --input IMAGES_FOLDER_OR_IMAGE_PATH --output OUTPUT_FOLDER --maps --glb --ply ``` ```bash python scripts/infer.py --input IMAGES_FOLDER_OR_IMAGE_PATH --output OUTPUT_FOLDER --show ``` -------------------------------- ### Run web demo Source: https://github.com/zbw001/tapip3d/blob/main/third_party/megasam/MoGe/README.md Run the web demo using Gradio. Add --share for public sharing. ```bash python scripts/app.py # --share for Gradio public sharing ``` -------------------------------- ### Comma-initializer example Source: https://github.com/zbw001/tapip3d/blob/main/third_party/megasam/base/thirdparty/eigen/doc/TutorialMatrixClass.dox Shows how to set matrix and vector coefficients using the comma-initializer syntax. ```cpp #include #include using namespace Eigen; int main() { Matrix3f m; m << 1, 2, 3, 4, 5, 6, 7, 8, 9; std::cout << m; } ``` ```text 1 2 3 4 5 6 7 8 9 ``` -------------------------------- ### Run demo.py for SE3/Sim3 registration Source: https://github.com/zbw001/tapip3d/blob/main/third_party/megasam/base/thirdparty/lietorch/examples/registration/readme.md Run the demo for SE3 or Sim3 registration. Requires Open3D and a downloaded model checkpoint (se3.pth or sim3.pth). ```bash python demo.py --transformation=SE3 --ckpt=se3.pth python demo.py --transformation=Sim3 --ckpt=sim3.pth ``` -------------------------------- ### Run demo.py for RGB-D SLAM Source: https://github.com/zbw001/tapip3d/blob/main/third_party/megasam/base/thirdparty/lietorch/examples/rgbdslam/readme.md Requires a GPU with at least 8GB memory. Download a TUM-RGBD sequence first. The --frame_rate flag controls subsampling rate; --go enables global optimization. With RTX-3090 and no visualization, --frame_rate <= 8.0 gives real-time performance. ```bash python demo.py --ckpt=lietorch_rgbdslam.pth --datapath= --frame_rate=8.0 --go --viz ``` -------------------------------- ### Transpose and conjugate example (tut_arithmetic_transpose_conjugate.cpp) Source: https://github.com/zbw001/tapip3d/blob/main/third_party/megasam/base/thirdparty/eigen/doc/TutorialMatrixArithmetic.dox Shows how to obtain the transpose, conjugate, and adjoint (conjugate transpose) of a matrix using transpose(), conjugate(), and adjoint(). For real matrices, conjugate() is a no-op and adjoint() equals transpose(). ```cpp #include #include using namespace Eigen; int main() { MatrixXcf a = MatrixXcf::Random(2,2); std::cout << "Here is the matrix a\n" << a << std::endl; std::cout << "Here is the matrix a^T\n" << a.transpose() << std::endl; std::cout << "Here is the conjugate of a\n" << a.conjugate() << std::endl; std::cout << "Here is the matrix a^*\n" << a.adjoint() << std::endl; } ``` ```cpp Here is the matrix a (-0.211234,0.680375) (0.59688,0.823295) (0.566198,-0.604897) (-0.329554,0.536459) Here is the matrix a^T (-0.211234,0.680375) (0.566198,-0.604897) (0.59688,0.823295) (-0.329554,0.536459) Here is the conjugate of a (-0.211234,-0.680375) (0.59688,-0.823295) (0.566198,0.604897) (-0.329554,-0.536459) Here is the matrix a^* (-0.211234,-0.680375) (0.566198,0.604897) (0.59688,-0.823295) (-0.329554,-0.536459) ``` -------------------------------- ### RowMajor output for extract_patches example Source: https://github.com/zbw001/tapip3d/blob/main/third_party/megasam/base/thirdparty/lietorch/eigen/unsupported/Eigen/CXX11/src/Tensor/README.md Output of the extract_patches example when the data layout is RowMajor. The set of patches is the same as in ColMajor but indexed differently. ```text patch index: 0 0 1 4 5 patch index: 1 1 2 5 6 patch index: 2 2 3 6 7 patch index: 3 4 5 8 9 patch index: 4 5 6 9 10 patch index: 5 6 7 10 11 ``` -------------------------------- ### ColMajor output for extract_patches example Source: https://github.com/zbw001/tapip3d/blob/main/third_party/megasam/base/thirdparty/lietorch/eigen/unsupported/Eigen/CXX11/src/Tensor/README.md Output of the extract_patches example when the data layout is ColMajor. The set of patches is the same as in RowMajor but indexed differently. ```text patch index: 0 0 1 4 5 patch index: 1 4 5 8 9 patch index: 2 1 2 5 6 patch index: 3 5 6 9 10 patch index: 4 2 3 6 7 patch index: 5 6 7 10 11 ``` -------------------------------- ### Run inference with known depths and camera parameters (dexycb.npz) Source: https://github.com/zbw001/tapip3d/blob/main/README.md Runs inference on an .npz file containing rgb, depths, intrinsics, and extrinsics, operating in an aligned global frame to generate world-coordinate trajectories. The example file must be downloaded and placed in demo_inputs/. ```bash python inference.py --input_path demo_inputs/dexycb.npz --checkpoint checkpoints/tapip3d_final.pth --resolution_factor 2 ``` -------------------------------- ### Get total element count with size() Source: https://github.com/zbw001/tapip3d/blob/main/third_party/megasam/base/thirdparty/lietorch/eigen/unsupported/Eigen/CXX11/src/Tensor/README.md Shows how to get the total number of elements in a Tensor using size(). The output is printed to cout. ```cpp Eigen::Tensor a(3, 4); cout << "Size: " << a.size(); => Size: 12 ``` -------------------------------- ### Naive implementation with temporary Source: https://github.com/zbw001/tapip3d/blob/main/third_party/megasam/base/thirdparty/eigen/doc/InsideEigenExample.dox This shows what a naive C++ library would do, introducing a temporary and causing two loops. ```cpp VectorXf tmp = v + w; VectorXf u = tmp; ``` -------------------------------- ### List Easy trajectories for first environment Source: https://github.com/zbw001/tapip3d/blob/main/third_party/megasam/base/thirdparty/tartanair_tools/TartanAir_Sample.ipynb List the 'Easy' trajectories for the first environment (abandonedfactory). The output shows 10 trajectories. ```python diff_level = 'Easy' env_ind = 0 trajlist = get_trajectory_list(envlist[env_ind], easy_hard = diff_level) print('Find {} trajectories in {}'.format(len(trajlist), envlist[env_ind]+diff_level)) print(trajlist) ``` ```text Output: Find 10 trajectories in abandonedfactory/Easy ['abandonedfactory/Easy/P000/', 'abandonedfactory/Easy/P001/', 'abandonedfactory/Easy/P002/', 'abandonedfactory/Easy/P004/', 'abandonedfactory/Easy/P005/', 'abandonedfactory/Easy/P006/', 'abandonedfactory/Easy/P008/', 'abandonedfactory/Easy/P009/', 'abandonedfactory/Easy/P010/', 'abandonedfactory/Easy/P011/'] ``` -------------------------------- ### Dot product and cross product Source: https://github.com/zbw001/tapip3d/blob/main/third_party/megasam/base/thirdparty/eigen/doc/TutorialMatrixArithmetic.dox Demonstrates the dot() and cross() methods for vectors. The example and its output are provided. ```cpp \include tut_arithmetic_dot_cross.cpp ``` ```text \verbinclude tut_arithmetic_dot_cross.out ``` -------------------------------- ### Install lapacke on macOS with MacPorts Source: https://github.com/zbw001/tapip3d/blob/main/third_party/megasam/base/thirdparty/eigen/doc/UsingBlasLapackBackends.dox Required for Mac users to use the LAPACK version shipped with the Accelerate framework. After installation, link with -framework Accelerate and /opt/local/lib/lapack/liblapacke.dylib. ```bash sudo port install lapack ``` -------------------------------- ### Clone and set up conda environment Source: https://github.com/zbw001/tapip3d/blob/main/third_party/megasam/base/thirdparty/lietorch/README.md Clone the repository with submodules and set up a conda environment with dependencies. Requires CUDA >= 10.1 and PyTorch >= 1.8. ```bash git clone --recursive https://github.com/princeton-vl/lietorch.git cd lietorch conda create -n lie_env conda activate lie_env conda install scipy pyyaml pytorch torchvision torchaudio cudatoolkit=10.2 -c pytorch ``` -------------------------------- ### Get Tensor dimensions with dimensions() Source: https://github.com/zbw001/tapip3d/blob/main/third_party/megasam/base/thirdparty/lietorch/eigen/unsupported/Eigen/CXX11/src/Tensor/README.md Shows how to get the dimensions of a Tensor using dimensions() and access the size and individual dimension values. The output is printed to cout. ```cpp Eigen::Tensor a(3, 4); const Eigen::Tensor::Dimensions& d = a.dimensions(); cout << "Dim size: " << d.size << ", dim 0: " << d[0] << ", dim 1: " << d[1]; => Dim size: 2, dim 0: 3, dim 1: 4 ``` -------------------------------- ### N-D Scaling initialization Source: https://github.com/zbw001/tapip3d/blob/main/third_party/megasam/base/thirdparty/eigen/doc/TutorialGeometry.dox Shows the various ways to construct an N-D scaling transformation. ```cpp Scaling(sx, sy) Scaling(sx, sy, sz) Scaling(s) Scaling(vecN) ``` -------------------------------- ### Compile camera tracking extensions Source: https://github.com/zbw001/tapip3d/blob/main/third_party/megasam/README.md Compiles the extensions for the camera tracking module. Run from the project root. ```bash cd base; python setup.py install ``` -------------------------------- ### Transpose aliasing example Source: https://github.com/zbw001/tapip3d/blob/main/third_party/megasam/base/thirdparty/eigen/doc/TopicAliasing.dox Example from TutorialMatrixArithmetic showing aliasing when transposing a matrix in-place. The output shows the issue, and by default Eigen uses a run-time assertion to detect it. ```cpp #include #include using namespace Eigen; int main() { Matrix2i a; a << 1, 2, 3, 4; std::cout << "Here is the matrix a:\n" << a << std::endl; a = a.transpose(); // !!! do NOT do this !!! std::cout << "and the result of the aliasing effect:\n" << a << std::endl; } ``` ```text Here is the matrix a: 1 2 3 4 and the result of the aliasing effect: 1 2 2 4 ``` -------------------------------- ### Initialization Source: https://github.com/zbw001/tapip3d/blob/main/third_party/megasam/base/thirdparty/lietorch/eigen/doc/AsciiQuickReference.txt Initialize matrices and vectors with comma-separated values, fill with a constant, or use static factory methods. ```APIDOC ## Initialization ### Description Initialize matrices and vectors using comma-initializers, fill with a constant, or create identity, zero, ones, random, and linearly spaced matrices/vectors. ### Methods - `A << ...` - Comma-initializer to set matrix elements row by row. - `A.fill(value)` - Fill all elements of `A` with the given value. - `MatrixXd::Identity(rows, cols)` - Create an identity matrix of given size. - `C.setIdentity(rows, cols)` - Resize and set `C` to identity. - `MatrixXd::Zero(rows, cols)` - Create a zero matrix. - `C.setZero(rows, cols)` - Resize and set `C` to zero. - `MatrixXd::Ones(rows, cols)` - Create a matrix of ones. - `C.setOnes(rows, cols)` - Resize and set `C` to ones. - `MatrixXd::Random(rows, cols)` - Create a matrix with uniform random numbers in (-1, 1). - `C.setRandom(rows, cols)` - Resize and set `C` to random values. - `VectorXd::LinSpaced(size, low, high)` - Create a vector with `size` linearly spaced values from `low` to `high`. - `v.setLinSpaced(size, low, high)` - Resize and set `v` to linearly spaced values. - `VectorXi::LinSpaced(size, low, high)` - Create an integer vector with linearly spaced values. ### Example ```cpp A << 1, 2, 3, 4, 5, 6, 7, 8, 9; B << A, A, A; // B is three horizontally stacked A's. A.fill(10); // Fill A with all 10's. MatrixXd::Identity(rows,cols); C.setIdentity(rows,cols); MatrixXd::Zero(rows,cols); C.setZero(rows,cols); MatrixXd::Ones(rows,cols); C.setOnes(rows,cols); MatrixXd::Random(rows,cols); C.setRandom(rows,cols); VectorXd::LinSpaced(size,low,high); v.setLinSpaced(size,low,high); VectorXi::LinSpaced(((hi-low)/step)+1, low, low+step*(size-1)); ``` ```