Pymanopt

https://pymanopt.org
Admin
Pymanopt is a Python toolbox for optimization on Riemannian manifolds using automatic...
Tokens:129,147
Snippets:800
Trust Score:6.3
Update:6 months ago
Show doc for...
Tokens:
Raw
### Install Pymanopt with backend

Source: https://pymanopt.org/docs/stable/_sources/quickstart.rst.txt

Install the package via pip, specifying the desired automatic differentiation backend.

```bash
$ pip install "pymanopt[autograd]" # or [jax], [tensorflow], [torch]
```

--------------------------------

### Install Development Dependencies

Source: https://pymanopt.org/docs/stable/CONTRIBUTING.html

Install the runtime and development dependencies for Pymanopt using pip. This command installs the package in editable mode.

```bash
pip install -e ".[all]"
```

--------------------------------

### Optimization output log

Source: https://pymanopt.org/docs/stable/_sources/quickstart.rst.txt

Example output generated by running the Pymanopt optimization process.

```none
Optimizing...
Iteration    Cost                       Gradient norm
---------    -----------------------    --------------
   1         +1.1041943339110254e+00    5.65626470e-01
   2         +5.2849633289004561e-01    8.90742722e-01
   3         -8.0741058657312559e-01    2.23937710e+00
   4         -1.2667369971251594e+00    1.59671326e+00
   5         -1.4100298597091836e+00    1.11228845e+00
   6         -1.5219408277812505e+00    2.45507203e-01
   7         -1.5269956262562046e+00    6.81712914e-02
   8         -1.5273114803528709e+00    3.40941735e-02
   9         -1.5273905588875487e+00    1.70222768e-02
  10         -1.5274100956128560e+00    8.61140952e-03
  11         -1.5274154319869837e+00    3.90706914e-03
  12         -1.5274156215853507e+00    3.62943721e-03
  13         -1.5274162595152783e+00    2.47643452e-03
  14         -1.5274168030609154e+00    3.66398414e-04
  15         -1.5274168133149475e+00    1.45210081e-04
```

--------------------------------

### Pymanopt Setup for GMM Optimization

Source: https://pymanopt.org/docs/stable/examples/notebooks/mixture_of_gaussians.html

Sets up the Pymanopt problem for optimizing the reparameterized log-likelihood of a Gaussian Mixture Model. It defines the manifold, the cost function using autograd, and the optimizer.

```python
import sys

sys.path.insert(0, "../..")

from autograd.scipy.special import logsumexp

import pymanopt
from pymanopt import Problem
from pymanopt.manifolds import Euclidean, Product, SymmetricPositiveDefinite
from pymanopt.optimizers import SteepestDescent

# Assuming N, D, K, samples are defined

# (1) Instantiate the manifold
manifold = Product([SymmetricPositiveDefinite(D + 1, k=K), Euclidean(K - 1)])

# (2) Define cost function
# The parameters must be contained in a list theta.
@pymanopt.function.autograd(manifold)
def cost(S, v):
    # Unpack parameters
    nu = np.append(v, 0)

    logdetS = np.expand_dims(np.linalg.slogdet(S)[1], 1)
    y = np.concatenate([samples.T, np.ones((1, N))], axis=0)

    # Calculate log_q
    y = np.expand_dims(y, 0)

    # 'Probability' of y belonging to each cluster
    log_q = -0.5 * (np.sum(y * np.linalg.solve(S, y), axis=1) + logdetS)

    alpha = np.exp(nu)
    alpha = alpha / np.sum(alpha)
    alpha = np.expand_dims(alpha, 1)

    loglikvec = logsumexp(np.log(alpha) + log_q, axis=0)
    return -np.sum(loglikvec)


problem = Problem(manifold, cost)

# (3) Instantiate a Pymanopt optimizer
optimizer = SteepestDescent(verbosity=1)
```

--------------------------------

### Install Pre-commit Hook

Source: https://pymanopt.org/docs/stable/CONTRIBUTING.html

Install the pre-commit hook to enforce PEP8 coding style consistency. This hook runs flake8 on staged files before committing.

```bash
pip install -e ".[dev]"
```

--------------------------------

### Pymanopt Optimizers Overview

Source: https://pymanopt.org/docs/stable/_sources/optimizers.rst.txt

This section provides an overview of the different optimization algorithms available in Pymanopt.

```APIDOC
## Pymanopt Optimization Modules

### Optimizers

This module contains various optimization algorithms for manifold optimization.

#### Optimizer

.. automodule:: pymanopt.optimizers.optimizer

#### Conjugate Gradients

.. automodule:: pymanopt.optimizers.conjugate_gradient

#### Nelder-Mead Algorithm

.. automodule:: pymanopt.optimizers.nelder_mead

#### Particle Swarms

.. automodule:: pymanopt.optimizers.particle_swarm

#### Steepest Descent

.. automodule:: pymanopt.optimizers.steepest_descent

#### Riemannian Trust Regions Algorithm

.. automodule:: pymanopt.optimizers.trust_regions

### Line-Search Methods

This module contains line-search algorithms used in conjunction with other optimizers.

.. automodule:: pymanopt.optimizers.line_search
```

--------------------------------

### GET /manifold/random_point

Source: https://pymanopt.org/docs/stable/manifolds.html

Returns a randomly chosen point on the manifold.

```APIDOC
## GET /manifold/random_point

### Description
Returns a random point on the manifold.

### Method
GET

### Endpoint
/manifold/random_point

### Response
#### Success Response (200)
- **point** (object) - A randomly chosen point on the manifold.
```

--------------------------------

### GET /manifold/dist

Source: https://pymanopt.org/docs/stable/manifolds.html

Calculates the geodesic distance between two points on the manifold.

```APIDOC
## GET /manifold/dist

### Description
The geodesic distance between two points on the manifold.

### Method
GET

### Endpoint
/manifold/dist

### Parameters
#### Query Parameters
- **point_a** (object) - Required - The first point on the manifold.
- **point_b** (object) - Required - The second point on the manifold.

### Response
#### Success Response (200)
- **distance** (float) - The distance between `point_a` and `point_b` on the manifold.
```

--------------------------------

### Class: Product

Source: https://pymanopt.org/docs/stable/_modules/pymanopt/manifolds/product.html

Initializes a new Product manifold instance from a sequence of manifolds.

```APIDOC
## Constructor: Product(manifolds)

### Description
Creates a Cartesian product manifold from a collection of individual manifolds. Nested product manifolds are not supported.

### Parameters
#### Request Body
- **manifolds** (Sequence[Manifold]) - Required - A collection of manifolds to form the product.

### Request Example
{
  "manifolds": [manifold1, manifold2]
}
```

--------------------------------

### Pymanopt API Overview

Source: https://pymanopt.org/docs/stable/api-reference.html

Overview of the core modules available in the Pymanopt library for Riemannian optimization.

```APIDOC
## Pymanopt API Reference

### Description
The Pymanopt API is organized into several core modules that facilitate optimization on manifolds.

### Modules
- **Manifolds**: Defines the geometry of the search space (e.g., Sphere, Stiefel, Grassmann, Symmetric Positive Definite Matrices).
- **Optimization**: Contains solvers such as Conjugate Gradients, Nelder-Mead, Particle Swarms, Steepest Descent, and Riemannian Trust Regions.
- **Automatic Differentiation**: Provides backends for computing gradients automatically.
- **Problem**: Defines the objective function and manifold constraints.
- **Tools**: Includes diagnostic and testing utilities.
```

--------------------------------

### GET /manifold/random_tangent_vector

Source: https://pymanopt.org/docs/stable/manifolds.html

Returns a random vector in the tangent space at a specified point.

```APIDOC
## GET /manifold/random_tangent_vector

### Description
Returns a random vector in the tangent space at `point`.

### Method
GET

### Endpoint
/manifold/random_tangent_vector

### Parameters
#### Query Parameters
- **point** (object) - Required - A point on the manifold.

### Response
#### Success Response (200)
- **tangent_vector** (object) - A randomly chosen tangent vector in the tangent space at `point`.
```

--------------------------------

### Initialize Product Manifold

Source: https://pymanopt.org/docs/stable/_modules/pymanopt/manifolds/product.html

Instantiate a Product manifold by providing a sequence of individual manifolds. Nested product manifolds are not supported.

```python
Product(manifolds: Sequence[Manifold])
```

--------------------------------

### Enable Pre-commit Hook

Source: https://pymanopt.org/docs/stable/CONTRIBUTING.html

Enable the pre-commit hook after installing pre-commit. This ensures flake8 is run automatically on staged files.

```bash
pre-commit install
```

--------------------------------

### ParticleSwarm Initialization Parameters

Source: https://pymanopt.org/docs/stable/_modules/pymanopt/optimizers/particle_swarm.html

The __init__ method for the ParticleSwarm optimizer sets up the core parameters for the algorithm. Default values for max_cost_evaluations, max_iterations, and population_size are determined dynamically based on the manifold's dimension during the run method.

```python
def __init__(
    self,
    max_cost_evaluations=None,
    max_iterations=None,
    population_size=None,
    nostalgia=1.4,
    social=1.4,
    *args,
    **kwargs,
):
    super().__init__(*args, **kwargs)

    self._max_cost_evaluations = max_cost_evaluations
    self._max_iterations = max_iterations
    self._population_size = population_size
    self._nostalgia = nostalgia
    self._social = social
```

--------------------------------

### Get Zero Vector on Product Manifold

Source: https://pymanopt.org/docs/stable/_modules/pymanopt/manifolds/product.html

Returns the zero vector at a given point on the Product manifold. The result is a `_ProductTangentVector`.

```python
def zero_vector(self, point):
        return self._dispatch("zero_vector", reduction=_ProductTangentVector)(
            point
        )
```

--------------------------------

### Initialize ParticleSwarm Optimizer

Source: https://pymanopt.org/docs/stable/_modules/pymanopt/optimizers/particle_swarm.html

Initializes the ParticleSwarm optimizer with various parameters controlling the optimization process. Default values are set based on manifold dimension if not provided.

```python
from pymanopt.optimizers import ParticleSwarm

pso = ParticleSwarm(
    max_cost_evaluations=1000,
    max_iterations=500,
    population_size=30,
    nostalgia=1.5,
    social=1.5
)
```

--------------------------------

### Solve dominant eigenvector problem

Source: https://pymanopt.org/docs/stable/_sources/quickstart.rst.txt

A minimal example demonstrating how to find the dominant eigenvector of a symmetric matrix using the Sphere manifold and Autograd backend.

```python
import autograd.numpy as anp
import pymanopt

anp.random.seed(42)

dim = 3
manifold = pymanopt.manifolds.Sphere(dim)

matrix = anp.random.normal(size=(dim, dim))
matrix = 0.5 * (matrix + matrix.T)

@pymanopt.function.autograd(manifold)
def cost(point):
    return -point @ matrix @ point

problem = pymanopt.Problem(manifold, cost)

optimizer = pymanopt.optimizers.SteepestDescent()
result = optimizer.run(problem)

eigenvalues, eigenvectors = anp.linalg.eig(matrix)
dominant_eigenvector = eigenvectors[:, eigenvalues.argmax()]

print("Dominant eigenvector:", dominant_eigenvector)
print("Pymanopt solution:", result.point)
```

--------------------------------

### pymanopt.optimizers.nelder_mead

Source: https://pymanopt.org/docs/stable/_modules/pymanopt/optimizers/nelder_mead.html

Documentation for the Nelder-Mead optimizer module in Pymanopt.

```APIDOC
## pymanopt.optimizers.nelder_mead

### Description
This module provides the implementation of the Nelder-Mead optimization algorithm, a derivative-free optimization method used for finding the local minimum of a function on a manifold.

### Module Path
pymanopt.optimizers.nelder_mead
```

--------------------------------

### ComplexCircle Exponential Map

Source: https://pymanopt.org/docs/stable/_modules/pymanopt/manifolds/complex_circle.html

Computes the exponential map of a tangent vector at a given point on the ComplexCircle manifold. This operation moves along the geodesic starting from the point in the direction of the tangent vector.

```python
def exp(self, point, tangent_vector):
    tangent_vector_abs = np.abs(tangent_vector)
    mask = tangent_vector_abs > 0
    not_mask = np.logical_not(mask)
    tangent_vector_new = np.zeros(self._dimension)
    tangent_vector_new[mask] = point[mask] * np.cos(
        tangent_vector_abs[mask]
    ) + tangent_vector[mask] * (
        np.sin(tangent_vector_abs[mask]) / tangent_vector_abs[mask]
    )
    tangent_vector_new[not_mask] = point[not_mask]
    return tangent_vector_new
```

--------------------------------

### Nelder-Mead Run Method (Python)

Source: https://pymanopt.org/docs/stable/_modules/pymanopt/optimizers/nelder_mead.html

Executes the Nelder-Mead optimization algorithm on a given Pymanopt problem. It handles initialization of the simplex, cost evaluation, and iteration until a stopping criterion is met.

```python
def run(self, problem, *, initial_point=None) -> OptimizerResult:
    """Run Nelder-Mead algorithm.

    Args:
        problem: Pymanopt problem class instance exposing the cost function
            and the manifold to optimize over.
        initial_point: Initial point on the manifold.
            If no value is provided then a starting point will be randomly
            generated.

    Returns:
        Local minimum of the cost function, or the most recent iterate if
        algorithm terminated before convergence.
    """
    manifold = problem.manifold
    objective = problem.cost

    # Choose proper default algorithm parameters. We need to know about the
    # dimension of the manifold to limit the parameter range, so we have to
    # defer proper initialization until this point.
    dim = manifold.dim
    if self._max_cost_evaluations is None:
        self._max_cost_evaluations = max(1000, 2 * dim)
    if self._max_iterations is None:
        self._max_iterations = max(2000, 4 * dim)

    # If no initial simplex x is given by the user, generate one at random.
    num_points = int(dim + 1)
    if initial_point is None:
        x = [manifold.random_point() for _ in range(num_points)]
    elif (
        tools.is_sequence(initial_point)
        and len(initial_point) != num_points
    ):
        x = initial_point
    else:
        raise ValueError(
            "The initial simplex `initial_point` must be a sequence of "
            f"{num_points} points"
        )

    # Compute objective-related quantities for x, and setup a function
    # evaluations counter.
    costs = np.array([objective(xi) for xi in x])
    cost_evaluations = dim + 1

    # Sort simplex points by cost.
    order = np.argsort(costs)
    costs = costs[order]
    x = [x[i] for i in order]

    # Iteration counter (at any point, iteration is the number of fully
    # executed iterations so far).
    iteration = 0

    start_time = time.time()

    self._initialize_log()

    while True:
        iteration += 1

        if self._verbosity >= 2:
            print(
                f"Cost evals: {cost_evaluations:7d}	"
                f"Best cost: {costs[0]:+.8e}"
            )

        # Sort simplex points by cost.
        order = np.argsort(costs)
        costs = costs[order]
        x = [x[i] for i in order]

        stopping_criterion = self._check_stopping_criterion(
            start_time=start_time,
            iteration=iteration,
            cost_evaluations=cost_evaluations,
        )
        if stopping_criterion:

```

--------------------------------

### Clone Pymanopt Repository

Source: https://pymanopt.org/docs/stable/CONTRIBUTING.html

Clone the Pymanopt repository to your local machine. Replace `<your-username>` with your GitHub username.

```bash
git clone git@github.com:<your-username>/pymanopt.git
```

--------------------------------

### Generate MoG Samples and Plot

Source: https://pymanopt.org/docs/stable/_sources/examples/notebooks/mixture_of_gaussians.ipynb.txt

Generates N samples from a Mixture of Gaussians model with specified means, covariances, and mixture probabilities. It then visualizes these samples using a scatter plot. Ensure matplotlib is installed for plotting.

```python
import autograd.numpy as np


np.set_printoptions(precision=2)
import matplotlib.pyplot as plt


%matplotlib inline

# Number of data points
N = 1000

# Dimension of each data point
D = 2

# Number of clusters
K = 3

pi = [0.1, 0.6, 0.3]
mu = [np.array([-4, 1]), np.array([0, 0]), np.array([2, -1])]
Sigma = [
    np.array([[3, 0], [0, 1]]),
    np.array([[1, 1.0], [1, 3]]),
    0.5 * np.eye(2),
]

components = np.random.choice(K, size=N, p=pi)
samples = np.zeros((N, D))
# For each component, generate all needed samples
for k in range(K):
    # indices of current component in X
    indices = k == components
    # number of those occurrences
    n_k = indices.sum()
    if n_k > 0:
        samples[indices, :] = np.random.multivariate_normal(
            mu[k], Sigma[k], n_k
        )

colors = ["r", "g", "b", "c", "m"]
for k in range(K):
    indices = k == components
    plt.scatter(
        samples[indices, 0],
        samples[indices, 1],
        alpha=0.4,
        color=colors[k % K],
    )
plt.axis("equal")
plt.show()
```

--------------------------------

### Initialize TrustRegions Optimizer

Source: https://pymanopt.org/docs/stable/_modules/pymanopt/optimizers/trust_regions.html

Configures the Riemannian Trust-Regions algorithm with hyperparameters for the trust region radius and inner iteration constraints.

```python
def __init__(
        self,
        miniter=3,
        kappa=0.1,
        theta=1.0,
        rho_prime=0.1,
        use_rand=False,
        rho_regularization=1e3,
        *args,
        **kwargs,
    ):
        """Riemannian Trust-Regions algorithm.

        Second-order method that approximates the objective function by a
        quadratic surface and then updates the current estimate based on that.
        Also included is the truncated (Steihaug-Toint) conjugate gradient
        algorithm.
        """
        super().__init__(*args, **kwargs)

        self.miniter = miniter
        self.kappa = kappa
        self.theta = theta
        self.rho_prime = rho_prime
        self.use_rand = use_rand
        self.rho_regularization = rho_regularization
```

--------------------------------

### Nelder-Mead Optimizer Initialization (Python)

Source: https://pymanopt.org/docs/stable/_modules/pymanopt/optimizers/nelder_mead.html

Initializes the Nelder-Mead optimizer with customizable parameters for cost evaluations, iterations, and simplex manipulation. Default parameters are adjusted based on manifold dimension.

```python
class NelderMead(Optimizer):
    """Nelder-Mead alglorithm.

    Perform optimization using the derivative-free Nelder-Mead minimization
    algorithm.

    Args:
        max_cost_evaluations: Maximum number of allowed cost function
            evaluations.
        max_iterations: Maximum number of allowed iterations.
        reflection: Determines how far to reflect away from the worst vertex:
            stretched (reflection > 1), compressed (0 < reflection < 1),
            or exact (reflection = 1).
        expansion: Factor by which to expand the reflected simplex.
        contraction: Factor by which to contract the reflected simplex.
    """

    def __init__(
        self,
        max_cost_evaluations=None,
        max_iterations=None,
        reflection=1,
        expansion=2,
        contraction=0.5,
        *args,
        **kwargs,
    ):
        super().__init__(*args, **kwargs)

        self._max_cost_evaluations = max_cost_evaluations
        self._max_iterations = max_iterations
        self._reflection = reflection
        self._expansion = expansion
        self._contraction = contraction
```

--------------------------------

### ParticleSwarm Run Method Logic

Source: https://pymanopt.org/docs/stable/_modules/pymanopt/optimizers/particle_swarm.html

The run method initializes PSO parameters, generates the initial population (or uses a provided one), and sets up initial velocities and personal bests before entering the main optimization loop.

```python
def run(self, problem, *, initial_point=None) -> OptimizerResult:
    manifold = problem.manifold
    objective = problem.cost

    dim = manifold.dim
    if self._max_cost_evaluations is None:
        self._max_cost_evaluations = max(5000, 2 * dim)
    if self._max_iterations is None:
        self._max_iterations = max(500, 4 * dim)
    if self._population_size is None:
        self._population_size = min(40, 10 * dim)

    if initial_point is None:
        x = [
            manifold.random_point()
            for i in range(int(self._population_size))
        ]
    elif tools.is_sequence(initial_point):
        if len(initial_point) != self._population_size:
            print(
                "The population size was forced to the size of "
                "the given initial population"
            )
            self._population_size = len(initial_point)
        x = initial_point
    else:
        raise ValueError("The initial population must be iterable")

    y = list(x)
    xprev = list(x)
    v = [manifold.random_tangent_vector(xi) for xi in x]

    costs = np.array([objective(xi) for xi in x])
    fy = list(costs)
    cost_evaluations = self._population_size

    imin = costs.argmin()
    fbest = costs[imin]
    xbest = x[imin]

    if self._verbosity >= 2:
        iteration_format_length = int(np.log10(self._max_iterations)) + 1
        column_printer = printer.ColumnPrinter(
            columns=[
                ("Iteration", f"{iteration_format_length}d"),
                ("Cost evaluations", "7d"),
                ("Best cost", "+.8e"),
            ]
        )
    else:
        column_printer = printer.VoidPrinter()

    column_printer.print_header()

    self._initialize_log()

    iteration = 0
    start_time = time.time()

    while True:
        iteration += 1

        column_printer.print_row([iteration, cost_evaluations, fbest])

        for i, xi in enumerate(x):  # noqa

```

--------------------------------

### Initialize Unitary Group Manifold

Source: https://pymanopt.org/docs/stable/_modules/pymanopt/manifolds/group.html

Constructor for the Unitary group manifold, supporting product manifolds U(n)^k.

```python
    def __init__(self, n: int, *, k: int = 1, retraction: str = "qr"):
        self._n = n
        self._k = k

        if k == 1:
            name = f"Unitary group U({n})"
        elif k > 1:
            name = f"Unitary group U({n})^{k}"
        else:
            raise ValueError("k must be an integer no less than 1.")
        dimension = int(k * n**2)
        super().__init__(name, dimension, retraction)
```

--------------------------------

### PSDFixedRank Class Initialization

Source: https://pymanopt.org/docs/stable/manifolds.html

Initializes the manifold of fixed-rank positive semidefinite matrices of size n x n and rank k.

```APIDOC
## PSDFixedRank(n, k)

### Description
Initializes the manifold of fixed-rank positive semidefinite (PSD) matrices.

### Parameters
#### Path Parameters
- **n** (int) - Required - Number of rows and columns of a point in the ambient space.
- **k** (int) - Required - Rank of matrices in the ambient space.
```

--------------------------------

### Run Pymanopt Optimization

Source: https://pymanopt.org/docs/stable/examples/notebooks/mixture_of_gaussians.html

Execute the optimization process on a defined problem.

```python
Xopt = optimizer.run(problem).point
```

--------------------------------

### Pymanopt Optimization Output

Source: https://pymanopt.org/docs/stable/quickstart.html

This output shows the progress of the SteepestDescent optimizer in Pymanopt, detailing the cost and gradient norm at each iteration until convergence.

```text
Optimizing...
Iteration    Cost                       Gradient norm
---------    -----------------------    --------------
   1         +1.1041943339110254e+00    5.65626470e-01
   2         +5.2849633289004561e-01    8.90742722e-01
   3         -8.0741058657312559e-01    2.23937710e+00
   4         -1.2667369971251594e+00    1.59671326e+00
   5         -1.4100298597091836e+00    1.11228845e+00
   6         -1.5219408277812505e+00    2.45507203e-01
   7         -1.5269956262562046e+00    6.81712914e-02
   8         -1.5273114803528709e+00    3.40941735e-02
   9         -1.5273905588875487e+00    1.70222768e-02
  10         -1.5274100956128560e+00    8.61140952e-03
  11         -1.5274154319869837e+00    3.90706914e-03
  12         -1.5274156215853507e+00    3.62943721e-03
  13         -1.5274162595152783e+00    2.47643452e-03
  14         -1.5274168030609154e+00    3.66398414e-04
  15         -1.5274168133149475e+00    1.45210081e-04
  16         -1.5274168150025758e+00    4.96142583e-05
  17         -1.5274168150483476e+00    4.42317042e-05
  18         -1.5274168151841643e+00    2.13915041e-05
  19         -1.5274168152087644e+00    1.36422863e-05
  20         -1.5274168152220804e+00    6.25780214e-06
  21         -1.5274168152229037e+00    5.48381052e-06
  22         -1.5274168152252021e+00    2.16996083e-06
  23         -1.5274168152255774e+00    7.52279600e-07
Terminated - min grad norm reached after 23 iterations, 0.01 seconds.
```

--------------------------------

### Initialize HermitianPositiveDefinite Manifold

Source: https://pymanopt.org/docs/stable/_modules/pymanopt/manifolds/positive_definite.html

Constructor for the Hermitian positive definite manifold, accepting matrix size n and product geometry count k.

```python
    def __init__(self, n: int, *, k: int = 1):
        self._n = n
        self._k = k

        if k == 1:
            name = f"Manifold of Hermitian positive definite {n}x{n} matrices"
        else:
            name = (
                f"Product manifold of {k} "
                f"Hermitian positive definite {n}x{n} matrices"
            )
        dimension = int(k * n * (n + 1))
        super().__init__(name, dimension)
```

--------------------------------

### Run Pymanopt Test Suite

Source: https://pymanopt.org/docs/stable/CONTRIBUTING.html

Execute the Pymanopt test suite with coverage reporting. This command verifies that all existing tests pass and checks code coverage.

```bash
pytest -v --cov-report= --cov=src/pymanopt tests
```

--------------------------------

### Print Ground Truth Parameters

Source: https://pymanopt.org/docs/stable/examples/notebooks/mixture_of_gaussians.html

Display the ground truth parameters for comparison.

```python
print(mu[0])
print(Sigma[0])
print(mu[1])
print(Sigma[1])
print(mu[2])
print(Sigma[2])
print(pi[0])
print(pi[1])
print(pi[2])
```

--------------------------------

### Initialize Hermitian Quotient Manifold

Source: https://pymanopt.org/docs/stable/_modules/pymanopt/manifolds/psd.html

Constructor for the quotient manifold of Hermitian matrices, setting the manifold name and dimension based on matrix size n and rank k.

```python
def __init__(self, n, k):
    name = f"Quotient manifold of Hermitian {n}x{n} matrices of rank {k}"
    dimension = 2 * k * n - k * k
    super().__init__(n, k, name, dimension)
```

--------------------------------

### Initialize Positive Manifold

Source: https://pymanopt.org/docs/stable/_modules/pymanopt/manifolds/positive.html

Initializes the Positive manifold for matrices of size m x n, optionally as a product of k matrices. Use this to define the space for optimization.

```python
Positive(m, n, k=1, use_parallel_transport=False)
```

--------------------------------

### Print Optimization Iteration Details

Source: https://pymanopt.org/docs/stable/_modules/pymanopt/optimizers/trust_regions.html

Prints detailed information about each iteration of the optimization process, including convergence status, iteration count, and gradient norm. Useful for monitoring progress.

```python
print(f"{accstr:.3s} {trstr:.3s}    k: {iteration:5d}     "
                    f"num_inner: {numit:5d}     {srstr:s}")
                print(f"       f(x) : {fx:+e}     |grad| : {norm_grad:e}")
                print(f"        rho : {rho:e}")
```

--------------------------------

### Print Ground Truth and Inferred Parameters

Source: https://pymanopt.org/docs/stable/_sources/examples/notebooks/mixture_of_gaussians.ipynb.txt

Displays the ground truth and inferred parameters for comparison.

```python
print(mu[0])
print(Sigma[0])
print(mu[1])
print(Sigma[1])
print(mu[2])
print(Sigma[2])
print(pi[0])
print(pi[1])
print(pi[2])
```

```python
print(mu1hat)
print(Sigma1hat)
print(mu2hat)
print(Sigma2hat)
print(mu3hat)
print(Sigma3hat)
print(pihat[0])
print(pihat[1])
print(pihat[2])
```

--------------------------------

### Problem Class Initialization

Source: https://pymanopt.org/docs/stable/_modules/pymanopt/core/problem.html

Initializes a Problem object to define a Riemannian optimization problem.

```APIDOC
## Problem Class

### Description
Defines a Riemannian optimization problem by specifying the manifold, cost function, and optionally its derivatives (gradient and Hessian).

### Method
__init__

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
- **manifold** (Manifold) - Required - The manifold on which to optimize.
- **cost** (Function) - Required - A callable representing the cost function, decorated with a pymanopt backend decorator.
- **euclidean_gradient** (Optional[Function]) - Optional - The Euclidean gradient of the cost function.
- **riemannian_gradient** (Optional[Function]) - Optional - The Riemannian gradient of the cost function.
- **euclidean_hessian** (Optional[Function]) - Optional - The Euclidean Hessian of the cost function.
- **riemannian_hessian** (Optional[Function]) - Optional - The Riemannian Hessian of the cost function.
- **preconditioner** (Optional[Callable]) - Optional - A callable for preconditioning.

### Request Example
```python
from pymanopt import Problem
from pymanopt.manifolds import Sphere
from pymanopt.function import numpy as pymanopt_numpy

# Define a manifold
manifold = Sphere(3)

# Define a cost function (e.g., sum of squares)
@pymanopt_numpy
def cost_function(x):
    return np.sum(x**2)

# Create a Problem instance
problem = Problem(manifold=manifold, cost=cost_function)
```

### Response
#### Success Response (200)
N/A (This is a class constructor)

#### Response Example
N/A
```

--------------------------------

### Pymanopt Tools Modules

Source: https://pymanopt.org/docs/stable/_sources/tools.rst.txt

Overview of the available tool modules for Pymanopt, covering multi-dimensional array handling, diagnostics, and testing utilities.

```APIDOC
## Pymanopt Tools Modules

### Description
This section provides access to the internal utility modules used within Pymanopt for mathematical operations, diagnostics, and testing.

### Modules
- **pymanopt.tools.multi**: Provides tools for handling multi-dimensional array operations.
- **pymanopt.tools.diagnostics**: Provides diagnostic utilities for verifying optimization problems and manifold properties.
- **pymanopt.tools.testing**: Provides testing utilities for validating Pymanopt implementations.
```

--------------------------------

### Visualize Sample Data for GMM

Source: https://pymanopt.org/docs/stable/examples/notebooks/mixture_of_gaussians.html

Visualizes the generated sample data points, colored by their component assignment. This helps in understanding the distribution of the synthetic data.

```python
import matplotlib.pyplot as plt

colors = ["r", "g", "b", "c", "m"]
for k in range(K):
    indices = k == components
    plt.scatter(
        samples[indices, 0],
        samples[indices, 1],
        alpha=0.4,
        color=colors[k % K],
    )
plt.axis("equal")
plt.show()
```

--------------------------------

### Initialize Elliptope Manifold

Source: https://pymanopt.org/docs/stable/_modules/pymanopt/manifolds/psd.html

Constructor for the Elliptope manifold, which represents fixed-rank PSD matrices with unit diagonal elements.

```python
def __init__(self, n, k):
    self._n = n
    self._k = k

    name = (
        f"Quotient manifold of {n}x{n} psd matrices of rank {k} "
        "with unit diagonal elements"
    )
    dimension = int(n * (k - 1) - k * (k - 1) / 2)
    super().__init__(name, dimension)
```

--------------------------------

### Class: pymanopt.core.problem.Problem

Source: https://pymanopt.org/docs/stable/problem.html

Constructor for the Problem class used to define a Riemannian optimization problem.

```APIDOC
## Class: pymanopt.core.problem.Problem

### Description
Initializes a new Riemannian optimization problem instance.

### Parameters
- **manifold** (pymanopt.manifolds.manifold.Manifold) - Required - Manifold to optimize over.
- **cost** (pymanopt.autodiff.Function) - Required - A callable decorated with a decorator from pymanopt.functions which takes a point on a manifold and returns a real scalar.
- **euclidean_gradient** (Optional[pymanopt.autodiff.Function]) - Optional - The Euclidean gradient of the cost function.
- **riemannian_gradient** (Optional[pymanopt.autodiff.Function]) - Optional - The Riemannian gradient of the cost function.
- **euclidean_hessian** (Optional[pymanopt.autodiff.Function]) - Optional - The Euclidean Hessian of the cost function.
- **riemannian_hessian** (Optional[pymanopt.autodiff.Function]) - Optional - The Riemannian Hessian of the cost function.
- **preconditioner** (Optional[Callable]) - Optional - A preconditioner function.
```

--------------------------------

### Conjugate Gradient Optimizer Initialization

Source: https://pymanopt.org/docs/stable/_modules/pymanopt/optimizers/conjugate_gradient.html

Initializes the ConjugateGradient optimizer, allowing selection of the beta rule, Powell's restart strategy parameter, and a line searcher. Defaults to Hestenes-Stiefel beta rule and AdaptiveLineSearcher.

```python
class ConjugateGradient(Optimizer):
    """Riemannian conjugate gradient method.

    Perform optimization using nonlinear conjugate gradient method with
    line_searcher.
    This method first computes the gradient of the cost function, and then
    optimizes by moving in a direction that is conjugate to all previous search
    directions.

    Args:
        beta_rule: Conjugate gradient beta rule used to construct the new
            search direction. Valid choices are ``{"Fletcher-Reeves",
            "PolakRibiere", "HestenesStiefel", "HagerZhang"}``.
        orth_value: Parameter for Powell's restart strategy.
            An infinite value disables this strategy.
        line_searcher: The line search method.

    Notes:
        See [HZ2006]_ for details about Powell's restart strategy.
    """

    def __init__(
        self,
        beta_rule: str = "HestenesStiefel",
        orth_value=np.inf,
        line_searcher=None,
        *args,
        **kwargs,
    ):
        super().__init__(*args, **kwargs)

        try:
            self._beta_update = BETA_RULES[beta_rule]
        except KeyError:
            raise ValueError(
                f"Invalid beta rule '{beta_rule}'. Should be one of "
                f"{list(BETA_RULES.keys())}."
            )
        self._beta_rule = beta_rule
        self._orth_value = orth_value

        if line_searcher is None:
            self._line_searcher = AdaptiveLineSearcher()
        else:
            self._line_searcher = line_searcher
        self.line_searcher = None
```

--------------------------------

### JAX Backend Decorator

Source: https://pymanopt.org/docs/stable/autodiff.html

Decorator for using the JAX backend with Pymanopt. It requires a manifold instance.

```python
pymanopt.function.jax(_manifold_)
```

--------------------------------

### Custom Line Search for MoG with Pymanopt

Source: https://pymanopt.org/docs/stable/_sources/examples/notebooks/mixture_of_gaussians.ipynb.txt

This class implements a backtracking line-search that checks for close-to-singular matrices, preventing issues in MoG model fitting. It requires the objective function, manifold, current point, descent direction, and directional derivative as inputs.

```python
class LineSearchMoG:
    """
    Back-tracking line-search that checks for close to singular matrices.
    """

    def __init__(
        self,
        contraction_factor=0.5,
        optimism=2,
        sufficient_decrease=1e-4,
        max_iterations=25,
        initial_step_size=1,
    ):
        self.contraction_factor = contraction_factor
        self.optimism = optimism
        self.sufficient_decrease = sufficient_decrease
        self.max_iterations = max_iterations
        self.initial_step_size = initial_step_size

        self._oldf0 = None

    def search(self, objective, manifold, x, d, f0, df0):
        """
        Function to perform backtracking line-search.
        Arguments:
            - objective
                objective function to optimise
            - manifold
                manifold to optimise over
            - x
                starting point on the manifold
            - d
                tangent vector at x (descent direction)
            - df0
                directional derivative at x along d
        Returns:
            - step_size
                norm of the vector retracted to reach newx from x
            - newx
                next iterate suggested by the line-search
        """
        # Compute the norm of the search direction
        norm_d = manifold.norm(x, d)

        if self._oldf0 is not None:
            # Pick initial step size based on where we were last time.
            alpha = 2 * (f0 - self._oldf0) / df0
            # Look a little further
            alpha *= self.optimism
        else:
            alpha = self.initial_step_size / norm_d
        alpha = float(alpha)

        # Make the chosen step and compute the cost there.
        newx, newf, reset = self._newxnewf(x, alpha * d, objective, manifold)
        step_count = 1

        # Backtrack while the Armijo criterion is not satisfied
        while (
            newf > f0 + self.sufficient_decrease * alpha * df0
            and step_count <= self.max_iterations
            and not reset
        ):

            # Reduce the step size
            alpha = self.contraction_factor * alpha

            # and look closer down the line
            newx, newf, reset = self._newxnewf(
                x, alpha * d, objective, manifold
            )

            step_count = step_count + 1

        # If we got here without obtaining a decrease, we reject the step.
        if newf > f0 and not reset:
            alpha = 0
            newx = x

        step_size = alpha * norm_d

        self._oldf0 = f0

        return step_size, newx

    def _newxnewf(self, x, d, objective, manifold):
        newx = manifold.retraction(x, d)
        try:
            newf = objective(newx)
        except np.linalg.LinAlgError:
            replace = np.asarray(
                [
                    np.linalg.matrix_rank(newx[0][k, :, :])
                    != newx[0][0, :, :].shape[0]
                    for k in range(newx[0].shape[0])
                ]
            )
            x[0][replace, :, :] = manifold.random_point()[0][replace, :, :]
            return x, objective(x), True
        return newx, newf, False
```

--------------------------------

### TrustRegions.run

Source: https://pymanopt.org/docs/stable/_modules/pymanopt/optimizers/trust_regions.html

Executes the Riemannian Trust-Regions optimization algorithm on a given problem.

```APIDOC
## TrustRegions.run

### Description
Runs the Riemannian Trust-Regions optimization algorithm to find the minimum of a cost function on a manifold.

### Parameters
#### Path Parameters
- **problem** (Object) - Required - The optimization problem containing the manifold, cost function, gradient, and Hessian.

#### Query Parameters
- **initial_point** (Point) - Optional - The starting point on the manifold. If not provided, a random point is generated.
- **mininner** (int) - Optional - Minimum number of inner iterations for the conjugate gradient solver.
- **maxinner** (int) - Optional - Maximum number of inner iterations. Defaults to the manifold dimension.
- **Delta_bar** (float) - Optional - Maximum trust region radius.
- **Delta0** (float) - Optional - Initial trust region radius.

### Response
#### Success Response (200)
- **OptimizerResult** (Object) - An object containing the optimization results, including the final point and cost.
```

--------------------------------

### Elliptope Manifold Constructor

Source: https://pymanopt.org/docs/stable/manifolds.html

Initializes the Elliptope manifold, which consists of fixed-rank positive semidefinite (PSD) matrices with unit diagonal elements.

```APIDOC
## _class _pymanopt.manifolds.psd.Elliptope(_n_, _k_)

### Description
Manifold of fixed-rank PSD matrices with unit diagonal elements.

### Method
Constructor

### Endpoint
Not applicable (class instantiation)

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
None

### Request Example
```python
Elliptope(n=5, k=3)
```

### Response
#### Success Response (200)
- **manifold_instance** (Elliptope) - An instance of the Elliptope manifold.

#### Response Example
```python
# Returns an Elliptope manifold object
```

### Parameters
* **n** (int) - Number of rows and columns of a point in the ambient space.
* **k** (int) - Rank of matrices in the ambient space.

### Note
A point X on the manifold is parameterized as X=YY⊤ where Y is a matrix of size `n x k` and rank `k`. As such, X is symmetric, positive semidefinite with rank `k`.
```

--------------------------------

### Define and Solve Optimization Problem

Source: https://pymanopt.org/docs/stable/_sources/examples/notebooks/mixture_of_gaussians.ipynb.txt

Instantiates a product manifold, defines a cost function using autograd, and runs a steepest descent optimizer.

```python
import sys


sys.path.insert(0, "../..")

from autograd.scipy.special import logsumexp

import pymanopt
from pymanopt import Problem
from pymanopt.manifolds import Euclidean, Product, SymmetricPositiveDefinite
from pymanopt.optimizers import SteepestDescent


# (1) Instantiate the manifold
manifold = Product([SymmetricPositiveDefinite(D + 1, k=K), Euclidean(K - 1)])

# (2) Define cost function
# The parameters must be contained in a list theta.
@pymanopt.function.autograd(manifold)
def cost(S, v):
    # Unpack parameters
    nu = np.append(v, 0)

    logdetS = np.expand_dims(np.linalg.slogdet(S)[1], 1)
    y = np.concatenate([samples.T, np.ones((1, N))], axis=0)

    # Calculate log_q
    y = np.expand_dims(y, 0)

    # 'Probability' of y belonging to each cluster
    log_q = -0.5 * (np.sum(y * np.linalg.solve(S, y), axis=1) + logdetS)

    alpha = np.exp(nu)
    alpha = alpha / np.sum(alpha)
    alpha = np.expand_dims(alpha, 1)

    loglikvec = logsumexp(np.log(alpha) + log_q, axis=0)
    return -np.sum(loglikvec)


problem = Problem(manifold, cost)

# (3) Instantiate a Pymanopt optimizer
optimizer = SteepestDescent(verbosity=1)

# let Pymanopt do the rest
Xopt = optimizer.run(problem).point
```
Tokens:
Raw